Mi pare che JMenu e JMenuItem supportino i tag html.
http://java.sun.com/docs/books/tutor...ents/html.html