There are many different places to listen to jazz.
Cotton Club Tokyo may be one of the most famous place:
I have been to MADURO, a jazz bar in Roppongi. It had a quite nice ambience and music was good.
BLUES ALLEY JAPAN Meguro 1-3-14 Meguro, Tokyo Meguro Leon International B1F Tel: 03-5496-4381