Politeness, cleanliness and respectfulness are assets in Japan.
Especially when you are visiting at a temple or shrine, as they have a cultural and religious values.
The most common thing I see foreigners making Japanese uncomfortable is the loudness of their voice. Especially in the train, Japanese tend to be quiet, even if you are speaking. So if you speak loudly, it is considered that they are not following manners.
Also, don't talk on the phone in the train. It is not allowed in Japanese culture, too.
However, about the loudness in the train, drunken people are an exception though :P