You're sick. As a courtesy to others you wear a mask to prevent transmission of your germs.
You have allergies. Allergies are very common in Japan. The most common allergy is to airborne cedar pollen. Masks help people with such allergies.
Your coworkers are sick. This is a secret motivation to wear a mask that few people will admit to. Tired of everyone giving you a cold — just wear a mask.
Foreigners and MasksMost newcomers to Japan find masks amusing. However, with time it's natural to adhere to local customs.
Why You Have to Wear A MaskIf you're sick and you don't wear a mask you're breaking local customs. Customs related to sanitation are deeply ingrained in every culture.
It's like not taking a shower for three days — people might not say anything but it's not good for your social life or career.
The MysteryJapanese culture generally doesn't consider masks to be unappealing. There's no reason to feel self conscious in a mask (unless your friends back home see you).