I know to some people this might seem gross, but I am amazed at how many men that do not wash their hands in the restrooms. I don't know if they feel it is less manly or what the reasoning is.
There was that old joke where the Army guy told the Marine, "We were taught to wash our hands after going to the restroom." The Marine's response was, "We were taught not to pee on our hand in the restroom."
Maybe that's why.
But, with all the viruses and colds going around, maybe just maybe, swallow your manhood and just give the hand washing a try. You never know, you might like it. I know everyone else will.