How interesting! There’s likely some scent or taste peculiar to you and other men that Stormy finds particularly attractive. It may be a type of cologne or soap, or perhaps your arms have more hair than female visitors and he’s attracted to grooming that. Grooming of course can be a sign of affection but the “power grooming” behavior also can be a way confident cats try to control interaction. Rather than you petting him, he’s determined to be the one in charge and “pet” you instead.
More...