I think I might know...
Would you ever consider getting a second pet?
I think that may be the problem... he's lonely, if you don't want babies or would be willing to sell them then get another male (nuetered) or a female (spayed) There are too many unwanted kittens, people have thousands to choose from, they don't want more, even if they'll take them.