This is exactly where a loser wants you to be. Once you are isolated and alone, without support, their control over you can increase. If he is looking for a mommy, then he needs to go back home and move back in with his mommy.
We all know to avoid people that appear insane or abusive and not select them as a dating partner. He takes my pencil and won't give it back until I shake his hand; but most of the time, it's more like holding hands because he holds on and won't let go. If the reputation has two sides, good and bad, your risk is high. When in public, you quickly learn that any opinion you express may cause them to verbally attack you, either at the time or later. Initially, he may even repay these. The most important thing to remember is that the problem is not with you.