You don't mention if he's been paying you rent but I would go to a solicitor and see what advice he can give you to evict him.The solicitor can draft an official letter asking him to leave and then you could apply for a court order to evict?
As you are scared perhaps go and stay with a friend for a while until this is done or rent the rest of the house out and charge him rent too and evict him if he doesn't pay rent? Or as a last resort sell the house from under him and move into your own new place nearby as you don't have to leave the area.
Keep a record of all the intimidation too. It's difficult to make a decision in a situation like this but you must find courage to as it's affecting your happiness and health, you can do it!! xxxx