Should i break it off with the 50 yr old? (I am 17)?

Yes, find someone your own age. It is clear that he has problems and is a creep. You need to cut off all contact with him.