Defamation without having any evidence

Hi My relative friend is defaming without having any evidence, that I and his wife is having affair, but it is not true, of course we talk over phone sometime just normal conversation as friends nothing romantic, he has threaten me of murder,. Recently he got a recording of our phone call in which his wife and me are talking as her mother in law is harassing her so just abused that's it. Now he is threatenning me to leave the city else will murder me. He keep harassing his wife daily and torture with belt. She is afraid and can't do much. Please guide how should I proceed, as my reputation is destroying day by day. Pls guide