Is saying 'You will pay the price' to someone a criminal offence?

On 31st March,at around 12:A.M I got a call,I picked up the call & said 'Hello' for about 8 seconds & then the person on the other side hung up. I was in the mid of my sleep & couldn't understand who was he,so I whatsapped that person,I asked him why did he call & he replied it was by mistake,I asked him if we have talked before on some social sites or anything ,he said no. I jokingly said 'I like hot guys',so that he never calls me again if he is someone who did it intentionally,after that I called him,so that I could recognize his voice because for some reason I wasn't feeling comfortable because of that call,so I called him & he said he did that by mistake & I said it's strange that you dialed a wrong number from a mobile that too at post midnight.He abruptly got cranky & flurried abusive words on me in Hindi for about half a month,I really got upset & I only said that you don't even know me but still abusing me, 'You will pay the price' ,I said that repeatedly for 2-3 times & I hung up.