Issues with the Recovery of loan

Good morning sir/ma’am I’ve lended about a crore rupees to the borrower. There has been a loan agreement made about it. And within 6months, person is supposed to repay the principal amount. And as a surity I’ve got a house document which is 40 lakhs worth. From last year onwards I’ve stopped receiving interest for the amount given. The reason being mentioned by borrower is that due to COVID the interest will not be paid. I’ve exempted from taking interest and have been asking to repay the principal amount. Borrower has begun to raise the issue that- asking him to repay money at the moment is not possible and is scaring me that he would file a complaint against me for pressurising him to repay the money back. I’ve got the loan agreement notary done. My question is- does the law support this act of my borrower scaring me to stop asking about the amount as he claims he will file a case against me for pressurising him about money. What legal action can I take about it ?