Colleague Harassing Me Mentally To Return Money

Took money from colleague to start a side venture, he gave me money, without any form of agreement (solely on his own confidence). He transferred me money using online banking (UPI). Now due to some of his personal reason he is asking me to refund the money quickly. I want to give him money but I will be able to return only in installments. Now he is pressurising me to return money urgently and blackmailing me saying that he would file FIR against me and giving me political pressure as well, saying he has sources with politicians. We have no agreement with each other, can he file FIR against me? What can I do for my own protection. I have intention to return back his money, but I need some time. Please give your advice and suggestions.