father gave child in orphanage and asking him back after 21 years

Hello Sir/Madam, In 2013 a man had approached me because I ran an Orphanage wanting to admit his child under our care because his wife had died during birth, the baby was 10days old and was in a very harrowing state of health, we felt pity and accepted the Child in the best interest of caring for the baby. 21/2 years later the Father approaches us seeking to take the Child back, but we feel that he is not in a capable position to take care of the kid, however due to pressure we gave in and he forcefully removed the Child from our home.What are our options, as we know the Child needs morecare