Company asking for payment of non served notice period

Private employment is governed by your employment contract.

For a specific answer to your question, and to look at what all option your have, your employment contract need to be looked into. Generally, there are 2 options available to every employee at the time of resignation.

Either you can serve the notice period specified in the  employment contract or you can pay your monthly salary for the notice period month(s) as mentioned in your employment contract / letter.