The mere fact that the OP asked this question here without any suggested solution would indicate to me that there is most likely a lack of understanding the complexity of what is being attempted, if custom coded. Even using standard open source libraries this will be a major project. I have experienced the difficulties many developers have with maintaining a simple comms connection reliably and these connections will be a lot more complicated to manage. Add to that compression, encryption and protocol(s), memory management and limited processing power in handsets and it would certainly be a very interesting project.
Somewhere you will need a server to do initial call routing(look up target, check credentials etc etc) otherwise network performance will be crippled, which will be a project on its own.
You would need voicemail servers, ADC/DAC for voice, call logging, 'SMS' store and forward.
Security will also be a major issue.
Or adapt your requirements and use an off the shelf solution like Skype (I don't like it either, but it works). Or use Viber. This just requires getting to know the products you use.