Hi, I'm new to OpenSIPS but been running Kamailio on Kubernetes in production on different projects. The same applies to Asterisk and RTPengine.
The networking issue can be easily solved using another networking plugin for k8s like macvlan. You can find a video from a couple of years ago here describing some of the issues we had at EvoSip and how we solved them: https://www.youtube.com/watch?v=QOB998rAyUM Although I would not use anycast anymore, it works but it's a pain to deploy and maintain. So much complexity...I like simple things ;) RTPEngine can be run in kernel space without issues and with the same performances as on VMs. Working ad Wazo we developed an opensource solution of a C4 running on k8s and it was running 600cps without issues with one SBC and two Router nodes with two RTPs. Here are a few articles describing some of the implementations we've done there: https://wazo-platform.org/blog/wazo-platform-c4-overview https://wazo-platform.org/blog/kamailio-ha-dispatcher-and-dmq https://wazo-platform.org/blog/kamailio-routing-with-rtjson-and-http-async-client All of our implementations have an SBC doing only some basic filtering and dispatching the requests to another layer of multiple "routers" that are applying the routing logic. For the SBC I would recommend an HA installation as you would do with VMs, for the rest of the infra you can have everything on the k8s local networking and scale as much as you want without issues. Mind that the SBC should do just the balancing of the SIP messages, all the logic should be on the "routing" layer beneath. We almost always had an API that did the routing logic. So the "router" nodes would query the API which responded what to do with the SIP packet and where to route it. Much easier to maintain and develop and obviously scale. For auto-scaling we used a custom k8s controller in the EvoSIP project, but in Wazo we used Consul for service discovery and populating the dispatcher list in Kamailio which is a better solution IMHO because you can run it on VMs also. I can confirm that it worked like a charm. When you would scale the RTPs or Routers the dispatcher and RTP list will be updated and an rpc call will trigger the reload in Kamailio without restarting it or changing things "manually". Pretty neat. I suppose you can do the same with OpenSIPS, but as told I'm pretty new with it. Just did an integration of a Rating Billing System with the rest_client module and tracer. Please mind that I don't work anymore with the guys at Wazo so I'm not sure about the status of their C4 project which had also a helm chart for deploying it in k8s. Not sure if it is maintained now. We will be publishing some articles regarding your questions in the next months and probably release a Canyan C4 as an open-source project as we did with the Rating Engine. So to recap with your question, yes it is possible and yes it can run in production but it's not quite that simple to do if you have no experience with k8s. You can do it with a default k8s deployment but will not have the performances as you would have with VM as the other guys here mentioned. To have those performances and the scalability and all the nice features of k8s you need some customization of the k8s cluster and moving your logic on an API in order to drive multiple instances of your OpenIPS/Kamailio as "stateless" components to be able to scale. With a deployment on k8s you than have multiple advantages as you can add or remove nodes, have them partially deployed on-premises or in the cloud as you need them. Mind that the RTPs should be as near as possible to the client in order to have a good MOS and so a good QoS. The rest of the infra is messaging so you can deploy it wherever you like. You can, for example, have some nodes on-premises near the client and use the nodeSelector k8s feature to deploy RTPs there. To conclude I would suggest you to try it out. It has multiple advantages that a standard VM deployment does not. It will make your life easier later ;) For any other questions don't hesitate to contact me here or in private. Regards, Alex On Sat, May 2, 2020 at 7:56 AM H Yavari via Users <users@lists.opensips.org> wrote: > > Hi to all, > > As you know docker and K8s, are growing quickly. So we dockerized Asterisk > and OpenSIPS also. > But I see some community members are against it. They have some reasons like > NAT, RTP ports and performance. > > Do you agree with them ? > Is there any successful large scale OpenSIPS cluster based on K8s ? > > > Thanks for sharing your experiences. > > > Regards, > HY > _______________________________________________ > Users mailing list > Users@lists.opensips.org > http://lists.opensips.org/cgi-bin/mailman/listinfo/users _______________________________________________ Users mailing list Users@lists.opensips.org http://lists.opensips.org/cgi-bin/mailman/listinfo/users