Just a quick question:
How does kamailio acquire memory?
I have started kamailio with 32mb Pkg and 1.5gb shared memory but
when I look at free , it does not seem to match. Only 265 is used.
Does it reside in cache buff?
root 11195 11171 0 18:06 ? 00:00:00
/usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
/usr/local/etc/kamailio//kamailio.cfg -M 32 -m 1536
[root@ ~]# free -m
total used free shared buff/cache available
Mem: 7822 265 3881 377 3675 6862
Swap: 5119 0 5119
[root@msteams-sip-01 ~]#
On Wed, Jun 7, 2023 at 12:32 AM Kaufman <[email protected]> wrote:
>
> While it's not "official" my recollection is that `make uninstall` will
> provide a list of the files to be removed to uninstall kamailio. I've done
> this and validated what I want to keep/delete to make a simple 'uninstall'
> shell script (i.e. I'd want to keep my config files in /etc/kamailio).
>
> You should be able to git pull the desired branch/tag into your working
> directory, run `make distclean`, rerun `./configure` with whatever parameters
> you want, and `make` all without any impact to the running software at all.
> The bulk of the compilation time is running `make`, so there's no reason
> this can't be done ahead of time.
>
> I have, at various times, run the uninstall script and `make install` before
> restarting kamailio and had success, but I would guess the "correct" way to
> do this is to stop kamailio, run the uninstall script, run `make install`,
> then start Kamailio.
>
> You might also consider building packages from source depending if there's
> supporting code for your distribution. While this is slightly more work, it
> does leave you with a simpler installation pattern.
>
>
> - Kaufman
>
> -----Original Message-----
> From: Muhammad Danish Moosa <[email protected]>
> Sent: Tuesday, June 6, 2023 7:26 AM
> To: Henning Westerholt <[email protected]>
> Cc: Kamailio (SER) - Users Mailing List <[email protected]>
> Subject: [SR-Users] Re: Kamailio - could not allocate private memory from pkg
> pool
>
> CAUTION: This email originated from outside the organization. Do not click
> links or open attachments unless you recognize the sender and know the
> content is safe.
>
>
> Hi Henning,
>
> All good,I will re-examine the changes. There were positive changes after
> that. Before that , I had to restart kamailio around every week,
>
> Quick question, is there any cleaner procedure to upgrade from 5.5.3 to
> 5.5.6. I have compiled from sources. Please understand these are production
> systems and should be done with 0 or minimal outage.
>
> Danish
>
>
>
> On Tue, Jun 6, 2023 at 10:14 PM Henning Westerholt <[email protected]> wrote:
> >
> > Hello Muhammad,
> >
> > maybe its related to the changes you've done, maybe its some load increase
> > or similar, but hard to say from the outside.
> >
> > As said, maybe try the minor update first, and then if its persists, lets
> > go the memory debugging route.
> >
> > Cheers,
> >
> > Henning
> >
> > --
> > Henning Westerholt –
> > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fskal
> > atan.de%2Fblog%2F&data=05%7C01%7Cbkaufman%40bcmone.com%7C1c9af403b8b14
> > 266a53608db668a59c4%7Cafc1818e7b6848568913201b9396c4fc%7C1%7C0%7C63821
> > 6516674656903%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2lu
> > MzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=irgb9nc1zWh0X
> > Ssq5JuKlApvUZBlT5nvwpzYKwjHFOg%3D&reserved=0
> > Kamailio services –
> > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgila
> > wa.com%2F&data=05%7C01%7Cbkaufman%40bcmone.com%7C1c9af403b8b14266a5360
> > 8db668a59c4%7Cafc1818e7b6848568913201b9396c4fc%7C1%7C0%7C6382165166746
> > 56903%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJB
> > TiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=KhlWLRiIZwEAb2JvgfUaS
> > gX7kk5WKTWUAfJGiE%2BhLGo%3D&reserved=0
> >
> > > -----Original Message-----
> > > From: Muhammad Danish Moosa <[email protected]>
> > > Sent: Dienstag, 6. Juni 2023 13:58
> > > To: Henning Westerholt <[email protected]>
> > > Cc: Kamailio (SER) - Users Mailing List
> > > <[email protected]>
> > > Subject: Re: [SR-Users] Re: Kamailio - could not allocate private
> > > memory from pkg pool
> > >
> > > Thanks for the response. I really appreciate it.
> > >
> > > This system was tested thoroughly in the lab but problems only
> > > appeared in production. Actually it was working smoothly for 3
> > > months after I fixed some issues like external system calls etc and this
> > > appeared all of sudden.
> > >
> > >
> > >
> > >
> > >
> > > On Tue, Jun 6, 2023 at 9:40 PM Henning Westerholt <[email protected]> wrote:
> > > >
> > > > Hello,
> > > >
> > > > first step should be probably to update to the latest version of
> > > > the 5.5.x branch,
> > > e.g., 5.5.6. There have been several fixes, some also related to
> > > memory leaks included if I remember correctly.
> > > >
> > > > If the problem persists, it might be a new memory leak. This can
> > > > be found by
> > > enabling memory debugging, preferable on a test system or lab system.
> > > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > > www.kamailio.org%2Fwiki%2Ftutorials%2Ftroubleshooting%2Fmemory&dat
> > > > a=05%7C01%7Cbkaufman%40bcmone.com%7C1c9af403b8b14266a53608db668a59
> > > > c4%7Cafc1818e7b6848568913201b9396c4fc%7C1%7C0%7C638216516674656903
> > > > %7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBT
> > > > iI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3c61kNO7YPT6EYTlnl
> > > > LgQARPKyQQ9KJjQkualoxUvJ0%3D&reserved=0
> > > >
> > > > Cheers,
> > > >
> > > > Henning
> > > >
> > > > --
> > > > Henning Westerholt –
> > > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > > skalatan.de%2Fblog%2F&data=05%7C01%7Cbkaufman%40bcmone.com%7C1c9af
> > > > 403b8b14266a53608db668a59c4%7Cafc1818e7b6848568913201b9396c4fc%7C1
> > > > %7C0%7C638216516674656903%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw
> > > > MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C
> > > > &sdata=irgb9nc1zWh0XSsq5JuKlApvUZBlT5nvwpzYKwjHFOg%3D&reserved=0
> > > > Kamailio services –
> > > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > > gilawa.com%2F&data=05%7C01%7Cbkaufman%40bcmone.com%7C1c9af403b8b14
> > > > 266a53608db668a59c4%7Cafc1818e7b6848568913201b9396c4fc%7C1%7C0%7C6
> > > > 38216516674656903%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ
> > > > IjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=K
> > > > hlWLRiIZwEAb2JvgfUaSgX7kk5WKTWUAfJGiE%2BhLGo%3D&reserved=0
> > > >
> > > > > -----Original Message-----
> > > > > From: Muhammad Danish Moosa <[email protected]>
> > > > > Sent: Dienstag, 6. Juni 2023 10:46
> > > > > To: Kamailio (SER) - Users Mailing List
> > > > > <[email protected]>
> > > > > Subject: [SR-Users] Re: Kamailio - could not allocate private
> > > > > memory from pkg pool
> > > > >
> > > > > Mainly if I am looking at the private memory usage , it keeps
> > > > > increasing very slowly but does not go back even after the calls
> > > > > are literally zero. I suspect some UDP listener process are not
> > > > > releasing the
> > > memory.
> > > > >
> > > > > If I consider that my kamailio cfg and lua code has problem with
> > > > > variables or something , I want to understand how it works when
> > > > > calls are released. My understanding is usage should be minimum
> > > > > when calls are released. Looking for feedback please?
> > > > >
> > > > > [roottmp]# kamcmd pkg.stats |grep real_used
> > > > > real_used: 836144
> > > > > real_used: 1017328
> > > > > real_used: 1028656
> > > > > real_used: 1020864
> > > > > real_used: 1020632
> > > > > real_used: 1006120
> > > > > real_used: 1014536
> > > > > real_used: 1023984
> > > > > real_used: 1017584
> > > > > real_used: 835784
> > > > > real_used: 835784
> > > > > real_used: 835784
> > > > > real_used: 844344
> > > > > real_used: 835784
> > > > > real_used: 901344
> > > > > real_used: 835784
> > > > > real_used: 1259728
> > > > > real_used: 1196688
> > > > > real_used: 1181320
> > > > > real_used: 1166624
> > > > > real_used: 1216032
> > > > > real_used: 1173464
> > > > > real_used: 1168552
> > > > > real_used: 1235528
> > > > > real_used: 917904
> > > > >
> > > > >
> > > > >
> > > > > For short term I increase the pvt memory allocation or keep
> > > > > restarting kamailio after every week or fortnight but I am
> > > > > looking for more stable solution. I am using kamailio 5.5.3
> > > > >
> > > > > On Tue, Jun 6, 2023 at 1:21 PM Muhammad Danish Moosa
> > > > > <[email protected]> wrote:
> > > > > >
> > > > > > Any recommendations?
> > > > > > I have good settings for pkg and shm but I can increase more
> > > > > >
> > > > > > [root1 ~]# ps -ef |grep kam
> > > > > > root 1816 1 0 09:48 ? 00:00:00
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1817 1816 0 09:48 ? 00:00:03
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1818 1816 0 09:48 ? 00:00:03
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1819 1816 0 09:48 ? 00:00:03
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1820 1816 0 09:48 ? 00:00:03
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1821 1816 0 09:48 ? 00:00:03
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1822 1816 0 09:48 ? 00:00:03
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1823 1816 0 09:48 ? 00:00:03
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1824 1816 0 09:48 ? 00:00:03
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1825 1816 0 09:48 ? 00:00:00
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1826 1816 0 09:48 ? 00:00:06
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1827 1816 0 09:48 ? 00:00:00
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1828 1816 0 09:48 ? 00:00:00
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1829 1816 0 09:48 ? 00:00:00
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1830 1816 0 09:48 ? 00:00:00
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1831 1816 0 09:48 ? 00:00:00
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1832 1816 0 09:48 ? 00:00:03
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1833 1816 0 09:48 ? 00:00:02
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1834 1816 0 09:48 ? 00:00:02
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1835 1816 0 09:48 ? 00:00:01
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1836 1816 0 09:48 ? 00:00:02
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1837 1816 0 09:48 ? 00:00:02
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1838 1816 0 09:48 ? 00:00:02
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1839 1816 0 09:48 ? 00:00:03
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 1840 1816 0 09:48 ? 00:00:00
> > > > > > /usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
> > > > > > /usr/local/etc/kamailio//kamailio.cfg -M 16 -m 1024
> > > > > > root 3100 3062 0 13:13 pts/1 00:00:00 grep --color=auto
> > > > > > kam
> > > > > > [root@msteams-sip-01 ~]# free -m
> > > > > > total used free shared buff/cache
> > > > > > available
> > > > > > Mem: 7822 221 4055 386 3545
> > > > > > 6900
> > > > > > Swap: 5119 0 5119
> > > > > >
> > > > > > On Tue, Jun 6, 2023 at 1:19 PM Muhammad Danish Moosa
> > > > > > <[email protected]> wrote:
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I have been kamailio services working for few months and
> > > > > > > suddenly I got this error.
> > > > > > >
> > > > > > > ERROR: <core> [core/kemi.c:2923]: sr_kemi_pv_sets(): unable
> > > > > > > to set pv [$rp]
> > > > > > > ERROR: <core> [core/pvapi.c:352]: pv_cache_add(): could not
> > > > > > > allocate private memory from pkg pool
> > > > > > > ERROR: <core> [core/kemi.c:2914]: sr_kemi_pv_sets(): cannot
> > > > > > > get pv spec for
> > > > > > > [$dlg_var([email protected])]
> > > > > > > ERROR: <core> [core/data_lump.c:544]: dup_lump_list_r():
> > > > > > > could not allocate private memory from pkg pool
> > > > > > > ERROR: <core> [core/data_lump.c:582]: dup_lump_list_r():
> > > > > > > out of mem
> > > > > > > ERROR: tm [t_fwd.c:1759]: t_forward_nonack(): failure to
> > > > > > > add branches
> > > > > > > ERROR: <core> [core/msg_translator.c:2501]:
> > > > > > > build_res_buf_from_sip_req(): could not allocate private
> > > > > > > memory from pkg pool
> > > > > > > ERROR: sl [sl_funcs.c:414]: sl_reply_error(): stateless
> > > > > > > error reply
> > > > > > > used: I'm terribly sorry, server error occurred (1/SL)
> > > > > > > ERROR: <core> [core/msg_translator.c:2501]:
> > > > > > > build_res_buf_from_sip_req(): could not allocate private
> > > > > > > memory from pkg pool
> > > > > > > ERROR: tm [t_reply.c:475]: _reply_light(): response
> > > > > > > building failed
> > > > > > > WARNING: tm [t_lookup.c:1499]: t_unref(): generation of a
> > > > > > > delayed stateful reply failed
> > > > > > >
> > > > > > > Issue was fixed after the restart of kamailio.
> > > > > > >
> > > > > > > --
> > > > > > > Muhammad Danish Moosa
> > > > > > >
> > > > > > > " The core of mans' spirit comes from new experiences. "___
> > > > > > > Christopher McCandless
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Muhammad Danish Moosa
> > > > > >
> > > > > > " The core of mans' spirit comes from new experiences. "___
> > > > > > Christopher McCandless
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Muhammad Danish Moosa
> > > > >
> > > > > " The core of mans' spirit comes from new experiences. "___
> > > > > Christopher McCandless
> > > > > __________________________________________________________
> > > > > Kamailio - Users Mailing List - Non Commercial Discussions To
> > > > > unsubscribe send an email to [email protected]
> > > > > Important: keep the mailing list in the recipients, do not reply
> > > > > only to the
> > > sender!
> > > > > Edit mailing list options or unsubscribe:
> > >
> > >
> > >
> > > --
> > > Muhammad Danish Moosa
> > >
> > > " The core of mans' spirit comes from new experiences. "___
> > > Christopher McCandless
>
>
>
> --
> Muhammad Danish Moosa
>
> " The core of mans' spirit comes from new experiences. "___ Christopher
> McCandless __________________________________________________________
> Kamailio - Users Mailing List - Non Commercial Discussions To unsubscribe
> send an email to [email protected]
> Important: keep the mailing list in the recipients, do not reply only to the
> sender!
> Edit mailing list options or unsubscribe:
> __________________________________________________________
> Kamailio - Users Mailing List - Non Commercial Discussions
> To unsubscribe send an email to [email protected]
> Important: keep the mailing list in the recipients, do not reply only to the
> sender!
> Edit mailing list options or unsubscribe:
--
Muhammad Danish Moosa
" The core of mans' spirit comes from new experiences. "___
Christopher McCandless
__________________________________________________________
Kamailio - Users Mailing List - Non Commercial Discussions
To unsubscribe send an email to [email protected]
Important: keep the mailing list in the recipients, do not reply only to the
sender!
Edit mailing list options or unsubscribe:
