### Description
Kamailio pipelimit module causes high CPU load (20 - 30 %) on servers with > 48
GB RAM even if Kamailio is idle (no active calls).
strace shows that every second `open /pro/stat` is called even if
timer_interval is set to 10 seconds
`modparam("pipelimit", "timer_interval", 10)`
`1539343630.677932 open("/proc/stat", O_RDONLY) = 6`
`1539343630.678278 open("/sys/devices/system/cpu/online", O_RDONLY|O_CLOEXEC) =
6`
`1539343630.678418 open("/proc/net/udp", O_RDONLY) = 6`
`1539343630.679448 open("/proc/net/tcp", O_RDONLY) = 6`
`1539343630.722522 open("/proc/net/tcp", O_RDONLY) = 6`
`1539343630.753095 open("/proc/net/udp", O_RDONLY) = 6`
`1539343630.753838 open("/proc/net/tcp", O_RDONLY) = 6`
`1539343630.783710 open("/proc/net/tcp", O_RDONLY) = 6`
`1539343631.677966 open("/proc/stat", O_RDONLY) = 6`
`1539343631.678378 open("/sys/devices/system/cpu/online", O_RDONLY|O_CLOEXEC) =
6`
`1539343631.678534 open("/proc/net/udp", O_RDONLY) = 6`
`1539343631.679784 open("/proc/net/tcp", O_RDONLY) = 6`
`1539343631.717329 open("/proc/net/tcp", O_RDONLY) = 6`
`1539343631.760763 open("/proc/net/udp", O_RDONLY) = 6`
`1539343631.761786 open("/proc/net/tcp", O_RDONLY) = 6`
`1539343631.805426 open("/proc/net/tcp", O_RDONLY) = 6`
strace shows that with 8 GB RAM the read syscall takes 100us (average) - with
48GB it takes 586us (average)!
see 5 seconds interval strace for 48GB:
`2018-09-21 07:55:38,201`
`strace: Process 6907 attached`
`strace: Process 6907 detached`
`% time seconds usecs/call calls errors syscall`
`------ ----------- ----------- --------- --------- ----------------`
`99.79 1.259694 586 2148 read`
` 0.06 0.000761 19 41 open`
` 0.04 0.000469 8 61 61 pause`
` 0.04 0.000462 13 36 munmap`
` 0.03 0.000336 8 41 close`
` 0.02 0.000257 7 36 mmap`
` 0.02 0.000234 3 80 61 rt_sigreturn`
` 0.01 0.000113 3 36 fstat`
` 0.00 0.000017 3 5 kill`
`------ ----------- ----------- --------- --------- ----------------`
`100.00 1.262343 2484 122 total`
`2018-09-21 07:55:43,225`
In the documentation
(https://kamailio.org/docs/modules/5.1.x/modules/pipelimit.html) I found the
following:
> As reading the CPU load average is relatively expensive (opening /proc/stat,
> parsing it, etc), this only happens once every timer_interval seconds and
> consequently the FEEDBACK value is only at these intervals recomputed. This
> in turn makes it difficult for the drop rate to adjust quickly. Worst case
> scenarios are request rates going up/down instantly by thousands - it takes
> up to 20 seconds for the controller to adapt to the new request rate.
IMHO the doc is misleading because the periodic timer used for this is
hardcoded with 1000ms.
see
`( 349 pipelimit.c ) timer_add(pl_timer, MS_TO_TICKS(1000)); /* Start it after
1000ms */`
Conclusion:
Pipelimit polls periodically every second /proc/stats and /proc/net/udp|tcp.
The modparam timer_interval does not affect the timer period.
On servers with >48 GB this causes high CPU load.
#### Reproduction
always reproducible.
### Possible Solutions
Use modparam timer_interval to initialize the `pl_timer` interval
### Additional Information
version: kamailio 5.1.2 (x86_64/linux)
flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, DISABLE_NAGLE,
USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC,
TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE,
USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16,
MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: unknown
* **Operating System**: Centos 7.4
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/1675_______________________________________________
Kamailio (SER) - Development Mailing List
[email protected]
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
