wedhornsipgate created an issue (kamailio/kamailio#4426)
**Description:**
We're missing a KEMI function in the htable module that provides the same
functionality as the PV function `$shtcn(table=>pattern)` to count the number
of keys in an htable that match a given pattern (prefix/suffix/regex). The goal
is to let KEMI scripts perform pattern counts without constructing dynamic PV
expressions, which currently causes unbounded pv_cache growth in PKG memory.
**Use case / motivation:**
We implemented rolling-window rate limiting by writing short-lived, per-call
entries into an htable and then counting keys that match a number-derived
pattern. Using $shtcn() works well for the logic, but in KEMI (JS/TypeScript)
we must call it via pv.gete("$shtcn(...)"). Because the PV string varies per
INVITE (different pattern/key each time), these dynamic PV expressions
accumulate in the PV cache, growing PKG/private memory over time.
We’d like a native KEMI function to perform the same count/match so we can
avoid the PV layer entirely.
**Environment**
Kamailio: 5.8.6 on Debian 12
Scripting: KEMI (JavaScript/TypeScript)
Symptom: steady PKG/private memory increase over ~24h under load when using
pv.gete("$shtcn(...)"). Disabling the $shtcn() call stops the growth.
Related log line: the one discussed in
[#3440](https://github.com/kamailio/kamailio/issues/3440) (“pv cache … pkg
memory may get filled with pv declarations”).
Current line in our script:
```if (ksr.pv.gete(`$shtcn(${htable_count}=>%~${number})`) > threshold) { ...
}```
**Current workaround**
Restarting the Kamailio every day or disabling rate limit on instances with
KEMI.
**Idea for API**
Minimal, PV-parity:
```
// KEMI export
int shtcn_match(str table, str pattern, int mode /* 0=exact, 1=prefix,
2=suffix, 3=regex */);
```
JS example:
`const n = KSR.htable.shtcn_match("from_number_count", number, 3); // regex`
More explicit helpers (avoids integer modes):
```
KSR.htable.shtcn_exact(table, key) -> int
KSR.htable.shtcn_prefix(table, prefix) -> int
KSR.htable.shtcn_suffix(table, suffix) -> int
KSR.htable.shtcn_regex(table, pattern) -> int
```
**Behavioral notes / expectations**
Should mirror $shtcn() semantics where reasonable (esp. regex behavior), but
returning an integer directly to KEMI.
Must not allocate PV expressions or interact with pv_cache.
**Why this belongs in htable KEMI**
$shtcn() is already offered at the PV level; parity in KEMI keeps feature
consistency across scripting engines.
It enables memory-safe implementations of rolling-window rate limits and
similar patterns (without touching PV).
**Alternatives considered**
Replace pattern counting with per-bucket counters (sht_inc), which leads to
other problems like increment/decrement handling and locking.
Use ratelimit/pike modules—useful for IP/method limits, but our use case is
PAI/From/To based and currently hinges on pattern counts.
**Acceptance criteria**
A KEMI function exists to count matching keys in an htable without PV parsing.
Verified that repeated calls with dynamic patterns do not grow PV cache / PKG
memory.
Basic tests for exact/prefix/suffix/regex modes (and skip_expired if
implemented).
Thank you!
This would let KEMI users keep $shtcn()-style logic with the same performance
characteristics, but without the PV-cache side effects.
--
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/4426
You are receiving this because you are subscribed to this thread.
Message ID: <kamailio/kamailio/issues/[email protected]>_______________________________________________
Kamailio - Development Mailing List -- [email protected]
To unsubscribe send an email to [email protected]
Important: keep the mailing list in the recipients, do not reply only to the
sender!
