Should I open a GH issue and move the conversation to there? Thanks Henning.
On Mon, Nov 25, 2019 at 10:36 AM Henning Westerholt <[email protected]> wrote: > Hi Joel, > > > > strange. One reason might be the simple query that the dispatcher module > is doing, it is basically doing a full table load. Maybe it makes sense to > disable the warning in this particular case. > > > > I can’t dig into it right now more, let’s see if somebody of the > developers of this module might be able to comment here. > > > > Cheers, > > > > Henning > > > > -- > > Henning Westerholt – https://skalatan.de/blog/ > > Kamailio services – https://gilawa.com > > Kamailio Merchandising – https://skalatan.de/merchandising > > > > *From:* Joel Serrano <[email protected]> > *Sent:* Monday, November 25, 2019 7:19 PM > *To:* Henning Westerholt <[email protected]> > *Cc:* Kamailio (SER) - Users Mailing List <[email protected]> > *Subject:* Re: [SR-Users] Avoid full table scan for dispatcher table on > db_redis using ds_reload() from config script > > > > Hi Henning, > > > > I did read the lengthy explanation.. I also tried your suggestion for the > dispatcher keys modparam but still same problem: > > > > Nov 25 18:01:13 test COPS[17825]: DEBUG: db_redis [redis_dbase.c:1761]: > db_redis_query(): querying prefix (table) 'dispatcher' > Nov 25 18:01:13 test COPS[17825]: DEBUG: db_redis [redis_dbase.c:1811]: > db_redis_query(): no columns given to build query keys, falling back to > full table scan > Nov 25 18:01:13 test COPS[17825]: DEBUG: <core> [db_res.c:119]: > db_new_result(): allocate 56 bytes for result set at 0x7efe768040b0 > Nov 25 18:01:13 test COPS[17825]: DEBUG: <core> [db_res.c:156]: > db_allocate_columns(): allocate 40 bytes for result names at 0x7efe76804150 > Nov 25 18:01:13 test COPS[17825]: DEBUG: <core> [db_res.c:167]: > db_allocate_columns(): allocate 20 bytes for result types at 0x7efe768041e0 > Nov 25 18:01:13 test COPS[17825]: WARNING: db_redis [redis_dbase.c:1098]: > db_redis_perform_query(): performing full table scan on table 'dispatcher' > while performing query > > > > I've tried: > > > > modparam("db_redis", "keys", > "version=entry:table_name;dispatcher=entry:id") > > > > And also separated in different declarations: > > > > modparam("db_redis", "keys", "version=entry:table_name") > > modparam("db_redis", "keys", "dispatcher=entry:id") > > > > In both cases it's still doing a full table scan and I continue getting > the warning in the logfile. > > > > For the sake of testing I did one more attempt with : > > > > modparam("db_redis", "keys", "version=entry:table_name") > > modparam("db_redis", "keys", "dispatcher=entry:destination") > > > > But same results... > > > > I do see in the logs that the modparam is applied correctly: > > > > Nov 25 18:16:09 test COPS[18195]: DEBUG: db_redis [redis_table.c:200]: > db_redis_print_all_tables(): table dispatcher > Nov 25 18:16:09 test COPS[18195]: DEBUG: db_redis [redis_table.c:203]: > db_redis_print_all_tables(): schema: > Nov 25 18:16:09 test COPS[18195]: DEBUG: db_redis [redis_table.c:208]: > db_redis_print_all_tables(): description: 3 > Nov 25 18:16:09 test COPS[18195]: DEBUG: db_redis [redis_table.c:208]: > db_redis_print_all_tables(): attrs: 3 > Nov 25 18:16:09 test COPS[18195]: DEBUG: db_redis [redis_table.c:208]: > db_redis_print_all_tables(): flags: 0 > Nov 25 18:16:09 test COPS[18195]: DEBUG: db_redis [redis_table.c:208]: > db_redis_print_all_tables(): destination: 3 > Nov 25 18:16:09 test COPS[18195]: DEBUG: db_redis [redis_table.c:208]: > db_redis_print_all_tables(): priority: 0 > Nov 25 18:16:09 test COPS[18195]: DEBUG: db_redis [redis_table.c:208]: > db_redis_print_all_tables(): setid: 0 > Nov 25 18:16:09 test COPS[18195]: DEBUG: db_redis [redis_table.c:208]: > db_redis_print_all_tables(): id: 0 > Nov 25 18:16:09 test COPS[18195]: DEBUG: db_redis [redis_table.c:214]: > db_redis_print_all_tables(): entry keys: > Nov 25 18:16:09 test COPS[18195]: DEBUG: db_redis [redis_table.c:217]: > db_redis_print_all_tables(): id > > > > But it seems not to change the full table scan behavior... > > > > Any other suggestions I can try? > > > > I appreciate your help on this. > > > > Thanks, > > Joel. > > > > > > > > On Mon, Nov 25, 2019 at 12:21 AM Henning Westerholt <[email protected]> > wrote: > > Hello Joel, > > > > You probably read already the lengthy explanation at the module overview > docs: > > > > > https://kamailio.org/docs/modules/devel/modules/db_redis.html#db_redis.sec.overview > > > > A small correction about my previous comment – as the dispatcher module > does a full table load anyway during start/re-load, the warning is indeed > harmless, the number of records in the table does not matter here. > > > > Haven’t tried it myself right now, but what about simply: > > > > modparam("db_redis", "keys", "version=entry:table_name;dispatcher=entry:id") > > > > Cheers, > > > > Henning > > > > -- > > Henning Westerholt – https://skalatan.de/blog/ > > Kamailio services – https://gilawa.com > > Kamailio Merchandising – https://skalatan.de/merchandising > > > > *From:* Joel Serrano <[email protected]> > *Sent:* Monday, November 25, 2019 12:39 AM > *To:* Henning Westerholt <[email protected]> > *Cc:* Kamailio (SER) - Users Mailing List <[email protected]> > *Subject:* Re: [SR-Users] Avoid full table scan for dispatcher table on > db_redis using ds_reload() from config script > > > > Hi Henning, > > > > That’s the thing, I don’t really understand the docs, thus way opened this > thread. > > > > Can you give me an example of what the keys modparam for dispatcher table > would look like? I would really appreciate it as I don’t get it after > reading the docs. > > > > I don’t have many gateways as this is a test env but still I want to do > things the correct way. Otherwise every time I reload I get a warning in > the logs and I rather avoid that warning if I can. > > > > Thanks, > > Joel. > > > > On Sat, Nov 23, 2019 at 00:34 Henning Westerholt <[email protected]> wrote: > > Hello Joel, > > > > you already quoted the keys modparam docs. 😊 You need to add an entry > for the dispatcher table to keys as well. > > > > How many records do you actually have in the dispatcher table? If it is a > low number (like < 100) or so, it should work also fine without it. IMHO > the warning is there in case people forget to add it on huge tables. And of > course, it is “the right thing to do” from an implementation point of view. > > > > Cheers, > > > > Henning > > > > -- > > Henning Westerholt – https://skalatan.de/blog/ > > Kamailio services – https://gilawa.com > > Kamailio Merchandising – https://skalatan.de/merchandising > > > > *From:* sr-users <[email protected]> *On Behalf Of *Joel > Serrano > *Sent:* Saturday, November 23, 2019 1:18 AM > *To:* Kamailio (SER) - Users Mailing List <[email protected]> > *Subject:* Re: [SR-Users] Avoid full table scan for dispatcher table on > db_redis using ds_reload() from config script > > > > Just in case: > > > > Kam version - latest nightly deb (5.4.0~dev1+0~20191122005600.1540+buster) > > > > Redis config: > > > > loadmodule "db_redis.so" > > modparam("db_redis", "keys", "version=entry:table_name") > > ... > > modparam("dispatcher", "db_url", "redis://X.X.X.X:6379/2") > > > > > > Thanks! > > Joel. > > > > On Fri, Nov 22, 2019 at 1:48 PM Joel Serrano <[email protected]> wrote: > > Hello, > > > > I'm trying out redis as db backend, and right now I only have dispatcher > records there... Every so often, I do a ds_reload() from within Kam config > script, and I see in logs: > > > Nov 22 20:36:35 test COPS[25531]: WARNING: db_redis [redis_dbase.c:1098]: > db_redis_perform_query(): performing full table scan on table 'dispatcher' > while performing query > > After enabling debug logs: > > > > Nov 22 21:15:35 test COPS[26661]: DEBUG: db_redis [redis_dbase.c:1761]: > db_redis_query(): querying prefix (table) 'dispatcher' > Nov 22 21:15:35 test COPS[26661]: DEBUG: db_redis [redis_dbase.c:1811]: > db_redis_query(): no columns given to build query keys, falling back to > full table scan > Nov 22 21:15:35 test COPS[26661]: DEBUG: <core> [db_res.c:119]: > db_new_result(): allocate 56 bytes for result set at 0x7f0be4273898 > Nov 22 21:15:35 test COPS[26661]: DEBUG: <core> [db_res.c:156]: > db_allocate_columns(): allocate 40 bytes for result names at 0x7f0be4273938 > Nov 22 21:15:35 test COPS[26661]: DEBUG: <core> [db_res.c:167]: > db_allocate_columns(): allocate 20 bytes for result types at 0x7f0be42739c8 > Nov 22 21:15:35 test COPS[26661]: WARNING: db_redis [redis_dbase.c:1098]: > db_redis_perform_query(): performing full table scan on table 'dispatcher' > while performing query > > Given the msg... "no columns given to build query keys, falling back to > full table scan", I assume I'm missing a keys modparam.. but I don't know > how to build it, and the docs (to me) seem confusing? > > > > > https://kamailio.org/docs/modules/devel/modules/db_redis.html#db_redis.p.keys > > > > Can anyone help me understand how to build the keys modparam for > dispatcher table? Or if the reason for this is different, what do I need to > do to avoid the full table scan and the warning in the log? > > > > Thanks, > > Joel. > > > > > >
_______________________________________________ Kamailio (SER) - Users Mailing List [email protected] https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
