Re: Kannel Redis - can I force the destination in the key name?

[Robert Chen]

Thanks for the clarification. Turned out that in my case, some of the
message IDs would get recycled given enough time and volume. So in order to
generate unique redis keys, I still had to include the destination.

In case anyone is interested, I ended up making a local change to
gw/smsc/smsc_smpp.c, by setting the use_dst flag to 1 for the dlr_find and
dlr_add method calls. Just had to be mindful that the redis key only uses
the last 7 characters of the destination.

@@ -1535,7 +1535,7 @@

 dlrmsg = dlr_find(smpp->conn->id,

 tmp, /* smsc message id */

 destination_addr, /* destination */

-dlrstat, 0);

+dlrstat, 1);



 octstr_destroy(msgid);

 } else

@@ -1854,7 +1854,7 @@

  * order to get it logged to access-log.

  */

 if (DLR_IS_ENABLED_DEVICE(msg->sms.dlr_mask)) {

-dlr_add(smpp->conn->id, tmp, msg, 0);

+dlr_add(smpp->conn->id, tmp, msg, 1);

 octstr_destroy(tmp);

 } else {

 octstr_destroy(msg->sms.foreign_id);


On Tue, Nov 1, 2016 at 5:33 PM, Donald Jackson 
wrote:

> Ts is just the old variable name, the remote message id is used for that
> field, not a time stamp. So as long as your supplier provides a unique
> message ID you won't have collisions
>
>
> On Tuesday, 01 November 2016, Robert Chen  thorntechnologies.com> wrote:
>
>> Hi, I'm trying to set up Kannel with Redis. My concern is that some DLRs
>> would be lost due to timestamp collisions. And my question is whether
>> hard-coding dlr_redis_add to use the destination (at least for my personal
>> installation) is the best approach.
>>
>> dlr_redis.c:
>> Changing:
>> if (entry->use_dst && entry->destination)
>> To this:
>> if (entry->destination) { ... }
>>
>> Here's my train of thought:
>>
>> I'm load-balancing Kannel. And according to community recommendations,
>> each instance is using the same smsc-id:
>>
>>- http://kannel.6189.n7.nabble.com/How-to-Multiple-SMSC-s-conf
>>iguration-and-how-to-Load-balancing-B-W-Muliple-SMSC-s-td17874.html
>>
>> 
>>- https://francispereira.com/kannel-setting-up-active-active-l
>>oad-balanced-smpp-gateways.html
>>
>> 
>>- http://stackoverflow.com/questions/10322269/kannelroute-mess
>>age-through-2-smsc
>>
>> The current Kannel setup is using 1.4.4 with MySQL. I'm in the process of
>> migrating the whole thing to cloud hosting, but using Redis this time.
>>
>> I'm worried that DLRs could be lost if they come back with the same
>> timestamp. Within dlr_redis.c, it looks like it does one of two things:
>>
>> 1. If the REDIS_PRECHECK flag is set, it uses HSETNX (only creating a key
>> if it does not exist)
>> 2. Otherwise, it uses HMSET (overriding whatever dictionary is tied to
>> that key)
>>
>> Either way, it looks like one of the DLRs will get lost if there's ever a
>> conflict.
>>
>> Within dlr_redis.c, it looks like the key can be one of two formats:
>>
>> 1. ::
>> 2. :::
>>
>> After analyzing a MySQL CSV dump containing 1 million messages, I'm
>> finding rows with identical timestamps. And for Redis, this could spell a
>> key collision. But, I can get all the keys to be unique by including the
>> destination.
>>
>> I'm trying to figure out how to force Kannel to include the destination.
>> According to the documentation, it seems like this is determined by the
>> SMSC:
>>
>> “Some SMSCs also append the destination to the DLR keyname resulting DLR
>> keynames in the format :::.”
>>
>> It seems as though cimd2 and emi are the SMSC types that append the
>> destination.
>>
>> gateway-1.4.4/gw $ grep -iR dlr_add .
>> ./smsc/http/clickatell.c:dlr_add(conn->id, msgid, msg, 0);
>> ./smsc/http/generic.c:dlr_add(conn->id, msgid,
>> msg, 0);
>> ./smsc/http/generic.c:dlr_add(conn->id, msgid, msg, 0);
>> ./smsc/http/xidris.c:dlr_add(conn->id, mid, msg, 0);
>> ./smsc/smsc_at.c:dlr_add(privdata->conn->id,
>> dlrmsgid, msg, 0);
>> ./smsc/smsc_cgw.c:dlr_add(conn->id, ts, msg, 0);
>> ./smsc/smsc_cimd2.c:dlr_add(conn->name, ts, msg, 1);
>> ./smsc/smsc_emi.c: dlr_add((conn->id ? conn->id : privdata->name),
>> ts, m, 1);
>> ./smsc/smsc_fake.c:dlr_add(conn->id, tmp, sms, 0);
>> ./smsc/smsc_http.c:dlr_add(conn->id, mid, msg, 0);
>> ./smsc/smsc_loopback.c:dlr_add(conn->id, mid, sms, 0);
>> ./smsc/smsc_oisd.c:dlr_add(conn->name, ts, msg, 0);
>> ./smsc/smsc_smpp.c:dlr_add(smpp->conn->id, tmp, msg,
>> 0);
>> ./smsc/smsc_soap.c:dlr_add(conn->id, 

Re: Kannel Redis - can I force the destination in the key name?

[Donald Jackson]

Ts is just the old variable name, the remote message id is used for that
field, not a time stamp. So as long as your supplier provides a unique
message ID you won't have collisions

On Tuesday, 01 November 2016, Robert Chen 
wrote:

> Hi, I'm trying to set up Kannel with Redis. My concern is that some DLRs
> would be lost due to timestamp collisions. And my question is whether
> hard-coding dlr_redis_add to use the destination (at least for my personal
> installation) is the best approach.
>
> dlr_redis.c:
> Changing:
> if (entry->use_dst && entry->destination)
> To this:
> if (entry->destination) { ... }
>
> Here's my train of thought:
>
> I'm load-balancing Kannel. And according to community recommendations,
> each instance is using the same smsc-id:
>
>- http://kannel.6189.n7.nabble.com/How-to-Multiple-SMSC-s-conf
>iguration-and-how-to-Load-balancing-B-W-Muliple-SMSC-s-td17874.html
>
> 
>- https://francispereira.com/kannel-setting-up-active-active-
>load-balanced-smpp-gateways.html
>
> 
>- http://stackoverflow.com/questions/10322269/kannelroute-
>message-through-2-smsc
>
> The current Kannel setup is using 1.4.4 with MySQL. I'm in the process of
> migrating the whole thing to cloud hosting, but using Redis this time.
>
> I'm worried that DLRs could be lost if they come back with the same
> timestamp. Within dlr_redis.c, it looks like it does one of two things:
>
> 1. If the REDIS_PRECHECK flag is set, it uses HSETNX (only creating a key
> if it does not exist)
> 2. Otherwise, it uses HMSET (overriding whatever dictionary is tied to
> that key)
>
> Either way, it looks like one of the DLRs will get lost if there's ever a
> conflict.
>
> Within dlr_redis.c, it looks like the key can be one of two formats:
>
> 1. ::
> 2. :::
>
> After analyzing a MySQL CSV dump containing 1 million messages, I'm
> finding rows with identical timestamps. And for Redis, this could spell a
> key collision. But, I can get all the keys to be unique by including the
> destination.
>
> I'm trying to figure out how to force Kannel to include the destination.
> According to the documentation, it seems like this is determined by the
> SMSC:
>
> “Some SMSCs also append the destination to the DLR keyname resulting DLR
> keynames in the format :::.”
>
> It seems as though cimd2 and emi are the SMSC types that append the
> destination.
>
> gateway-1.4.4/gw $ grep -iR dlr_add .
> ./smsc/http/clickatell.c:dlr_add(conn->id, msgid, msg, 0);
> ./smsc/http/generic.c:dlr_add(conn->id, msgid,
> msg, 0);
> ./smsc/http/generic.c:dlr_add(conn->id, msgid, msg, 0);
> ./smsc/http/xidris.c:dlr_add(conn->id, mid, msg, 0);
> ./smsc/smsc_at.c:dlr_add(privdata->conn->id,
> dlrmsgid, msg, 0);
> ./smsc/smsc_cgw.c:dlr_add(conn->id, ts, msg, 0);
> ./smsc/smsc_cimd2.c:dlr_add(conn->name, ts, msg, 1);
> ./smsc/smsc_emi.c: dlr_add((conn->id ? conn->id : privdata->name),
> ts, m, 1);
> ./smsc/smsc_fake.c:dlr_add(conn->id, tmp, sms, 0);
> ./smsc/smsc_http.c:dlr_add(conn->id, mid, msg, 0);
> ./smsc/smsc_loopback.c:dlr_add(conn->id, mid, sms, 0);
> ./smsc/smsc_oisd.c:dlr_add(conn->name, ts, msg, 0);
> ./smsc/smsc_smpp.c:dlr_add(smpp->conn->id, tmp, msg,
> 0);
> ./smsc/smsc_soap.c:dlr_add(conn->id, octstr_imm(tmpid), msg, 0);
> ./smsc/smsc_soap_parlayx.c:dlr_add(conn->id, mid, sms, 0);
> ./smsc/smsc_soap_parlayx.c:dlr_add(conn->id, mid, msg, 0);
> The problem though is that I'm using smpp. (I verified against my SMSC
> that the format coming back is indeed ::)
>
> So I guess my question is, should I just hard-code my local install to
> append the destination? Or is there a better approach or some configuration
> flat I'm just not seeing?
>
> Thanks!
> Robert Chen
>


--