Re: Kannel Redis - can I force the destination in the key name?

2016-11-01 Thread Donald Jackson
Ts is just the old variable name, the remote message id is used for that
field, not a time stamp. So as long as your supplier provides a unique
message ID you won't have collisions

On Tuesday, 01 November 2016, Robert Chen 
wrote:

> Hi, I'm trying to set up Kannel with Redis. My concern is that some DLRs
> would be lost due to timestamp collisions. And my question is whether
> hard-coding dlr_redis_add to use the destination (at least for my personal
> installation) is the best approach.
>
> dlr_redis.c:
> Changing:
> if (entry->use_dst && entry->destination)
> To this:
> if (entry->destination) { ... }
>
> Here's my train of thought:
>
> I'm load-balancing Kannel. And according to community recommendations,
> each instance is using the same smsc-id:
>
>- http://kannel.6189.n7.nabble.com/How-to-Multiple-SMSC-s-conf
>iguration-and-how-to-Load-balancing-B-W-Muliple-SMSC-s-td17874.html
>
> 
>- https://francispereira.com/kannel-setting-up-active-active-
>load-balanced-smpp-gateways.html
>
> 
>- http://stackoverflow.com/questions/10322269/kannelroute-
>message-through-2-smsc
>
> The current Kannel setup is using 1.4.4 with MySQL. I'm in the process of
> migrating the whole thing to cloud hosting, but using Redis this time.
>
> I'm worried that DLRs could be lost if they come back with the same
> timestamp. Within dlr_redis.c, it looks like it does one of two things:
>
> 1. If the REDIS_PRECHECK flag is set, it uses HSETNX (only creating a key
> if it does not exist)
> 2. Otherwise, it uses HMSET (overriding whatever dictionary is tied to
> that key)
>
> Either way, it looks like one of the DLRs will get lost if there's ever a
> conflict.
>
> Within dlr_redis.c, it looks like the key can be one of two formats:
>
> 1. ::
> 2. :::
>
> After analyzing a MySQL CSV dump containing 1 million messages, I'm
> finding rows with identical timestamps. And for Redis, this could spell a
> key collision. But, I can get all the keys to be unique by including the
> destination.
>
> I'm trying to figure out how to force Kannel to include the destination.
> According to the documentation, it seems like this is determined by the
> SMSC:
>
> “Some SMSCs also append the destination to the DLR keyname resulting DLR
> keynames in the format :::.”
>
> It seems as though cimd2 and emi are the SMSC types that append the
> destination.
>
> gateway-1.4.4/gw $ grep -iR dlr_add .
> ./smsc/http/clickatell.c:dlr_add(conn->id, msgid, msg, 0);
> ./smsc/http/generic.c:dlr_add(conn->id, msgid,
> msg, 0);
> ./smsc/http/generic.c:dlr_add(conn->id, msgid, msg, 0);
> ./smsc/http/xidris.c:dlr_add(conn->id, mid, msg, 0);
> ./smsc/smsc_at.c:dlr_add(privdata->conn->id,
> dlrmsgid, msg, 0);
> ./smsc/smsc_cgw.c:dlr_add(conn->id, ts, msg, 0);
> ./smsc/smsc_cimd2.c:dlr_add(conn->name, ts, msg, 1);
> ./smsc/smsc_emi.c: dlr_add((conn->id ? conn->id : privdata->name),
> ts, m, 1);
> ./smsc/smsc_fake.c:dlr_add(conn->id, tmp, sms, 0);
> ./smsc/smsc_http.c:dlr_add(conn->id, mid, msg, 0);
> ./smsc/smsc_loopback.c:dlr_add(conn->id, mid, sms, 0);
> ./smsc/smsc_oisd.c:dlr_add(conn->name, ts, msg, 0);
> ./smsc/smsc_smpp.c:dlr_add(smpp->conn->id, tmp, msg,
> 0);
> ./smsc/smsc_soap.c:dlr_add(conn->id, octstr_imm(tmpid), msg, 0);
> ./smsc/smsc_soap_parlayx.c:dlr_add(conn->id, mid, sms, 0);
> ./smsc/smsc_soap_parlayx.c:dlr_add(conn->id, mid, msg, 0);
> The problem though is that I'm using smpp. (I verified against my SMSC
> that the format coming back is indeed ::)
>
> So I guess my question is, should I just hard-code my local install to
> append the destination? Or is there a better approach or some configuration
> flat I'm just not seeing?
>
> Thanks!
> Robert Chen
>


--


Kannel Redis - can I force the destination in the key name?

2016-11-01 Thread Robert Chen
Hi, I'm trying to set up Kannel with Redis. My concern is that some DLRs
would be lost due to timestamp collisions. And my question is whether
hard-coding dlr_redis_add to use the destination (at least for my personal
installation) is the best approach.

dlr_redis.c:
Changing:
if (entry->use_dst && entry->destination)
To this:
if (entry->destination) { ... }

Here's my train of thought:

I'm load-balancing Kannel. And according to community recommendations, each
instance is using the same smsc-id:

   - http://kannel.6189.n7.nabble.com/How-to-Multiple-SMSC-s-
   configuration-and-how-to-Load-balancing-B-W-Muliple-SMSC-s-td17874.html
   

   - https://francispereira.com/kannel-setting-up-active-
   active-load-balanced-smpp-gateways.html
   

   - http://stackoverflow.com/questions/10322269/
   kannelroute-message-through-2-smsc
   


The current Kannel setup is using 1.4.4 with MySQL. I'm in the process of
migrating the whole thing to cloud hosting, but using Redis this time.

I'm worried that DLRs could be lost if they come back with the same
timestamp. Within dlr_redis.c, it looks like it does one of two things:

1. If the REDIS_PRECHECK flag is set, it uses HSETNX (only creating a key
if it does not exist)
2. Otherwise, it uses HMSET (overriding whatever dictionary is tied to that
key)

Either way, it looks like one of the DLRs will get lost if there's ever a
conflict.

Within dlr_redis.c, it looks like the key can be one of two formats:

1. ::
2. :::

After analyzing a MySQL CSV dump containing 1 million messages, I'm finding
rows with identical timestamps. And for Redis, this could spell a key
collision. But, I can get all the keys to be unique by including the
destination.

I'm trying to figure out how to force Kannel to include the destination.
According to the documentation, it seems like this is determined by the
SMSC:

“Some SMSCs also append the destination to the DLR keyname resulting DLR
keynames in the format :::.”

It seems as though cimd2 and emi are the SMSC types that append the
destination.

gateway-1.4.4/gw $ grep -iR dlr_add .
./smsc/http/clickatell.c:dlr_add(conn->id, msgid, msg, 0);
./smsc/http/generic.c:dlr_add(conn->id, msgid, msg,
0);
./smsc/http/generic.c:dlr_add(conn->id, msgid, msg, 0);
./smsc/http/xidris.c:dlr_add(conn->id, mid, msg, 0);
./smsc/smsc_at.c:dlr_add(privdata->conn->id,
dlrmsgid, msg, 0);
./smsc/smsc_cgw.c:dlr_add(conn->id, ts, msg, 0);
./smsc/smsc_cimd2.c:dlr_add(conn->name, ts, msg, 1);
./smsc/smsc_emi.c: dlr_add((conn->id ? conn->id : privdata->name), ts,
m, 1);
./smsc/smsc_fake.c:dlr_add(conn->id, tmp, sms, 0);
./smsc/smsc_http.c:dlr_add(conn->id, mid, msg, 0);
./smsc/smsc_loopback.c:dlr_add(conn->id, mid, sms, 0);
./smsc/smsc_oisd.c:dlr_add(conn->name, ts, msg, 0);
./smsc/smsc_smpp.c:dlr_add(smpp->conn->id, tmp, msg, 0);
./smsc/smsc_soap.c:dlr_add(conn->id, octstr_imm(tmpid), msg, 0);
./smsc/smsc_soap_parlayx.c:dlr_add(conn->id, mid, sms, 0);
./smsc/smsc_soap_parlayx.c:dlr_add(conn->id, mid, msg, 0);
The problem though is that I'm using smpp. (I verified against my SMSC that
the format coming back is indeed ::)

So I guess my question is, should I just hard-code my local install to
append the destination? Or is there a better approach or some configuration
flat I'm just not seeing?

Thanks!
Robert Chen