Re: Patch: EMI UUCP DLR

Thu, 24 Jun 2010 11:43:47 -0700 

Dear Alex,
That patch probably predated me. Can i take a look at it? I would be interested to see why it was rejected. EMI is the largest SMS provider, and it is a pitty that kannel treats it as second-grade. 


As pointed by several users, the like is not that expensive, since it is the 
last condition in the query, at which point results are already narrowed 
down to a handful. Still I am planning to implement it only when necessary. 
Or maybe I will just use your patch.


BR,
Nikos

----- Original Message ----- 
From: "Alexander Malysh" <[email protected]>

To: "Alejandro Guerrieri" <[email protected]>
Cc: "Nikos Balkanas" <[email protected]>; <[email protected]>
Sent: Thursday, June 24, 2010 7:33 PM
Subject: Re: Patch: EMI UUCP DLR


Hi,


I'm posted patch for this issue for years now but it was not accepted. We 
use this patch and it works very well.

We trying to skip first up to 10 chars or 7 if number is not large enough.


And yes we use LIKE but we use oracle and create reverse index so that it 
can be used with like '%XXX'.
Even with other engines you will not have any issues because the index on 
dst ist not really needed due to the
fact that you use not only dst in where clause and DB will take other 
indexes from the conditions.


Thanks,
Alexander Malysh

Am 24.06.2010 um 17:53 schrieb Alejandro Guerrieri:


I've seen all kind of crap coming from carriers so far, so you could never 
be 100% sure that it'd support all cases.


Also, if you take only the last 5 digits, you have 2 problems:


1. 5 digits would not cover all cases, there might be cases (quite 
unlikely, true, but still possible) where two different numbers would 
match the same last 5 digits.


2. You'd forced to do queries looking like this:

... AND dst LIKE '%12345'


That would mean not using an index for the dst column because the 
conditions starts with a "wildcard" (at least on MySQL, not sure on other 
engines). Put a couple million records on the dlr table and you have a 
problem ;)



Unless, of course, you'd normalize the data you store in first place, and 
then do a direct comparison. You'd need to add an index with both columns 
to speed things up anyway.


Regards,

Alex
--
Alejandro Guerrieri
[email protected]



On 24/06/2010, at 17:27, Nikos Balkanas wrote:


Hi Alex,

As usual your 2 cents are worth a lifetime :-)


It is true that I should probably make it configurable. Bear in mind that 
by taking just the last 5 digits, I avoid all prefix issues (as 
demonstrated by Victor). You have much more experience than me with SMS. 
How safe would that be?


BR,
Nikos

----- Original Message ----- From: "Alejandro Guerrieri" 
<[email protected]>

To: "Nikos Balkanas" <[email protected]>
Cc: <[email protected]>; <[email protected]>
Sent: Thursday, June 24, 2010 6:12 PM
Subject: Re: Patch: EMI UUCP DLR


Actually, you could face tons of problems if you add DST in all cases.


The format could be way different than what you've sent. Assuming it'll 
be even similar to what you've sent is far from accurate. Some carriers 
could add prefixes of all kinds or require you to add prefixes for 
billing purposes what wouldn't come back on the DLR at all.



If you add that functionality I think it should be user-configurable and 
also you should be able to disabled it completely to revert back to what 
it is now.



Many, many people using SMPP is fine with the current approach and 
wouldn't need the extra logic at all.


Just my 2 cents :)

Regards,

Alex
--
Alejandro Guerrieri
[email protected]



On 24/06/2010, at 17:01, Nikos Balkanas wrote:


Hi,


This patch will be rejected as we need a global resolution
of the problem. I've already posted this patch a couple of years ago.
(http://www.mail-archive.com/[email protected]/msg05349.html)


I didn't know about your patch, I joined kannel a bit after, I think :-)

My patch is different and more global. It handles cases when no dst is 
known and when it is known. It also patches all dbs supported by kannel. 
However, I was not aware that dst could come in with less digits than 
sent, so I will have to address this. Thanks for pointing this out. 
Instead of using the whole dst for the query, I could just use the last 
5 digits and like. However, that would mean extra processing in the case 
where dst is known (like...) therefore I would like a solid confirmation 
that indeed source can be different in the DLR than destination in the 
MT sent.



Also your patch does not fully-fixes this issue.
If 2 MT are sended on the same second.
Which DLR will you get ? (probably the first one inserted on your db)



Nope. That was what this patch was created for. If dst is known, then 
dst (along with ts and smsc-id) are used for the query. Therefore it 
will discriminate between 2 MTs at the same time from their destination. 
It is assumed, that if you don't use unique ts (ie timestamp) you will 
at least include a dst.


BR,
Nikos


----- Original Message ----- From: "Vincent CHAVANIS" 
<[email protected]>

To: <[email protected]>
Sent: Thursday, June 24, 2010 5:27 PM
Subject: Re: Patch: EMI UUCP DLR


Hi Nikos,

This patch will be rejected as we need a global resolution
of the problem. I've already posted this patch a couple of years ago.
(http://www.mail-archive.com/[email protected]/msg05349.html)

Also your patch does not fully-fixes this issue.
If 2 MT are sended on the same second.
Which DLR will you get ? (probably the first one inserted on your db)


Vincent.

--
Telemaque - 06560 SOPHIA-ANTIPOLIS - (FR)
Service Technique/Reseau - NOC
Direction du Developpement xMS+
http://www.telemaque.fr/
[email protected]

Le 24/06/2010 12:14, Nikos Balkanas a Γ©crit :

Hi,

This patch to dlr_mysql, dlr_pgsql, dlr_oracle, dlr_mssql, dlr_mem &
dlr_sdb adds support for destination number on dlr_get, dlr_remove &
dlr_update. This is to fix a long known issue with EMI DLRs, where they
use non-unique ts (timestamp) and kannel mismatches DLRs. It compiles
clean, but I cannot test it.

Notes:


1) It needs dst NULL if it doesn't exist (some SMScs don't include it 
in

DLRs). Going through the code this is valid for smpp & EMI.
2) dlr_oracle.c already had implemented dst in querries. How come it
didn't conflict with said SMScs? Aligned it with rest.
3) There is no dlr_mem_update. Shouldn't there be?

@Luis: Could you please test it?

BR,
Nikos
----- Original Message ----- From: Luis Tiago Rico
To: [email protected]
Sent: Monday, June 14, 2010 12:28 PM
Subject: RE: DLR matches wrong message


This is a known problem to Kannel.
Kannel, using EMI connection, cannot guarantee DLR matching of messages
sent at the same timestamp.

Why?
Because it does not compare the MSISDN! It only queries DLR

�²β�¬ο�½database�²β�¬β�Ά, with the timestamp (even 
that is not in milliseconds)

and SMSC.

I think there is a patch for this problem (don�²β�¬β�Άt know 
where). Or if
you manage, you can change the Kannel source code, and add the MSISDN 
to
the database query�²β�¬Β¦. Some of these days I will have to do 
it for myself.



And why this haven�²β�¬β�Άt been included in the last 
release?

Because this does not work for all SMSC. Not all of them send back the
MSISDN�²β�¬Β¦

Hope my explanation was correct!

Best Regards,
Tiago Rico

From: [email protected] [mailto:[email protected]] On
Behalf Of Konstantin Vayner
Sent: segunda-feira, 14 de Junho de 2010 09:04
To: Kannel Users
Subject: DLR matches wrong message

Hi everyone,

I'm using emi/ucp smsc under kannel 1.4.3. Here's smsc group
configuration (sensitive information masked):

group = smsc
smsc = emi
smsc-id = ucp_smsc
log-file = "/var/log/kannel/ucp_smsc.log"
log-level = 0
host = 1.2.3.4
port = 1234
smsc-username = "678"
smsc-password = XYZPWZYX
keepalive = 50
idle-timeout = 60
source-addr-autodetect = yes
allowed-smsc-id = ucp_smsc
flow-control = 1
throughput = 8

The problem is: messages sent on same timestamp mix up in DLRs even
though they are sent to different destinations. E.g. sending to X and Y

at the same time makes kannel send both DLRs to a dlr-url supplied for 
X.


Here's the log:

2010-06-13 11:15:33 [8444] [16] DEBUG: EMI2[ucp_smsc]: emi2 parsing
packet: < 04/00043/R/51/A//0541234567:130610111626/43 >
2010-06-13 11:15:33 [8444] [16] DEBUG: DLR[internal]: Adding DLR
smsc=ucp_smsc, ts=130610111626, src=031111111, dst=0541234567, mask=31,
boxc=

2010-06-13 11:15:33 [8444] [16] DEBUG: SMSC[ucp_smsc]: creating DLR 
message

2010-06-13 11:15:33 [8444] [16] DEBUG: SMSC[ucp_smsc]: DLR =
http://127.0.0.1/imsc/interfaces/kannel_http/dlr.php?msg_id=32490951&dlr=%d&reason=%A

2010-06-13 11:15:33 [8444] [16] DEBUG: EMI2[ucp_smsc]: Got packet from
the main socket
2010-06-13 11:15:33 [8444] [16] DEBUG: EMI2[ucp_smsc]: emi2 parsing
packet: <
73/00239/O/53/678/0541234567/////////////130610111626/1/107/130610111626/3//4D65737361676520666F7220303534343937363830352C2077697468206964656E74696669636174696F6E2031303036313331313136323620686173206265656E206275666665726564/////////////F4



2010-06-13 11:15:33 [8444] [16] DEBUG: DLR[internal]: Looking for DLR
smsc=ucp_smsc, ts=130610111626, dst=0541234567, type=4
2010-06-13 11:15:33 [8444] [16] DEBUG: DLR[internal]: created DLR
message for URL
<http://127.0.0.1/imsc/interfaces/kannel_http/dlr.php?msg_id=32490951&dlr=%d&reason=%A>


2010-06-13 11:15:33 [8444] [16] DEBUG: DLR[internal]: DLR not 
destroyed,

still waiting for other delivery report
2010-06-13 11:15:33 [8444] [16] DEBUG: EMI2[ucp_smsc]: emi2 sending
packet: < 73/00020/R/53/A///A0 >
2010-06-13 11:15:33 [8444] [16] DEBUG: EMI2[ucp_smsc]: emi2 sending
packet: <
05/00376/O/51/0549988770/036532407//1//7///////1306102100//////4/1072/005400530054002D0042005A003A000A05D905EA05E805EA002005E405D905E705D305D505E005D505EA000A05D105E105DA000A0032002C003400330036002C003000350036002E00320032002005E905D7000A00310033002F00300036002000310031003A00310033002000680074007400700073003A002F002F007700770077002E0070////1//////0106050003030201020108///3F



2010-06-13 11:15:33 [8444] [16] DEBUG: EMI2[ucp_smsc]: Got packet from
the main socket
2010-06-13 11:15:33 [8444] [16] DEBUG: EMI2[ucp_smsc]: emi2 parsing
packet: < 05/00043/R/51/A//0549988770:130610111626/46 >
2010-06-13 11:15:33 [8444] [16] DEBUG: DLR[internal]: Adding DLR
smsc=ucp_smsc, ts=130610111626, src=032222222, dst=0549988770, mask=31,
boxc=
2010-06-13 11:15:33 [8444] [16] DEBUG: EMI2[ucp_smsc]: emi2 sending
packet: <
06/00182/O/51/0549988770/036532407///////////1306102100//////4/0304/006100790070006F0061006C0069006D002E0063006F002E0069006C002F00620061006E006B////1//////0106050003030202020108///B4



2010-06-13 11:15:33 [8444] [16] DEBUG: EMI2[ucp_smsc]: Got packet from
the main socket
2010-06-13 11:15:33 [8444] [16] DEBUG: EMI2[ucp_smsc]: emi2 parsing
packet: < 06/00043/R/51/A//0549988770:130610111627/48 >

2010-06-13 11:15:33 [8444] [16] DEBUG: SMSC[ucp_smsc]: creating DLR 
message

2010-06-13 11:15:33 [8444] [16] DEBUG: SMSC[ucp_smsc]: DLR =
http://127.0.0.1/imsc/interfaces/kannel_http/dlr.php?msg_id=32490499&dlr=%d&reason=%A

**this one is still correct**

... and then, a few seconds later ...

2010-06-13 11:15:46 [8444] [16] DEBUG: EMI2[ucp_smsc]: emi2 parsing
packet: <
89/00295/O/53/678/0549988770/////////////130610111626/0/000/130610111638/3//4D65737361676520666F7220303534343937333538352C2077697468206964656E74696669636174696F6E2031303036313331313136323620686173206265656E2064656C697665726564206F6E20323031302D30362D31332061742031313A31363A33382E/////////////A0



2010-06-13 11:15:46 [8444] [16] DEBUG: DLR[internal]: Looking for DLR
smsc=ucp_smsc, ts=130610111626, dst=0549988770, type=1
2010-06-13 11:15:46 [8444] [16] DEBUG: DLR[internal]: created DLR
message for URL
<http://127.0.0.1/imsc/interfaces/kannel_http/dlr.php?msg_id=32490951&dlr=%d&reason=%A>

**this one went out to a dlr-url of wrong message**




































					Reply via email to