Alejandro, while it's true the dst index wont be used, this is only a third column in index - first ones being smsc and ts , both on exact match, so only the messages fired on same smsc id and same ts will be analyzed by wildcard, which isnt that bad
RE:multipart - right now kannel is only storing one dlr for the whole multipart, correct me if i'm wrong... About amount of digits to compare - i think this should be configurable per smsc as well, this way we can define exactly how many digits after prefix there are and matching would be (almost?) ideal Regards, Konstantin On Thu, Jun 24, 2010 at 6:53 PM, Alejandro Guerrieri <[email protected]>wrote: > I've seen all kind of crap coming from carriers so far, so you could never > be 100% sure that it'd support all cases. > > Also, if you take only the last 5 digits, you have 2 problems: > > 1. 5 digits would not cover all cases, there might be cases (quite > unlikely, true, but still possible) where two different numbers would match > the same last 5 digits. > > 2. You'd forced to do queries looking like this: > > ... AND dst LIKE '%12345' > > That would mean not using an index for the dst column because the > conditions starts with a "wildcard" (at least on MySQL, not sure on other > engines). Put a couple million records on the dlr table and you have a > problem ;) > > Unless, of course, you'd normalize the data you store in first place, and > then do a direct comparison. You'd need to add an index with both columns to > speed things up anyway. > > Regards, > > Alex > -- > Alejandro Guerrieri > [email protected] > > > > On 24/06/2010, at 17:27, Nikos Balkanas wrote: > > > Hi Alex, > > > > As usual your 2 cents are worth a lifetime :-) > > > > It is true that I should probably make it configurable. Bear in mind that > by taking just the last 5 digits, I avoid all prefix issues (as demonstrated > by Victor). You have much more experience than me with SMS. How safe would > that be? > > > > BR, > > Nikos > > ----- Original Message ----- From: "Alejandro Guerrieri" < > [email protected]> > > To: "Nikos Balkanas" <[email protected]> > > Cc: <[email protected]>; <[email protected]> > > Sent: Thursday, June 24, 2010 6:12 PM > > Subject: Re: Patch: EMI UUCP DLR > > > > > > Actually, you could face tons of problems if you add DST in all cases. > > > > The format could be way different than what you've sent. Assuming it'll > be even similar to what you've sent is far from accurate. Some carriers > could add prefixes of all kinds or require you to add prefixes for billing > purposes what wouldn't come back on the DLR at all. > > > > If you add that functionality I think it should be user-configurable and > also you should be able to disabled it completely to revert back to what it > is now. > > > > Many, many people using SMPP is fine with the current approach and > wouldn't need the extra logic at all. > > > > Just my 2 cents :) > > > > Regards, > > > > Alex > > -- > > Alejandro Guerrieri > > [email protected] > > > > > > > > On 24/06/2010, at 17:01, Nikos Balkanas wrote: > > > >> Hi, > >> > >>> This patch will be rejected as we need a global resolution > >>> of the problem. I've already posted this patch a couple of years ago. > >>> (http://www.mail-archive.com/[email protected]/msg05349.html) > >> > >> I didn't know about your patch, I joined kannel a bit after, I think :-) > >> My patch is different and more global. It handles cases when no dst is > known and when it is known. It also patches all dbs supported by kannel. > However, I was not aware that dst could come in with less digits than sent, > so I will have to address this. Thanks for pointing this out. Instead of > using the whole dst for the query, I could just use the last 5 digits and > like. However, that would mean extra processing in the case where dst is > known (like...) therefore I would like a solid confirmation that indeed > source can be different in the DLR than destination in the MT sent. > >> > >>> Also your patch does not fully-fixes this issue. > >>> If 2 MT are sended on the same second. > >>> Which DLR will you get ? (probably the first one inserted on your db) > >> > >> Nope. That was what this patch was created for. If dst is known, then > dst (along with ts and smsc-id) are used for the query. Therefore it will > discriminate between 2 MTs at the same time from their destination. It is > assumed, that if you don't use unique ts (ie timestamp) you will at least > include a dst. > >> > >> BR, > >> Nikos > >> > >> ----- Original Message ----- From: "Vincent CHAVANIS" < > [email protected]> > >> To: <[email protected]> > >> Sent: Thursday, June 24, 2010 5:27 PM > >> Subject: Re: Patch: EMI UUCP DLR > >> > >> > >> Hi Nikos, > >> > >> This patch will be rejected as we need a global resolution > >> of the problem. I've already posted this patch a couple of years ago. > >> (http://www.mail-archive.com/[email protected]/msg05349.html) > >> > >> Also your patch does not fully-fixes this issue. > >> If 2 MT are sended on the same second. > >> Which DLR will you get ? (probably the first one inserted on your db) > >> > >> > >> Vincent. > >> > >> -- > >> Telemaque - 06560 SOPHIA-ANTIPOLIS - (FR) > >> Service Technique/Reseau - NOC > >> Direction du Developpement xMS+ > >> http://www.telemaque.fr/ > >> [email protected] > >> > >> Le 24/06/2010 12:14, Nikos Balkanas a écrit : > >>> Hi, > >>> > >>> This patch to dlr_mysql, dlr_pgsql, dlr_oracle, dlr_mssql, dlr_mem & > >>> dlr_sdb adds support for destination number on dlr_get, dlr_remove & > >>> dlr_update. This is to fix a long known issue with EMI DLRs, where they > >>> use non-unique ts (timestamp) and kannel mismatches DLRs. It compiles > >>> clean, but I cannot test it. > >>> > >>> Notes: > >>> > >>> 1) It needs dst NULL if it doesn't exist (some SMScs don't include it > in > >>> DLRs). Going through the code this is valid for smpp & EMI. > >>> 2) dlr_oracle.c already had implemented dst in querries. How come it > >>> didn't conflict with said SMScs? Aligned it with rest. > >>> 3) There is no dlr_mem_update. Shouldn't there be? > >>> > >>> @Luis: Could you please test it? > >>> > >>> BR, > >>> Nikos > >>> ----- Original Message ----- From: Luis Tiago Rico > >>> To: [email protected] > >>> Sent: Monday, June 14, 2010 12:28 PM > >>> Subject: RE: DLR matches wrong message > >>> > >>> > >>> This is a known problem to Kannel. > >>> Kannel, using EMI connection, cannot guarantee DLR matching of messages > >>> sent at the same timestamp. > >>> > >>> Why? > >>> Because it does not compare the MSISDN! It only queries DLR > >>> β€�database’, with the timestamp (even that is not in > milliseconds) > >>> and SMSC. > >>> I think there is a patch for this problem (don’t know where). Or > if > >>> you manage, you can change the Kannel source code, and add the MSISDN > to > >>> the database query…. Some of these days I will have to do it for > myself. > >>> > >>> And why this haven’t been included in the last release? > >>> Because this does not work for all SMSC. Not all of them send back the > >>> MSISDN… > >>> > >>> Hope my explanation was correct! > >>> > >>> Best Regards, > >>> Tiago Rico > >>> > >>> From: [email protected] [mailto:[email protected]] On > >>> Behalf Of Konstantin Vayner > >>> Sent: segunda-feira, 14 de Junho de 2010 09:04 > >>> To: Kannel Users > >>> Subject: DLR matches wrong message > >>> > >>> Hi everyone, > >>> > >>> I'm using emi/ucp smsc under kannel 1.4.3. Here's smsc group > >>> configuration (sensitive information masked): > >>> > >>> group = smsc > >>> smsc = emi > >>> smsc-id = ucp_smsc > >>> log-file = "/var/log/kannel/ucp_smsc.log" > >>> log-level = 0 > >>> host = 1.2.3.4 > >>> port = 1234 > >>> smsc-username = "678" > >>> smsc-password = XYZPWZYX > >>> keepalive = 50 > >>> idle-timeout = 60 > >>> source-addr-autodetect = yes > >>> allowed-smsc-id = ucp_smsc > >>> flow-control = 1 > >>> throughput = 8 > >>> > >>> The problem is: messages sent on same timestamp mix up in DLRs even > >>> though they are sent to different destinations. E.g. sending to X and Y > >>> at the same time makes kannel send both DLRs to a dlr-url supplied for > X. > >>> > >>> Here's the log: > >>> > >>> 2010-06-13 11:15:33 [8444] [16] DEBUG: EMI2[ucp_smsc]: emi2 parsing > >>> packet: < 04/00043/R/51/A//0541234567:130610111626/43 > > >>> 2010-06-13 11:15:33 [8444] [16] DEBUG: DLR[internal]: Adding DLR > >>> smsc=ucp_smsc, ts=130610111626, src=031111111, dst=0541234567, mask=31, > >>> boxc= > >>> 2010-06-13 11:15:33 [8444] [16] DEBUG: SMSC[ucp_smsc]: creating DLR > message > >>> 2010-06-13 11:15:33 [8444] [16] DEBUG: SMSC[ucp_smsc]: DLR = > >>> > http://127.0.0.1/imsc/interfaces/kannel_http/dlr.php?msg_id=32490951&dlr=%d&reason=%A > >>> > >>> 2010-06-13 11:15:33 [8444] [16] DEBUG: EMI2[ucp_smsc]: Got packet from > >>> the main socket > >>> 2010-06-13 11:15:33 [8444] [16] DEBUG: EMI2[ucp_smsc]: emi2 parsing > >>> packet: < > >>> > 73/00239/O/53/678/0541234567/////////////130610111626/1/107/130610111626/3//4D65737361676520666F7220303534343937363830352C2077697468206964656E74696669636174696F6E2031303036313331313136323620686173206265656E206275666665726564/////////////F4 > >>> > > >>> 2010-06-13 11:15:33 [8444] [16] DEBUG: DLR[internal]: Looking for DLR > >>> smsc=ucp_smsc, ts=130610111626, dst=0541234567, type=4 > >>> 2010-06-13 11:15:33 [8444] [16] DEBUG: DLR[internal]: created DLR > >>> message for URL > >>> < > http://127.0.0.1/imsc/interfaces/kannel_http/dlr.php?msg_id=32490951&dlr=%d&reason=%A > > > >>> > >>> 2010-06-13 11:15:33 [8444] [16] DEBUG: DLR[internal]: DLR not > destroyed, > >>> still waiting for other delivery report > >>> 2010-06-13 11:15:33 [8444] [16] DEBUG: EMI2[ucp_smsc]: emi2 sending > >>> packet: < 73/00020/R/53/A///A0 > > >>> 2010-06-13 11:15:33 [8444] [16] DEBUG: EMI2[ucp_smsc]: emi2 sending > >>> packet: < > >>> > 05/00376/O/51/0549988770/036532407//1//7///////1306102100//////4/1072/005400530054002D0042005A003A000A05D905EA05E805EA002005E405D905E705D305D505E005D505EA000A05D105E105DA000A0032002C003400330036002C003000350036002E00320032002005E905D7000A00310033002F00300036002000310031003A00310033002000680074007400700073003A002F002F007700770077002E0070////1//////0106050003030201020108///3F > >>> > > >>> 2010-06-13 11:15:33 [8444] [16] DEBUG: EMI2[ucp_smsc]: Got packet from > >>> the main socket > >>> 2010-06-13 11:15:33 [8444] [16] DEBUG: EMI2[ucp_smsc]: emi2 parsing > >>> packet: < 05/00043/R/51/A//0549988770:130610111626/46 > > >>> 2010-06-13 11:15:33 [8444] [16] DEBUG: DLR[internal]: Adding DLR > >>> smsc=ucp_smsc, ts=130610111626, src=032222222, dst=0549988770, mask=31, > >>> boxc= > >>> 2010-06-13 11:15:33 [8444] [16] DEBUG: EMI2[ucp_smsc]: emi2 sending > >>> packet: < > >>> > 06/00182/O/51/0549988770/036532407///////////1306102100//////4/0304/006100790070006F0061006C0069006D002E0063006F002E0069006C002F00620061006E006B////1//////0106050003030202020108///B4 > >>> > > >>> 2010-06-13 11:15:33 [8444] [16] DEBUG: EMI2[ucp_smsc]: Got packet from > >>> the main socket > >>> 2010-06-13 11:15:33 [8444] [16] DEBUG: EMI2[ucp_smsc]: emi2 parsing > >>> packet: < 06/00043/R/51/A//0549988770:130610111627/48 > > >>> 2010-06-13 11:15:33 [8444] [16] DEBUG: SMSC[ucp_smsc]: creating DLR > message > >>> 2010-06-13 11:15:33 [8444] [16] DEBUG: SMSC[ucp_smsc]: DLR = > >>> > http://127.0.0.1/imsc/interfaces/kannel_http/dlr.php?msg_id=32490499&dlr=%d&reason=%A > >>> > >>> **this one is still correct** > >>> > >>> ... and then, a few seconds later ... > >>> > >>> 2010-06-13 11:15:46 [8444] [16] DEBUG: EMI2[ucp_smsc]: emi2 parsing > >>> packet: < > >>> > 89/00295/O/53/678/0549988770/////////////130610111626/0/000/130610111638/3//4D65737361676520666F7220303534343937333538352C2077697468206964656E74696669636174696F6E2031303036313331313136323620686173206265656E2064656C697665726564206F6E20323031302D30362D31332061742031313A31363A33382E/////////////A0 > >>> > > >>> 2010-06-13 11:15:46 [8444] [16] DEBUG: DLR[internal]: Looking for DLR > >>> smsc=ucp_smsc, ts=130610111626, dst=0549988770, type=1 > >>> 2010-06-13 11:15:46 [8444] [16] DEBUG: DLR[internal]: created DLR > >>> message for URL > >>> < > http://127.0.0.1/imsc/interfaces/kannel_http/dlr.php?msg_id=32490951&dlr=%d&reason=%A > > > >>> > >>> **this one went out to a dlr-url of wrong message** > >>> > >> > >> > >> > >> > > > > >
