Hi Nikos,
Not sure what part of my message your "Nope" applies to.
With regards to the architecture, sqlbox as it stands today is definitely
not compliant. It has no ACK generation at ALL and acts just as a
go-between, intercepting messages from bearerbox, processing them then
passing them to smsbox, and vice-versa. ACKs (if any) must come from a
connected smsbox.
My proposed patch fixes that, by generating the ACK if there's no smsbox
configured.
However, based on your description of the architecture is does little to
bring sqlbox into compliance. That would take quite a bit more re-work -
sqlbox would need to ACK all messages and then decide which messages from
smsbox it should pass back to bearerbox. I'm likely not the right person to
take on that task given my limited exposure to the code to date.
On the smsbox-route issue, this is new for me. I've been running Kannel for
a long time and have never had this group in the config. My reading of the
users' guide is that this section is only needed if you want to override the
default round-robin behaviour. In my case, each message has its msgc_id set
to the specific smsbox ("sqlbox") where it needs to route to.
Looking at bb_boxc.c/route_incoming_to_boxc, the routing of incoming
messages looks to me like this order of events:
- If no smsbox is connected then store the message
- If the message's boxc_id is set, then route it direct to that box
- If there's a smsbox-route config section, then follow the rules there
- Pick a random box from the list of connected boxes
Having a hard time seeing why smsbox-route is mandatory. My main objection
to setting it anyway is that I have a lot of SMSC connections and I'd prefer
not to maintain a list of them in a string in this section. Leads to
configuration issues down the track if updating that line is missed while
adding/removing a SMSC.
My code is vanilla from the svn head, except the new ACK changes I've made
to sqlbox.
Thanks,
Toby
-----Original Message-----
From: Nikos Balkanas [mailto:[email protected]]
Sent: Tuesday, 10 August 2010 4:40 PM
To: Toby Phipps; 'Rene Kluwen'; [email protected]
Subject: Re: Life without smsbox
Nope. It disrupts architecture. Once a box receives a Msg *, it must
generate an ACK, so that bearerbox knows that it is the other box's
responsibility how to handle it and delete it from its own queue. ACK should
be returned whenever it is accepted, indepenedent of the logic thereafter.
It is now sqlbox's responsibility and it should figure out whether to send
it upstream, or handle it itself.
smsbox-route should be even if there is only 1 connected box. I don't follow
you saying that without it you get no warnings. Did you change the sources?
BR,
Nikos
----- Original Message -----
From: "Toby Phipps" <[email protected]>
To: "'Rene Kluwen'" <[email protected]>; "'Nikos Balkanas'"
<[email protected]>; <[email protected]>
Sent: Tuesday, August 10, 2010 8:14 AM
Subject: RE: Life without smsbox
> Rene,
>
> As the subject line suggests, I'm not actually running a smsbox, and the
> smsbox-id in both sqlbox and smsbox groups is set to the same value as
> recommended by Nikos (below) in order to force the routing. Maybe I
> misunderstood his directions?
>
> In any case, the "smsbox_list empty" warning turned out to be a red
> herring.
> It turns out that it appeared in the logs only a couple of times when I
> was
> tweaking the configuration and sqlbox wasn't running. With the current
> configuration (with the smsbox-route group removed), DLRs are routing
> correctly to sqlbox, and I've confirmed this with traces in the sqlbox
> code,
> the lack of the "smsbox_list empty" warning, and 100% MT-to-DLR matching
> in
> the sqlbox tables. I'm guessing this routing is happening either because
> I'm
> forcing the routing (setting boxc_id in every MT submitted), or because
> there's only one box connected, and bearerbox routes it there.
>
> So, with the new ACK code I've added to sqlbox, things appear to be
> running
> very smoothly. I'm seeing both MT submission notification and true DLRs
> coming through to sqlbox, and the bearerbox logs and status output show
> that
> they're being properly ACKed and purged from the local store. I'm quite
> happy with this config.
>
> With regards to the sqlbox ACK generation patch, I've been thinking about
> what should trigger the sqlbox-originated ACK. I originally coded it to
> generate the ACK if there was no smsbox connected to it, but on second
> thought this doesn't seem like a great idea as there could be situations
> where smsbox should be running but isn't, so the ACKs SHOULD be delegated
> and messages queued. My next thought was to use the presence of the
> smsbox-port config parameter (sqlbox group) to trigger the ACK (sqlbox
> generates the ACK if sqlbox-port is not present in the config and it
> delegates the ACK if it is present). However, looking at the code, the
> smsbox-port parameter is defaulted to 13005 if not present, so this would
> break backwards compatibility.
>
> So, I'm thinking of adding a new config parameter instead - something like
> "run-without-smsbox" or "sqlbox-generates-ack". Any suggestions or
> thoughts?
>
> Thanks,
> Toby.
>
> -----Original Message-----
> From: Rene Kluwen [mailto:[email protected]]
> Sent: Tuesday, 10 August 2010 3:56 AM
> To: 'Nikos Balkanas'; 'Toby Phipps'; [email protected]
> Subject: RE: Life without smsbox
>
> True. You have the same smsbox-id for both smsbox and sqlbox.
> It should be different.
>
> But... probably sqlbox isn't going to send an ACK. At least I didn't put
> it
> in the code. Maybe someone else.
>
> == Rene
>
> -----Original Message-----
> From: Nikos Balkanas [mailto:[email protected]]
> Sent: Monday, 09 August, 2010 19:07
> To: Toby Phipps; 'Rene Kluwen'; [email protected]
> Subject: Re: Life without smsbox
>
> I reiterate. SQLbox shouldn't have such an error. Your smsbox-route is not
> correct, you need to specify smsc-id. Read User's guide about it. You
> should
>
> still be getting that WARNING.
>
> Solve that, and you should be set.
>
> BR,
> Nikos
> ----- Original Message -----
> From: "Toby Phipps" <[email protected]>
> To: "'Nikos Balkanas'" <[email protected]>; "'Rene Kluwen'"
> <[email protected]>; <[email protected]>
> Sent: Monday, August 09, 2010 7:59 PM
> Subject: RE: Life without smsbox
>
>
>> Hmm, OK. Interesting. I've reverted to the SVN head of sqlbox and
>> configured
>> the following:
>>
>> group = smsbox
>> bearerbox-host = localhost
>> smsbox-id = sqlbox
>>
>> group = smsbox-route
>> smsbox-id = sqlbox
>>
>> group = sqlbox
>> id = sqlbox
>> smsbox-id = sqlbox
>> bearerbox-port = 13100
>> bearerbox-host = localhost
>> # smsbox-port = 13200
>> sql-log-table = smsgw_kannel_sms_result
>> sql-insert-table = smsgw_kannel_sms_queue
>> log-file = /usr/local/kannel/log/sqlbox.log
>> log-level = 0
>>
>> Now, I see the following in bearerbox.log:
>>
>> 2010-08-10 00:53:11 [1691] [13] DEBUG: boxc_receiver: sms received
>> 2010-08-10 00:53:11 [1691] [13] DEBUG: send_msg: sending msg to boxc:
>> <sqlbox>
>> 2010-08-10 00:53:11 [1691] [14] DEBUG: send_msg: sending msg to boxc:
>> <sqlbox>
>> 2010-08-10 00:53:11 [1691] [14] DEBUG: boxc_sender: sent message to
>> <127.0.0.1>
>> 2010-08-10 00:53:17 [1691] [14] DEBUG: send_msg: sending msg to boxc:
>> <sqlbox>
>> 2010-08-10 00:53:17 [1691] [14] DEBUG: boxc_sender: sent message to
>> <127.0.0.1>
>>
>> The lines at 00:53:11 are written when the MT is sent out. The lines at
>> 00:53:17 are written when the DLR arrives.
>>
>> And... unfortunately the queuing is back - here's the relevant status
>> output:
>>
>> SMS: received 0 (0 queued), sent 2 (0 queued), store size 4
>> SMS: inbound (0.00,0.00,0.00) msg/sec, outbound (0.01,0.01,0.01) msg/sec
>>
>> DLR: received 4, sent 0
>> DLR: inbound (0.02,0.01,0.01) msg/sec, outbound (0.00,0.00,0.00) msg/sec
>> DLR: 262 queued, using mysql storage
>>
>> Box connections:
>> smsbox:sqlbox, IP 127.0.0.1 (4 queued), (on-line 0d 0h 4m 39s)
>>
>> I still chalk the queuing up to a missing ACK from sqlbox - the following
>> line is missing from the logs:
>> DEBUG: boxc_receiver: got ack
>>
>> When I had smsbox running and calling a dummy URL to get rid of DLRs,
>> that
>> ack would appear each time a message (MT or DLR) was sent to sqlbox.
>>
>> So, I'm still convinced that the DLR routing to sqlbox is happening, and
>> that bearerbox is just getting upset as it gets no ACK from sqlbox. My
>> hypothesis regarding the "WARNING: smsbox_list empty!" message is that
>> bearerbox isn't getting a smsbox registration message (as sqlbox doesn't
>> generate one, and relies on the non-existent smsbox to do so), but is
>> still
>> routing both MT notifications and DLRs to sqlbox as it's been instructed
>> to
>> do. I'd be happy to be proved wrong though!
>>
>> Thanks,
>> Toby.
>>
>>
>> -----Original Message-----
>> From: Nikos Balkanas [mailto:[email protected]]
>> Sent: Tuesday, 10 August 2010 12:31 AM
>> To: Toby Phipps; 'Rene Kluwen'; [email protected]
>> Subject: Re: Life without smsbox
>>
>> Nope. What you see is entirely different. send_msg is not sending the DLR
>> to
>>
>> sqlbox, but rather responding an ACK to the received MT. I don't know if
>> there is a problem with sqlbox, but I bet I would have heard about it if
>> it
>> were, but revert to the svn version and try smsbox-route. The warning you
>> get is very crucial, you are missing incoming DLRs. You need to fix that,
>> before anything else.
>>
>> BR,
>> Nikos
>> ----- Original Message -----
>> From: "Toby Phipps" <[email protected]>
>> To: "'Nikos Balkanas'" <[email protected]>; "'Rene Kluwen'"
>> <[email protected]>; <[email protected]>
>> Sent: Monday, August 09, 2010 7:24 PM
>> Subject: RE: Life without smsbox
>>
>>
>>> Hi Nikos,
>>>
>>> I'll certainly try what you suggest, but I'm not quite sure that's the
>>> issue. Although bearerbox may think it doesn't have any smsboxes
>>> connected,
>>> it's still happily routing messages to sqlbox - here's a brief extract
>>> from
>>> the bearerbox log:
>>>
>>> 2010-08-09 23:46:52 [24133] [4] WARNING: smsbox_list empty!
>>> 2010-08-09 23:46:52 [24133] [4] DEBUG: Thread 4
>>> (gw/bb_boxc.c:sms_to_smsboxes) terminates.
>>> 2010-08-09 23:46:52 [24133] [14] DEBUG: send_msg: sending msg to boxc:
>>> <sqlbox>
>>>
>>> sqlbox is also getting each message as intended, and I can see it
>>> arriving
>>> in its log, and I can also see bearerbox queuing them in the status
>>> output.
>>>
>>> smsbox:sqlbox, IP 127.0.0.1 (110 queued), (on-line 0d 0h 0m 50s)
>>>
>>> The piece that's missing really seems to be the ACK from sqlbox to
>>> bearerbox
>>> for each DLR message it receives. bearerbox is successfully sending each
>>> DLR
>>> to sqlbox, and sqlbox is successfully logging it to the DB. However, as
>>> sqlbox is sending no ACK back, bearerbox has no way of knowing that
>>> sqlbox
>>> received the DLR and is therefore queuing it locally, and periodically
>>> re-sending to sqlbox.
>>>
>>> Now I've read the sqlbox code, it seems that it's a thin shell and
>>> doesn't
>>> implement ANY of the ACK processing that smsbox does. It simply logs to
>>> the
>>> DB, passes messages to smsbox and hopes that smsbox will pass it back an
>>> ACK
>>> that it can then pass to bearerbox so that the loop can be closed.
>>> However,
>>> without an smsbox connection, sqlbox is simply processing and dumping
>>> each
>>> message from bearerbox, leading to the unnecessary queuing in bearerbox.
>>>
>>> Anyway, that's what I've worked out from a few hours of looking through
>>> the
>>> code. This is nothing compared to the years that everyone else has with
>>> it,
>>> and I'm happy to be proved wrong!
>>>
>>> Just as an aside, between the time I sent my earlier email and I saw
>>> your
>>> response, I implemented a very rudimentary #2 (ACK in sqlbox) and it
>>> seems
>>> to have resolved the issue. All I'm doing is checking on each message
>>> receipt inside sqlbox to see if a smsbox is connected. If so then
>>> nothing
>>> changes - if not, then sqlbox returns the successful ACK.
>>>
>>> Thanks,
>>> Toby
>>>
>>> -----Original Message-----
>>> From: Nikos Balkanas [mailto:[email protected]]
>>> Sent: Tuesday, 10 August 2010 12:08 AM
>>> To: Toby Phipps; 'Rene Kluwen'; [email protected]
>>> Subject: Re: Life without smsbox
>>>
>>> Hi,
>>>
>>> You are missing an smsbox-route group. Once you configure that, your DLR
>>> problem will disappear. You need to set an smsbox-id in your smsbox
>>> group
>>> (I
>>>
>>> remeber you already have a dummy one), that corresponds to the id in
>>> your
>>> sqlbox. Then configure that to your smsbox-route group. Right now your
>>> bb
>>> is
>>>
>>> getting the DLRs, but doesn't know where to send them. Granted it is
>>> pretty
>>> lame to have only 1 smsbox connected to it, and not know what to do with
>>> it.
>>>
>>> It could be fixed sometime.
>>>
>>> Do not proceed with (2).
>>>
>>> BR,
>>> Nikos
>>> ----- Original Message -----
>>> From: "Toby Phipps" <[email protected]>
>>> To: "'Nikos Balkanas'" <[email protected]>; "'Rene Kluwen'"
>>> <[email protected]>; <[email protected]>
>>> Sent: Monday, August 09, 2010 6:17 PM
>>> Subject: RE: Life without smsbox
>>>
>>>
>>>> Hi Nikos,
>>>>
>>>> Thanks again for the feedback. You hit the nail on the head - I am
>>>> seeing
>>>> "WARNING: smsbox_list empty!" throughout the bearerbox log. Looks like
>>>> running sqlbox alone is not enough to truly emulate a smsbox.
>>>>
>>>> So, I delved into the source for sqlbox, and from what I can see, it
>>>> has
>>>> no
>>>> ACK response logic at all. It receives the DLR (or MO) from bearerbox,
>>>> writes it to the DB and then passes it to a connected smsbox. It
>>>> appears
>>>> to
>>>> rely on smsbox to generate the ACK - there's logic in there to pass
>>>> smsbox
>>>> messages back to bearerbox (which I assume would include the ACKs), but
>>>> nothing to handle the situation where there is no smsbox connected. In
>>>> this
>>>> case, the message gets written to the DB then simply dropped, hence the
>>>> queuing in bearerbox that I'm seeing.
>>>>
>>>> So, I'm looking at two options and would very much appreciate advice on
>>>> which one would be most appropriate:
>>>>
>>>> 1. Configure a "dummy" smsbox to receive/ACK the messages from sqlbox
>>>> but
>>>> do
>>>> nothing else them. Not sure if I can configure smsbox to do this (I'll
>>>> hit
>>>> the doc), whether I'll need to patch it to provide a "ACK and drop"
>>>> mode,
>>>> or
>>>> whether I should have it call a dummy destination to avoid the
>>>> patching.
>>>>
>>>> 2. Enhance sqlbox to generate its own ACK if it doesn't have a
>>>> connected
>>>> smsbox to pass the MO/DLR onto.
>>>>
>>>> Any suggestions? If not, I'll probably start on #2 as it seems much
>>>> cleaner
>>>> to me and avoids the unnecessary overhead of a dummy smsbox process and
>>>> associated IPC. Who is the appropriate sqlbox owner that I should run
>>>> these
>>>> changes by? I'm sure I'm not the only one that has no need for smsbox.
>>>>
>>>> Thanks,
>>>> Toby.
>>>>
>>>> -----Original Message-----
>>>> From: Nikos Balkanas [mailto:[email protected]]
>>>> Sent: Monday, 9 August 2010 2:17 AM
>>>> To: Toby Phipps; 'Rene Kluwen'; [email protected]
>>>> Subject: Re: Life without smsbox
>>>>
>>>> Hi,
>>>>
>>>> The only way for bearerbox to remove the DLR from store is to receive
>>>> an
>>>> upstream ACK (smsbox or sqlbox). That is the architectural spec. If
>>>> this
>>>> is
>>>> not happenning, sqlbox has a bug. You could check for that in maximum
>>>> detail
>>>>
>>>> bb (and sqlbox) logs. If there is a problem with routing or boxc ids in
>>>> bb,
>>>> you should get a lot of warnings like "Warning: smsbox-list empty!",
>>>> which
>>>> will also build up DLR store queue.
>>>>
>>>> BR,
>>>> Nikos
>>>>
>>>>
>>>> ----- Original Message -----
>>>> From: "Toby Phipps" <[email protected]>
>>>> To: "'Rene Kluwen'" <[email protected]>; <[email protected]>
>>>> Sent: Sunday, August 08, 2010 3:26 PM
>>>> Subject: RE: Life without smsbox
>>>>
>>>>
>>>>> Just a follow-up - I also tried setting "smsbox-id = sqlbox" in the
>>>>> "sqlbox"
>>>>> section, and sqlbox starts but I get the same results, which is that
>>>>> the
>>>>> bearerbox store size increments by one for each DLR received, as
>>>>> viewed
>>>>> in
>>>>> the status output. For example:
>>>>>
>>>>> SMS: received 0 (0 queued), sent 13 (0 queued), store size 85
>>>>>
>>>>> Right now, my sqlbox.conf looks like this:
>>>>>
>>>>> group = sqlbox
>>>>> id = sqlbox
>>>>> smsbox-id = sqlbox
>>>>> bearerbox-port = 13100
>>>>> bearerbox-host = localhost
>>>>> # smsbox-port = 13200
>>>>> sql-log-table = smsgw_kannel_sms_result
>>>>> sql-insert-table = smsgw_kannel_sms_queue
>>>>> log-file = /usr/local/kannel/log/sqlbox.log
>>>>> log-level = 0
>>>>>
>>>>> group = mysql-connection
>>>>> id = sqlbox
>>>>> host = localhost
>>>>> username = <removed>
>>>>> password = <removed>
>>>>> database = <removed>
>>>>> max-connections = 3
>>>>>
>>>>> As you can see, the smsbox-port is commented out, meaning that I don't
>>>>> want
>>>>> sqlbox to try to talk to smsbox. I've tried with this line commented
>>>>> out,
>>>>> or
>>>>> uncommented (but pointed to a non-listening port). Both result in the
>>>>> same
>>>>> behaviour,
>>>>>
>>>>> Any more ideas anyone? I'm about to start delving into the source...
>>>>>
>>>>> Thanks,
>>>>> Toby.
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>
>
>
