Re: Blocked ports for MO (generic http smsc)

2013-07-23 Thread Andreas Fink
in this context the key question is if your smsc configs for http smsc's shares 
the same receive port because for every http-smsc sending is a http client 
while receiving is a http server. Every http-smsc must have its own http-server 
as they dont know of each other. If you have two http-smscs with the same 
receive port, the described outcome could very well be as one http-smsc 
receiver thread would steal tasks from the other.

could that be your issue?

On 22.07.2013, at 22:12, Brian McCavour bmccav...@yahoo.com wrote:

 
 Hi Andreas,
 
 Thanks for your response. Sorry I was not more clear :) 
 I only have 1 http server. The modifications I've done is to add custom types 
 of http smsc. I never touched any of the underlying code however, I just copy 
 pasted the generic implementation (in smsc_http.c), and made new send/receive 
 functions per type. Then I added a few other configuration options for the 
 smsc. So all of the threading / networking code is unmodified (except what 
 I've added to try and fix this issue)
 
 From what I can understand, this 1 http server thread in kannel will do:
 Look for a new server socket, add 1 new server socket to the list of file 
 descriptors if a new socket found
 Poll given the list of FD
 Handle data
 Check if we need to close any sockets
 During startup as we're looping through SMSC in CFG (bb_smscconn 
 smsc2_start), for each http smsc it calls http_open_port_if, which should (I 
 believe?) wakeup the server thread, basicaly restarting back at 1. again.
 
 So what I expect to see, when I have 4 http smsc in my .conf is 1, 2, 1, 2, 
 1, 2, 1, 2, which each new socket connection interupting and adding its 
 socket to the list of FD, before going back to polling.
 
 What I am actually seeing through, is that it will block during the poll, and 
 the new sockets being added still in the main thread get added to the 
 new_server_sockets correctly, but the polling thread never wakes up to 
 pickup this new socket. So the fix that I did was just to move the start of 
 the http server from attempting to start it everytime you try and open an 
 http port, to just openeing it 1 time after reading in all the smsc from CFG. 
 The I changed from adding 1 socket per itteration, to add all new server 
 sockets to the FD list in 1 iteration.
 
 Maybe I am just really misunderstanding what is happening, but my hack did 
 fix the problem, so it must somehow be related.
 
 What are your thoughts?
 
 thx,
 Brian
 
 
 
 From: Andreas Fink af...@list.fink.org
 To: Brian McCavour bmccav...@yahoo.com 
 Cc: devel Devel devel@kannel.org 
 Sent: Monday, July 22, 2013 3:37:17 PM
 Subject: Re: Blocked ports for MO (generic http smsc)
 
 If I understand you right, you have some code to run virtually multiple http 
 servers which do different things?
 You must be aware that gwlib is build the way that it runs multiple threads 
 polling of the same incoming port. In other words if you run on port 80 a min 
 webserver to serve documents, then every thread waits on something to happen 
 and if an incoming data arrives any of the worker thread catches it and 
 processes it.
 As in many places inside kannel/gwlib this is done using internal pipes where 
 the waiting threads are sitting inside a poll and the thread who discovered 
 there's work to do actually writes a byte into this pipe and thus waking up 
 any or all of the waiting threads.
 
 Now if you run multiple servers, lets say on port 80 and port 443, you need 
 to separate this mechanism. Which means every group of listening threads for 
 either port must have its own pipe its listening to. If I guess correctly, 
 you have an extension built which runs multiple http servers and does 
 something with it
 
 I have used gwlib with multiple http servers without a problem. I believe 
 gwlib is doing the right thing but you might need to create a second 
 instance. But without seeing your code modifications, its impossible to 
 answer that.
 
 On 22.07.2013, at 21:20, Brian McCavour bmccav...@yahoo.com wrote:
 
 Hi,
 
 I was investigating an issue I had on my customized Kannel where only the 
 first Http SMSC, of many http smsc, would be able to receive MO. The other 
 sockets would simply hang (or so it appeared).
 
 I discovered that basically the http server thread was getting blocked on 
 the thread poll, and until I send a message through to a port that is 
 already in the list, it ill never pickup any of those requests.
 Basically, in http.c, the function server_thread() picks up the first http 
 SMSC, adds it to struct pollfd tab[MAX_SERVERS];
 Then if ((ret = gwthread_poll(tab, n, -1.0)) == -1)   will block 
 waiting for data on the socket.
 
 But if there is another server to be added afterwards, this gets called from 
 http.c: int http_open_port_if(int port, int ssl, Octstr *interface);
 Now it looks like this should wake up that polling thread, and add the new 
 port to tab data structure above, and go back to waiting 

Re: Blocked ports for MO (generic http smsc)

2013-07-23 Thread Brian McCavour

Each SMSC has its own port.I forgot to mention, this is working off of 1.4.3. 
Maybe the http implementaiton has changed since then? 
I only see 1 server thread starting for all http ports. (including the admin 
port)

I'm going to go back and try and reproduce it on release 1.4.3 with debug logs, 
then I can post the config.





 From: Andreas Fink af...@list.fink.org
To: Brian McCavour bmccav...@yahoo.com 
Cc: devel Devel devel@kannel.org 
Sent: Tuesday, July 23, 2013 2:07:11 AM
Subject: Re: Blocked ports for MO (generic http smsc)
 


in this context the key question is if your smsc configs for http smsc's shares 
the same receive port because for every http-smsc sending is a http client 
while receiving is a http server. Every http-smsc must have its own http-server 
as they dont know of each other. If you have two http-smscs with the same 
receive port, the described outcome could very well be as one http-smsc 
receiver thread would steal tasks from the other.

could that be your issue?



On 22.07.2013, at 22:12, Brian McCavour bmccav...@yahoo.com wrote:


Hi Andreas,

Thanks for your response. Sorry I was not more clear :) 
I only have 1 http server. The modifications I've done is to add custom types 
of http smsc. I never touched any of the underlying code however, I just copy 
pasted the generic implementation (in smsc_http.c), and made new send/receive 
functions per type. Then I added a few other configuration options for the 
smsc. So all of the threading / networking code is unmodified (except what 
I've added to try and fix this issue)

From what I can understand, this 1 http server thread in kannel will do:

   1. Look for a new server socket, add 1 new server socket to the list of 
 file descriptors if a new socket found

   2. Poll given the list of FD
   3. Handle data
   4. Check if we need to close any sockets
During startup as we're looping through SMSC in CFG (bb_smscconn smsc2_start), 
for each http smsc it calls http_open_port_if, which should (I believe?) 
wakeup the server thread, basicaly restarting back at 1. again.

So what I expect to see, when I have 4 http smsc in my .conf is 1, 2, 1, 2, 1, 
2, 1, 2, which each new socket connection interupting and adding its socket to 
the list of FD, before going back to polling.

What I am actually seeing through, is that it will block during the poll, and 
the new sockets being added still in the main thread get added to the 
new_server_sockets correctly, but the polling thread never wakes up to 
pickup this new socket. So the fix that I did was just to move the start of 
the http server from attempting to start it everytime you try and open an http 
port, to just openeing it 1 time after reading in all the smsc from CFG. The I 
changed from adding 1 socket per itteration, to add all new server sockets to 
the FD list in 1
 iteration.

Maybe I am just really misunderstanding what is happening, but my hack did 
fix the problem, so it must somehow be related.

What are your thoughts?

thx,
Brian








 From: Andreas Fink af...@list.fink.org
To: Brian McCavour bmccav...@yahoo.com 
Cc: devel Devel devel@kannel.org 
Sent: Monday, July 22, 2013 3:37:17 PM
Subject: Re: Blocked ports for MO (generic http smsc)
 


If I understand you right, you have some code to run virtually multiple http 
servers which do different things?
You must be aware that gwlib is build the way that it runs multiple threads 
polling of the same incoming port. In other words if you run on port 80 a min 
webserver to serve documents, then every thread waits on something to happen 
and if an incoming data arrives any of the worker thread catches it and 
processes it.
As in many places inside kannel/gwlib this is done using internal pipes where 
the waiting threads are sitting inside a poll and the thread who discovered 
there's work to do actually writes a byte into this pipe and thus waking up 
any or all of the waiting threads.


Now if you run multiple servers, lets say on port 80 and port 443, you need to 
separate this mechanism. Which means every group of listening threads for 
either port must have its own pipe its listening to. If I guess correctly, you 
have an extension built which runs multiple http servers and does something 
with it


I have used gwlib with multiple http servers without a problem. I believe 
gwlib is doing the right thing but you might need to create a second instance. 
But without seeing your code modifications, its impossible to answer that.


On 22.07.2013, at 21:20, Brian McCavour bmccav...@yahoo.com wrote:


Hi,


I was investigating an issue I had on my customized Kannel where only the 
first Http SMSC, of many http smsc, would be able to receive MO. The other 
sockets would simply hang (or so it appeared).


I discovered that basically the http server thread was getting blocked on the 
thread poll, and until I send a message through to a port that is 

Re: Blocked ports for MO (generic http smsc)

2013-07-23 Thread Brian McCavour


I'm still not able to reproduce it with generic http smsc's. I think it must be 
just a racing condition somewhere when the loading time for smsc is long.
I just checked the 1.5.0 source, and someone has made one of the 2 changes I 
made. (To pickup all new server ports instead of 1 per loop iteration.)

So I'm going to try and remove my other change and maybe this 1 change is 
enough
Either way I'm running out of time for this unfortunately :( so if it works 
fine for me after the fix and no one else has this issue... so be it :D

Thanks again for your thoughts.
== Brian





 From: Brian McCavour bmccav...@yahoo.com
To: Andreas Fink af...@list.fink.org 
Cc: devel Devel devel@kannel.org 
Sent: Tuesday, July 23, 2013 7:59:22 AM
Subject: Re: Blocked ports for MO (generic http smsc)
 



Each SMSC has its own port.I forgot to mention, this is working off of 1.4.3. 
Maybe the http implementaiton has changed since then? 
I only see 1 server thread starting for all http ports. (including the admin 
port)

I'm going to go back and try and reproduce it on release 1.4.3 with debug logs, 
then I can post the config.





 From: Andreas Fink af...@list.fink.org
To: Brian McCavour bmccav...@yahoo.com 
Cc: devel Devel devel@kannel.org 
Sent: Tuesday, July 23, 2013 2:07:11 AM
Subject: Re: Blocked ports for MO (generic http smsc)
 


in this context the key question is if your smsc configs for http smsc's shares 
the same receive port because for every http-smsc sending is a http client 
while receiving is a http server. Every http-smsc must have its own http-server 
as they dont know of each other. If you have two http-smscs with the same 
receive port, the described outcome could very well be as one http-smsc 
receiver thread would steal tasks from the other.

could that be your issue?



On 22.07.2013, at 22:12, Brian McCavour bmccav...@yahoo.com wrote:


Hi Andreas,

Thanks for your response. Sorry I was not more clear :) 
I only have 1 http server. The modifications I've done is to add custom types 
of http smsc. I never touched any of the underlying code however, I just copy 
pasted the generic implementation (in smsc_http.c), and made new send/receive 
functions per type. Then I added a few other configuration options for the 
smsc. So all of the threading / networking code is unmodified (except what 
I've added to try and fix this issue)

From what I can understand, this 1 http server thread in kannel will do:

   1. Look for a new server socket, add 1 new server socket to the list of 
 file descriptors if a new socket found

   2. Poll given the list of FD
   3. Handle data
   4. Check if we need to close any sockets
During startup as we're looping through SMSC in CFG (bb_smscconn smsc2_start), 
for each http smsc it calls http_open_port_if, which should (I believe?) 
wakeup the server thread, basicaly restarting back at 1. again.

So what I expect to see, when I have 4 http smsc in my .conf is 1, 2, 1, 2, 1, 
2, 1, 2, which each new socket connection interupting and adding its socket to 
the list of FD, before going back to polling.

What I am actually seeing through, is that it will block during the poll, and 
the new sockets being added still in the main thread get added to the 
new_server_sockets correctly, but the polling thread never wakes up to 
pickup this new socket. So the fix that I did was just to move the start of 
the http server from attempting to start it everytime you try and open an http 
port, to just openeing it 1 time after reading in all the smsc from CFG. The I 
changed from adding 1 socket per itteration, to add all new server sockets to 
the FD list in 1
 iteration.

Maybe I am just really misunderstanding what is happening, but my hack did 
fix the problem, so it must somehow be related.

What are your thoughts?

thx,
Brian








 From: Andreas Fink af...@list.fink.org
To: Brian McCavour bmccav...@yahoo.com 
Cc: devel Devel devel@kannel.org 
Sent: Monday, July 22, 2013 3:37:17 PM
Subject: Re: Blocked ports for MO (generic http smsc)
 


If I understand you right, you have some code to run virtually multiple http 
servers which do different things?
You must be aware that gwlib is build the way that it runs multiple threads 
polling of the same incoming port. In other words if you run on port 80 a min 
webserver to serve documents, then every thread waits on something to happen 
and if an incoming data arrives any of the worker thread catches it and 
processes it.
As in many places inside kannel/gwlib this is done using internal pipes where 
the waiting threads are sitting inside a poll and the thread who discovered 
there's work to do actually writes a byte into this pipe and thus waking up 
any or all of the waiting threads.


Now if you run multiple servers, lets say on port 80 and port 443, you need to 
separate this mechanism. Which means every group of listening 

Re: Blocked ports for MO (generic http smsc)

2013-07-23 Thread Brian McCavour

Sorry for the spam guys. Last one

I found the resolution. This is fixed in 1.5.0, bug #525
See more details @ https://redmine.kannel.org/issues/525

Lessons learned... always checks latest code changes!





 From: Brian McCavour bmccav...@yahoo.com
To: Andreas Fink af...@list.fink.org 
Cc: devel Devel devel@kannel.org 
Sent: Tuesday, July 23, 2013 8:52:24 AM
Subject: Re: Blocked ports for MO (generic http smsc)
 




I'm still not able to reproduce it with generic http smsc's. I think it must be 
just a racing condition somewhere when the loading time for smsc is long.
I just checked the 1.5.0 source, and someone has made one of the 2 changes I 
made. (To pickup all new server ports instead of 1 per loop iteration.)

So I'm going to try and remove my other change and maybe this 1 change is 
enough
Either way I'm running out of time for this unfortunately :( so if it works 
fine for me after the fix and no one else has this issue... so be it :D

Thanks again for your thoughts.
== Brian





 From: Brian McCavour bmccav...@yahoo.com
To: Andreas Fink af...@list.fink.org 
Cc: devel Devel devel@kannel.org 
Sent: Tuesday, July 23, 2013 7:59:22 AM
Subject: Re: Blocked ports for MO (generic http smsc)
 



Each SMSC has its own port.I forgot to mention, this is working off of 1.4.3. 
Maybe the http implementaiton has changed since then? 
I only see 1 server thread starting for all http ports. (including the admin 
port)

I'm going to go back and try and reproduce it on release 1.4.3 with debug logs,
 then I can post the config.





 From: Andreas Fink af...@list.fink.org
To: Brian McCavour bmccav...@yahoo.com 
Cc: devel Devel devel@kannel.org 
Sent: Tuesday, July 23, 2013 2:07:11 AM
Subject: Re: Blocked ports for MO (generic http smsc)
 


in this context the key question is if your smsc configs for http smsc's shares 
the same receive port because for every http-smsc sending is a http client 
while receiving is a http server. Every http-smsc must have its own http-server 
as they dont know of each other. If you have two http-smscs with the same 
receive port, the described outcome could very well be as one http-smsc 
receiver thread would steal tasks from the other.

could that be your issue?



On 22.07.2013, at 22:12, Brian McCavour bmccav...@yahoo.com wrote:


Hi Andreas,

Thanks for your response. Sorry I was not more clear :) 
I only have 1 http server. The modifications I've done is to add custom types 
of http smsc. I never touched any of the underlying code however, I just copy 
pasted the generic implementation (in smsc_http.c), and made new send/receive 
functions per type. Then I added a few other configuration options for the 
smsc. So all of the threading / networking code is unmodified (except what 
I've added to try and fix this issue)

From what I can understand, this 1 http server thread in kannel will do:

   1. Look for a new server socket, add 1 new server socket to the list of 
 file descriptors if a new socket found

   2. Poll given the list of FD
   3. Handle data
   4. Check if we need to close any sockets
During startup as we're looping through SMSC in CFG (bb_smscconn smsc2_start), 
for each http smsc it calls http_open_port_if, which should (I believe?) 
wakeup the server thread, basicaly restarting back at 1. again.

So what I expect to see, when I have 4 http smsc in my .conf is 1, 2, 1, 2, 1, 
2, 1, 2, which each new socket connection interupting and adding its socket to 
the list of FD, before going back to polling.

What I am actually seeing through, is that it will block during the poll, and 
the new sockets being added still in the main thread get added to the 
new_server_sockets correctly, but the polling thread never wakes up to 
pickup this new socket. So the fix that I did was just to move the start of 
the http server from attempting to start it everytime you try and open an http 
port, to just openeing it 1 time after reading in all the smsc from CFG. The I 
changed from adding 1 socket per itteration, to add all new server sockets to 
the FD list in 1
 iteration.

Maybe I am just really misunderstanding what is happening, but my hack did 
fix the problem, so it must somehow be related.

What are your thoughts?

thx,
Brian








 From: Andreas Fink af...@list.fink.org
To: Brian McCavour bmccav...@yahoo.com 
Cc: devel Devel devel@kannel.org 
Sent: Monday, July 22, 2013 3:37:17 PM
Subject: Re: Blocked ports for MO (generic http smsc)
 


If I understand you right, you have some code to run virtually multiple http 
servers which do different things?
You must be aware that gwlib is build the way that it runs multiple threads 
polling of the same incoming port. In other words if you run on port 80 a min 
webserver to serve documents, then every thread waits on something to happen 
and if an incoming data arrives any of 

RE: Proposal for Real-time Routing in opensmppbox and bearerbox

2013-07-23 Thread Porter, Kelvin
Hi,

Responses below.

Regards,

Kelvin R. Porter

From: spameden [mailto:spame...@gmail.com]
Sent: Monday, July 22, 2013 6:23 PM
To: Porter, Kelvin
Cc: devel@kannel.org; us...@kannel.org
Subject: Re: Proposal for Real-time Routing in opensmppbox and bearerbox



2013/7/23 Porter, Kelvin 
kelvin.por...@h3net.commailto:kelvin.por...@h3net.com
Hi,

In the straightforward implementation, the load on the database would be 1 
query per message sent (MT) and 1 query per message received (MO) (excluding 
situations where loopback occurred).

A caching component could be added separately afterwards. The trade-offs would 
involve added code complexity, and the increasing latency in having database 
changes affect message routing.

Different applications (e.g., A2P vs. P2P) might or might not see much benefit 
depending on how long the caching interval is, the rate of back-and-forth 
conversation, and the size of the cache.

I am not familiar with all aspects of the code base, is there a caching 
mechanism that can be reused?

I think there is no caching mechanism at all and grace reload is not 
implemented yet. Stipe was posting a patch earlier giving kannel the ability to 
gracefully reload it's configuration, but I think it's still bugy and needs to 
be cleaned up before accepted in the codebase.
[KRP] That would be great.  Is it for both opensmppbox and bearerbox?  How 
would you trigger the reload on opensmppbox?
The problem getting configuration from SQL is 1 more query per MT, if you have 
busy site with huge traffic (100-200 MT /sec) it could affect on your database, 
so caching definitely needed somehow. Alternatively you can look into storing 
configuration into memcached.
[KRP] 100 - 200 T/sec does not seem that like that many transactions for one of 
our servers.
[KRP] I can definitely look into a caching component as a subsequent 
enhancement.
The route mechanism you're wanting can be implemented without database as well 
using add-smsc / remove-smsc commands for example. I'm using myself this 2 
hooks for failsafe configuration (when 1 of the smsc goes down, I change with 
sed 'allowed-smsc-id' directive and re-add via remove-smsc, add-smsc commands 
specific smsc - everything done automatically via monit with additional 
scripts).
[KRP] Unfortunately, that will not address my needs.  I need to be able to add 
and remove numbers dynamically and affect their routing accordingly.
I even heard someone implemented whole kannel configuration in the database, if 
you could make it - it would be awesome, because it would be MUCH easier to 
code web interface for kannel to configure it more easily.
[KRP] Well, you could not implement the entire configuration in a database, 
because the process needs configuration information to find the database.  
Also, currently, the configuration is read in once at start up time.  The code 
would need to be changed to consult the database or keep in sync with it 
dynamically.  I am proposing an enhancement that would dynamically consult the 
database (when configured to do so).  Migrating the code to dynamic 
configuration would be useful, I believe; however, moving some or all of the 
code in this direction would be a significant undertaking.
I think that if caching is added that it should be configurable as to size and 
duration.

Regards,

Kelvin R. Porter

From: spameden [mailto:spame...@gmail.com]
Sent: Monday, July 22, 2013 3:55 PM
To: Porter, Kelvin
Cc: devel@kannel.orgmailto:devel@kannel.org; 
us...@kannel.orgmailto:us...@kannel.org
Subject: Re: Proposal for Real-time Routing in opensmppbox and bearerbox

Interesting idea, but what about load on the database? Will it be cached or 
it's gonna get results from the database everytime SMS arrives?

2013/7/23 Porter, Kelvin 
kelvin.por...@h3net.commailto:kelvin.por...@h3net.com
Hi,

I have included a proposed enhancement for routing message based on database 
contents below.

I am looking for feedback.

Please share your thoughts as to whether this would be of interest.

Thank you.

Regards,

Kelvin R. Porter


I would like to propose an enhancement to the source code to kannel bearerbox 
and opensmppbox.  The enhancement would supercede the existing configuration 
groups:

smsbox-route in the bearerbox, and
smsc-route in the oppensmppbox.

The enhancement dynamically consults a database table/view to determine the 
mapping based on the sender, receiver and original connection (smsc or box).  
The purpose of this enhancement is to allow dynamically changing the message 
routing without requiring a change to the configuration file(s) and then the 
subsequent restart of the opensmppbox and/or bearerbox.

The enhancement works by taking advantage of the database access functionality 
currently used for storing DLRs in a database.

The DLR code currently supports the following databases: mysql, oracle, 
pgsql, mssql, and sdb (where  SDB is the simplified DB interface).  The 
internal option refers to storing DLRs in memory 

Re: Proposal for Real-time Routing in opensmppbox and bearerbox

2013-07-23 Thread spameden
Hi!


2013/7/23 Porter, Kelvin kelvin.por...@h3net.com

 Hi,

 ** **

 Responses below.

 ** **

 Regards,

 ** **

 Kelvin R. Porter

 ** **

 *From:* spameden [mailto:spame...@gmail.com]
 *Sent:* Monday, July 22, 2013 6:23 PM

 *To:* Porter, Kelvin
 *Cc:* devel@kannel.org; us...@kannel.org
 *Subject:* Re: Proposal for Real-time Routing in opensmppbox and bearerbox
 

 ** **

 ** **

 ** **

 2013/7/23 Porter, Kelvin kelvin.por...@h3net.com

 Hi,

  

 In the straightforward implementation, the load on the database would be 1
 query per message sent (MT) and 1 query per message received (MO)
 (excluding situations where loopback occurred). 

  

 A caching component could be added separately afterwards. The trade-offs
 would involve added code complexity, and the increasing latency in having
 database changes affect message routing.

  

 Different applications (e.g., A2P vs. P2P) might or might not see much
 benefit depending on how long the caching interval is, the rate of
 back-and-forth conversation, and the size of the cache.

  

 I am not familiar with all aspects of the code base, is there a caching
 mechanism that can be reused?

 ** **

 I think there is no caching mechanism at all and grace reload is not
 implemented yet. Stipe was posting a patch earlier giving kannel the
 ability to gracefully reload it's configuration, but I think it's still
 bugy and needs to be cleaned up before accepted in the codebase.

 [KRP] That would be great.  Is it for both opensmppbox and bearerbox?  How
 would you trigger the reload on opensmppbox?


Bearerbox only. Here is a patch - http://pastebin.com/v4rSRHrJ (I hope
Stipe wouldnt mind posting it).

 

 The problem getting configuration from SQL is 1 more query per MT, if you
 have busy site with huge traffic (100-200 MT /sec) it could affect on your
 database, so caching definitely needed somehow. Alternatively you can look
 into storing configuration into memcached.

 [KRP] 100 – 200 T/sec does not seem that like that many transactions for
 one of our servers.


Well, in this case you get 100-200 additional queries, it was just an
example and rly depends on your database workload.

 

 [KRP] I can definitely look into a caching component as a subsequent
 enhancement.



 The route mechanism you're wanting can be implemented without database as
 well using add-smsc / remove-smsc commands for example. I'm using myself
 this 2 hooks for failsafe configuration (when 1 of the smsc goes down, I
 change with sed 'allowed-smsc-id' directive and re-add via remove-smsc,
 add-smsc commands specific smsc - everything done automatically via monit
 with additional scripts).

 [KRP] Unfortunately, that will not address my needs.  I need to be able to
 add and remove numbers dynamically and affect their routing accordingly. *
 ***

 I even heard someone implemented whole kannel configuration in the
 database, if you could make it - it would be awesome, because it would be
 MUCH easier to code web interface for kannel to configure it more easily.*
 ***

 [KRP] Well, you could not implement the entire configuration in a
 database, because the process needs configuration information to find the
 database.

True, ofc, you need to provide connection settings as in everywhere.

 Also, currently, the configuration is read in once at start up time.  The
 code would need to be changed to consult the database or keep in sync with
 it dynamically.  I am proposing an enhancement that would dynamically
 consult the database (when configured to do so).  Migrating the code to
 dynamic configuration would be useful, I believe; however, moving some or
 all of the code in this direction would be a significant undertaking.

I'm not against your proposal, just thinking about potential problems you
might get .. The best plan is to make different engines, i.e. MySQL /
PostgreSQL / Oracle / etc.

Only dynamic variables like smsbox-route and various smsc settings should
be kept into the database to operate them on the fly I think.. Other data
like DLR / bearerbox core settings can be static in the configuration I
think.

Anyways, I'm waiting for your version of the patched opensmppbox /
bearerbox to test out! :)

 I think that if caching is added that it should be configurable as to size
 and duration.

  

 Regards,

  

 Kelvin R. Porter

  

 *From:* spameden [mailto:spame...@gmail.com spame...@gmail.com]
 *Sent:* Monday, July 22, 2013 3:55 PM
 *To:* Porter, Kelvin
 *Cc:* devel@kannel.org; us...@kannel.org
 *Subject:* Re: Proposal for Real-time Routing in opensmppbox and bearerbox
 

  

 Interesting idea, but what about load on the database? Will it be cached
 or it's gonna get results from the database everytime SMS arrives?

  

 2013/7/23 Porter, Kelvin kelvin.por...@h3net.com

 *Hi,*

 * *

 *I have included a proposed enhancement for routing message 

Re: Proposal for Real-time Routing in opensmppbox and bearerbox

2013-07-23 Thread Rinor Hoxha
How about using (implementing or adapting) HTTP-based callbacks.
So you could implement routing, billing or whatever you may need based
on callback responses.

Br, Rinor
 

On 07/22/2013 10:45 PM, Porter, Kelvin wrote:

 *Hi,*

 * *

 *I have included a proposed enhancement for routing message based on
 database contents below.*

 * *

 *I am looking for feedback.*

 * *

 *Please share your thoughts as to whether this would be of interest.*

 * *

 *Thank you.*

 * *

 *Regards,*

 * *

 *Kelvin R. Porter*

 * *

  

 I would like to propose an enhancement to the source code to kannel
 bearerbox and opensmppbox.  The enhancement would supercede the
 existing configuration groups:

  

 smsbox-route in the bearerbox, and

 smsc-route in the oppensmppbox.

  

 The enhancement dynamically consults a database table/view to
 determine the mapping based on the sender, receiver and original
 connection (smsc or box).  The purpose of this enhancement is to allow
 dynamically changing the message routing without requiring a change to
 the configuration file(s) and then the subsequent restart of the
 opensmppbox and/or bearerbox.

  

 The enhancement works by taking advantage of the database access
 functionality currently used for storing DLRs in a database.

  

 The DLR code currently supports the following databases: mysql,
 oracle, pgsql, mssql, and sdb (where  SDB is the simplified DB
 interface).  The internal option refers to storing DLRs in memory
 and would not apply to this enhancement.  I can write the code for
 these databases, but may require some assistance from the kannel
 community in testing them.

  

 The enhancement adds two new table parameters to the group dlr-db:

  

 table-smsc specifies the name of the table/view mapping a message to
 a smsc-id, and

 table-box specifies the name of the table/view mapping a message to
 a (sms)box.

  

 The enhancement is selectively enabled by configuring the parameters
 above.

  

 The following existing fields parameters of the group dlr-db are
 re-used:

  

 field-source specifies the name of the field that matches the sender
 of a message, and

 field-destination specifies the name of the field that matches the
 receiver of a message, and

 field-smsc specifies the name of the field that matches the smsc, and

 field-boxc-id specifies the name of the field that matches the name
 of the box.

  

 The following are two routines added to the dlr.h interface:

  

 /** Given message, then map to smsc id. */

 Octstr * map_to_smsc(Msg *msg);

  

 /** Given message, then map to boxc-id. */

 Ocstr *map_to_box(Msg *msg);

  

 If these methods return a (non-NULL) result, then they supercede the
 configuration-based routing.  If the results are NULL, then the
 default configured routing can be applied.

  

 The queries for this the match look something like the following:

  

 1.   To determine the smsc...

 SELECT field-smsc FROM table-smsc WHERE ((field-sender = NULL)
 OR (field-sender = ?)) AND

 ((field-destination = NULL) OR (field-destination = ?)) AND

 ((field-boxc-id = NULL) OR (field-boxc-id = ?))

  

 2.   To determine the (sms)box...

 SELECT field-boxc-id FROM table-box WHERE ((field-sender =
 NULL) OR (field-sender = ?)) AND

 ((field-destination = NULL) OR (field-destination = ?)) AND

 ((field-smsc = NULL) OR (field-smsc = ?))

  

 The queries are written in this fashion where a column with a NULL
 value will always match (in essence a row with a NULL in a column will
 always match).  That way a subset of the column values (e.g., receiver
 only or sender and boxc-id ) can be used to match as a criteria.

  

 New debug level logs would be added to indicate the input to the
 queries and the result.  In addition, certain configuration panics
 might be added like attempting to configure table-smsc  or
 table-box in conjunction with the internal database option.

 This enhancement would enable the directing of messages on the fly.  I
 think that the approach would serve as a good foundation in case
 additional routing criteria were desired (i.e., language/encoding
 based routing).