Re: XML Post and dlr-url

Mon, 15 Mar 2010 12:35:46 -0700

Sorry, it seems I missunderstood your point. You say that dlr-url should be unescaped for all possible XML entities (with libxml or otherwise). I agree with that in principle, but my problem is that I cannot imagine any other XML char in dlr-url (other than &) that could create a problem. I would appreciate it if you could give me an example. Otherwise, I hope you understand that getting the whole libxml for just 1 char is not worthwhile (performance & resource wise). 

BR,
Nikos
----- Original Message ----- From: "Nikos Balkanas" <[email protected]> 
To: "Alexander Malysh" <[email protected]>
Cc: <[email protected]>
Sent: Monday, March 15, 2010 8:23 PM
Subject: Re: XML Post and dlr-url



Hi,
Do you appreciate how heavy/slow libxml is? Besides, that section doesn't use libxml. What you suggest would need to rewrite the whole section. I am just patching the existing status. If you prefer to have kannel's POST xml err until someone rewrites the whole section, be my guest. 

BR,
Nikos
----- Original Message ----- From: "Alexander Malysh" <[email protected]> 
To: "Nikos Balkanas" <[email protected]>
Cc: <[email protected]>
Sent: Monday, March 15, 2010 6:19 PM
Subject: Re: XML Post and dlr-url


-1 because this is rather hack as fix...
you should use libxml unescape functions or implement it but not only for one char. 

Thanks,
Alexander Malysh

Am 15.03.2010 um 17:08 schrieb Nikos Balkanas:


So, Alex, what do you say about committing it?

BR,
Nikos
----- Original Message ----- From: "oscar cassetti" <[email protected]> 
To: "Nikos Balkanas" <[email protected]>
Cc: <[email protected]>
Sent: Monday, March 15, 2010 10:03 AM
Subject: Re: XML Post and dlr-url


I have been tested this patch for the past week and it looks good.

Kind Regards
Oscar
On Sat, Feb 27, 2010 at 11:17 AM, Nikos Balkanas <[email protected]> wrote:
OK. It should take me ~10' to fix it and create the patch. Then you can see 
it. I just didn't want to be rude by denying you the opportunity.

BR,
Nikos
----- Original Message ----- From: "oscar cassetti"
<[email protected]>
To: "Nikos Balkanas" <[email protected]>
Cc: <[email protected]>
Sent: Saturday, February 27, 2010 11:55 AM
Subject: Re: XML Post and dlr-url


Me? I don't think so I told you I would have a look and I had.
However, my idea of uri-escaping the dlr-url element Ξ’ wasn't any good.
I am not sure how to un-escape &amp; with &. Is there a function ?

Thanks
Oscar
On Sat, Feb 27, 2010 at 9:49 AM, Nikos Balkanas <[email protected]> wrote: 

We were waiting for you. Didn't you say you would do it?

BR,
Nikos
----- Original Message ----- From: "oscar cassetti"
<[email protected]>
To: <[email protected]>
Sent: Saturday, February 27, 2010 11:03 AM
Subject: Re: XML Post and dlr-url


Any development on this?

On Mon, Feb 22, 2010 at 1:32 PM, Nikos Balkanas <[email protected]>
wrote:


Well, this is just it. A workaround. Since solution is simple &
straightforward, I think the best thing is to make kannel XML fully
compatible.

BR,
Nikos

----- Original Message -----
From: Konstantin Vayner
To: oscar cassetti ; Nikos Balkanas
Cc: [email protected]
Sent: Monday, February 22, 2010 2:55 PM
Subject: Re: XML Post and dlr-url
Hi guys,
Another possible workaround may be using PHP configuration (in case
you're
using php as a server side scripting).
Take a look at this php.ini option:
http://php.net/manual/en/ini.core.php#ini.arg-separator.input
testcase:
pon...@localhost:~/www$ cat .htaccess
php_value arg_separator.input "&;"
pon...@localhost:~/www$ cat dump_request.php
<?php var_dump($_REQUEST); ?>
pon...@localhost:~/www$ lynx -source
'http://localhost/~poncha/dump_request.php?foo=bar&amp;bar=baz'
array(3) {
["foo"]=>
string(3) "bar"
["amp"]=>
string(0) ""
["bar"]=>
string(3) "baz"
}
Regards,
Konstantin
On Mon, Feb 22, 2010 at 8:07 AM, Nikos Balkanas <[email protected]>
wrote:


That was my suggestion, too. But verify with debugger to make sure.
Do not use the same approach for <udh>. There is no need and you should 
not require POST XML to be urlencoded. Just scan the value in dlr-url
for
&amp; and replace with '&'.

BR,
Nikos
----- Original Message ----- From: "oscar cassetti"
<[email protected]>
To: "Nikos Balkanas" <[email protected]>
Cc: <[email protected]>
Sent: Monday, February 22, 2010 1:48 AM
Subject: Re: XML Post and dlr-url


Hi,

I believe the problem is in the functions static void
get_x_kannel_from_xml in smsbox.c
around the line 804 (source code kannel 1.4.3)

get_tag(tmp, octstr_imm("dlr-url"), dlr_url, 0, 0);
O_DESTROY(tmp);

I think that that the same strategy adopted for the tag <ud> could be
used also for the dlr-url tag assuming that this will be uri-encoded.


Oscar



On Sun, Feb 21, 2010 at 6:35 PM, Nikos Balkanas <[email protected]>
wrote:
Sure. Start by checking the get_tag function in smsbox.c. Verify with 
debugger, since it is only my guess looking at the code.

BR,
Nikos
----- Original Message ----- From: "oscar cassetti"
<[email protected]>
To: "Nikos Balkanas" <[email protected]>
Sent: Sunday, February 21, 2010 10:38 AM
Subject: Re: XML Post and dlr-url
Yes, I can use a workaround by passing only one parameter to GET query 
and use a internal separator within the string to actually separate
the different parts.

If you can point me where the problem is (like function or source
file) I could I have a look myself.

Oscar
On Sun, Feb 21, 2010 at 2:26 AM, Nikos Balkanas <[email protected]> 
wrote:
OK. Looks like a bug. I'll take a look at it when I get some time. In 
the
meantime can you live with the illegal XML?

Nikos
----- Original Message ----- From: "oscar cassetti"
<[email protected]>
To: "Nikos Balkanas" <[email protected]>
Cc: <[email protected]>
Sent: Saturday, February 20, 2010 2:18 PM
Subject: Re: XML Post and dlr-url


But that's what I did in the second test. I didn't URL-escape
anything, I just let the Perl XML writer escape & with &amp; which is 
the right thing to do.
However when kannel parses the <dlr-url> it doesn't un-escape the
&amp;
to
&.
As a result the query URL used by kannel looks like this one below


2010-02-19 21:11:44 [1381] [9] DEBUG: Parsing URL




`http://localhost/dump_me.php?send_history_id=13853220&amp;to_number=000000000000&amp;type=8':

which produces on my entry point this

GET =
Array
(
[send_history_id] => 13853220
[amp;to_number] => 000000000000
[amp;type] => 8
)

Note the amp;to_number for example. If the dlr-url was escaped
properly this one should have just been to_number .

Finally in third and last test I produced an illegal XML by not
escaping the & and it worked:


2010-02-19 21:28:18 [1381] [9] DEBUG: Parsing URL




`http://localhost/dump_me.php?send_history_id=13853221&to_number=3538706323
46&type=8':

and on my entry point
GET =
Array
(
[send_history_id] => 13853221
[to_number] => 000000000000
[type] => 8
)
The issue is that I need to use an invalid XML to produced the right 
result. The problem is related to non un-escaping the <dlr-url>
element.

Oscar
On Sat, Feb 20, 2010 at 2:09 AM, Nikos Balkanas <[email protected]> 
wrote:
So, in XML POST you shouldn't urlencode anything. And it will work. 
What's
your issue then?

Nikos
----- Original Message ----- From: "oscar cassetti"
<[email protected]>
To: <[email protected]>
Sent: Saturday, February 20, 2010 12:36 AM
Subject: Re: XML Post and dlr-url


Hi,

I tried few things and I believe I found a bug in the way the
<dlr-url> is parsed.
(1)
First of all I tried to URI-escape the entire dlr-url and it didn't 
work.
Basically I passed this:





<dlr-url>http%3A%2F%2Flocalhost%2Fdump_me.php%3Fsend_history_id%3D13853219%26to_number%3D000000000000%26t
ype%3D%25d</dlr-url>

And the same appears in the logs
2010-02-19 21:03:32 [1318] [3] DEBUG: XMLParsing: tag <dlr-url> value 
<http%3A%2F%2Flocalhost%2Fdump_me.php%3Fsend_history_id

When Kannel is going to parse the url it rejects it:

2010-02-19 21:03:32 [1318] [3] DEBUG: Status: 400 Answer: <DLR-URL
field misformed, rejected>

In this case the message was not sent.
(2)
In a second test I didn't escape URI-escape. I let the Perl XML
Writer
to escape the content of <dlr-url> .
In this case & is substituted by &amp; This can also be seen in
smsbox
logs.
2010-02-19 21:11:43 [1381] [3] DEBUG: XMLParsing: tag <dlr-url> value 





<http://localhost/dump_me.php?send_history_id=13853220&amp;to_number=000000000000&amp;type=%d>
However when kannel parses the URL the &amp; is not changed back to & 
.
So the result is the following URL is used.

2010-02-19 21:11:44 [1381] [9] DEBUG: Parsing URL





`http://localhost/dump_me.php?send_history_id=13853220&amp;to_number=000000000000&amp;type=8':
I wrote a simple script to catch the content of the GET when kannel 
query the <dlr-url> and the result is

GET =
Array
(
[send_history_id] => 13853220
[amp;to_number] => 000000000000
[amp;type] => 8
)

(3)
Finally I tried to overwrite the <dlr-ulr> so I substituted &amp;
with
&
so the request looks like this one below:

<?xml version="1.0" encoding="UTF-8"?>





<message><submit><da><number>000000000000</number></da><oa><number>000000000000</number></oa><ud>%E2%82%ACee</ud><statusrequest><dlr-mask>31</dlr-mask><dlr-url>http://localhost/dump_me.php?send_history_id=13853221&to_number=000000000000&type=%d</dlr-url></statusrequest><!--
request from application to Kannel





--><from><username>xxxxxxx</username><password>xxxxxxx</password></from></submit></message>

Note that this XML is technically wrong. However this produces the
right result as ti can be seen from the logs:
2010-02-19 21:28:17 [1381] [3] DEBUG: XMLParsing: tag <dlr-url> value 





<http://localhost/dump_me.php?send_history_id=13853221&to_number=000000000000&type=%d>

2010-02-19 21:28:18 [1381] [9] DEBUG: Parsing URL





`http://localhost/dump_me.php?send_history_id=13853221&to_number=3538706323
46&type=8':
2010-02-19 21:28:18 [1381] [9] DEBUG: Scheme: http://
2010-02-19 21:28:18 [1381] [9] DEBUG: Host: localhost
2010-02-19 21:28:18 [1381] [9] DEBUG: Port: 80
2010-02-19 21:28:18 [1381] [9] DEBUG: Username: (null)
2010-02-19 21:28:18 [1381] [9] DEBUG: Password: (null)
2010-02-19 21:28:18 [1381] [9] DEBUG: Path: /dump_me.php
2010-02-19 21:28:18 [1381] [9] DEBUG: Query:
send_history_id=13853221&to_number=353870632346&type=8
2010-02-19 21:28:18 [1381] [9] DEBUG: Fragment: (null)
And also from my entry point I was able to get the three parameters 
correctly

GET =
Array
(
[send_history_id] => 13853221
[to_number] => 000000000000
[type] => 8
)
In summary I believe that an incorrect XML produces the right result. 
The problem looks to me e that the <dlr-ulr> is not escaped
correctly.

Here some details
Kannel smsbox version 1.4.3
xml2-config --version 2.6.32

Any suggestion?

Thank you,
Oscar
On Thu, Feb 18, 2010 at 3:09 PM, Nikos Balkanas <[email protected]> 
wrote:


Well, urlencoding is not right, still. In your URL ytou need to
encode
/,
&
and ?. Don't need to encode = or 8. But I think you want to encode 
&type=%d.
That should be %26type=%%d
So is this working or not? What do you mean by 3 parmaters, not 2? 
This
is
the dlr-url you send in your XML.

BR,
Nikos

Reply via email to