> ----- Original Message ----- > From: "Christian V." <[EMAIL PROTECTED]> > To: [email protected] > Subject: Re: De-Chunking > Date: Wed, 08 Nov 2006 09:59:08 +0100 > > > Christian V. wrote: > > Nick Kew wrote: > >> On Tue, 07 Nov 2006 11:24:05 +0100 "Christian V." > >> <[EMAIL PROTECTED]> wrote: > >> > >>> Hi , > >>> > >>> i 'm running a third-party web service authentication module that > >>> hangs when the request coming from the client is splitted out in > >>> different chunks. I don't have access to the module and to the > >>> client neither, so I'm thinking to write an input filter that > >>> collects all the chunks and pass'em to the downstream filter or > >>> handler . Is that possible? > >> > >> It's possible, yes. > >> > >> Whether it'll fix the problem for you is not certain. I'd suggest > >> starting with a quick hack (or a dechunking proxy in front of your > >> server) to test it first, if you really can't get the source. > >> > > > > > > Maybe the Proxy will fix it but it will not be the solution, so i > > think i'm gonna write the module-filter. But i need to know how > > Apache handle multi chunk request, as im not able to find this > > information. Is request coming entirely to my filter in the form of > > bucket&brigades then passed to down-streams module or brigades are > > passed down as soon as they come? (I hope i explained it well) > > > > Tnx a lot, Chris. > > > > > > > > > Let me explain well as im not 100% sure the problem is the 3rd > party module, and other people may had met the same issue: > > > > [ CLIENT ] --> [ APACHE R.PROXY (SSL) + 3rd MODULE ] --> [WEB SERVICE] > > The Web Service receives requests from both Java and .net clients. > > Our problem is the following. > > The .Net clients (we have one in c# and one in VB, both programmed with > visual studio 2005 ) will split the client's XML request in multiple > 1024 byte packets. This happens only over HTTPS, and causes problems > with the 3rd party module > > To debug this we have programmed an apache module on the reverse proxy > that dumps the stream of data as it receives it from the clients, and > our .Net client, over HTTPS, splits it up in multiple chunks, as seen here: > > [Tue Oct 31 15:09:56 2006] [notice] (IN) bucketdumper: mode READBYTES; > blocking; 8192 bytes > [Tue Oct 31 15:09:57 2006] [notice] (IN) - (AFTER bucket_read) > -\tbucketdumper:\tbytes: 1024 - lenght read: 1024 - data: <?xml > version="1.0" encoding="utf-8"?><soap:Envelope > xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"; > xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"; > xmlns:tns="http://www.acme.com.com/wsdl/HelloMoto.wsdl"; > xmlns:types="http://www.acme.com.com/wsdl/HelloMoto.wsdl/encodedTypes"; > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > xmlns:xsd="http://www.w3.org/2001/XMLSchema";><soap:Body > soap:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/";><q1:sayHello > xmlns:q1="urn:examples:HelloMoto"><TAG1 > xsi:type="xsd:string">TES</TAG1><TAG2 > xsi:type="xsd:string">TES</TAG2><TAG3 > xsi:type="xsd:string">TES</TAG3><TAG4 > xsi:type="xsd:string">TES</TAG4><TAG5 > xsi:type="xsd:string">TES</TAG5><TAG6 > xsi:type="xsd:string">TES</TAG6><TAG7 > xsi:type="xsd:string">TES</TAG7><TAG8 > xsi:type="xsd:string">TES</TAG8><TAG9 > xsi:type="xsd:string">TES</TAG9><TAG10 > xsi:type="xsd:string">TES</TAG10><TAG11 > xsi:type="xsd:string">TES</TAG11><TAG12 > xsi:type="xsd:string">TEST</TAG12></q1:sayHello></soap:Body></soap:Envelope > - > [Tue Oct 31 15:09:57 2006] [notice] (IN) Complete Bucket : > [Tue Oct 31 15:09:57 2006] [notice] (IN) bucketdumper: mode READBYTES; > blocking; 8192 bytes > [Tue Oct 31 15:09:58 2006] [notice] (IN) - (AFTER bucket_read) > -\tbucketdumper:\tbytes: 1 - lenght read: 1 - data: > - > [Tue Oct 31 15:09:58 2006] [notice] (IN) - (AFTER bucket_read) > -\tbucketdumper:\tbytes: 0 - lenght read: 0 - data: > - > > Note how the XML is 1025 bytes long, and gets send in one 1024 byte > packet first, followed by a second 1 byte packet (that contains just the > final ">"). > > This does not happen over HTTP, where the entire XML arrives in one 1025 > byte long data chunk. > > Also, our Java clients do not split up the XML when posting in HTTPS, > independently of how long it is. Here is a request made by one of our > java clients: > > > [Tue Oct 31 15:12:57 2006] [notice] (IN) bucketdumper: mode READBYTES; > blocking; 8192 bytes > [Tue Oct 31 15:12:57 2006] [notice] (IN) - (AFTER bucket_read) > -\tbucketdumper:\tbytes: 4333 - lenght read: 4333 - data: <?xml > version="1.0" encoding="UTF-8"?><soapenv:Envelope > xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"; > xmlns:xsd="http://www.w3.org/2001/XMLSchema"; > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";><soapenv:Body><beregnLivorno > xmlns="http://prognosiRiservata.acme.com";><startinput><ns1:channel > xmlns:ns1="http://";>LIV</ns1:channel><ns2:clientId > xmlns:ns2="http://";>TEST</ns2:clientId><ns3:clientPassword > xsi:nil="true" xmlns:ns3="http://"/><ns4:test > xmlns:ns4="http://";>false</ns4:test><ns5:useCache > ---------------------------------------- > ---------------------------------------- > mlns:ns36="http://data.prognosiRiservata.acme.com";>true</ns36:returnerBeskrivelser><ns37:returnerThreadSide > xmlns:ns37="http://Livorno.data.prognosiRiservata.acme.com";>true</ns37:returnerThreadSide><ns38:returnerResultaterAvtale > xmlns:ns38="http://Livorno.data.prognosiRiservata.acme.com";>true</ns38:returnerResultaterAvtale></prognosiRiservatainput></beregnLivorno></soapenv:Body></soapenv:Envelope> > - > [Tue Oct 31 15:12:57 2006] [notice] (IN) Complete Bucket : > [Tue Oct 31 15:12:57 2006] [notice] (IN) bucketdumper: mode READBYTES; > blocking; 8192 bytes > [Tue Oct 31 15:12:57 2006] [notice] (IN) - (AFTER bucket_read) > -\tbucketdumper:\tbytes: 0 - lenght read: 0 - data: - > > Notice how the XML (4333 bytes long) is sent in one entire chunk. > > --- > > I realized that my initial idea of having an input filter is not > applicable as long as the 3rd party module lives in the > authentication/authorization phase, accessing data before any input filter. > > Ummm... > > The DeChunking-Proxy hypothesis seems to be, at present, the only valid > solution! :-( > > > Chris.
Hi Chris, To me it looks like the term chunked is causing confusion. HTTP chunked transfer encoding is a specific method of avoiding the need to know the content-length up front. It involves inserting a length before each "chunk" instead of specifying the complete body length with the C-L header. I would avoid using the term "chunk" in HTTP conversations to avoid this interpretation. >From what you've written I see a TCP stream being read in portions. Another >common cause of confusion is to talk about TCP packets. Although data may >arrive at a receiving application in portions which appear to correspond to >packets, a TCP application MUST be written without assumptions about how much >data will arrive after a read(). In Apache, the data you see in a bucket brigade will reflect the data which was read() from the TCP socket. Say an application write()s 10 bytes. The receiver may read() 1 byte 10 times, or it may get all 10 bytes with 1 read() call, or any other combination. The point is TCP is a stream with no defined message boundaries. This is similar to a file, the read()er would not be aware what size the write()r had used for outputting portions of the data. I think your issue here is that you have an application which is expecting the data to by read() in one operation. TCP does not provide this, therefore your application may already unreliable. Even the working client/server could be forced to break. For example if a portion of data normally read() in one operation were to pass over a network hop with a small MTU, it may get fragmented and appear as the result of two read() operations for the receiver. Hope that's all clear. Good luck, Paul -- _______________________________________________ Surf the Web in a faster, safer and easier way: Download Opera 9 at http://www.opera.com Powered by Outblaze
