Here is our chat on the CXF mailing list [11:07] --> You have joined the channel #cxf (~ [EMAIL PROTECTED]). [11:12] <kmalhi> When i invoke a web service, sometimes I get the following exception, com.ctc.wstx.exc.WstxEOFException: Unexpected EOF in prolog [11:12] <kmalhi> any ideas what it might be, I will be sending the complete trace in the users list, just wanted to see if anybody can help me with this quickly
[11:55] <dkulp> dain: ping [14:31] <dain> dkulp: still looking for me? [14:33] <dkulp> dain: Kind of... [14:33] <dkulp> A CXF user is having problems with CXF in OpenEJB if http chunking is on. [14:34] <dkulp> It works fine if chunking is turned off. [14:34] <dain> I thought I fixed that [14:34] <dkulp> http://www.nabble.com/EOF-in-prolog-exception-tt18261760.html [14:35] <dkulp> Might be an older version of OpenESB. He doesn't mention which version. [14:36] <dkulp> I just don't know the details on how to respond. [14:37] <dain> I'll handle it [14:37] <dkulp> Thanks. [14:38] <dkulp> kmalhi: ^^ [14:40] <dain> dkulp: while I have you here [14:40] <dain> for the cxf client code are you using java.net.URL or something like commons http client [14:41] <dkulp> HTTPUrlConnection for HTTP [14:41] <dain> any problems with it? [14:42] <dkulp> Not in JDK 1.5 [14:42] <dain> that is what we use in openejb for ejbd over http [14:42] <kmalhi> dain: Its me having issues [14:42] <dkulp> I hear there were all kinds of issues with it in older JDK's, but it has worked fine for us with JDK 1.5. [14:42] <dain> dkulp: thanks fro the info [14:42] <dain> there were [14:43] <dain> kmalhi: what version of OpenEJB are you using? [14:43] <kmalhi> built it from trunk [14:43] <dain> and is that just the unmodified ws example code? [14:43] <dkulp> Our security folks looked at commons http, but had problems with the API's which made it hard (or impossible) to do some of the SSL stuff they needed. [14:44] <dain> I happy to hear it works good... was thinking I needed to add an optional commons http for "scalability" [14:44] <kmalhi> dain: had to switch off chunking at the client to get rid of the issue, if we turn chunking on, it is very erratic and 7 out of 10 times, you will get an error [14:44] <dain> less work for me [14:45] <dain> kmalhi: what os are you on [14:45] <kmalhi> ubuntu linux [14:45] <kmalhi> sun jdk 5 [14:45] <kmalhi> mnour and manugeorge were having the same issues on RHEL [14:46] <dain> I added chunking support to the built-in Openejb http server [14:46] <dain> but the whole code is kind of a hack [14:47] <dain> if you have some time, can you try jetty [14:47] <kmalhi> I saw the chunking support, I did open the RFC 2616 and in section 19.6 they having explained how to parse the TE:chunked. I did make some changes to do that exact parsing (e.g. we did not calculate content lenght if chunking is enabled etc), but it still didnt work [14:48] <dain> just add jetty-6.1.7.jar jetty-util-6.1.7.jar to the lib dir [14:48] <kmalhi> I think the issue might be that we are closing the InputStream after every request (this is just a guess) and chunking probably requires the inputstream to remain open [14:49] <kmalhi> okay , will try that [14:49] <dkulp> It's possible that CXF isn't reading the whole request either. I've found several cases where we aren't reading the closing soap:body/soap:envelope tags. [14:49] <dain> I'm just curious [14:50] <dkulp> Possible that it's expecting more reading to be occuring? [14:52] <kmalhi> dkulp: I saw that when CXF closes the HttpConduit, at one point it just reads the first 25 bytes of all the closing tags. The comment in there says that it does not want to read GB's of data which it does not need. [14:53] <dkulp> Yea, to prevent a DOS attack. [14:53] <kmalhi> okay [14:53] <dain> dkulp: the problem is when cxf tries to read the input stream [14:53] <kmalhi> brb [14:53] <dkulp> WIth chunked, you can just keep sending data and more data, etc... [14:53] <dain> specifically calling nextTag [15:08] <kmalhi> dain: I ran into this issue when running the CalculatorTest in examples/simple-webservice. I dont know how to include jetty and jetty util here. [15:09] <dain> are you running from the command line? [15:09] <kmalhi> yes, mvn clean install -Dtest=CalculatorTest [15:10] <dain> one sec.... my ide is refreshing [15:10] <kmalhi> okay [15:13] <dain> kmalhi: add this dependency to the pom http://rifers.org/paste/show/7598 [15:14] <-- bsnyder has left this server ("When the going gets weird, the weird turn pro."). [15:18] <kmalhi> dain: Works fine now, so its our chunking handling stuff probably. I still think that closing the inputstream when TE:chunked comes in, could be the issue [15:19] <dain> do you remember where that code is? [15:19] <kmalhi> Yes, its in the HttpRequestImpl.readBody() -- in openejb-http [15:22] <kmalhi> got to go, will be back in 3 hours [15:23] <dain> thanks [15:23] <dain> I should be here when you get back [15:42] <dain> kmalhi: can you back out the jetty addition and add this line to HttpRequestImpl:453 [15:42] <dain> System.out.println("BODY CHUNKED: " + new String(body)); [15:43] <dain> after the "body = out.toByteArray();" statement in the chunk section [15:43] <dain> then re run the test and see let me know what the body was (it will appear in the console output) [19:25] <kmalhi> dain:ping [19:26] <kmalhi> dain:posted the results at http://rifers.org/paste/show/7602 [19:26] <kmalhi> dain:keep in mind that i also enabled the cxf logging so that you could see the details of the http request also [19:27] <dain> earlier you mentioned that we close the stream... I can't seem to find that code [19:27] <dain> what is the file/line number? [19:28] <kmalhi> look at http://rifers.org/paste/show/7603 . i have highlighted the system out [19:30] <kmalhi> sorry, I think that the inputstream is part of the HttpRequestImpl. For every request we create a new instance of HttpRequestImpl, my guess was that when this object goes out of scope, the stream instance would be closed automatically on garbage collection. That might be the reason of the erratic behaviour that sometimes the test runs fine and sometimes it gives errors [19:32] <dain> that is exactly the same output I get [19:32] <dain> (without the exception) [19:32] <dain> oh wait [19:33] <dain> is the "BODY ==" line (the empty one) the input that causes the exception? [19:34] <kmalhi> yes, another thing i was thinking was the second system.out , where the body is empty. I checked the woodstox source code and what it does is looks for a EOF character before any xml start tag. What might be happening during chunking is that each chunk is probably being passed as a separate stream to woodstox and it treats as a completely new xml structure. If this chunk is empty, then writing to this second stream might simply be writing [19:34] <kmalhi> an EOF. [19:34] <dain> nope [19:34] <dain> we read the entire input into memory and then hand it off to the cxf code [19:35] <dain> my guess is the the chunk code has a bug where it thinks there are 0 bytes of input [19:36] <dain> if you could somehow capture the real input on the wire I could fix the chunking code [19:36] <dain> or maybe you can debug the chunk reading code and "see" the bug [19:36] <kmalhi> okay, maybe I will try and use tcp mon [19:36] <kmalhi> debugging is pretty painful, setting up tcpmon might be easier [19:36] <kmalhi> :) [20:02] <kmalhi> dain: you are right, gimme a minute and I will post the output on rifers [20:07] <kmalhi> dain: I have posted the results of the test http://rifers.org/paste/show/7604 . I routed the request to 4205 -- which is where tcp mon was running and it re-routed it to 4204. the highlighted areas are not part of the data on the wire and i just used it to highlight each request and response [20:08] <kmalhi> dain:Got to go for a while, will be back in 45 minutes [21:11] <dain> kmalhi: maybe the responce isn't getting flushed [21:11] <dain> try adding out.flush() to HttpResponseImpl.writeMessage(...) [21:11] <dain> make sure that you flush "out" the DataOutputStream [21:12] <dain> you need to change the type of out to DataOutputStream [21:13] <dain> like this http://rifers.org/paste/show/7605 [21:14] <dain> if that doesn't work, try writing the contents to a ByteArrayOutputstream and then dumping it to the console... just to see what is going on [21:16] <dain> like this http://rifers.org/paste/show/7606 [21:16] <dain> gotta run [23:23] <kmalhi> dain: that fixed the issue, client was waiting for a valid response which was not being flushed fully, causing the client to hang On Fri, Jul 4, 2008 at 8:10 AM, Karan Malhi <[EMAIL PROTECTED]> wrote: > Just start chatting on IRC with Dain and your issues will magically > resolve :) . > > Okay, so here are detailed steps:- > > Ran the CalculatorTest in examples/simple-webservice. The test failed > I looked at the stack trace and just followed the trace through the source > code. > Also downloaded the sources of CXF and Woodstox (both were not needed , but > I learnt a few things looking through them). > Googled the exceptions on the stack trace to find out why the client would > hang for a while. Bumped into something called HttpClientPolicy for CXF. > Copied an example Client policy from the CXF website and pasted in our web > service example test case > The test case ran successfully. Would always run the test case multiple > times because a single successful run did not guarantee sucess (now we know > why ) > Started commenting out each line from the HttpClientPolicy code (to figure > out which policy was fixing the issue) and re-running the test > Found out that chunking was the issue. > Read RFC 2616, sections 3 and 19.6 for chunking. > Found out our code which involved chunking - couldnt find anything wrong in > there > Found out CXF code involing chunking and couldnt find anything wrong in > there too > Mohammed came on IRC and we started to collaborate as he had some > suggestions on where things could go wrong > Joined the CXF IRC and users list . Pinged the IRC and sent stack trace to > users list > Dan Kulp started helping me. Dan pinged Dain on IRC and then Dain started > helping me out figure it. > Dain immediately told me to replace our HttpServer with Jetty, and the > error went away. This meant the problem was with our HTTPServer Chunking > part and not CXF (I was parallelly taking the longer route of replacing our > HttpServer code with commons HTTP core -- which I stopped as I saw success > with Jetty and knew it was our HttpServer) > Then we started dumping the body of the HttpMessage on the console > (something which i had already tried earlier too, but that didnt help) > Then we ran TCP mon to find out the data travelling back and forth on the > wire and thats where Dain caught the issue that the response being sent back > by the HttpServer is not complete i.e. our HttpServer was not flushing the > response properly. > > We exchanged a lot of console output and code samples on rifers.org. You > can look at those here > http://www.rifers.org/paste/show/7602 . Also look at 7603,7604,7605,7606 > > So it was a team effort and I alone could not have figured it out. But I > thoroughly enjoyed it :) > > > > On Fri, Jul 4, 2008 at 2:58 AM, Jacek Laskowski <[EMAIL PROTECTED]> > wrote: > >> On Fri, Jul 4, 2008 at 5:46 AM, <[EMAIL PROTECTED]> wrote: >> > Author: kmalhi >> > Date: Thu Jul 3 20:46:02 2008 >> > New Revision: 673899 >> > >> > URL: http://svn.apache.org/viewvc?rev=673899&view=rev >> > Log: >> > Fixed OPENEJB-843. >> > Http response was not being flushed fully causing the client to hang. >> > Fixed by flushing the response. >> > Thanks to Dain >> >> Could you let us know how you figured it out? It's not easy to fix >> such issues so any tips for hunting them down would be greatly >> appreciated. >> >> Jacek >> >> -- >> Jacek Laskowski >> Notatnik Projektanta Java EE - http://www.JacekLaskowski.pl >> > > > > -- > Karan Singh Malhi -- Karan Singh Malhi
