Re: AJP Flush Packet causing text/plain output
I encountered a similar problem, where one servlet had a bug where the headers would be set _after_ the data had been sent. The result was that unrelated responses would come back as text/plain. Once I found the offending code and fixed it, the issue has not surfaced again. Mark On 9/24/07, Larry Reisler [EMAIL PROTECTED] wrote: We recently switched our development JBOSS instance from 4.05GA to 4.21GA, where we are have been using mod_jk for connecting an Apache front end server (2.2) to the Tomcat AppServer. We have noticed periodic times when the apache web server will return data with a content-type of plain/text (the server default) instead of the real content-type, and a chunked encoding (even if the AppServer was producing a non-chunked content). We tried switch to mod_proxy_ajp, but got the same result. A bit of sleuthing (2 days X 2 engineers) revealed that the AJP connection on Tomcat is sending a SEND_BODY_CHUNK (which I assume is a flush packet) periodically that seem to be confusing mod_jk. We can see in the mod_jk and mod_proxy_ajp debug log that the correct headers are coming from the Tomcat server. However, it seems like mod_jk and mod_proxy_ajp are losing all the header information regarding the packet (including the Powered by headers, content-type, content-length, cookies, etc.). In normal operation, for a sample small transaction, we would see this sequence in response: SEND_HEADERS SEND_BODY_CHUNK END_RESPONSE When the output would come out as text/plain, we would see this sequence: SEND_BODY_CHUNK SEND_HEADERS SEND_BODY_CHUNK END_RESPONSE Is there something we should have configured differently so that mod_jk or mod_proxy_ajp will behave better? - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: AJP Flush Packet causing text/plain output
this is probably what you run into http://issues.apache.org/bugzilla/show_bug.cgi?id=43478 Filip Mark Deneen wrote: I encountered a similar problem, where one servlet had a bug where the headers would be set _after_ the data had been sent. The result was that unrelated responses would come back as text/plain. Once I found the offending code and fixed it, the issue has not surfaced again. Mark On 9/24/07, Larry Reisler [EMAIL PROTECTED] wrote: We recently switched our development JBOSS instance from 4.05GA to 4.21GA, where we are have been using mod_jk for connecting an Apache front end server (2.2) to the Tomcat AppServer. We have noticed periodic times when the apache web server will return data with a content-type of plain/text (the server default) instead of the real content-type, and a chunked encoding (even if the AppServer was producing a non-chunked content). We tried switch to mod_proxy_ajp, but got the same result. A bit of sleuthing (2 days X 2 engineers) revealed that the AJP connection on Tomcat is sending a SEND_BODY_CHUNK (which I assume is a flush packet) periodically that seem to be confusing mod_jk. We can see in the mod_jk and mod_proxy_ajp debug log that the correct headers are coming from the Tomcat server. However, it seems like mod_jk and mod_proxy_ajp are losing all the header information regarding the packet (including the Powered by headers, content-type, content-length, cookies, etc.). In normal operation, for a sample small transaction, we would see this sequence in response: SEND_HEADERS SEND_BODY_CHUNK END_RESPONSE When the output would come out as text/plain, we would see this sequence: SEND_BODY_CHUNK SEND_HEADERS SEND_BODY_CHUNK END_RESPONSE Is there something we should have configured differently so that mod_jk or mod_proxy_ajp will behave better? - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: AJP Flush Packet causing text/plain output
On 9/24/07, Larry Reisler [EMAIL PROTECTED] wrote: We recently switched our development JBOSS instance from 4.05GA to 4.21GA Which AJP connector is used in JBoss exactly ? Normally, the two connectors in org.apache.coyote.ajp cannot be doing what you see. Rémy - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: AJP Flush Packet causing text/plain output
Hi Larry, Larry Reisler wrote: We recently switched our development JBOSS instance from 4.05GA to 4.21GA, where we are have been using mod_jk for connecting an Apache front end server (2.2) to the Tomcat AppServer. We have noticed Would you mind telling us the JK version? periodic times when the apache web server will return data with a content-type of plain/text (the server default) instead of the real content-type, and a chunked encoding (even if the AppServer was producing a non-chunked content). We tried switch to mod_proxy_ajp, but got the same result. A bit of sleuthing (2 days X 2 engineers) revealed that the AJP connection on Tomcat is sending a SEND_BODY_CHUNK (which I assume is a flush packet) periodically that seem to be confusing mod_jk. We can see in the mod_jk and mod_proxy_ajp debug log that the correct headers are coming from the Tomcat server. However, it seems like mod_jk and mod_proxy_ajp are losing all the header information regarding the packet (including the Powered by headers, content-type, content-length, cookies, etc.). In normal operation, for a sample small transaction, we would see this sequence in response: SEND_HEADERS SEND_BODY_CHUNK END_RESPONSE When the output would come out as text/plain, we would see this sequence: SEND_BODY_CHUNK SEND_HEADERS SEND_BODY_CHUNK END_RESPONSE Is there something we should have configured differently so that mod_jk or mod_proxy_ajp will behave better? I'll have to check, but I wouldn't be to surprised, if mod_jk expects SEND_HEADERS first. Could you please open an issue with bugzilla? That helps us keeping track. There's no option at the moment, and I think we need to fix this. At the moment I'm not sure about where this needs fixing, in JK or in the AJP connector on the Tomcat side (or both). It would be very helpful, if you could attach your JK debug log, the JK configuration and the Tomcat connector configuration to the bug entry. Is there a chance to get a simple reproduction case (small webapp)? Regards, Rainer - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: AJP Flush Packet causing text/plain output
Thanks for your reply. There is a holiday here right now, so I'm not sure if I will be able to get to file the Bugzilla issue before the holiday break. We tried a few different versions of mod_jk (the early ones had other issues), but all the latest ones showed the problem. We are currently using mod_jk 1.2.25. In desperation, I was able to create a patch to mod_jk that seems to prevent the symptom from occurring. Here is the diff for the file jk_ajp_common.c: 1742,1751d1741 // Removing extra flush buffer if we do not need it. if (headeratclient == JK_FALSE) { int code = (int)jk_b_pget_byte(op-reply,0); unsigned int len = (unsigned int)jk_b_pget_int(op-reply,1); if ((code == JK_AJP13_SEND_BODY_CHUNK) (len == 0)) { jk_log(l, JK_LOG_DEBUG, Received flushbuffer -- ignoring); continue; } } - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: AJP Flush Packet causing text/plain output
Larry Reisler wrote: Thanks for your reply. There is a holiday here right now, so I'm not sure if I will be able to get to file the Bugzilla issue before the holiday break. We tried a few different versions of mod_jk (the early ones had other issues), but all the latest ones showed the problem. We are currently using mod_jk 1.2.25. In desperation, I was able to create a patch to mod_jk that seems to prevent the symptom from occurring. Here is the diff for the file jk_ajp_common.c: 1742,1751d1741 // Removing extra flush buffer if we do not need it. if (headeratclient == JK_FALSE) { int code = (int)jk_b_pget_byte(op-reply,0); unsigned int len = (unsigned int)jk_b_pget_int(op-reply,1); if ((code == JK_AJP13_SEND_BODY_CHUNK) (len == 0)) { jk_log(l, JK_LOG_DEBUG, Received flushbuffer -- ignoring); continue; } } Thanks for the patch. I'll check and likely commit. The core problem maybe is inside the Tomcat AJP connector (shouldn't send flush packets before sending headers) or in the webapp (which might trigger flushes to early), but it will be definitely good to be tolerant on the JK side. Thanks! Regards, Rainer - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: AJP Flush Packet causing text/plain output
Jim Jagielski wrote: Thanks for the patch. I'll check and likely commit. The core problem maybe is inside the Tomcat AJP connector (shouldn't send flush packets before sending headers) or in the webapp (which might trigger flushes to early), but it will be definitely good to be tolerant on the JK side. +1... I'll look at the mod_proxy_ajp side. Right, the problem is in the webapp probably. The first flush shouldn't occur before initial packet is send. However the explicit flushes (send_body with size 0) are very important. Think the patch is safe, although not tested. Regards, Mladen - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: AJP Flush Packet causing text/plain output
On a related note Has there been any jk connector releases for Apache HTTP 2.2? Thanks M-- - Original Message - From: Mladen Turk [EMAIL PROTECTED] To: Tomcat Users List users@tomcat.apache.org Sent: Tuesday, September 25, 2007 11:04 AM Subject: Re: AJP Flush Packet causing text/plain output Jim Jagielski wrote: Thanks for the patch. I'll check and likely commit. The core problem maybe is inside the Tomcat AJP connector (shouldn't send flush packets before sending headers) or in the webapp (which might trigger flushes to early), but it will be definitely good to be tolerant on the JK side. +1... I'll look at the mod_proxy_ajp side. Right, the problem is in the webapp probably. The first flush shouldn't occur before initial packet is send. However the explicit flushes (send_body with size 0) are very important. Think the patch is safe, although not tested. Regards, Mladen - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: AJP Flush Packet causing text/plain output
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Martin, Martin Gainty wrote: On a related note Has there been any jk connector releases for Apache HTTP 2.2? Any reasonably recent version of mod_jk will work for Apache httpd 2.2. As always, building your own binary is the best strategy, but binaries are also available from the Apache website. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFG+Vha9CaO5/Lv0PARAjonAJ0SmPXZVzsUMX99pHdAC0H/nTi+HQCfVAaw Wrkv/L3B2a+vl/pbayeSPHE= =LUO9 -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: AJP Flush Packet causing text/plain output
Hi Lary, Larry Reisler wrote: Thanks for your reply. There is a holiday here right now, so I'm not sure if I will be able to get to file the Bugzilla issue before the holiday break. Maybe this time we are faster than our request ticketing ... We tried a few different versions of mod_jk (the early ones had other issues), but all the latest ones showed the problem. We are currently using mod_jk 1.2.25. In desperation, I was able to create a patch to mod_jk that seems to prevent the symptom from occurring. Here is the diff for the file jk_ajp_common.c: 1742,1751d1741 // Removing extra flush buffer if we do not need it. if (headeratclient == JK_FALSE) { int code = (int)jk_b_pget_byte(op-reply,0); unsigned int len = (unsigned int)jk_b_pget_int(op-reply,1); if ((code == JK_AJP13_SEND_BODY_CHUNK) (len == 0)) { jk_log(l, JK_LOG_DEBUG, Received flushbuffer -- ignoring); continue; } } I'm a little reluctant to use your approach. It should work, but the peeking for the message code and length does not belong into the get reply function. The best place for the patch would be in the callback handling, but there we don't know, if we already received the headers. So either we pass some additional info to the callback handling (status info about the request processing stage), or we prevent the flushing later. I decided to move an already existing flag response_started from the web server private data to the public service struct and use it inside the callback handler. This patch was committted (http://svn.apache.org/viewvc?view=revrevision=579349). You can find a simple download form of it under http://people.apache.org/~rjung/mod_jk-dev/patches/flush-before-header.patch The patch looks lengthy, but in fact did only small changes. It would be helpful, if you could try it with your reproduction scenario. Regards, Rainer - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: AJP Flush Packet causing text/plain output
I understand your hesitation. I agree that my method was poor in terms of encapsulation, but was elegant in terms of its size, that it did not change any of the function signatures, and that it was easy to understand. The holiday has begun here, and I am off for this week -- I will try to test the patch ASAP. Thanks so much for your help. Larry. -Original Message- From: Rainer Jung [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 25, 2007 8:58 PM To: Tomcat Users List Subject: Re: AJP Flush Packet causing text/plain output Hi Lary, Larry Reisler wrote: Thanks for your reply. There is a holiday here right now, so I'm not sure if I will be able to get to file the Bugzilla issue before the holiday break. Maybe this time we are faster than our request ticketing ... We tried a few different versions of mod_jk (the early ones had other issues), but all the latest ones showed the problem. We are currently using mod_jk 1.2.25. In desperation, I was able to create a patch to mod_jk that seems to prevent the symptom from occurring. Here is the diff for the file jk_ajp_common.c: 1742,1751d1741 // Removing extra flush buffer if we do not need it. if (headeratclient == JK_FALSE) { int code = (int)jk_b_pget_byte(op-reply,0); unsigned int len = (unsigned int)jk_b_pget_int(op-reply,1); if ((code == JK_AJP13_SEND_BODY_CHUNK) (len == 0)) { jk_log(l, JK_LOG_DEBUG, Received flushbuffer -- ignoring); continue; } } I'm a little reluctant to use your approach. It should work, but the peeking for the message code and length does not belong into the get reply function. The best place for the patch would be in the callback handling, but there we don't know, if we already received the headers. So either we pass some additional info to the callback handling (status info about the request processing stage), or we prevent the flushing later. I decided to move an already existing flag response_started from the web server private data to the public service struct and use it inside the callback handler. This patch was committted (http://svn.apache.org/viewvc?view=revrevision=579349). You can find a simple download form of it under http://people.apache.org/~rjung/mod_jk-dev/patches/flush-before-header.patch The patch looks lengthy, but in fact did only small changes. It would be helpful, if you could try it with your reproduction scenario. Regards, Rainer - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: AJP Flush Packet causing text/plain output
FYI mod_jk that works (Apache -t says Syntax ok) http://archive.apache.org/dist/jakarta/tomcat-connectors/jk/binaries/win32/j k-1.2.6/ filename:mod_jk_1.2.6_2.0.50.dll filesize: 86016 (bookmark this site) Still looking for the mod_jk that works with Apache 2.2 Have a good holiday M-- - Original Message - Wrom: NBOHMKHJYFMYXOEAIJJPHSCRTNHGSWZIDR To: Tomcat Users List users@tomcat.apache.org Cc: Oren Hurvitz [EMAIL PROTECTED] Sent: Tuesday, September 25, 2007 4:45 PM Subject: RE: AJP Flush Packet causing text/plain output I understand your hesitation. I agree that my method was poor in terms of encapsulation, but was elegant in terms of its size, that it did not change any of the function signatures, and that it was easy to understand. The holiday has begun here, and I am off for this week -- I will try to test the patch ASAP. Thanks so much for your help. Larry. -Original Message- Wrom: EXCAXZOWCONEUQZAAFXISHJEXXIMQZUIVOTQNQEMSFDU Sent: Tuesday, September 25, 2007 8:58 PM To: Tomcat Users List Subject: Re: AJP Flush Packet causing text/plain output Hi Lary, Larry Reisler wrote: Thanks for your reply. There is a holiday here right now, so I'm not sure if I will be able to get to file the Bugzilla issue before the holiday break. Maybe this time we are faster than our request ticketing ... We tried a few different versions of mod_jk (the early ones had other issues), but all the latest ones showed the problem. We are currently using mod_jk 1.2.25. In desperation, I was able to create a patch to mod_jk that seems to prevent the symptom from occurring. Here is the diff for the file jk_ajp_common.c: 1742,1751d1741 // Removing extra flush buffer if we do not need it. if (headeratclient == JK_FALSE) { int code = (int)jk_b_pget_byte(op-reply,0); unsigned int len = (unsigned int)jk_b_pget_int(op-reply,1); if ((code == JK_AJP13_SEND_BODY_CHUNK) (len == 0)) { jk_log(l, JK_LOG_DEBUG, Received flushbuffer -- ignoring); continue; } } I'm a little reluctant to use your approach. It should work, but the peeking for the message code and length does not belong into the get reply function. The best place for the patch would be in the callback handling, but there we don't know, if we already received the headers. So either we pass some additional info to the callback handling (status info about the request processing stage), or we prevent the flushing later. I decided to move an already existing flag response_started from the web server private data to the public service struct and use it inside the callback handler. This patch was committted (http://svn.apache.org/viewvc?view=revrevision=579349). You can find a simple download form of it under http://people.apache.org/~rjung/mod_jk-dev/patches/flush-before-header.patch The patch looks lengthy, but in fact did only small changes. It would be helpful, if you could try it with your reproduction scenario. Regards, Rainer - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: AJP Flush Packet causing text/plain output
This has been given the bug # http://issues.apache.org/bugzilla/show_bug.cgi?id=43478 Rainer Jung [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] Hi Lary, Larry Reisler wrote: Thanks for your reply. There is a holiday here right now, so I'm not sure if I will be able to get to file the Bugzilla issue before the holiday break. Maybe this time we are faster than our request ticketing ... We tried a few different versions of mod_jk (the early ones had other issues), but all the latest ones showed the problem. We are currently using mod_jk 1.2.25. In desperation, I was able to create a patch to mod_jk that seems to prevent the symptom from occurring. Here is the diff for the file jk_ajp_common.c: 1742,1751d1741 // Removing extra flush buffer if we do not need it. if (headeratclient == JK_FALSE) { int code = (int)jk_b_pget_byte(op-reply,0); unsigned int len = (unsigned int)jk_b_pget_int(op-reply,1); if ((code == JK_AJP13_SEND_BODY_CHUNK) (len == 0)) { jk_log(l, JK_LOG_DEBUG, Received flushbuffer -- ignoring); continue; } } I'm a little reluctant to use your approach. It should work, but the peeking for the message code and length does not belong into the get reply function. The best place for the patch would be in the callback handling, but there we don't know, if we already received the headers. So either we pass some additional info to the callback handling (status info about the request processing stage), or we prevent the flushing later. I decided to move an already existing flag response_started from the web server private data to the public service struct and use it inside the callback handler. This patch was committted (http://svn.apache.org/viewvc?view=revrevision=579349). You can find a simple download form of it under http://people.apache.org/~rjung/mod_jk-dev/patches/flush-before-header.patch The patch looks lengthy, but in fact did only small changes. It would be helpful, if you could try it with your reproduction scenario. Regards, Rainer - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]