Re: Long running requests and timeouts...resolved...
Just FYI to those that were following this thread, this was a Cocoon (2.1.5) issue that caused this, not Tomcat. Turns out that the Cocoon 2.1.5 ResourceReader sets some response headers willy nilly. If you have another component that uses a resolver to access a pipeline that uses the reader (eg. using the cocoon: pseudo protocol, in our case it was a custom PDF concatenation serializer that called out to a sub-pipeline to get a separator page pdf), then the reader sets the Content-Length http reponse header in the original response object. If the originating component/pipeline generates output that is longer than what the reader grabbed, then it gets truncated. Not sure why Cocoon doesn't create a new response object when you try to resolve a local cocoon:/ resource, but it doesn't. Creating a customized ResourceReader with all response header setting commented out resolved the issue for me. Andrzej I have a situation where some requests that get sent to Tomcat are very long running (basically batch operations). I've been testing with a request that takes just over 7 minutes to process and returns and XML document as a response. The problem I'm having is that the response gets truncated. It's always truncated at a consistent spot. It always sends back exactly 3207 characters in the response body and this is consistent using Firefox or a commons- httpClient based script. Shorter running requests don't truncate anything, regardless of how long the response might be. I'm running Tomcat 5.0.28 and my Connector in the Tomcat server.xml file looks like: Connectorport=80 maxThreads=150 minSpareThreads=25 maxSpareThreads=75 enableLookups=false redirectPort=8443 acceptCount=100 debug=0 disableUploadTimeout=true tcpNoDelay=true connectionLinger=-1 connectionTimeout=600 connectionUploadTimeout=600 keepAlive=true maxKeepAliveRequests=-1 serverSocketTimeout=0/ I've tried all sorts of timeout values in the aboveincluding zero values, but the behaviour is consistent. Long running invocations truncate the response body to 3207 characters. I've also tried changing the client side timeout values, but again, to no avail. I'm kinda stumped as to what might cause this, especially given the very consistent response body length that is always returned. Almost seems like some strange buffering issue that is timer-related. Any ideas? Andrzej Jan Taramina Chaeron Corporation: Enterprise System Solutions http://www.chaeron.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Long running requests and timeouts?
I have a situation where some requests that get sent to Tomcat are very long running (basically batch operations). I've been testing with a request that takes just over 7 minutes to process and returns and XML document as a response. The problem I'm having is that the response gets truncated. It's always truncated at a consistent spot. It always sends back exactly 3207 characters in the response body and this is consistent using Firefox or a commons- httpClient based script. Shorter running requests don't truncate anything, regardless of how long the response might be. I'm running Tomcat 5.0.28 and my Connector in the Tomcat server.xml file looks like: Connector port=80 maxThreads=150 minSpareThreads=25 maxSpareThreads=75 enableLookups=false redirectPort=8443 acceptCount=100 debug=0 disableUploadTimeout=true tcpNoDelay=true connectionLinger=-1 connectionTimeout=600 connectionUploadTimeout=600 keepAlive=true maxKeepAliveRequests=-1 serverSocketTimeout=0/ I've tried all sorts of timeout values in the aboveincluding zero values, but the behaviour is consistent. Long running invocations truncate the response body to 3207 characters. I've also tried changing the client side timeout values, but again, to no avail. I'm kinda stumped as to what might cause this, especially given the very consistent response body length that is always returned. Almost seems like some strange buffering issue that is timer-related. Any ideas? Andrzej Jan Taramina Chaeron Corporation: Enterprise System Solutions http://www.chaeron.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Long running requests and timeouts?
I have noticed that you have posted this before. I don't have an answer but a couple of suggestions. First have you tried turning up the debug level to 99 and seeing if anything shows up in the logs. With any luck it might point to what is giving you such grief. Second. If all else fails, could you set up the process to write the file to a temp directory. Then after x minutes make a call for the file? So: Call to kick off process. Process runs and writes the file to the temp dir, overwriting any previous version. After 10 minutes download file. Third. Run the process with a tool to watch the process and see where it is dying. Fourth. Cause an error to get a stack trace. Then walk through the source code for each class in the process until you find the cause. Like I said, I know it's no answer but maybe it will lead to one. Doug - Original Message - From: Andrzej Jan Taramina [EMAIL PROTECTED] To: tomcat-user@jakarta.apache.org Sent: Tuesday, January 25, 2005 2:37 PM Subject: Long running requests and timeouts? I have a situation where some requests that get sent to Tomcat are very long running (basically batch operations). I've been testing with a request that takes just over 7 minutes to process and returns and XML document as a response. The problem I'm having is that the response gets truncated. It's always truncated at a consistent spot. It always sends back exactly 3207 characters in the response body and this is consistent using Firefox or a commons- httpClient based script. Shorter running requests don't truncate anything, regardless of how long the response might be. I'm running Tomcat 5.0.28 and my Connector in the Tomcat server.xml file looks like: Connector port=80 maxThreads=150 minSpareThreads=25 maxSpareThreads=75 enableLookups=false redirectPort=8443 acceptCount=100 debug=0 disableUploadTimeout=true tcpNoDelay=true connectionLinger=-1 connectionTimeout=600 connectionUploadTimeout=600 keepAlive=true maxKeepAliveRequests=-1 serverSocketTimeout=0/ I've tried all sorts of timeout values in the aboveincluding zero values, but the behaviour is consistent. Long running invocations truncate the response body to 3207 characters. I've also tried changing the client side timeout values, but again, to no avail. I'm kinda stumped as to what might cause this, especially given the very consistent response body length that is always returned. Almost seems like some strange buffering issue that is timer-related. Any ideas? Andrzej Jan Taramina Chaeron Corporation: Enterprise System Solutions http://www.chaeron.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Re: Long running requests and timeouts?
Geachte relatie, Het door u gebruikte e-mailadres is niet meer actief. U kunt uw e-mailbericht sturen naar [EMAIL PROTECTED] of dit bericht beantwoorden. Bedankt voor uw medewerking, Met vriendelijke groet, ATP Hypotheken Het Spoor 40 3994 AK Houten Tel. 030 750 25 33 Fax. 030 750 25 88 [EMAIL PROTECTED] -- DIT IS EEN AUTOMATISCH GEGENEREERD E-MAILBERICHT -- - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Long running requests and timeouts?
I have a situation where some requests that get sent to Tomcat are very long running (basically batch operations). I've been testing with a request that takes just over 7 minutes to process and returns and XML document as a response. The problem I'm having is that the response gets truncated. It's always truncated at a consistent spot (3207 characters for some reason, that's what Firefox reports). Shorter running requests don't truncate anything, regardless of how long the response might be. I'm running Tomcat 5.0.28 and my Connector in the Tomcat server.xml file looks like: Connector port=80 maxThreads=150 minSpareThreads=25 maxSpareThreads=75 enableLookups=false redirectPort=8443 acceptCount=100 debug=0 connectionTimeout=2 disableUploadTimeout=true / Is the connectionTimeout value the one that might be contributing to this behaviour? Are there any other timeout settings that might affect this? Thanks! Andrzej Jan Taramina Chaeron Corporation: Enterprise System Solutions http://www.chaeron.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Long running requests and timeouts?
I can't answer your specific question, but the scenario you describe raises a red flag with me in general... When you potentially have such long-running requests, it's usual a Bad Idea(tm) to just allow them to run like this as part of a request. I think it's fair to say that most people would say you should be taking some sort of queueing or multithread approach, coupled with polling. I don't know your usage pattern, but especially if there could be a number of such requests coming in at once, you are tieing up server resources this way. You also start running into situations like you mention with timeouts (I'm actually surprised the browser itself didn't time out after a few minutes). It also doesn't give a very good appearance to the user... it seems like the system has just frozen, which it actually hasn't. If it's feasible, I think you may save yourself some trouble by rearchitecting this rather than trying to solve this problem. You can do something as simple as this... (1) Have a class with two static HashMaps. One HashMap we'll call THREADS and it will store a reference to an object (more on this in a second), keyed by userID... In the other that we'll call STATUS you will store a Boolean, also keyed by userID. (2) When one of these long-running requests comes in, spawn a thread to handle the XML generation, and put a reference to the thread in that THREADS HashMap for the userID your servicing (might want to use sessionID instead, it's up to you)... In the STATUS HashMap, store a false Boolean (false means the request hasn't completed yet). (3) Return a Plase Wait page to the user. Use a simple JavaScript timeout to call a status checking function every few seconds that checks the value of the Boolean. If false, just return that Please Wait page again. (4) When the thread completes, it should set that Boolean to true for that userID. (5) When the status checking function finds True, it should (a) get the reference to the thread from the THREADS HashMap, (b) get the XML from the thread (just store it in a class-level variable and expose it with a getter), (b) set some value in the thread object that indicates the thread should die, (c) return the XML to the browser. Does that all make sense? I don't know if your in a position to rearchitect what your doing, but if you are, I very much suggest doing so. Hope this helps! -- Frank W. Zammetti Founder and Chief Software Architect Omnytex Technologies http://www.omnytex.com Andrzej Jan Taramina wrote: I have a situation where some requests that get sent to Tomcat are very long running (basically batch operations). I've been testing with a request that takes just over 7 minutes to process and returns and XML document as a response. The problem I'm having is that the response gets truncated. It's always truncated at a consistent spot (3207 characters for some reason, that's what Firefox reports). Shorter running requests don't truncate anything, regardless of how long the response might be. I'm running Tomcat 5.0.28 and my Connector in the Tomcat server.xml file looks like: Connector port=80 maxThreads=150 minSpareThreads=25 maxSpareThreads=75 enableLookups=false redirectPort=8443 acceptCount=100 debug=0 connectionTimeout=2 disableUploadTimeout=true / Is the connectionTimeout value the one that might be contributing to this behaviour? Are there any other timeout settings that might affect this? Thanks! Andrzej Jan Taramina Chaeron Corporation: Enterprise System Solutions http://www.chaeron.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] . - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
AW: Long running requests and timeouts?
Hi Andrzej, one possible workaround is compress the output, so the response is compressed with zip compresseion and your response gets shorter. You need a response filter for that, if this is an option an you need more details let me know. Cheers Bernhard -Ursprüngliche Nachricht- Von: Andrzej Jan Taramina [mailto:[EMAIL PROTECTED] Gesendet: Montag, 17. Januar 2005 16:25 An: tomcat-user@jakarta.apache.org Betreff: Long running requests and timeouts? I have a situation where some requests that get sent to Tomcat are very long running (basically batch operations). I've been testing with a request that takes just over 7 minutes to process and returns and XML document as a response. The problem I'm having is that the response gets truncated. It's always truncated at a consistent spot (3207 characters for some reason, that's what Firefox reports). Shorter running requests don't truncate anything, regardless of how long the response might be. I'm running Tomcat 5.0.28 and my Connector in the Tomcat server.xml file looks like: Connector port=80 maxThreads=150 minSpareThreads=25 maxSpareThreads=75 enableLookups=false redirectPort=8443 acceptCount=100 debug=0 connectionTimeout=2 disableUploadTimeout=true / Is the connectionTimeout value the one that might be contributing to this behaviour? Are there any other timeout settings that might affect this? Thanks! Andrzej Jan Taramina Chaeron Corporation: Enterprise System Solutions http://www.chaeron.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Long running requests and timeouts?
Frank: I don't know your usage pattern, but especially if there could be a number of such requests coming in at once, you are tieing up server resources this way. You also start running into situations like you mention with timeouts (I'm actually surprised the browser itself didn't time out after a few minutes). It also doesn't give a very good appearance to the user... it seems like the system has just frozen, which it actually hasn't. These requests are not going to be called by users through a browser, since they are Web Services/REST types of requests. The only time they might get called through a browser is for testing purposes, in which case the tester knows that they will be very long running and that's not at issue. The timeout seems to be affecting scripted invocations of the request as well (using the commons-http library), so it doesn't seem to be a browser timeout issue as far as I can tell. If it's feasible, I think you may save yourself some trouble by rearchitecting this rather than trying to solve this problem. You can do something as simple as this... Does that all make sense? I don't know if your in a position to rearchitect what your doing, but if you are, I very much suggest doing so. Hope this helps! This does make sense, except for the user part, since users will never access the URL's involved directly. Furthermore, these long running tasks will be called by a scheduler, off hours, and typically there will be very few such requests a day (maybe 2 or 3 max). The requests are really single overnight batch processing runs. However, deadlines preclude rearchitecting the solution to make it asynchronous as you suggest (it's currently synchronous) in the short term. Hence my looking for a way to easily fix the timeout issue that causes the response to be truncated when Tomcat services a long running request. Thanks for the inputmost appreciated. Andrzej Jan Taramina Chaeron Corporation: Enterprise System Solutions http://www.chaeron.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Long running requests and timeouts?
I have the same problem. If you find a solution, please post and email me. What I found out with my application is that when tomcat runs low on memory, it will start to behave erratically. For example: scheduled job won't run; running processes seem to terminate/timeout... I don't know exactly what's happening, but enough memory usually keeps these problems away... Peiyun -Original Message- From: Andrzej Jan Taramina [mailto:[EMAIL PROTECTED] Sent: January 17, 2005 1:22 PM To: Tomcat Users List Cc: [EMAIL PROTECTED] Subject: Re: Long running requests and timeouts? Frank: I don't know your usage pattern, but especially if there could be a number of such requests coming in at once, you are tieing up server resources this way. You also start running into situations like you mention with timeouts (I'm actually surprised the browser itself didn't time out after a few minutes). It also doesn't give a very good appearance to the user... it seems like the system has just frozen, which it actually hasn't. These requests are not going to be called by users through a browser, since they are Web Services/REST types of requests. The only time they might get called through a browser is for testing purposes, in which case the tester knows that they will be very long running and that's not at issue. The timeout seems to be affecting scripted invocations of the request as well (using the commons-http library), so it doesn't seem to be a browser timeout issue as far as I can tell. If it's feasible, I think you may save yourself some trouble by rearchitecting this rather than trying to solve this problem. You can do something as simple as this... Does that all make sense? I don't know if your in a position to rearchitect what your doing, but if you are, I very much suggest doing so. Hope this helps! This does make sense, except for the user part, since users will never access the URL's involved directly. Furthermore, these long running tasks will be called by a scheduler, off hours, and typically there will be very few such requests a day (maybe 2 or 3 max). The requests are really single overnight batch processing runs. However, deadlines preclude rearchitecting the solution to make it asynchronous as you suggest (it's currently synchronous) in the short term. Hence my looking for a way to easily fix the timeout issue that causes the response to be truncated when Tomcat services a long running request. Thanks for the inputmost appreciated. Andrzej Jan Taramina Chaeron Corporation: Enterprise System Solutions http://www.chaeron.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Long running requests and timeouts?
These requests are not going to be called by users through a browser, since they are Web Services/REST types of requests. The only time they might get called through a browser is for testing purposes, in which case the tester knows that they will be very long running and that's not at issue. I got into this conversation late so forgive me if this has already been covered but have you considered putting a webservice engine on both ends and sending some kind of call-back from the called process when it's done? On Mon, 2005-01-17 at 14:21, Jiang, Peiyun wrote: I have the same problem. If you find a solution, please post and email me. What I found out with my application is that when tomcat runs low on memory, it will start to behave erratically. For example: scheduled job won't run; running processes seem to terminate/timeout... I don't know exactly what's happening, but enough memory usually keeps these problems away... Peiyun -Original Message- From: Andrzej Jan Taramina [mailto:[EMAIL PROTECTED] Sent: January 17, 2005 1:22 PM To: Tomcat Users List Cc: [EMAIL PROTECTED] Subject: Re: Long running requests and timeouts? Frank: I don't know your usage pattern, but especially if there could be a number of such requests coming in at once, you are tieing up server resources this way. You also start running into situations like you mention with timeouts (I'm actually surprised the browser itself didn't time out after a few minutes). It also doesn't give a very good appearance to the user... it seems like the system has just frozen, which it actually hasn't. These requests are not going to be called by users through a browser, since they are Web Services/REST types of requests. The only time they might get called through a browser is for testing purposes, in which case the tester knows that they will be very long running and that's not at issue. The timeout seems to be affecting scripted invocations of the request as well (using the commons-http library), so it doesn't seem to be a browser timeout issue as far as I can tell. If it's feasible, I think you may save yourself some trouble by rearchitecting this rather than trying to solve this problem. You can do something as simple as this... Does that all make sense? I don't know if your in a position to rearchitect what your doing, but if you are, I very much suggest doing so. Hope this helps! This does make sense, except for the user part, since users will never access the URL's involved directly. Furthermore, these long running tasks will be called by a scheduler, off hours, and typically there will be very few such requests a day (maybe 2 or 3 max). The requests are really single overnight batch processing runs. However, deadlines preclude rearchitecting the solution to make it asynchronous as you suggest (it's currently synchronous) in the short term. Hence my looking for a way to easily fix the timeout issue that causes the response to be truncated when Tomcat services a long running request. Thanks for the inputmost appreciated. Andrzej Jan Taramina Chaeron Corporation: Enterprise System Solutions http://www.chaeron.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Long running requests and timeouts?
Ok Andrzej, sorry I couldn't be of any real help. I see what I suggested wouldn't apply in your situation, I was just taking a guess anyway. And certainly I understand the pressure of deadlines :) Perhaps my approach might help someone else that's just lurking, but good luck in solving your problem. I too will be watching to see what your solution is, just out of curiosity. -- Frank W. Zammetti Founder and Chief Software Architect Omnytex Technologies http://www.omnytex.com Andrzej Jan Taramina wrote: Frank: I don't know your usage pattern, but especially if there could be a number of such requests coming in at once, you are tieing up server resources this way. You also start running into situations like you mention with timeouts (I'm actually surprised the browser itself didn't time out after a few minutes). It also doesn't give a very good appearance to the user... it seems like the system has just frozen, which it actually hasn't. These requests are not going to be called by users through a browser, since they are Web Services/REST types of requests. The only time they might get called through a browser is for testing purposes, in which case the tester knows that they will be very long running and that's not at issue. The timeout seems to be affecting scripted invocations of the request as well (using the commons-http library), so it doesn't seem to be a browser timeout issue as far as I can tell. If it's feasible, I think you may save yourself some trouble by rearchitecting this rather than trying to solve this problem. You can do something as simple as this... Does that all make sense? I don't know if your in a position to rearchitect what your doing, but if you are, I very much suggest doing so. Hope this helps! This does make sense, except for the user part, since users will never access the URL's involved directly. Furthermore, these long running tasks will be called by a scheduler, off hours, and typically there will be very few such requests a day (maybe 2 or 3 max). The requests are really single overnight batch processing runs. However, deadlines preclude rearchitecting the solution to make it asynchronous as you suggest (it's currently synchronous) in the short term. Hence my looking for a way to easily fix the timeout issue that causes the response to be truncated when Tomcat services a long running request. Thanks for the inputmost appreciated. Andrzej Jan Taramina Chaeron Corporation: Enterprise System Solutions http://www.chaeron.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Long running requests and timeouts?
I have a situation where some requests that get sent to Tomcat are very long running (basically batch operations). I've been testing with a request that takes just over 7 minutes to process and returns and XML document as a response. The problem I'm having is that the response gets truncated. It's always truncated at a consistent spot (3207 characters for some reason, that's what Firefox reports). Shorter running requests don't truncate anything, regardless of how long the response might be. I'm running Tomcat 5.0.28 and my Connector in the Tomcat server.xml file looks like: Connector port=80 maxThreads=150 minSpareThreads=25 maxSpareThreads=75 enableLookups=false redirectPort=8443 acceptCount=100 debug=0 connectionTimeout=2 disableUploadTimeout=true / Is the connectionTimeout value the one that might be contributing to this behaviour? Are there any other timeout settings that might affect this? Thanks! Andrzej Jan Taramina Chaeron Corporation: Enterprise System Solutions http://www.chaeron.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
