On 27/10/2020 04:43, Eric Robinson wrote: <snip/>
>>>>> Any changes in the Nginx configuration in the relevant timescale? >>>>> >>> >>> The last change to the nginx config files was on 8/21. The first >>> report of problems from the users in question was on 9/16. There is >>> another set of users on a different tomcat instance who reported >>> issues around 8/26, 5 days after nginx config change. It seems >>> unlikely to be related. Also, I can't imagine what nginx could be >>> sending that would induce the upstream tomcat to behave this way. If there is some sort of retaining references to request/response/etc. at the root of this then that sort of issue is very sensitive to timing. For example, I have had reliable reproduction cases in the past that stopped working with the addition of a single debug statement. Any sort of configuration change might have changed the timing sufficiently to trigger the issue At this point, I'd say the Nginx config change might be a potential trigger if the root cause is retaining references. >>>>> Any updates to the application in the relevant timescale? >>>>> >>> >>> Their application was patched to a newer version on 6/5. That seems far enough away to be unlikely. >>>>> Any features users started using that hadn't been used before in >>>>> that timescale? >>> >>> That one I couldn't answer, as we are only the hosting facility and we >>> are not in the loop when it comes to the users' workflow, but it seems >>> unlikely given the nature of their business. Fair enough. That one was a bit of a shot in the dark. <snip/> >>> 1. Now that you have provided this patch, should I still enable >>> RECYCLE_FACADES=true? I'd recommend yes. At least until the issue is resolved. >>> 2. The servers in question are multi-tenanted. There are 17 instances >>> of tomcat, each running on a different set of ports and out of a >>> separate directory tree, but they are all started with essentially the >>> same init script, which exports certain site-specific path variables >>> and executes tomcat from the specified folder structure. Can you think >>> of any potential issues where making this change for one instance >>> could have a negative effect on any of the other instances? Probably >>> not, but just being careful. I will have these changes implemented >>> during our nightly maintenance window will begin to gather relevant logs >> first thing tomorrow! I can't think of any side effects. >>> >>> --Eric >> >> Mark, the changes have been made per your instructions and tomcat has >> been restarted. The debug0.log, and debug0.log.lck files were created in the >> directory, but they currently both have 0 bytes. Drat. That suggests something isn't quite right as the logs should start filling up as soon as a single request is made. I'll double check my instructions if you could double check your end. Konstantin noted there was no source provided. I've pushed the branch to https://github.com/markt-asf/tomcat/tree/debug-7.0.72 so you can see the changes I made. > Also, RECYCLE_FACADES has been enabled and I confirmed that it is referenced > in the logs as follows... > > INFO: Command line argument: > -Dorg.apache.catalina.connector.RECYCLE_FACADES=true Great. Mark --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
