Re: [Resin-interest] Multiple instances of the same webapp after redeploy in 4.0.16 (strange!!!)
Scott Ferguson wrote (2011-04-05 01:25): > On 04/04/2011 10:02 AM, Mattias Jiderhamn wrote: >> Scott Ferguson wrote (2011-04-04 18:47): >>> On 04/04/2011 12:43 AM, Mattias Jiderhamn wrote: While evaluating Resin 4.0.16 we are seeing something that makes me really concerned. It appears as if after a redeploy there can be two instances of our main app running. I was made aware of this by noticing that timed servlets (>>> ... />) were having concurrency issues and logging the same event twice within the same second. This made me add some extra logging in a servlet, and I can actually see that at some reploys, the same web app is initialized twice - in different, concurrent threads and different classloaders. For example one instance may be initialized by Thread[http://*:8080-776,5,main] and at the same time (give or take a few seconds) another one by Thread[resin-366,5,main]. I cannot deliberately recreate it by simply redeploying (which makes it less likely to be a configuration issue). But it happens about daily on our test server; possibly issuing requests on a session in the app while redeploying is a factor. We are not using versioning. dependency-check-interval is set to 20 seconds. The "extra" app does not appear in the admin console. How can we figure out what is going on here??? >>> >>> Are you using version="true" and/or versioned web-apps? With >>> versioned web-apps, Resin is supposed to have two versions running >>> at the same time while it migrates sessions from the old version to >>> the new version. >> >> As stated above, we are not using versioning. >> (And if we we're, should the old version be re-initialized when >> deploying a new version...?) > > Do you have logs for these events? The WebApp start/stop should be > logged at the info level and be in the jvm-default.log. > > I can't yet see how this is happening, though I have found a related > issue with 503s. > > The main thing I'd be looking for is: > > {thread1} WebApp[/foo] start > {thread2} WebApp[/foo] stop > {thread2} WebApp[/foo] start > {threadX} WebApp[/foo] stop > > VS > > {thread1} WebApp[/foo] start > {thread2} WebApp[/foo] start > ... > > In other words, an actual simultaneous start compared to an erroneous > restart. Yes, there are multiple WebApp starts in the log: [2011-04-05 15:03:51.905] WebApp[production/webapp/default/ROOT] starting ... [2011-04-05 15:03:59.062] WebApp[production/webapp/default/ROOT] starting (the app takes more than 8 seconds to initialize, so they are concurrent inits) I haven't configured the default logs to include the thread, but from my own logging, I can see that the first start is in Thread[http://*:8080-343,5,main] and the second in Thread[http://*:8080-349,5,main], in different classloaders I caught the above starts with "finer" logging level. Should I extract the part around this time and send to you, Scott? -- ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Multiple instances of the same webapp after redeploy in 4.0.16 (strange!!!)
On 04/04/2011 10:02 AM, Mattias Jiderhamn wrote: > Scott Ferguson wrote (2011-04-04 18:47): >> On 04/04/2011 12:43 AM, Mattias Jiderhamn wrote: >>> While evaluating Resin 4.0.16 we are seeing something that makes me >>> really concerned. It appears as if after a redeploy there can be two >>> instances of our main app running. >>> >>> I was made aware of this by noticing that timed servlets (>> ... />) were having concurrency issues and logging the same event >>> twice within the same second. >>> This made me add some extra logging in a servlet, >>> and I can actually see that at some reploys, the same web app is >>> initialized twice - in different, concurrent threads and different >>> classloaders. >>> For example one instance may be initialized by >>> Thread[http://*:8080-776,5,main] and at the same time (give or take >>> a few seconds) another one by Thread[resin-366,5,main]. >>> >>> I cannot deliberately recreate it by simply redeploying (which makes >>> it less likely to be a configuration issue). But it happens about >>> daily on our test server; possibly issuing requests on a session in >>> the app while redeploying is a factor. >>> >>> We are not using versioning. >>> dependency-check-interval is set to 20 seconds. >>> The "extra" app does not appear in the admin console. >>> >>> How can we figure out what is going on here??? >>> >> >> Are you using version="true" and/or versioned web-apps? With >> versioned web-apps, Resin is supposed to have two versions running at >> the same time while it migrates sessions from the old version to the >> new version. > > As stated above, we are not using versioning. > (And if we we're, should the old version be re-initialized when > deploying a new version...?) Do you have logs for these events? The WebApp start/stop should be logged at the info level and be in the jvm-default.log. I can't yet see how this is happening, though I have found a related issue with 503s. The main thing I'd be looking for is: {thread1} WebApp[/foo] start {thread2} WebApp[/foo] stop {thread2} WebApp[/foo] start {threadX} WebApp[/foo] stop VS {thread1} WebApp[/foo] start {thread2} WebApp[/foo] start ... In other words, an actual simultaneous start compared to an erroneous restart. -- Scott ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Multiple instances of the same webapp after redeploy in 4.0.16 (strange!!!)
On 04/04/2011 10:02 AM, Mattias Jiderhamn wrote: > Scott Ferguson wrote (2011-04-04 18:47): >> On 04/04/2011 12:43 AM, Mattias Jiderhamn wrote: >>> While evaluating Resin 4.0.16 we are seeing something that makes me >>> really concerned. It appears as if after a redeploy there can be two >>> instances of our main app running. >>> >>> I was made aware of this by noticing that timed servlets (>> ... />) were having concurrency issues and logging the same event >>> twice within the same second. >>> This made me add some extra logging in a servlet, >>> and I can actually see that at some reploys, the same web app is >>> initialized twice - in different, concurrent threads and different >>> classloaders. >>> For example one instance may be initialized by >>> Thread[http://*:8080-776,5,main] and at the same time (give or take >>> a few seconds) another one by Thread[resin-366,5,main]. >>> >>> I cannot deliberately recreate it by simply redeploying (which makes >>> it less likely to be a configuration issue). But it happens about >>> daily on our test server; possibly issuing requests on a session in >>> the app while redeploying is a factor. >>> >>> We are not using versioning. >>> dependency-check-interval is set to 20 seconds. >>> The "extra" app does not appear in the admin console. >>> >>> How can we figure out what is going on here??? >>> >> >> Are you using version="true" and/or versioned web-apps? With >> versioned web-apps, Resin is supposed to have two versions running at >> the same time while it migrates sessions from the old version to the >> new version. > > As stated above, we are not using versioning. Thanks. I'd missed that. > (And if we we're, should the old version be re-initialized when > deploying a new version...?) No. Only the latest version should ever be initialized. Even if you were to restart Resin completely with multiple versions, only the latest version should be initialized. I've filed this as http://bugs.caucho.com/view.php?id=4474. -- Scott ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Multiple instances of the same webapp after redeploy in 4.0.16 (strange!!!)
Scott Ferguson wrote (2011-04-04 18:47): > On 04/04/2011 12:43 AM, Mattias Jiderhamn wrote: >> While evaluating Resin 4.0.16 we are seeing something that makes me >> really concerned. It appears as if after a redeploy there can be two >> instances of our main app running. >> >> I was made aware of this by noticing that timed servlets (> />) were having concurrency issues and logging the same event twice >> within the same second. >> This made me add some extra logging in a servlet, >> and I can actually see that at some reploys, the same web app is >> initialized twice - in different, concurrent threads and different >> classloaders. >> For example one instance may be initialized by >> Thread[http://*:8080-776,5,main] and at the same time (give or take a >> few seconds) another one by Thread[resin-366,5,main]. >> >> I cannot deliberately recreate it by simply redeploying (which makes >> it less likely to be a configuration issue). But it happens about >> daily on our test server; possibly issuing requests on a session in >> the app while redeploying is a factor. >> >> We are not using versioning. >> dependency-check-interval is set to 20 seconds. >> The "extra" app does not appear in the admin console. >> >> How can we figure out what is going on here??? >> > > Are you using version="true" and/or versioned web-apps? With versioned > web-apps, Resin is supposed to have two versions running at the same > time while it migrates sessions from the old version to the new version. As stated above, we are not using versioning. (And if we we're, should the old version be re-initialized when deploying a new version...?) -- ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Multiple instances of the same webapp after redeploy in 4.0.16 (strange!!!)
On 04/04/2011 12:43 AM, Mattias Jiderhamn wrote: > While evaluating Resin 4.0.16 we are seeing something that makes me > really concerned. It appears as if after a redeploy there can be two > instances of our main app running. > > I was made aware of this by noticing that timed servlets ( />) were having concurrency issues and logging the same event twice > within the same second. > This made me add some extra logging in a servlet, > and I can actually see that at some reploys, the same web app is > initialized twice - in different, concurrent threads and different > classloaders. > For example one instance may be initialized by > Thread[http://*:8080-776,5,main] and at the same time (give or take a > few seconds) another one by Thread[resin-366,5,main]. > > I cannot deliberately recreate it by simply redeploying (which makes > it less likely to be a configuration issue). But it happens about > daily on our test server; possibly issuing requests on a session in > the app while redeploying is a factor. > > We are not using versioning. > dependency-check-interval is set to 20 seconds. > The "extra" app does not appear in the admin console. > > How can we figure out what is going on here??? > Are you using version="true" and/or versioned web-apps? With versioned web-apps, Resin is supposed to have two versions running at the same time while it migrates sessions from the old version to the new version. -- Scott ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest