Re: git (yet again)
Regarding question b), I vote yes, whatever the organization of the development Regarding a), for my part I rather see something between A and B. I think of the following questions that should be answered first : - will there be 1 repo per major version like the current SVN setup ? or only branches in a unique repo ? - how to backport modifications from one major version to another ? is cherry-picking ok ? - where will the main repo be (the one committers will push to)? ASF or github ? where should we open and treat pull requests, ASF or github ? (currently having the SVN repo at ASF and pull requests at github is not convenient) Sylvain On 2 sept. 2014, at 18:41, Mark Thomas ma...@apache.org wrote: I've been looking at this again (anything to get a break from writing parsers for cookies) and chatting with some of the infra folks that look after the ASF's git repos. There are a couple of things we need to do: a) decide how we want to organise development in git b) decide if we want to move to git Now the decision we make for a) might influence some folks to make a different decision for b). On the other hand, there is no point debating a) if we are never going to move. So, how do folks want to approach this? A: Vote to move to git and then figure out how best to use it? or B: Agree our git workflows and then have a vote on moving to git with those workflows? I'm leaning towards A myself. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
buildbot stdio missing junit assertion failures ?
Hello all, In the last couple of days, several buildbot runs failed because of intermittent failures in TestStuckThreadDetectionValve. By following links provided in buildbot emails or the ones in http://ci.apache.org/builders/tomcat-trunk , I arrived in the large stdio file of the failing builds, for example http://ci.apache.org/builders/tomcat-trunk/builds/104/steps/compile_1/logs/stdio In such a file, TestStuckThreadDetectionValve was marked FAILED, with a line that said : [junit] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 17.436 sec So there was 1 failure, but I could not figure out on which test method it was, not what the failure was. After much time lost, I figured out that part of the test output was missing from the stdio and is actually sent to individual files like http://ci.apache.org/projects/tomcat/tomcat8/1596888/TEST-org.apache.catalina.valves.TestStuckThreadDetectionValve.NIO.txt Would it be possible to have assertion failures in stdio too ? I could not find any link to such individual files, actually I had to guess the URL by deriving a link provided by Konstantin a couple of days ago... Or did I miss something ? I also came accross this page http://vmgump.apache.org/gump/public/tomcat-trunk/tomcat-trunk-test-nio/ that looked promising to help me find the failure message but could not find any failure. Until I realized Gump is a different CI process than buildbot. Why are there 2 different ones ? Regards, Sylvain - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: buildbot stdio missing junit assertion failures ?
On 23 mai 2014, at 22:04, Mark Thomas ma...@apache.org wrote: On 23/05/2014 21:00, Sylvain Laurent wrote: Would it be possible to have assertion failures in stdio too ? No need. why not ? this would avoid having to jump to different files to find the culprit of a failed build... I could not find any link to such individual files, actually I had to guess the URL by deriving a link provided by Konstantin a couple of days ago... Or did I miss something ? Again, the buildbot is placing those files in the incorrect location. Once the correct location is used, a directory listing will be available. ok, I guess this is in the hands of infra team ? I also came accross this page http://vmgump.apache.org/gump/public/tomcat-trunk/tomcat-trunk-test-nio/that looked promising to help me find the failure message but could not find any failure. Until I realized Gump is a different CI process than buildbot. Why are there 2 different ones ? Why not use multiple CI systems? You increase your chances of detecting issues. We have certainly had issues that only one of the systems has found. Also, Gump has a slightly different purpose which is to check for interoperability between components. I also saw in http://tomcat.apache.org/ci.html#Jenkins a link to an experimental tomcat7 build by Jenkins but this does not seem to work. Are there plans to add a 3rd CI system ? - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Git (again)
Hello, Back in january/february there was a thread on moving to git with many +1 for it. Are there any concrete plans for this ? I'm looking forward to it as using SVN from Europe is so slow... or maybe I'm now too accustomed to git's speed ;-) I was wondering what the workflow would be for tomcat development with git : currently some commits are made on trunk, then potentially selectively merged to tomcat 7 then 6. With git, how would it look like ? cherry picks ? that's not very in the spirit of git. Or fixes on tomcat 6 or 7 and then merge to 8 ? In that case tomcat 8 would be branch master and tomcat 6 and 7 would be on their own branch and merges would look like 6 - 7 - master ? I'm quite used to such a workflow for enterprise app dev where we add features to master, very rarely to the old/stable version... but this is actually not the case with tomcat... Sylvain - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn commit: r1596415 - /tomcat/site/trunk/docs/ci.html
ok, I reverted (r1596664). Sorry, I had not checked the README, everything was explained... who can fix the buildbot config now ? Sylvain On 21 mai 2014, at 00:18, Konstantin Kolinko knst.koli...@gmail.com wrote: 2014-05-21 1:41 GMT+04:00 Sylvain Laurent slaur...@apache.org: so, I just have to fix /xdocs/ci.xml ? and the buildbot generates ci.html and commits it ? 1. No. The tomcat.apache.org web site is authored in XML (just like Tomcat documentation). See README.txt and build.xml in the root directory of /site. It is odd that you edited and committed generated HTML file only. 2. The old links were correct. The buildbot is misconfigured and publishes generated documentation snapshots, junit logs and coverage reports into a wrong place (all in the same directory instead of subdirectories). On 20 mai 2014, at 23:14, Konstantin Kolinko knst.koli...@gmail.com wrote: 2014-05-21 0:58 GMT+04:00 slaur...@apache.org: Author: slaurent Date: Tue May 20 20:58:55 2014 New Revision: 1596415 URL: http://svn.apache.org/r1596415 Log: fixed URLs to documentation snapshots -1. 1. This shall be fixed not here, but in Buildbot configuration. 2. It writes documentation and coverage (and logs) into the same place. Whether you are seeing docs or coverage depends on whether a build is running in this very moment. 3. ci.html only? (Without xml) Modified: tomcat/site/trunk/docs/ci.html Modified: tomcat/site/trunk/docs/ci.html URL: http://svn.apache.org/viewvc/tomcat/site/trunk/docs/ci.html?rev=1596415r1=1596414r2=1596415view=diff - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: buildbot failure in ASF Buildbot on tomcat-trunk
) [junit] at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:526) [junit] at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1033) [junit] at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:652) [junit] at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222) [junit] at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1565) [junit] at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1522) [junit] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [junit] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [junit] at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [junit] at java.lang.Thread.run(Thread.java:722) [junit] [junit] 21-May-2014 17:48:11.909 WARNING [ContainerBackgroundProcessor[StandardEngine[Tomcat].StandardHost[localhost].StandardContext[]]] org.apache.catalina.valves.StuckThreadDetectionValve.notifyStuckThreadCompleted Thread #34;http-nio-127.0.0.1-auto-2-exec-1#34; (id=28) was previously reported to be stuck but has completed. It was active for approximately 5,980 milliseconds. [junit] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 17.473 sec [junit] 21-May-2014 17:48:13.913 INFO [main] org.apache.coyote.AbstractProtocol.pause Pausing ProtocolHandler [#34;http-nio-127.0.0.1-auto-2-39963#34;] [junit] 21-May-2014 17:48:13.914 INFO [main] org.apache.catalina.core.StandardService.stopInternal Stopping service Tomcat [junit] 21-May-2014 17:48:13.921 INFO [main] org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler [#34;http-nio-127.0.0.1-auto-2-39963#34;] [junit] 21-May-2014 17:48:13.922 INFO [main] org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler [#34;http-nio-127.0.0.1-auto-2-39963#34;] /spanspan class=stderr[junit] Test org.apache.catalina.valves.TestStuckThreadDetectionValve FAILED There seems to be a mix and desynchronization of stdout and stderr, but apart from that, I cannot see where it failed. I also tried to look in http://vmgump.apache.org/gump/public/tomcat-trunk/tomcat-trunk-test-nio/gump_file/TEST-org.apache.catalina.valves.TestStuckThreadDetectionValve.NIO.txt.html but it's now a later (passing) build... any ideas on why it would fail without any error message ? Sylvain On 20 mai 2014, at 22:42, Sylvain Laurent slaur...@apache.org wrote: I think I fixed the problem, which was only in the test case. There was a timing issue (the bot seems to be even slower than my 5 years old mac), which provoked an assertion failure. It then provoked the class loading issue because the request was still stuck in the background when the test returned... I hope this will be ok with the next build... Sylvain On 20 mai 2014, at 21:14, Sylvain Laurent slaur...@apache.org wrote: I'll have a look at this right now. On 19 mai 2014, at 23:28, Konstantin Kolinko knst.koli...@gmail.com wrote: 2014-05-19 23:30 GMT+04:00 build...@apache.org: The Buildbot has detected a new failure on builder tomcat-trunk while building ASF Buildbot. Full details are available at: http://ci.apache.org/builders/tomcat-trunk/builds/89 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: bb-vm_ubuntu Build Reason: scheduler Build Source Stamp: [branch tomcat/trunk] 1595898 Blamelist: markt BUILD FAILED: failed compile_1 The failed test is org.apache.catalina.valves.TestStuckThreadDetectionValve x NIO. BIO and NIO2 runs were OK. http://ci.apache.org/projects/tomcat/tomcat8/1595898/ http://ci.apache.org/projects/tomcat/tomcat8/1595898/TEST-org.apache.catalina.valves.TestStuckThreadDetectionValve.NIO.txt A difference is that there was an attempt to load a class which failed on a stopped web application. Best regards, Konstantin Kolinko - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: buildbot failure in ASF Buildbot on tomcat-trunk
I'll have a look at this right now. On 19 mai 2014, at 23:28, Konstantin Kolinko knst.koli...@gmail.com wrote: 2014-05-19 23:30 GMT+04:00 build...@apache.org: The Buildbot has detected a new failure on builder tomcat-trunk while building ASF Buildbot. Full details are available at: http://ci.apache.org/builders/tomcat-trunk/builds/89 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: bb-vm_ubuntu Build Reason: scheduler Build Source Stamp: [branch tomcat/trunk] 1595898 Blamelist: markt BUILD FAILED: failed compile_1 The failed test is org.apache.catalina.valves.TestStuckThreadDetectionValve x NIO. BIO and NIO2 runs were OK. http://ci.apache.org/projects/tomcat/tomcat8/1595898/ http://ci.apache.org/projects/tomcat/tomcat8/1595898/TEST-org.apache.catalina.valves.TestStuckThreadDetectionValve.NIO.txt A difference is that there was an attempt to load a class which failed on a stopped web application. Best regards, Konstantin Kolinko - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: buildbot failure in ASF Buildbot on tomcat-trunk
I think I fixed the problem, which was only in the test case. There was a timing issue (the bot seems to be even slower than my 5 years old mac), which provoked an assertion failure. It then provoked the class loading issue because the request was still stuck in the background when the test returned... I hope this will be ok with the next build... Sylvain On 20 mai 2014, at 21:14, Sylvain Laurent slaur...@apache.org wrote: I'll have a look at this right now. On 19 mai 2014, at 23:28, Konstantin Kolinko knst.koli...@gmail.com wrote: 2014-05-19 23:30 GMT+04:00 build...@apache.org: The Buildbot has detected a new failure on builder tomcat-trunk while building ASF Buildbot. Full details are available at: http://ci.apache.org/builders/tomcat-trunk/builds/89 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: bb-vm_ubuntu Build Reason: scheduler Build Source Stamp: [branch tomcat/trunk] 1595898 Blamelist: markt BUILD FAILED: failed compile_1 The failed test is org.apache.catalina.valves.TestStuckThreadDetectionValve x NIO. BIO and NIO2 runs were OK. http://ci.apache.org/projects/tomcat/tomcat8/1595898/ http://ci.apache.org/projects/tomcat/tomcat8/1595898/TEST-org.apache.catalina.valves.TestStuckThreadDetectionValve.NIO.txt A difference is that there was an attempt to load a class which failed on a stopped web application. Best regards, Konstantin Kolinko - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn commit: r1596415 - /tomcat/site/trunk/docs/ci.html
so, I just have to fix /xdocs/ci.xml ? and the buildbot generates ci.html and commits it ? On 20 mai 2014, at 23:14, Konstantin Kolinko knst.koli...@gmail.com wrote: 2014-05-21 0:58 GMT+04:00 slaur...@apache.org: Author: slaurent Date: Tue May 20 20:58:55 2014 New Revision: 1596415 URL: http://svn.apache.org/r1596415 Log: fixed URLs to documentation snapshots -1. 1. This shall be fixed not here, but in Buildbot configuration. 2. It writes documentation and coverage (and logs) into the same place. Whether you are seeing docs or coverage depends on whether a build is running in this very moment. 3. ci.html only? (Without xml) Modified: tomcat/site/trunk/docs/ci.html Modified: tomcat/site/trunk/docs/ci.html URL: http://svn.apache.org/viewvc/tomcat/site/trunk/docs/ci.html?rev=1596415r1=1596414r2=1596415view=diff == --- tomcat/site/trunk/docs/ci.html (original) +++ tomcat/site/trunk/docs/ci.html Tue May 20 20:58:55 2014 @@ -263,14 +263,14 @@ prepared and published by ASF Buildbot, ul li -a href=http://ci.apache.org/projects/tomcat/tomcat8/docs/index.html; rel=nofollowTomcat trunk/a (8.0.x)/li +a href=http://ci.apache.org/projects/tomcat/tomcat8/index.html; rel=nofollowTomcat trunk/a (8.0.x)/li li -a href=http://ci.apache.org/projects/tomcat/tomcat7/docs/index.html; rel=nofollowTomcat 7.0.x/a +a href=http://ci.apache.org/projects/tomcat/tomcat7/index.html; rel=nofollowTomcat 7.0.x/a /li li -a href=http://ci.apache.org/projects/tomcat/tomcat6/docs/index.html; rel=nofollowTomcat 6.0.x/a +a href=http://ci.apache.org/projects/tomcat/tomcat6/index.html; rel=nofollowTomcat 6.0.x/a /li /ul - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [VOTE] Release Apache Tomcat 7.0.24
On 16 janv. 2012, at 10:44, Rainer Jung wrote: 1) Unit test failure due to missing target directory in src dist I noticed that test/webapp-3.0-virtual-library/target is missing from the src dist. So when rebuilding from src dist and running unit tests, I get a failure in TestVirtualContext, because some resource which is usually retrieved form the target dir is missing: Testcase: testVirtualClassLoader took 7.434 sec FAILED expected:200 but was:404 junit.framework.AssertionFailedError: expected:200 but was:404 at org.apache.catalina.loader.TestVirtualContext.assertPageContains(TestVirtualContext.java:302) at org.apache.catalina.loader.TestVirtualContext.assertPageContains(TestVirtualContext.java:294) at org.apache.catalina.loader.TestVirtualContext.testVirtualClassLoader(TestVirtualContext.java:100) AFAIK the root cause is, that we exclude **/target/** in the ant target dist-source from being copied. Note that there is also test/webapp-3.0-virtual-library/target. I don't know, whether we can simply drop this exclude, or should rename the directory. I CC'd Sylvain explicitely, maybe he can comment on it. If there are no side effect to dropping this exclude, you may do it (I really don't know enough of the build system of tomcat). Otherwise I can rename those directories so that it is not excluded. Just let me know. Sylvain - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Move to Maven? (WAS: Re: Publishing process for JARs for Maven Central)
On 20 déc. 2011, at 12:22, Mark Thomas wrote: Where I disagree is on whether a switch to Maven lowers that barrier to entry. I agree it lowers it for folks that already know Maven but don't know Ant but that it raises it for folks that know Ant but don't know Maven. Knowing ant does not mean knowing how to build a particular project. That's the big drawback of ant over maven : there's no convention/standard way of doing things. IMHO knowing maven is worth the investment and can be applied to so many projects that the return on investment is quite quick. Regarding maven for tomcat, I think it would lower the barrier of entry for new developers. In a perfect maven scenario, a developer would checkout tomcat sources in its IDE, wait for the dependencies to be downloaded and have a workspace ready to hack into tomcat using standard maven commands. so, I'm +1 to support any experiments around maven and then judge on the results. Sylvain - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn commit: r1209686 [1/2] - in /tomcat/trunk: java/org/apache/catalina/core/ java/org/apache/catalina/loader/ java/org/apache/catalina/startup/ java/org/apache/naming/resources/ test/org/apache/c
good catch. Actually I did perform a ant test to check that all tests pass, but the validate target is not run by default. Why isn't it enabled by default ? Sylvain On 2 déc. 2011, at 22:08, Mark Thomas wrote: On 02/12/2011 20:49, slaur...@apache.org wrote: Author: slaurent Date: Fri Dec 2 20:49:50 2011 New Revision: 1209686 URL: http://svn.apache.org/viewvc?rev=1209686view=rev Log: bug 51741: Eclipse WTP Serve modules without publishing broken with tc7, needs patch in tomcat https://issues.apache.org/bugzilla/show_bug.cgi?id=51741 You need to run checkstyle on this. There are a number of issues including: - files without license headers - files with license headers that do not use the expected layout - an import from o.a.catlina used within o.a.naming wich is not permitted Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: buildbot failure in ASF Buildbot on tomcat-trunk
should be fixed with r1209731 On 2 déc. 2011, at 22:19, build...@apache.org wrote: The Buildbot has detected a new failure on builder tomcat-trunk while building ASF Buildbot. Full details are available at: http://ci.apache.org/builders/tomcat-trunk/builds/2564 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: bb-vm_ubuntu Build Reason: scheduler Build Source Stamp: [branch tomcat/trunk] 1209686 Blamelist: slaurent BUILD FAILED: failed compile_1 sincerely, -The Buildbot - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
VirtualDirContext useless with JarScanner scanAllDirectories=true?
Hello, I'm working on https://issues.apache.org/bugzilla/show_bug.cgi?id=51741 (Eclipse WTP Serve modules without publishing broken with tc7, needs patch in tomcat) and though the modifications I'm bringing are small, I'm spending some time creating a junit test that checks that it is possible to configure a webapp + a library in a maven/eclipse layout and have tomcat directly use the files in the eclipse workspace and maven repository. In my setup, the org.apache.catalina.loader.VirtualWebappLoader is used, as well as the JarScanner with option scanAllDirectories=true. With the latter, it seems that org.apache.naming.resources.VirtualDirContext is useless in such an environment. Is it correct or am I missing something ? (I did not find any test for it and it's not documented in /webapps/docs/config/resources.xml). If it really is useless, should we deprecate it ? Sylvain - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn commit: r1163807 - in /tomcat: tc7.0.x/trunk/ trunk/
On 1 sept. 2011, at 12:11, Konstantin Kolinko wrote: 1. What happens when someone creates a tag (aka svn copy) in another svn client? Should one remove the value, or leave it as is? The value should be left as is. The next person that uses subclipse may update the value to include the reference to the new tag in the property value 2. It duplicates information that is already in repository. If it were just configuration value (e.g. see tags there), but it lists all nn tags. 3. The property value on /tomcat/trunk was wrong. Indeed, I should not have added it to /tomcat/trunk , only to /tomcat/tc7.0.x/trunk - it should not have been added there. You wouldn't list 7.0.x tags on trunk. - /tomcat/tc7.0.x/branches is not a branch A branch would be /tomcat/tc7.0.x/branches/name Yes, my mistake Also message in this commit did not reflect what it did. 4. When I performed a trunk-7.0.x merge (see another email from yesterday) this revision caused property update conflict on 7.0.x/trunk. Maybe because of 3. above. Yes, if I put the property only on /tomcat/tc7.0.x/trunk there won't be any conflict 5. What you, personally, use it for? What are your practical benefits? Seeing tag names in history? Yes, I find it very convenient to see the history of changes on a file and the version of tomcat that includes the change. If you really use it and will add all TOMCAT_7_0_x tags to /tomcat/tc7.0.x/trunk, and will maintain it onwards, I personally do not mind. ok, I will do it - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn commit: r1163804 - /tomcat/tc7.0.x/trunk/webapps/docs/changelog.xml
Le 31 août 2011 à 23:12, Mark Thomas ma...@apache.org a écrit : On 31/08/2011 21:37, slaur...@apache.org wrote: +section name=Tomcat 7.0.22 You left out the (markt) I thought it represented the person who finalized the release, so I left it out. + subsection name=Catalina +changelog + fix +bug51744/bug: Prevent application code from closing the associated +JNDI context while the application is running. (markt) + /fix + add +bug51741/bug: Fixes a problem with Eclipse WTP Serve modules without +publishing feature where applications failed to access resources when using +getResource() on the classloader. (slaurent) + /add +/changelog + /subsection + subsection name=Coyote +changelog +/changelog + /subsection We don't add sections to the changelog until they are required. These empty sections can be removed. Ok, noted - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn commit: r1163807 - in /tomcat: tc7.0.x/trunk/ trunk/
Le 31 août 2011 à 23:13, Mark Thomas ma...@apache.org a écrit : On 31/08/2011 21:38, slaur...@apache.org wrote: --- subclipse:tags (added) +++ subclipse:tags Wed Aug 31 20:38:53 2011 @@ -0,0 +1,2 @@ +932358,branches,/tomcat/tc7.0.x/branches,branch +1162976,TOMCAT_7_0_21,/tomcat/tc7.0.x/tags/TOMCAT_7_0_21,tag What is this? It looks like junk that needs to be removed to me. It's part of subclipse support for tags and branches : http://subclipse.tigris.org/branch_tag.html This is very convenient, and I don't think it bothers developers who don't use subclipse. Sylvain - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn commit: r1163802 - in /tomcat: tc7.0.x/trunk/java/org/apache/catalina/loader/WebappClassLoader.java trunk/java/org/apache/catalina/loader/WebappClassLoader.java
I'm using subclipse which contains a 1.6 svn client. I'm not an svn expert, I'll try to do as Mark recommends, I see the point of performing the merge after so that svn can keep track of things. Sylvain Le 1 sept. 2011 à 00:34, Konstantin Kolinko knst.koli...@gmail.com a écrit : 2011/9/1 Mark Thomas ma...@apache.org: On 31/08/2011 22:57, Konstantin Kolinko wrote: 2011/9/1 Mark Thomas ma...@apache.org: On 31/08/2011 21:35, slaur...@apache.org wrote: Author: slaurent Date: Wed Aug 31 20:35:22 2011 New Revision: 1163802 URL: http://svn.apache.org/viewvc?rev=1163802view=rev Log: Fix https://issues.apache.org/bugzilla/show_bug.cgi?id=51741 bug 51741: Eclipse WTP Serve modules without publishing broken with tc7, needs patch in tomcat This should have been applied just to /trunk and then merged to 7.0.x/trunk. This Sylvain's commit patches both trunk and 7.0.x at the same time. Thus there is no need to merge. I know what the patch did. The point is that committing like this messes with the mergeinfo. It may (or may not) cause us some headaches at some point in the future. Given we don't have to commit trunk and 7.0.x/trunk together, I'd rather we continued with the current practise of trunk first and then merge. I prefer this practice of updating trunk + 7.0.x at the same time for small patches, e.g. documentation. To check whether it causes problems or not: I did a merge of trunk to 7.0.x for revisions 1155369-HEAD, using svn 1.7 (a nightly of TortoiseSVN) I had to start the merge command twice, because of conflict in changelog.xml, and there was a conflict with subclipse:tags property, but in the end it completed successfully. Here is a list of modified files: svn status [[[ C . A +TOMCAT-NEXT.txt ? dir_conflicts.prej M build.xml X modules\jdbc-pool M build.properties.default M test\org\apache\coyote\ajp\TesterAjpMessage.java M test\org\apache\catalina\core\TestStandardContext.java M java\org\apache\coyote\http11\Http11Processor.java M java\org\apache\coyote\http11\Http11AprProtocol.java M java\org\apache\coyote\ajp\AjpAprProtocol.java M java\org\apache\coyote\ajp\AjpMessage.java M java\org\apache\jasper\JspCompilationContext.java M java\org\apache\tomcat\util\net\AprEndpoint.java M java\org\apache\catalina\Host.java M java\org\apache\catalina\startup\ContextConfig.java M java\org\apache\catalina\startup\ExpandWar.java M java\org\apache\catalina\startup\HostConfig.java M java\org\apache\catalina\core\StandardHost.java M java\org\apache\catalina\core\StandardContext.java M java\org\apache\catalina\ha\deploy\FarmWarDeployer.java M java\org\apache\catalina\manager\HTMLManagerServlet.java M java\org\apache\catalina\manager\ManagerServlet.java M res\maven\mvn.properties.default M res\ide-support\eclipse\stop-tomcat.launch M res\ide-support\eclipse\java-compiler-errors-warnings.txt M res\ide-support\eclipse\eclipse.project M res\ide-support\eclipse\start-tomcat.launch M webapps\docs\changelog.xml M RELEASE-NOTES M BUILDING.txt Performing status on external item at 'modules\jdbc-pool': Summary of conflicts: Property conflicts: 1 ]]] ../loader/WebappClassLoader.java is not on the list and that means that this change has been successfully skipped (and that is what svn:mergeinfo is for). I think that such skipping is called implicit mergeinfo. I cannot say how it would behave with 1.6 svn clients (I hope it works, but I heard that 1,7 has improvements). Svn 1.7 is rather near - the first release candidate was released today and release expected in a month (after the soak period). We already did such updates to trunk+branch when working on 6.0.x and I prefer to keep this practice. Best regards, Konstantin Kolinko - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn commit: r1163855 - in /tomcat: tc7.0.x/trunk/ trunk/
Would it really hurt to keep those properties?
Le 1 sept. 2011 à 00:45, kkoli...@apache.org a écrit :
Author: kkolinko
Date: Wed Aug 31 22:45:23 2011
New Revision: 1163855
URL: http://svn.apache.org/viewvc?rev=1163855view=rev
Log:
Revert r1163807:
Remove subclipse:tags property.
Modified:
tomcat/tc7.0.x/trunk/ (props changed)
tomcat/trunk/ (props changed)
Propchange: tomcat/tc7.0.x/trunk/
('subclipse:tags' removed)
Propchange: tomcat/trunk/
('subclipse:tags' removed)
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn commit: r1090441 - /tomcat/trunk/webapps/docs/changelog.xml
On 8 avr. 2011, at 22:58, Konstantin Kolinko wrote: + add +bug50306/bug: New StuckThreadDetectionValve to detect requests that take a +long time to process, which might indicate that their processing threads are +stuck. (slaurent) Based on a patch provided by TomLu. ? fixed. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
anyone using git-svn for tomcat ?
Hello, I'm trying out git at the moment and found this page http://wiki.apache.org/general/GitAtApache Is anyone here using git in front of svn ? any problem, advice ? Sylvain
Re: anyone using git-svn for tomcat ?
I just did my first commit on tomcat using git, it seems to work. does it bother anyone if I add a .gitignore file in the SVN repository ? Sylvain On 7 avr. 2011, at 22:57, Rainer Jung wrote: On 07.04.2011 22:35, Sylvain Laurent wrote: Hello, I'm trying out git at the moment and found this page http://wiki.apache.org/general/GitAtApache Is anyone here using git in front of svn ? any problem, advice ? I used it some time for the httpd sources. I like that it is easy to transfer commits from the local repos to the ASF svn including commit log entry. The patch format is also nice. git-svn uses the svn perl bindings which produce crashes every now and then, at least on my system. I didn't have any situation where the crash left over inconsistent files or incomplete transactions. The crashes happen during process cleanup. Overall I liked it. Regards, Rainer - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: anyone using git-svn for tomcat ?
On 7 avr. 2011, at 23:54, Konstantin Kolinko wrote: What content will be in your version of the file? mostly what we ave already ignored with svn props : eclipse-sepcific files, the logs, work, temp directories... Otherwise, one can also run git svn show-ignore .git/info/exclude which saves a list of things that are ignored by svn to a local file that will not be committed. Though it works, it is local to each repository and thus each developer should run the command too... - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: bindOnInit and maxConnections for AJP connectors
On 5 avr. 2011, at 11:50, Tim Whittington wrote: HTTP: use maxConnections. For keep alive situations, reduce maxConnections to something closer to maxThreads (the default config is 10,000 keepalive connections serviced by 200 threads with a 60 second keepalive timeout, which could lead to some large backlogs of connected sockets that take 50 minutes to get serviced) I think it was tc6 behavior, but it changed with tc7 : threads are now returned to the pool between requests, even with keep-alive connections. Sylvain - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn commit: r1044731 - in /tomcat/trunk: res/checkstyle/ res/maven/ test/org/apache/tomcat/util/threads/
On 11 déc. 2010, at 23:21, ma...@apache.org wrote: Author: markt Date: Sat Dec 11 22:21:02 2010 New Revision: 1044731 URL: http://svn.apache.org/viewvc?rev=1044731view=rev Log: TRy and sync up the deps in the POMs with what Checkstyle validates Removed: tomcat/trunk/test/org/apache/tomcat/util/threads/ -1 : this commit removed the DedicatedThreadExecutorTest that I added in r1044145 : http://svn.apache.org/viewvc/tomcat/trunk/test/org/apache/tomcat/util/threads/DedicatedThreadExecutorTest.java?view=logpathrev=1044145 Sylvain - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
running tests for tc6 with ant
Which ant file to use to run junit tests for tc6 ? BUILDING.TXT says that ant -f dist.xml release should run tests, but it doesn't seem to. I tried ant -f test/build.xml but it's buggy (incorrect path to build.properties) and runs only one test... Sylvain - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn commit: r1044145 - in /tomcat/trunk: java/org/apache/catalina/core/ java/org/apache/tomcat/util/threads/ test/org/apache/tomcat/util/threads/
Good catch! Actually I think my commit revealed a current bug in tc 6 7 : https://issues.apache.org/bugzilla/show_bug.cgi?id=50459 I fixed it with http://svn.apache.org/viewvc?rev=1044746view=rev . Now all tests pass (checked with ant clean test). Sylvain On 11 déc. 2010, at 21:34, Mark Thomas wrote: On 09/12/2010 22:11, slaur...@apache.org wrote: Author: slaurent Date: Thu Dec 9 22:11:27 2010 New Revision: 1044145 URL: http://svn.apache.org/viewvc?rev=1044145view=rev Log: bug 49159: Improve ThreadLocal memory leak clean-up https://issues.apache.org/bugzilla/show_bug.cgi?id=49159 -1. This breaks at least one unit test. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn commit: r1044746 - /tomcat/trunk/java/org/apache/catalina/core/StandardContext.java
oops, I just forgot a dot and a new line... fixed in rev 1044747. On 12 déc. 2010, at 01:13, Mark Thomas wrote: On 12/12/2010 00:10, slaur...@apache.org wrote: Author: slaurent Date: Sun Dec 12 00:10:39 2010 New Revision: 1044746 @@ -4862,10 +4862,10 @@ public class StandardContext extends Con if ((loader != null) (loader instanceof Lifecycle)) ((Lifecycle) loader).start(); -// Unbinding thread +// since the loader just started, the webapp classloader is now +// created by calling unbindThread and bindThread in a row, we +// setup the current Thread CCL to be the webapp classloader unbindThread(oldCCL); - -// Binding thread oldCCL = bindThread(); That new comment doesn't look right. The class loader is not created by calling unbindThread and bindThread in a row. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn commit: r1042482 - in /tomcat/trunk: ./ conf/ java/org/apache/catalina/core/ java/org/apache/catalina/loader/ java/org/apache/tomcat/util/threads/ res/confinstall/ webapps/ webapps/docs/ weba
On 7 déc. 2010, at 09:41, Rainer Jung wrote: Just in case Mark's remark a) wasn't clear: after building Tomcat you can debug inside the sub directory output/build which contains a full installation. I had understood Mark, but it's not practical for debugging... I prefer to debug tomcat within eclipse and use the classes compiled by eclipse. I'll just ignore the work and log directories when committing changes. Though if we could ignore them that would be nice... - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn commit: r1042786 - in /tomcat/trunk/java/org/apache: catalina/core/ catalina/loader/ tomcat/util/threads/
On 7 déc. 2010, at 17:35, Mark Thomas wrote: +1. Line length is an area I am becoming increasingly happy to be flexible. Aim for 80 but if 81 or 82 (or anything ~90) makes the code more readable then I'd be fine with it. As a rookie, I did not want to start bothering you with this, but since you bring the topic... I think 80 is somewhat outdated: screens are now bigger, and especially wider (16:9 vs 4:3). I think we should allow a greater line length to reduce the code height (the number of lines). My preference is 100 characters per line. Sylvain - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn commit: r1042495 - /tomcat/trunk/webapps/docs/changelog.xml
Are you using an automatic formatter ? if yes, which one ? Can you share its settings? On 6 déc. 2010, at 01:17, ma...@apache.org wrote: Author: markt Date: Mon Dec 6 00:17:43 2010 New Revision: 1042495 URL: http://svn.apache.org/viewvc?rev=1042495view=rev Log: Line-length Modified: tomcat/trunk/webapps/docs/changelog.xml Modified: tomcat/trunk/webapps/docs/changelog.xml URL: http://svn.apache.org/viewvc/tomcat/trunk/webapps/docs/changelog.xml?rev=1042495r1=1042494r2=1042495view=diff == --- tomcat/trunk/webapps/docs/changelog.xml (original) +++ tomcat/trunk/webapps/docs/changelog.xml Mon Dec 6 00:17:43 2010 @@ -38,7 +38,9 @@ /properties body -!-- General, Catalina, Coyote, Jasper, Cluster, Web applications, Extras, Other -- +!-- + General, Catalina, Coyote, Jasper, Cluster, Web applications, Extras, Other +-- section name=Tomcat 7.0.6 (markt) subsection name=Catalina changelog @@ -80,10 +82,10 @@ generate session IDs. (markt) /update add -bug50282/bug: Load codejavax.security.auth.login.Configuration/code -with codeJreMemoryLeakPreventionListener/code to avoid memory leak -when stopping a web application that would use JAAS. -(slaurent) +bug50282/bug: Load +codejavax.security.auth.login.Configuration/code with +codeJreMemoryLeakPreventionListener/code to avoid memory leak when +stopping a web application that would use JAAS. (slaurent) /add fix bug50351/bug: Fix the regression that broke BeanFactory resources - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn:eol-style info for our new committers
Thanks, and sorry for the incorrect svn properties and formatting. I'll try to compile a list of things to take care of for new committers and put it into the wiki. On 6 déc. 2010, at 12:00, Rainer Jung wrote: There's the useful http://www.apache.org/dev/svn-eol-style.txt to configure svn. It is linked together with other useful info in the version control section of http://www.apache.org/dev/ Regards, Rainer - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn commit: r1042482 - in /tomcat/trunk: ./ conf/ java/org/apache/catalina/core/ java/org/apache/catalina/loader/ java/org/apache/tomcat/util/threads/ res/confinstall/ webapps/ webapps/docs/ webap
I fixed most of your points except one, see below.
On 6 déc. 2010, at 01:09, Mark Thomas wrote:
On 05/12/2010 22:54, slaur...@apache.org wrote:
Author: slaurent
Date: Sun Dec 5 22:54:05 2010
New Revision: 1042482
URL: http://svn.apache.org/viewvc?rev=1042482view=rev
General comments:
Numerous new lines with length 80 chars.
Actually I had configured eclipse for 80 chars but the default line-wrapping
settings are such that it does not wrap in several cases...
ASF policy is to strongly discourage author tags.
fixed
Your svn client should be configured to set svn:eol-style native on new
text files.
OK, done for future new files.
Modified:
tomcat/trunk/ (props changed)
tomcat/trunk/conf/ (props changed)
tomcat/trunk/webapps/(props changed)
-1 to all these changes.
a) This is not a Tomcat instance for running. That is created in output.
b) It reverts a useful change I made earlier today
Sorry, I did not realize there could be changes on directories with SVN. For
debugging I launch tomcat from the base directory and this creates directories
likes log, work, etc...
I believe you fixed this.
Added:
tomcat/trunk/java/org/apache/catalina/core/ThreadLocalLeakPreventionListener.java
+private static final Log log = LogFactory
+.getLog(ThreadLocalLeakPreventionListener.class);
That is an odd place for a line-break that I find hard to read. I find
either of the following easier to read:
private static final Log log =
LogFactory.getLog(ThreadLocalLeakPreventionListener.class);
private static final Log log = LogFactory.getLog(
ThreadLocalLeakPreventionListener.class);
There are multiple instances of this throughout the commit.
Still the default eclipse formatter... We should really provide eclipse
formatter settings. Formatting code by hand is really a waste of time.
+public void lifecycleEvent(LifecycleEvent event) {
+try {
+Lifecycle lifecycle = event.getLifecycle();
+if (Lifecycle.AFTER_START_EVENT.equals(event.getType())
+ lifecycle instanceof Server) {
With the operator on the new line it is easy to miss what is going on.
Generally, Tomcat style is to put the operator on the first line. e.g.
if (Lifecycle.AFTER_START_EVENT.equals(event.getType())
lifecycle instanceof Server) {
There are multiple instances of this throughout the commit.
Still eclipse. But I did not manage to find a setting to change this behavior
:-(
+} catch (Exception e) {
+log.error(Exception processing event + event, e);
+}
Error messages really should be using a StringManager.
There are multiple instances of this throughout the commit.
done.
+protected void processContainerRemoveChild(Container parent, Container
child) {
+
+if (log.isDebugEnabled())
+log.debug(Process removeChild[parent= + parent + ,child=
++ child + ]);
+
+try {
+if (child instanceof Context) {
+Context context = (Context) child;
+context.removeLifecycleListener(this);
+} else if (child instanceof Host) {
+Host host = (Host) child;
+host.removeContainerListener(this);
+} else if (child instanceof Engine) {
+Engine engine = (Engine) child;
+engine.removeContainerListener(this);
+}
Why all this? Just use:
child.removeLifecycleListener(this);
No, it's not the same. The listener is interested in both LifeCycle events for
Context, and Container events for Host and Engine. These are not the same
things.
Does the fact the Container implements Lifecycle simplify any of the
other code here?
Can't tell.
Modified: tomcat/trunk/java/org/apache/catalina/loader/WebappClassLoader.java
-private void clearThreadLocalMap(Object map, Field internalTableField)
+private void checkThreadLocalMapForLeaks(Object map, Field
internalTableField)
throws NoSuchMethodException, IllegalAccessException,
NoSuchFieldException, InvocationTargetException {
Not all of those Exceptions are required after this commit.
Fixed.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn commit: r1042029 - in /tomcat: tc6.0.x/trunk/STATUS.txt trunk/java/org/apache/catalina/core/JreMemoryLeakPreventionListener.java trunk/webapps/docs/changelog.xml trunk/webapps/docs/config/list
ok, thanks for having added them for me. Sylvain On 4 déc. 2010, at 06:21, Konstantin Kolinko wrote: 2010/12/4 slaur...@apache.org: Author: slaurent Date: Fri Dec 3 22:19:11 2010 New Revision: 1042029 URL: http://svn.apache.org/viewvc?rev=1042029view=rev Log: bug 50282 : Load javax.security.auth.login.Configuration with JreMemoryLeakPreventionListener to avoid memory leak when stopping a webapp that would use JAAS. Modified: tomcat/tc6.0.x/trunk/STATUS.txt tomcat/trunk/java/org/apache/catalina/core/JreMemoryLeakPreventionListener.java tomcat/trunk/webapps/docs/changelog.xml tomcat/trunk/webapps/docs/config/listeners.xml Modified: tomcat/tc6.0.x/trunk/STATUS.txt URL: http://svn.apache.org/viewvc/tomcat/tc6.0.x/trunk/STATUS.txt?rev=1042029r1=1042028r2=1042029view=diff + +* Fix https://issues.apache.org/bugzilla/show_bug.cgi?id=50282 + Improve JreMemoryLeakPreventionListener to load + javax.security.auth.login.Configuration to avoid redeployment leak. + +1: slaurent + -1: You must include patch URL in your proposal. (What are we voting for?) Thus, you usually cannot mix update to STATUS and update to trunk in the same commit. Best regards, Konstantin Kolinko - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn commit: r1042022 - in /tomcat: tc6.0.x/trunk/STATUS.txt trunk/java/org/apache/catalina/session/StandardManager.java trunk/webapps/docs/changelog.xml
On 5 déc. 2010, at 18:58, Mark Thomas wrote: We haven't been good at this historically for debug messages but we should really use a StringManager here for i18n support. Though i can understand that i18n is useful for information and error message, is it really useful for debug level messages ?? Bugs should be added in ascending numerical order within the relevant section. Fixed - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
fixing subject of Bugzilla issues
Is it OK to change the subject field of bugzilla issues ? For instance the subject of https://issues.apache.org/bugzilla/show_bug.cgi?id=48837 is currently Memory leaks protection does not cure leaks triggered by JSP pages code After Mark applied the patch, I'd prefer to have the following subject which better reflects the final status/outcome of the bug : Memory leaks protection does not detect leaks triggered by JSP pages code Is there an established rule on this subject (pun intended) ? Sylvain - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
catalina-ant maven artifact for tc6 ?
Hello, I'm working on https://issues.apache.org/bugzilla/show_bug.cgi?id=46350 to backport what has been done for tc7 to tc6. In doing so I noticed that there's no artifact for catalina-ant . Is it a mistake or is it on purpose ? Sylvain - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: bugzilla question : bugs fixed in trunk but not yet in tc6
so, we lose the information of what bug was fixed in which version for tomcat 7 ? (apart from the comments) For instance for https://issues.apache.org/bugzilla/show_bug.cgi?id=49625 it is now marked for tomcat 6. And only the comments can tell us that it was fixed in 7.0.3 ?? Sylvain On 23 oct. 2010, at 00:25, Konstantin Kolinko wrote: 2010/10/23 Sylvain Laurent sylvain.laur...@m4x.org: How can you create a filter for bugs that are fixed in trunk, proposed to be fixed in tomcat 6, and waiting to be backported ? Do you use some special bug attribute ? What do you mean? If it is already fixed in TC7, we change the product that the bug is assigned to: we move it to the one where it is still present: to TC6, then to TC5. Best regards, Konstantin Kolinko - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: bugzilla question : bugs fixed in trunk but not yet in tc6
JIRA is available for Apache projects : http://issues.apache.org/jira/ On 23 oct. 2010, at 02:15, Josh Gooding wrote: They could be comfortable with Bugzilla too. I see JIRA has an open source license. (back pedal back pedal!!!) On Fri, Oct 22, 2010 at 8:10 PM, Josh Gooding josh.good...@gmail.comwrote: Not to step on anyones toes, but JIRA costs serious $$ after the initial 10 users. It gets expensive fast. Trac is good too though. I set up an awesome Trac system at work. - Josh On Fri, Oct 22, 2010 at 6:11 PM, Sylvain Laurent sylvain.laur...@m4x.orgwrote: How can you create a filter for bugs that are fixed in trunk, proposed to be fixed in tomcat 6, and waiting to be backported ? Do you use some special bug attribute ? Sylvain PS : Why not use JIRA ? It has many advantages over this aging bugzilla... - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
bugzilla question : bugs fixed in trunk but not yet in tc6
How can you create a filter for bugs that are fixed in trunk, proposed to be fixed in tomcat 6, and waiting to be backported ? Do you use some special bug attribute ? Sylvain PS : Why not use JIRA ? It has many advantages over this aging bugzilla... - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [PROPOSAL] Parallel deployment
I discovered this feature a while ago in Weblogic and I found this interesting. But as I always had classloader leaks, I had to restart the JVM anyway so that it defeated the purpose of the feature and I gave up dreaming of zero-downtime deployments. Now with tomcat memory leak protection features, this deployment option might be once again very interesting. Though I can see how this can be done for a standalone tomcat instance, how do you plan to do with clustered sessions ? Should the old version of the application wait for all sessions in the cluster to expire ? Sylvain On 21 oct. 2010, at 20:52, Mark Thomas wrote: All, As most of you are probably aware I work for SpringSource and SpringSource has an application server product - tc Server - that is heavily based on Apache Tomcat. When talking to prospective clients about tc Server, one of the things we often hear is WebSphere/WebLogic has XYZ feature - does tc Server? and one of those features is parallel deployment. Parallel deployment is essentially having two (or more) versions of the same application deployed side-by-side. Users with a session in an older version, will continue to use that older version until the session expires. Users without a session, or whose session expires, will use the latest version. Once there are no sessions using the older version it is undeployed. All versions of the application have the same context path so the transition is seamless to the users. Having looked at some implementation options, it quickly became apparent that Tomcat was the right place to implement this rather than as an 'add-on' in tc Server. I have managed to convince senior management that contributing an implementation of parallel deployment to the ASF is the way to go. I currently have a hacked together implementation [1] that works but is very ugly in places under the covers and there is still some significant work to go before it is robust enough to be put into trunk (it changes the mapper and that is an area I think we need to be very careful). A lot of the patch is dealing with the issues around the fact that we currently treat Context.getPath() and Context.getName() as the same thing but with parallel deployment these need to be treated differently. My proposal is therefore: - start committing some of the getPath() / getName() clean-up to trunk along with some of the other hooks - like adding a version attribute to Context that should be minimal risk - continue testing the current patch (e.g. with clustering) and modify as required - discuss the implementation details (like the file naming convention used) on the dev list - decide once we can see the patch without the clean-up if it is safe for 7.0.x or needs to be held back for 7.1.x Depending on how big this patch ends up looking then creating a branch for this work may make sense. I'd like to hold off creating a branch for a few days until the clean-up changes have been made to trunk so the branch can focus on the diffs required for the new feature. As always, any and all feedback welcome. Mark [1] http://people.apache.org/~markt/patches/2010-10-21-parallel-deployment.patch - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
welcome files, servlet mappings and jsps (was [VOTE] Release Apache Tomcat 7.0.4)
Actually Glassfish does not differentiate between implicit and explicit mappings. There's just a test on the servlet-name : When determining the wrapper to use for a welcome file, if the physical file is not found, it checks for servlet mappings. If there's such a match, it also checks that the servlet name is different from jsp (overridable by a System property). their source is here : https://grizzly.dev.java.net/source/browse/grizzly/trunk/code/modules/utils/src/main/java/com/sun/grizzly/util/http/mapper/Mapper.java?view=markup I believe we could do the same hack in tomcat 7. The spec might not be very clear but that behavior avoids big surprises for current applications... Sylvain On 18 oct. 2010, at 02:35, Mark Thomas wrote: On 17/10/2010 22:58, Caldarale, Charles R wrote: From: Mark Thomas [mailto:ma...@apache.org] Subject: Re: [VOTE] Release Apache Tomcat 7.0.4 There are no matches for the static files so all is OK for the first sentence. However there is a match (to *.jsp) the second time through so Tomcat passes /catalog/products/default.jsp to the JSP servlet which returns a 404. This might be the crux of the matter: section 12.2.1 of the spec states that *.jsp is an /implicit/ mapping (unless overridden by the webapp), and the second matching protocol in 10.10 seems to be referring to /explicit/ mappings (my interpretation, not definitive in the spec). Tomcat does not differentiate between implicit and explicit mappings, whereas the other containers seem to do so. Implicit seems to be a term used in section exclusively 12.2.1 (OK apart from a single mention in 12.1). It isn't used elsewhere in the Servlet spec so it isn't at all clear what - if any - special treatment implicit mappings are meant to have. 12.2.1 doesn't mention *.jspx which doesn't give me a great deal of confidence in the completeness or correctness of that section. Something else that requires some clarity for Servlet 3.1. I'll add it to my list. My previous suggestion regarding a new config option would at least allow users to control this behaviour until we get some clarity in the spec. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [VOTE] Release Apache Tomcat 7.0.4
Indeed, that's the problem I'm facing. After spending some time reading the spec and the comments in both bug reports, I'm inclined to believe that the current behavior of tomcat 7.0.4 is not spec compliant. I created a webapp that matches the example given in chapter 10.10 (Welcome files) of the spec. The files are the same, and my web.xml is like this : web-app xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance; xmlns=http://java.sun.com/xml/ns/javaee; xmlns:web=http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd; xsi:schemaLocation=http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd; id=WebApp_ID version=2.5 display-nametestWelcomeFiles/display-name welcome-file-list welcome-fileindex.html/welcome-file welcome-filedefault.jsp/welcome-file /welcome-file-list servlet description/description display-nameMyDefaultServlet/display-name servlet-nameMyDefaultServlet/servlet-name servlet-classtest.MyDefaultServlet/servlet-class /servlet servlet-mapping servlet-nameMyDefaultServlet/servlet-name url-pattern//url-pattern /servlet-mapping /web-app All the behaviors described in the spec are OK except this one : A request URI of /catalog/products/ will be passed to the “default” servlet, if any. Despite having declared a default servlet (as defined in section 12.2), it was not called when requesting catalog/products/ and I got a 404 instead... Sylvain On 16 oct. 2010, at 21:36, Mark Thomas wrote: On 15/10/2010 22:01, Sylvain Laurent wrote: I just played with a Roo application and I get a 404 with 7.0.4 whereas the very same application works OK with 6.0.29. I'll try to investigate this week-end. Best guess without seeing the app is https://issues.apache.org/bugzilla/show_bug.cgi?id=49422 Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [VOTE] Release Apache Tomcat 7.0.4
the same example works OK with jetty 7.1.6 and GlassFish 3.0.1... And there's also a difference of behavior between tc7.0.4 and those 2 others : Let's suppose we have the follwoing web.xml welcome-file-list welcome-filedefault.dummy/welcome-file /welcome-file-list servlet description/description servlet-nameMyDummyServlet/servlet-name servlet-classtest.MyDummyServlet/servlet-class /servlet servlet-mapping servlet-nameMyDummyServlet/servlet-name url-pattern*.dummy/url-pattern /servlet-mapping servlet description/description servlet-nameMyDefaultServlet/servlet-name servlet-classtest.MyDefaultServlet/servlet-class /servlet servlet-mapping servlet-nameMyDefaultServlet/servlet-name url-pattern//url-pattern /servlet-mapping When requesting the root of the context, tomcat 7 dispatches to MyDummyServlet whereas jetty and glassfish use MyDefaultServlet. On this one I'd say tomcat is right... A clarification of the Expert Group on the servlet spec would be welcome ! Sylvain On 17 oct. 2010, at 17:04, Sylvain Laurent wrote: Indeed, that's the problem I'm facing. After spending some time reading the spec and the comments in both bug reports, I'm inclined to believe that the current behavior of tomcat 7.0.4 is not spec compliant. I created a webapp that matches the example given in chapter 10.10 (Welcome files) of the spec. The files are the same, and my web.xml is like this : web-app xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance; xmlns=http://java.sun.com/xml/ns/javaee; xmlns:web=http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd; xsi:schemaLocation=http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd; id=WebApp_ID version=2.5 display-nametestWelcomeFiles/display-name welcome-file-list welcome-fileindex.html/welcome-file welcome-filedefault.jsp/welcome-file /welcome-file-list servlet description/description display-nameMyDefaultServlet/display-name servlet-nameMyDefaultServlet/servlet-name servlet-classtest.MyDefaultServlet/servlet-class /servlet servlet-mapping servlet-nameMyDefaultServlet/servlet-name url-pattern//url-pattern /servlet-mapping /web-app All the behaviors described in the spec are OK except this one : A request URI of /catalog/products/ will be passed to the “default” servlet, if any. Despite having declared a default servlet (as defined in section 12.2), it was not called when requesting catalog/products/ and I got a 404 instead... Sylvain On 16 oct. 2010, at 21:36, Mark Thomas wrote: On 15/10/2010 22:01, Sylvain Laurent wrote: I just played with a Roo application and I get a 404 with 7.0.4 whereas the very same application works OK with 6.0.29. I'll try to investigate this week-end. Best guess without seeing the app is https://issues.apache.org/bugzilla/show_bug.cgi?id=49422 Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [VOTE] Release Apache Tomcat 7.0.4
I just played with a Roo application and I get a 404 with 7.0.4 whereas the very same application works OK with 6.0.29. I'll try to investigate this week-end. Sylvain On 15 oct. 2010, at 10:47, Mark Thomas wrote: The proposed Apache Tomcat 7.0.4 release is now available for voting. It can be obtained from: http://people.apache.org/~markt/dev/tomcat-7/v7.0.4/ The svn tag is: http://svn.apache.org/repos/asf/tomcat/tc7.0.x/tags/TOMCAT_7_0_4/ As with previous votes, I have included a stable option below although my personal inclination is still to vote beta. The proposed 7.0.4 release is: [ ] Broken - do not release [ ] Alpha - go ahead and release as 7.0.4 Alpha [ ] Beta - go ahead and release as 7.0.4 Beta [ ] Stable - go ahead and release as 7.0.4 Stable This vote will run until 10.00 UTC Wednesday 20th October (3 working days). Cheers, Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Memory leak protection against ThreadLocal-related leaks : call for review
Hello, I contributed a patch (actually 2, for tc67) for https://issues.apache.org/bugzilla/show_bug.cgi?id=49159 to renew threads when an application is stopped. This allows to avoid some types of memory leak that were not cured by tomcat until now (or in an unsafe way and thus disabled by default) : http://wiki.apache.org/tomcat/MemoryLeakProtection#customThreadLocal , http://wiki.apache.org/tomcat/MemoryLeakProtection#webappClassInstanceAsThreadLocalValue , http://wiki.apache.org/tomcat/MemoryLeakProtection#webappClassInstanceAsThreadLocalIndirectValue and http://wiki.apache.org/tomcat/MemoryLeakProtection#threadLocalPseudoLeak I'd be glad if someone could review this and give some feedback. Sylvain - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn commit: r1006033 - in /tomcat/trunk: java/org/apache/catalina/startup/ContextConfig.java webapps/docs/changelog.xml
This commit broke all default webapps (ROOT, examples, docs). a 404 is returned
instead...
On 8 oct. 2010, at 23:37, ma...@apache.org wrote:
Author: markt
Date: Fri Oct 8 21:37:19 2010
New Revision: 1006033
URL: http://svn.apache.org/viewvc?rev=1006033view=rev
Log:
Fix https://issues.apache.org/bugzilla/show_bug.cgi?id=50059
Always make JAR resources available
Modified:
tomcat/trunk/java/org/apache/catalina/startup/ContextConfig.java
tomcat/trunk/webapps/docs/changelog.xml
Modified: tomcat/trunk/java/org/apache/catalina/startup/ContextConfig.java
URL:
http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/catalina/startup/ContextConfig.java?rev=1006033r1=1006032r2=1006033view=diff
==
--- tomcat/trunk/java/org/apache/catalina/startup/ContextConfig.java
(original)
+++ tomcat/trunk/java/org/apache/catalina/startup/ContextConfig.java Fri Oct
8 21:37:19 2010
@@ -1214,7 +1214,7 @@ public class ContextConfig
webXmlVersion = Double.parseDouble(webXml.getVersion());
}
-if (webXmlVersion = 3 !webXml.isMetadataComplete()) {
+if (webXmlVersion = 3) {
// Ordering is important here
// Step 1. Identify all the JARs packaged with the application
@@ -1222,67 +1222,78 @@ public class ContextConfig
// point.
MapString,WebXml fragments = processJarsForWebFragments();
-// Step 2. Order the fragments.
-SetWebXml orderedFragments =
-WebXml.orderWebFragments(webXml, fragments);
-
-// Step 3. Look for ServletContainerInitializer implementations
-ok = processServletContainerInitializers(orderedFragments);
-
-// Step 4. Process /WEB-INF/classes for annotations
-// This will add any matching classes to the typeInitializerMap
-if (ok) {
-URL webinfClasses;
-try {
-webinfClasses =
-
context.getServletContext().getResource(/WEB-INF/classes);
-processAnnotationsUrl(webinfClasses, webXml);
-} catch (MalformedURLException e) {
-
log.error(sm.getString(contextConfig.webinfClassesUrl), e);
+// Only need to process fragments and annotations if metadata is
+// not complete
+SetWebXml orderedFragments = null;
+if (!webXml.isMetadataComplete()) {
+// Step 2. Order the fragments.
+orderedFragments = WebXml.orderWebFragments(webXml,
fragments);
+
+// Step 3. Look for ServletContainerInitializer
implementations
+ok = processServletContainerInitializers(orderedFragments);
+
+// Step 4. Process /WEB-INF/classes for annotations
+// This will add any matching classes to the
typeInitializerMap
+if (ok) {
+URL webinfClasses;
+try {
+webinfClasses =
context.getServletContext().getResource(
+/WEB-INF/classes);
+processAnnotationsUrl(webinfClasses, webXml);
+} catch (MalformedURLException e) {
+log.error(sm.getString(
+contextConfig.webinfClassesUrl), e);
+}
}
-}
-
-// Step 5. Process JARs for annotations - only need to process
those
-// fragments we are going to use
-// This will add any matching classes to the typeInitializerMap
-if (ok) {
-processAnnotations(orderedFragments);
-}
-
-// Step 6. Merge web-fragment.xml files into the main web.xml
file.
-if (ok) {
-ok = webXml.merge(orderedFragments);
-}
-
-// Step 6.5 Convert explicitly mentioned jsps to servlets
-if (!false) {
-convertJsps(webXml);
-}
-
-// Step 7. Apply merged web.xml to Context
-if (ok) {
-webXml.configureContext(context);
-
-// Step 7a. Make the merged web.xml available to other
-// components, specifically Jasper, to save those components
-// from having to re-generate it.
-// TODO Use a ServletContainerInitializer for Jasper
-String mergedWebXml = webXml.toXml();
-context.getServletContext().setAttribute(
- org.apache.tomcat.util.scan.Constants.MERGED_WEB_XML,
-mergedWebXml);
-if (context.getLogEffectiveWebXml()) {
-log.info(web.xml:\n + mergedWebXml);
+
Re: 7.0.3 release plans
I think I now have a nice improvement to the memory leak protection (see BZ 49159). If you have time to review it, it would be a nice addition to this release... Sylvain On 30 sept. 2010, at 19:38, Mark Thomas wrote: I'm done with the async re-factoring. I want to run through a complete set of tests but I'm not expecting any major issues. My plan for 7.0.3 is currently: - fix as many bugs as I can between now and the weekend - run the tests (JUnit, TCK) over the weekend - assuming no failures, tag and roll the RC on Monday Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Draft blog entry for review
you could add a word on persistence of the JMX configuration. As far as I understand, the modifications done through JMX are not persisted, is that correct ? Sylvain On 15 sept. 2010, at 17:07, Mark Thomas wrote: As I mention recently, JMX is now looking pretty good. I have drafted a blog entry [1] on this topic that I'd like to publish later this week. Review / feedback welcome. Mark [1] https://blogs.apache.org/preview/tomcat/?previewEntry=tomcat_7_trunk_and_jmx - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: SharingWebappClassLoader proposal
I see 2 options for your case :
- either use JRebel which allows to reload classes even in jars that would be
the common or shared classloaded (I use it all the times, it's fabulous and
pays for itself very quickly)
- or try JBoss which has a Unified Class Loader, and as far as I understood it,
it tries to share classes between applications while allowing reloads (I never
used it in this way though)
Regards,
Sylvain
On 5 août 2010, at 12:57, Lohner Roland wrote:
Dear Mark, Rainer, Konstantin,
Thank you for your fast and detailed answers.
You are right, the proposal is erroneous. However my intention was not to
give an excellent implementation solving the problem, but to start a
conversation about it and to provide a 0th approach, which can be reworked,
refined by developers who have detailed insight in the Tomcat system
architecture.
You suggested the usage of the common or the shared class loader. Putting
all interfaces, and _all the classes they need_, into a separate JAR and
putting this JAR into ${catalina.base}/lib would really solve the problem.
But this solution has a serious disadvantage.
The disadvantage appears in development time. The common and the shared
class loaders – as far as I know – are not reloadable. This means if any of
the classes loaded by them changes the whole Tomcat server and so even all
another independent web applications must be restarted for the changes to
take effect. The number of shared interfaces and their dependent classes can
be remarkable. When these types change, a full restart is needed instead of
some web application reloads, which is supported by widely applied
development tools like Web Tools Platform (WTP) Project. The full restart
means extra overheat increasing the turnaround time of development cycles.
An another, only minor problem is that the deployment of the application
fallen to pieces must be done manually (increasing turnaround delay again).
WTP for example does not support the deployment of jar files in arbitrary
folders but only deployment of war files to a ‘webapps like’ folder.
Turnaround time of development cycles is a significant factor in the costs
of a software solution. Accordingly, the possibility of using shared classes
in a reloadable (and by development tools supported) way would be valuable
in Tomcat.
Regards,
Roland
2010/8/3 Konstantin Kolinko knst.koli...@gmail.com
2010/8/3 Lohner Roland loc...@gmail.com:
Dear Developers,
I am writing about a proposal for a new feature in Tomcat.
(...)
so the only solution at the
moment is to use a J2EE container and deploy an enterprise module which
causes unnecessary overheat.
I think the following should work:
1. Separate the interfaces, and _all the classes they need_,
into a separate JAR.
2. Put the JAR into ${catalina.base}/lib.
3. _Remove_ all those shared classes from web applications!
(Because otherwise classes loaded from web applications have priority
over the shared ones, as explained in
http://tomcat.apache.org/tomcat-6.0-doc/class-loader-howto.html
)
4. If you need some shared service, place it into JNDI
(GlobalNamingResources element in server.xml).
You can also add a Listener to server.xml to preload some classes or
instantiate any static object.
5. Restart Tomcat.
A less elaborate, hackish, but basically working implementation of this
special web application class loader is attached to this email.
The attachment was removed by mailing list software.
Maybe if you rename it to *.txt it will be allowed, but it is just a guess.
Anyway, if it works how you describe it it seeks for trouble. A web
application and be unloaded (stopped, reloaded) at any time. Since
that moment any classes loaded by its classloader should not be
accesses any more. Thus, no other web applications should keep
references to them.
Best regards,
Konstantin Kolinko
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: More sources of Tomcat memory leaks
IMHO, clearing threadlocals is a bad idea, it has even been disabled by default
because of concurrency issues.
Furthermore, the solution you propose would be limited to some type of classes
(Map in your case).
https://issues.apache.org/bugzilla/show_bug.cgi?id=49159 is still pending but
the solution proposed would cure this issue in a cleaner way.
Sylvain
On 29 juil. 2010, at 17:00, Arjen Knibbe wrote:
markt-2 wrote:
Please could you create a Bugzilla entry (against Tomact 7) for each
issue you identified so that this valuable work doesn't get forgotten
about?
95% of the work should just be a copy and paste.
I created three Bugzilla entries:
49667 for the JdbcLeakPreventer issue.
49668 for the ThreadLocal issue.
49669 for the PolicyFile issue.
The problem with the LoaderHandler is a question of patience: after an hour
or two, it will go away.
I searched for an upgrading to Tomcat 7 manual in the Tomcat documentation
to check if the new JreMemoryLeakPreventionListener is mentioned, but I
could not find any guidelines for upgrading...
markt-2 wrote:
PS How do you fancy trying to fix some of these issues?
I didn't try to write a subclass of WebappClassLoader. In stead, I used the
destroy() method of the HttpServlet to clean things up.
Clearing the PolicyFile reference is trivial:
/**
* Clear references from Policy.
*/
private void clearReferencesPolicy()
{
System.out.println(About to clear Policy references);
ClassLoader myLoader = this.getClass().getClassLoader();
try
{
Class? policyClass =
Class.forName(javax.security.auth.Policy);
Field contextClassLoaderField =
policyClass.getDeclaredField(contextClassLoader);
contextClassLoaderField.setAccessible(true);
Object contextClassLoaderObj =
contextClassLoaderField.get(null);
if (contextClassLoaderObj == myLoader)
{
System.out.println(Policy: setting contextClassLoader to
null);
contextClassLoaderField.set(null, null);
}
}
catch (Throwable e)
{
System.err.println(Clearing Policy references);
e.printStackTrace();
}
}
Clearing of the JDBC Drivers can be done by running the same loop twice:
/**
* Clear the JDBC drivers for the first time.
* This will load JDBC drivers that were loaded by other applications
* but not by this one, and can be found this class's loader.
*/
public void clearJdbcDrivers()
{
System.out.println(About to deregister JDBC drivers);
try
{
// This will list all drivers visible to this class loader
EnumerationDriver drivers = DriverManager.getDrivers();
while (drivers.hasMoreElements())
{
Driver driver = drivers.nextElement();
// Only unload the drivers this web app loaded
if (driver.getClass().getClassLoader() !=
this.getClass().getClassLoader()) {
continue;
}
System.out.println(Deregistering driver + driver);
DriverManager.deregisterDriver(driver);
}
}
catch (Exception e)
{
System.err.println(Deregistering JDBC drivers);
e.printStackTrace();
}
}
This is done first in the destroy method of the servlet, and again in the
JdbcLeakPreventer. That gives nasty warnings about JDBC Drivers being
forcibly unregistered. I suggest the loop should not run before, but just
after the loop in the JdbcLeakPreventer, without outputting messages.
Now for the real fancy thing: deeply clearing of the ThreadLocal references.
It starts with a modified copy of the methods in WebappClassLoader for
clearing ThreadLocal references:
/**
* Clear references to the ClassLoader in nested Maps and Collections
* of ThreadLocal objects.
*/
@SuppressWarnings(unchecked)
private void clearReferencesThreadLocalsDeep()
{
System.out.println(About to clear deep references from
ThreadLocals);
Thread[] threads = getThreads();
try
{
// Make the fields in the Thread class that store ThreadLocals
// accessible
Field threadLocalsField =
Thread.class.getDeclaredField(threadLocals);
threadLocalsField.setAccessible(true);
Field inheritableThreadLocalsField =
Thread.class.getDeclaredField(inheritableThreadLocals);
inheritableThreadLocalsField.setAccessible(true);
// Make the underlying array of ThreadLoad.ThreadLocalMap.Entry
objects
// accessible
Class? tlmClass =
Class.forName(java.lang.ThreadLocal$ThreadLocalMap);
Field tableField =
Re: [VOTE] Release Tomcat 7.0.0 based on Tomcat 7.0.0 RC3
Hello, how should one file bug reports for this RC3 ? in bugzilla or in the mailing list ? in BZ the only versions are trunk and unspecified... On 24 mai 2010, at 00:38, Mark Thomas wrote: All, The next Tomcat 7.0.0 release candiate is ready for testing. 7.0.0-RC3 can be obtained from: http://people.apache.org/~markt/dev/tomcat-7/v7.0.0-RC3/ The svn tag is: http://svn.apache.org/repos/asf/tomcat/tc7.0.x/tags/TOMCAT_7_0_0_RC3/ Please test this release candidate and provide feedback. The 7.0.0-RC3 tag is [ ] Broken - do not release [ ] Alpha - go ahead and release 7.0.0 Alpha based on 7.0.0-RC3 [ ] Beta - go ahead and release 7.0.0 Beta based on 7.0.0-RC3 [ ] Stable - go ahead and release 7.0.0 Stable based on 7.0.0-RC3 The following issues were noted: - Thanks, Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [VOTE] Release Tomcat 7.0.0 based on Tomcat 7.0.0 RC3
On 25 mai 2010, at 21:58, Mark Thomas wrote: On 25/05/2010 19:55, Sylvain Laurent wrote: Hello, how should one file bug reports for this RC3 ? in bugzilla or in the mailing list ? in BZ the only versions are trunk and unspecified... Trunk is fine for now since anything that affects an RC will almost certainly affect trunk. My instinct at this point is to keep the version list to just the released versions for now. If that causes problems we can easily start adding the RC versions. Mark OK, I filed one in BZ : https://issues.apache.org/bugzilla/show_bug.cgi?id=49340 By the way, speaking of RC3, https://issues.apache.org/bugzilla/show_bug.cgi?id=48971 [memory leak protection : stopping TimeThreads should be optional and disabled by default] is marked for 6.x but it also affects 7.x. It would be nice to take a look at it before 7.0.0 (and 6.0.27)... Sylvain - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Need advice to notify StandardExecutor when a webapp is stopped
I finally proposed a patch for trunk in
https://issues.apache.org/bugzilla/show_bug.cgi?id=49159 for this. Thanks for
reviewing it...
Sylvain
On 30 avr. 2010, at 18:27, Sylvain Laurent wrote:
On 30 avr. 2010, at 00:01, Pid wrote:
Are you saying that you want to stop processing requests each time a
webapp gets restarted, or that the thread pool is refreshed by
sequentially killing each thread and recreating it?
Something in between : I create a new pool with the same characteristics as
the current one, make it the current pool so that new requests are served by
the new pool, then cleanly shut the old pool down. When calling
ThreadPoolExecutor.shutdown(), it gracefully terminates all threads in the
pool after its associated TaskQueue is empty : Idle threads stop immediately
(and not sequentially), busy threads continue processing their current
request. If the TaskQueue is not empty, it means that there are no idle
threads, and so busy threads will continue processing tasks in the queue
until it becomes empty.
The renewThreads looks like (in StandardThreadExecutor) :
public void renewThreads() {
ThreadPoolExecutor oldExecutor;
synchronized (executorLock) { // to avoid renewing threads
concurrently
oldExecutor = executor;
ThreadPoolExecutor newExecutor=new
ThreadPoolExecutor(getMinSpareThreads(),
getMaxThreads(), maxIdleTime,
TimeUnit.MILLISECONDS,
taskqueue,
oldExecutor.getThreadFactory());
executor = newExecutor;
taskqueue.setParent(executor);
}
oldExecutor.shutdown();
//we don't wait for termination of the old pool, threads will
terminate when their work is done
}
I marked StandardThreadExecutor.executor and TaskQueue.parent as volatile to
propagate the change of executor instance to other threads without
synchronizing threads.
An improvement I can do is to pre-start some core threads in the new pool
before making it active. It would reduce the performance impact on the next
few requests.
Sylvain
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: clearReferencesThreads, Poller SunPKCS11-Solaris and strange context class loader
When you analyzed the heap dump, what path to GC roots was retaining the classloader in memory ? On 6 mai 2010, at 20:51, Rainer Jung wrote: While doing some testing with 6.0.26 I noticed, that when shutting it down it logs an error about thread Poller SunPKCS11-Solaris not being stopped. When I add the more explicit logging from tc6 trunk, it says the thread was started by /manager. The thread sits in java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at sun.security.pkcs11.SunPKCS11$TokenPoller.run(SunPKCS11.java:681) at java.lang.Thread.run(Thread.java:619) and is started for some version of the JDK (noted on Solaris for e.g. 1.5.0_22 and 1.6.0_3, not for 1.6.0_17) even without having anything related to keystores, https or similar configured in Tomcat (default config). The only non-default is using Log4J instead of Juli. What is strange, is that the check whether the context class loader of the thread is equal to the WebappClassLoader of the context to unload passes. I added an additional output by explicitely printing the contextName of the two loaders and in fact both print /manager. When deploying two instances of the manager and reloading the original instance with the additional one, I get the same warning. In a heap dump, it seems the context class loader of the thread is the system class loader and not of type WebappClassLoader. Any idea, why the context class loader test fails? Is there any reason why the context class loader of the thread might change when doing the manager reload or shutdown? Regards, Rainer - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn commit: r940089 - in /tomcat/trunk/java/org/apache/catalina/loader: LocalStrings.properties WebappLoader.java
I think that this commit broke the ability to stop and start a context. When I
start it after having stopped it, I get a
LifecycleException: An invalid Lifecycle transition was attempted
([before_start]) for component [WebappLoader[/testWeb]] in state [DESTROYED]
Sylvain
On 1 mai 2010, at 19:44, ma...@apache.org wrote:
Author: markt
Date: Sat May 1 17:44:26 2010
New Revision: 940089
URL: http://svn.apache.org/viewvc?rev=940089view=rev
Log:
Remove the controller - MBean registration will always happen in
init()/destroy() after Lifecycle refactoring
Fix a handful of Eclipse/FindBugs warnings
Modified:
tomcat/trunk/java/org/apache/catalina/loader/LocalStrings.properties
tomcat/trunk/java/org/apache/catalina/loader/WebappLoader.java
Modified: tomcat/trunk/java/org/apache/catalina/loader/LocalStrings.properties
URL:
http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/catalina/loader/LocalStrings.properties?rev=940089r1=940088r2=940089view=diff
==
--- tomcat/trunk/java/org/apache/catalina/loader/LocalStrings.properties
(original)
+++ tomcat/trunk/java/org/apache/catalina/loader/LocalStrings.properties Sat
May 1 17:44:26 2010
@@ -65,4 +65,5 @@ webappLoader.starting=Starting this Load
webappLoader.stopping=Stopping this Loader
webappLoader.failModifiedCheck=Error tracking modifications
webappLoader.copyFailure=Failed to copy resources
+webappLoader.mkdirFailure=Failed to create destination directory to copy
resources
webappLoader.namingFailure=Failed to access resource {0}
Modified: tomcat/trunk/java/org/apache/catalina/loader/WebappLoader.java
URL:
http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/catalina/loader/WebappLoader.java?rev=940089r1=940088r2=940089view=diff
==
--- tomcat/trunk/java/org/apache/catalina/loader/WebappLoader.java (original)
+++ tomcat/trunk/java/org/apache/catalina/loader/WebappLoader.java Sat May 1
17:44:26 2010
@@ -271,8 +271,8 @@ public class WebappLoader extends Lifecy
boolean oldDelegate = this.delegate;
this.delegate = delegate;
-support.firePropertyChange(delegate, new Boolean(oldDelegate),
- new Boolean(this.delegate));
+support.firePropertyChange(delegate, Boolean.valueOf(oldDelegate),
+ Boolean.valueOf(this.delegate));
}
@@ -332,8 +332,8 @@ public class WebappLoader extends Lifecy
boolean oldReloadable = this.reloadable;
this.reloadable = reloadable;
support.firePropertyChange(reloadable,
- new Boolean(oldReloadable),
- new Boolean(this.reloadable));
+ Boolean.valueOf(oldReloadable),
+ Boolean.valueOf(this.reloadable));
}
@@ -542,7 +542,6 @@ public class WebappLoader extends Lifecy
oname=new ObjectName(ctx.getEngineName() +
:type=Loader,path= +
path + ,host= + ctx.getParent().getName());
Registry.getRegistry(null, null).registerComponent(this,
oname, null);
-controller=oname;
} catch (Exception e) {
log.error(Error registering loader, e );
}
@@ -558,11 +557,8 @@ public class WebappLoader extends Lifecy
@Override
protected void destroyInternal() {
-if( controller==oname ) {
-// Self-registration, undo it
-Registry.getRegistry(null, null).unregisterComponent(oname);
-oname = null;
-}
+Registry.getRegistry(null, null).unregisterComponent(oname);
+oname = null;
}
/**
@@ -813,6 +809,7 @@ public class WebappLoader extends Lifecy
String path = libDir.getCanonicalPath();
classLoader.addPermission(path);
} catch (IOException e) {
+// Ignore
}
}
@@ -825,6 +822,7 @@ public class WebappLoader extends Lifecy
String path = libDir.getCanonicalPath();
classLoader.addPermission(path);
} catch (IOException e) {
+// Ignore
}
}
if (classesURL != null) {
@@ -833,6 +831,7 @@ public class WebappLoader extends Lifecy
String path = classesDir.getCanonicalPath();
classLoader.addPermission(path);
} catch (IOException e) {
+// Ignore
}
}
}
@@ -840,6 +839,7
Re: Need advice to notify StandardExecutor when a webapp is stopped
Interesting idea, really ! But I suspect it might break some things in some cases. I'm thinking about cross-context requests (where the requestDispatcher calls another web application in the same JVM with the same thread). If the context-specific table changes when dispatching the request to the other context, the latter won't see any threadlocal variables bound by the first context. I feel it might be problem for some webapps or some frameworks, but I cannot provide a concrete example. Maybe it's not a problem after all... Sylvain On 29 avr. 2010, at 23:57, Caldarale, Charles R wrote: From: Caldarale, Charles R Subject: RE: Need advice to notify StandardExecutor when a webapp is stopped This is an area where some byte code modification might be appropriate in a container environment, so that the ThreadLocal behavior could be modified to be ContextLocal instead. Thinking about this some more, byte code modification of the ThreadLocal class would not be needed. When a thread returns to the pool, any ThreadLocal objects it has could be moved to a context-specific table, using an MRU stack. When a thread is taken out of the pool to process a request for a context, it could then move the most recent set of ThreadLocal objects to itself. The context-specific table would be discarded whenever the webapp is stopped. This would allow performance acceleration for each webapp, and avoid pollution of the pool. - Chuck - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Need advice to notify StandardExecutor when a webapp is stopped
On 30 avr. 2010, at 00:01, Pid wrote:
Are you saying that you want to stop processing requests each time a
webapp gets restarted, or that the thread pool is refreshed by
sequentially killing each thread and recreating it?
Something in between : I create a new pool with the same characteristics as the
current one, make it the current pool so that new requests are served by the
new pool, then cleanly shut the old pool down. When calling
ThreadPoolExecutor.shutdown(), it gracefully terminates all threads in the pool
after its associated TaskQueue is empty : Idle threads stop immediately (and
not sequentially), busy threads continue processing their current request. If
the TaskQueue is not empty, it means that there are no idle threads, and so
busy threads will continue processing tasks in the queue until it becomes empty.
The renewThreads looks like (in StandardThreadExecutor) :
public void renewThreads() {
ThreadPoolExecutor oldExecutor;
synchronized (executorLock) { // to avoid renewing threads
concurrently
oldExecutor = executor;
ThreadPoolExecutor newExecutor=new
ThreadPoolExecutor(getMinSpareThreads(),
getMaxThreads(), maxIdleTime,
TimeUnit.MILLISECONDS,
taskqueue,
oldExecutor.getThreadFactory());
executor = newExecutor;
taskqueue.setParent(executor);
}
oldExecutor.shutdown();
//we don't wait for termination of the old pool, threads will
terminate when their work is done
}
I marked StandardThreadExecutor.executor and TaskQueue.parent as volatile to
propagate the change of executor instance to other threads without
synchronizing threads.
An improvement I can do is to pre-start some core threads in the new pool
before making it active. It would reduce the performance impact on the next few
requests.
Sylvain
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Need advice to notify StandardExecutor when a webapp is stopped
If I understand you and your comment here https://issues.apache.org/bugzilla/show_bug.cgi?id=49159#c2 you propose to clean only thread locals loaded by a web app. But how do you recognize them ? If the value bound to a threadlocal is an ArrayList, you won't be able to tell that it was loaded by a webapp. Furthermore, there is probably some frameworks out there that do save things in ThreadLocals in a manner that does not cause classloader leaks but improve their performance. Cleaning their threadlocals after each request would decrease their performance. I'm rather in favor of renewing the threads of the pool when a webapp is being stopped. I have a prototype (very early stage) with a new method in StandardThreaExecutor that recreates a ThreadPoolExecutor instance when invoked. With a specific LifecycleListener declared in server.xml which registers itself as a LifecycleListener on every Context, it calls this new renewThreads method when the after_stop event of a Context is fired. The renewThreads method is called on each Executor of each EndPoint of each Connector, provided it is a StandardThreadExecutor. It works well to avoid threadLocal-related leaks, and it has no impact for request processing. There's only an impact when an application is stopped : all threads in the pool(s) are shutdown (once they have no more work to do in the TaskQueue) and new threads are recreated. Thus for heavily loaded servers there is probably a decrease in throughput while new threads are being created, but in my opinion this is an acceptable price to pay for a welcome memory leak protection. Anyways, the feature can be disabled by simply commenting the listener in server.xml. I still have to polish the thing before proposing a patch : - make sure the lifecycleListener is registered for new Context/Host/Service added dynamically - check what happens when a Context fails to start - exception handling - various cleanups... Regarding tomcat 6 compatibility, I did not check yet. But having this feature for TC7 might be an incentive to upgrade ;-) Sylvain On 29 avr. 2010, at 01:36, Mark Thomas wrote: On 27/04/2010 15:57, Sylvain Laurent wrote: Do you mean one declared in server.xml ? or have the StandardThreadExecutor register itself as a LifeCycleListener ? In the latter case, my problem is how do I get a reference to StandardContext instances from the StandardThreadExecutor ? I'm not sure this is the way to go. It isn't going to work for Tomcat 6. I think cleaning the threads when they return to the pool may be the more viable option. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Need advice to notify StandardExecutor when a webapp is stopped
While experimenting with clearing threadlocal in afterExecute, I found out that tomcat itself use ThreadLocal in hope of improving performance : - org.apache.catalina.util.DateTool - org.apache.catalina.valves.AccessLogValve - org.apache.tomcat.util.http.ServerCookie - ... this is mainly to cache Date formatter or parsers... Sylvain On 29 avr. 2010, at 23:17, Caldarale, Charles R wrote: From: Sylvain Laurent [mailto:sylvain.laur...@gmail.com] On Behalf Of Sylvain Laurent Subject: Re: Need advice to notify StandardExecutor when a webapp is stopped Furthermore, there is probably some frameworks out there that do save things in ThreadLocals in a manner that does not cause classloader leaks but improve their performance. Cleaning their threadlocals after each request would decrease their performance. It would seem difficult to achieve any kind of consistent performance improvement using ThreadLocal objects in conjunction with a thread pool. If the server had a very limited number of webapps and the pool size were reasonably small, the webapp could show improvement, but otherwise there's little chance of a particular request being processed by a thread that has already been enhanced with a ThreadLocal for that webapp. To me, it's much cleaner and less disruptive to discard the ThreadLocal objects when a thread returns to the pool, rather than making all webapps suffer when any one of them is restarted. This is an area where some byte code modification might be appropriate in a container environment, so that the ThreadLocal behavior could be modified to be ContextLocal instead. - Chuck - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Need advice to notify StandardExecutor when a webapp is stopped
Hello, In order to provide a protection agains this type of memory leak http://wiki.apache.org/tomcat/MemoryLeakProtection#webappClassInstanceAsThreadLocalIndirectValue , I experimented with renewing the ThreadPoolExecutor in the org.apache.catalina.core.StandardThreadExecutor. It works when I invoke my renewThreads method through JMX, but now I'd like it to be invoked after a webapp is being stopped. What's the best way to do that ? a LifeCycle listener to be registered somewhere ? Thanks for your ideas. Sylvain
Re: Memory leak detection and JVMTI
I don't understand what would be the purpose of this feature. Is it in order to really force a GC when clicking on the Find leaks button of the manager ? If it's for that, I don't think it's worth the trouble. People really interested in fixing their leaking webapps should take a snapshot of the heap and analyze it to confirm (as advised by the manager page) and find the origin of the leak. Just my personal feeling... Sylvain On 22 avr. 2010, at 11:43, Rainer Jung wrote: Just a short note: I stumbled over the ForceGarbageCollection JVMTI function today. It reliably triggers a full GC and the call only returns after having run the GC. It will do so, even when System.gc() is disabled. It looks like it doesn't respect ExplicitGCInvokesConcurrent, so it will always do a stop-the-world collection leading to possibly long stop times but also to defragmented tenured space. As always, it is not expected, that the finalizers will be run. Maybe something we could add to tcnative. Since the call is stateless, adding it should be simple, just implementing Agent_OnLoad to initialize the JVMTI environmnt using GetEnv() and then making ForceGarbageCollection(env) publicly accessible. One must also register the JVMTi agent using e.g. the -agentlib startup option. I don't know, whether that would lead to a problem with duplicate library loading, in which case one would have to separate it from tcnative. Caution: this is all form looking at docs and code, no tests done yet. Regards, Rainer - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Other ideas for memory leak protection
Hello all, I wrote a wiki page to try and explain all the causes of webapp classloader leaks one might encounter : http://wiki.apache.org/tomcat/MemoryLeakProtection I'd be glad to have your feedback on it ! While creating the page, I thought about 2 things to improve the protection : 1) to fix leaks like Webapp class instance indirectly held through a ThreadLocal value ( http://wiki.apache.org/tomcat/MemoryLeakProtection#webappClassInstanceAsThreadLocalIndirectValue ), I think that the only efficient way would be to renew all the threads in the worker pool after an application is stopped. Unfortunately I don't see any way of doing this with the ThreadPoolExecutor that is now used by tomcat 7. We could create a new ThreadPoolExecutor instance, and stop all threads of the old one, but this would have a big performance impact for other running applications... Alternatively, we could clean all ThreadLocals in org.apache.tomcat.util.threads.ThreadPoolExecutor.afterExecute(Runnable, Throwable) . But I don't know the performance impacts that it would have on usual applications. 2) to fix leaks like Threads spawned by classes loaded by the common classloader ( http://wiki.apache.org/tomcat/MemoryLeakProtection#cclThreadSpawnedByCommonClassLoader ) while avoiding the side effects of stopping threads (see https://issues.apache.org/bugzilla/show_bug.cgi?id=48971 ), the expendable classloader can help. But before trying to provide a patch, I tried to simply nullify the CCL and inheritedAccessControlContext of the leaking thread : thread.setContextClassLoader(null); Field field = Thread.class.getDeclaredField(inheritedAccessControlContext); field.setAccessible(true); field.set(thread, null); I tested it, it works and is much simpler than the expendable classloader trick, but I really don't know how safe it is, especially if running with a security manager... What do you think ? Sylvain - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Feedback on tomcat memory leak protection
On 03/03/2010 21:12, Sylvain Laurent wrote: 3) a couple of months ago I had proposed on this very list a kind of memory leak protection for those leaks caused by threads with incorrect CCL. I called this the ExpendableClassLoader. Please have a look at my post back then : http://mail-archives.apache.org/mod_mbox/tomcat-dev/200903.mbox/%3cb1be8ffd-f13f-4b2b-b25a-83f2f855b...@m4x.org%3e Since I did not get any feedback about this idea at all, neither positive nor negative, I can only assume that my post got lost in the middle of the other ones. I still think that this ExpendableClassLoader would improve the memory leak protection. Actually it would make the JreMemoryLeakPreventionListener useless and developers would not have to think about which option of the JreMemoryLeakPreventionListener is useful to them. I suspect the lack of full source code (the Tomcat lists drop attachments) had something to do with it. The concept sounds promising and would be better than continually adding to the JreLeakPreventionListener. Should I try to pursue my investigation and provide a patch ? for TC 6 or 7 ? 4) Not tomcat specific, but it might interest you anyways : there's definitely a problem with Sun's server VM, even with the latest 6.0_18. Classloaders are not always collected by the GC and this can lead to an OOME in the perm gen. I managed to reproduce this with tomcat 6.0.24 and the petcare.war sample of Spring 3. When using the client VM, the WebAppClassLoader is always correctly GCed. When using the server VM (either 32 or 64 bits, on Windows or MacOS), the classloader is not collected and after a couple of redeployments, the perm gen is full and an OOME is raised. When analyzing the heap dump, I can see several instances of WebAppClassLoaders, and eclipse MAT (www.eclipse.org/mat) shows that there's no strong reference to them :-((( That sounds like http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6916498 No, it's actually much more severe. The bug you filed provokes a one time leak: only the first webapp that uses DocumentBuilderFactory.newInstance().newDocumentBuilder() will leak. The bug I see with the server VM is probably closer to http://bugs.sun.com/view_bug.do?bug_id=4957990 which is supposed to be fixed since 6.0u16, but the comment of 2009-12-09 says otherwise. Anyways, I filed a bug report to Sun, it's in review for now... Sylvain - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Feedback on tomcat memory leak protection
Hello, After reading Mark Thomas's interview at DZone http://java.dzone.com/articles/memory-leak-protection-tomcat and having experimented with classloader leaks issues, I'd like to share my thoughts with you : 1) first of all THANK YOU for providing some solutions to those problems. It's not perfect, but it's very valuable 2) About JreMemoryLeakPreventionListener : Here are other leaks I know of : 2.1) When using JNDI for LDAP access, if you use connection pooling, the JRE spawns a Thread whose context classloader is set to the one of the thread that initializes com.sun.jndi.ldap.LdapPoolManager. If it's a WebAppClassLoader, there's a leak. So, we could the initialization of com.sun.jndi.ldap.LdapPoolManager as an option for JreMemoryLeakPreventionListener. 2.2) Not JRE, but of the same kind : with Oracle JDBC driver 10.2.x, the first time statement timeouts are used, a thread is spawned with the same CCL leak. One can force the spawning by initializing oracle.jdbc.driver.OracleTimeoutThreadPerVM before starting webapps. We could provide a new *MemoryLeakPreventionListener for this kind of 3rd-party leak. 3) a couple of months ago I had proposed on this very list a kind of memory leak protection for those leaks caused by threads with incorrect CCL. I called this the ExpendableClassLoader. Please have a look at my post back then : http://mail-archives.apache.org/mod_mbox/tomcat-dev/200903.mbox/%3cb1be8ffd-f13f-4b2b-b25a-83f2f855b...@m4x.org%3e Since I did not get any feedback about this idea at all, neither positive nor negative, I can only assume that my post got lost in the middle of the other ones. I still think that this ExpendableClassLoader would improve the memory leak protection. Actually it would make the JreMemoryLeakPreventionListener useless and developers would not have to think about which option of the JreMemoryLeakPreventionListener is useful to them. 4) Not tomcat specific, but it might interest you anyways : there's definitely a problem with Sun's server VM, even with the latest 6.0_18. Classloaders are not always collected by the GC and this can lead to an OOME in the perm gen. I managed to reproduce this with tomcat 6.0.24 and the petcare.war sample of Spring 3. When using the client VM, the WebAppClassLoader is always correctly GCed. When using the server VM (either 32 or 64 bits, on Windows or MacOS), the classloader is not collected and after a couple of redeployments, the perm gen is full and an OOME is raised. When analyzing the heap dump, I can see several instances of WebAppClassLoaders, and eclipse MAT (www.eclipse.org/mat) shows that there's no strong reference to them :-((( Thank you for your attention, Sylvain - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Proposition for a WebAppClassLoader enhancement : ExpendableClassLoader
[I'm re-sending this e-mail as I did not have any response. Maybe it
was because of html format ?]
Hello,
Among the many possible causes of classloader leaks, one is the
context class loader of threads spawned during the execution of web
applications.
For instance, if the following servlet is executed at least once and
it is the first time the Graphics2D part of the JRE is used for the
current JVM, then the web app's classloader will never be garbage-
collected.
public class MyTomcatServlet extends HttpServlet {
@Override
protected void doGet(HttpServletRequest req, HttpServletResponse
response)
throws ServletException, IOException {
System.out.println(MyServlet.doGet : current CCL:
+
Thread.currentThread().getContextClassLoader());
BufferedImage image = new BufferedImage(20, 20,
BufferedImage.TYPE_INT_RGB);
Graphics2D g = image.createGraphics();
response.setContentType(image/png);
OutputStream out = response.getOutputStream();
ImageIO.write(image, png, out);
out.close();
out.flush();
g.dispose();
}
}
In order to work around such leaks, I created an
ExpendableClassLoader, which will leak instead of the normal
WebAppClassLoader, But the former loads no class at all.
Here is the code :
/**
* p
* A special ClassLoader to work around classloader (and thus
permgen) leaks
* caused by never-ending threads spawned during the execution of the
web app.
* If no care is taken, such threads have the webapp's classloader as
context
* classloader. So, if such a thread is still alive after the
application is
* undeployed, the application's classloader cannot be garbage-
collected.
* /p
* p
* When the application is undeployed, the reference to the delegated
* classloader is nullified so that the latter can be garbage-collected
* (provided it is not held by other means).
* /p
* p
* As of march 2009, there are several such leaking threads in
Sun's JRE
* library : Java2DDisposer {...@link see http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6489540
}, LDAP connection pool manager... A very simple way
* of provoking such a leak is with the following servlet code :
*
* pre
* public class MyServlet extends HttpServlet {
*
* #064;Override
* protected void doGet(HttpServletRequest req, HttpServletResponse
response)
* throws ServletException, IOException {
* BufferedImage image = new BufferedImage(20, 20,
* BufferedImage.TYPE_INT_RGB);
* Graphics2D g = image.createGraphics();
* response.setContentType(quot;image/pngquot;);
* OutputStream out = response.getOutputStream();
* ImageIO.write(image, quot;pngquot;, out);
* out.close();
* out.flush();
* g.dispose();
* }
* }
* /pre
*
* /p
* p
* By using this ExpendableClassLoader as the webapp's classloader
(and thus the
* context classloader), such dangling threads only keep a reference
to a very
* light classloader, drastically reducing the leak.
* /p
* p
* NOTE: the class has to be in the same package as {...@link
WebappClassLoader} in
* order to override package-protected methods.
* /p
*
* @author Sylvain LAURENT
*
*/
public class ExpendableClassLoader extends WebappClassLoader {
private WebappClassLoader delegatedClassLoader;
public ExpendableClassLoader(ClassLoader parentClassLoader) {
delegatedClassLoader = new WebappClassLoader(parentClassLoader);
}
public void stop() throws LifecycleException {
ClassLoader savedParentClassLoader =
delegatedClassLoader.getParent();
delegatedClassLoader.stop();
// release reference to the delegated classloader, potentially
// sacrificing the current instance
delegatedClassLoader = null;
// recreate a delegated classloader, just in case a dangling
Thread
// (e.g. JDK threads like Java2Disposer) needs to load a class
// we use the same parent CL as the previous delegate
delegatedClassLoader = new
WebappClassLoader(savedParentClassLoader);
}
public String toString() {
return ExpendableClassLoader@ + System.identityHashCode(this)
+ delegates to +
delegatedClassLoader.toString();
}
/* --- delegated methods - */
//skipped for brevity
}
I tested it with a small webapp and a custom context.xml to override
the default WebAppClassLoader and it does its job
Proposition for a WebAppClassLoader enhancement : ExpendableClassLoader
Hello,Among the many possible causes of classloader leaks, one is the context class loader of threads spawned during the execution of web applications.For instance, if the following servlet is executed at least once and it is the first time the Graphics2D part of the JRE is used for the current JVM, then the web app's classloader will never be garbage-collected.public class MyTomcatServlet extends HttpServlet { @Override protected void doGet(HttpServletRequest req, HttpServletResponse response) throws ServletException, IOException { System.out.println("MyServlet.doGet : current CCL:"+ Thread.currentThread().getContextClassLoader()); BufferedImage image = new BufferedImage(20, 20, BufferedImage.TYPE_INT_RGB); Graphics2D g = image.createGraphics(); response.setContentType("image/png"); OutputStream out = response.getOutputStream(); ImageIO.write(image, "png", out); out.close(); out.flush(); g.dispose(); }}In order to work around such leaks, I created an "ExpendableClassLoader", which will leak instead of the normal WebAppClassLoader, But the former loads no class at all.Here is the code :/*** p>* A special ClassLoader to work around classloader (and thus permgen) leaks* caused by never-ending threads spawned during the execution of the web app.* If no care is taken, such threads have the webapp's classloader as context* classloader. So, if such a thread is still alive after the application is* undeployed, the application's classloader cannot be garbage-collected.* /p>* p>* When the application is undeployed, the reference to the delegated* classloader is nullified so that the latter can be garbage-collected* (provided it is not held by other means).* /p>* p>* As of march 2009, there are several such "leaking" threads in Sun's JRE* library : Java2DDisposer {...@link see http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6489540}, LDAP connection pool manager... A very simple way* of provoking such a leak is with the following servlet code :** pre>* public class MyServlet extends HttpServlet {** #064;Override* protected void doGet(HttpServletRequest req, HttpServletResponse response)* throws ServletException, IOException {* BufferedImage image = new BufferedImage(20, 20,* BufferedImage.TYPE_INT_RGB);* Graphics2D g = image.createGraphics();* response.setContentType(quot;image/pngquot;);* OutputStream out = response.getOutputStream();* ImageIO.write(image, quot;pngquot;, out);* out.close();* out.flush();* g.dispose();* }* }* /pre>** /p>* p>* By using this ExpendableClassLoader as the webapp's classloader (and thus the* context classloader), such dangling threads only keep a reference to a very* light classloader, drastically reducing the leak.* /p>* p>* NOTE: the class has to be in the same package as {...@link WebappClassLoader} in* order to override package-protected methods.* /p>** @author Sylvain LAURENT**/public class ExpendableClassLoader extends WebappClassLoader { private WebappClassLoader delegatedClassLoader; public ExpendableClassLoader(ClassLoader parentClassLoader) { delegatedClassLoader = new WebappClassLoader(parentClassLoader); } public void stop() throws LifecycleException { ClassLoader savedParentClassLoader = delegatedClassLoader.getParent(); delegatedClassLoader.stop(); // release reference to the delegated classloader, potentially // sacrificing the current instance delegatedClassLoader = null; // recreate a delegated classloader, just in case a dangling Thread // (e.g. JDK threads like Java2Disposer) needs to load a class // we use the same parent CL as the previous delegate delegatedClassLoader = new WebappClassLoader(savedParentClassLoader); } public String toString() { return "ExpendableClassLoader@" + System.identityHashCode(this)+ " delegates to " + delegatedClassLoader.toString(); } /* --- delegated methods - *///skipped for brevity}And the java file is attached to the mail.
ExpendableClassLoader.java
Description: Binary data
I'd be glad to read your opinion on this hack !Sylvain
