Re: svn commit: r585313 - /tomcat/tc6.0.x/trunk/STATUS
On Wed, 2007-10-17 at 05:47 +0200, Peter Rossbach wrote: Hi Remy, I think the patch is incomplete this line seem not correct: +else + JRE_HOME=/usr +fi better +else + JRE_HOME=/usr/bin/java +fi The path to the Java executable is $JRE_HOME/bin/java, so /usr is the correct value. However, on older distributions (just tested with my F7), it will also pick up gij if it is installed (it does not really work). Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: svn commit: r585313 - /tomcat/tc6.0.x/trunk/STATUS
On Wed, 2007-10-17 at 07:28 -0400, Tim Funk wrote: There was a request in bugzilla to pass in a new env variable called JAVA_EXE (or similar). Could that be an acceptable alternative? Then it would make the bug submitter happy since he can symlink my_program_that_i_can_see_in_ps to java. That would be good too, with autodetection :) Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Mavenization (M10N) of Tomcat Build Process - Should Tomcat Be Migrated to Maven 2?
On Wed, 2007-10-17 at 16:21 -0400, Paul Shemansky wrote: It is not my intention to start a flame war between Ant and Maven users, but merely to propose Maven 2 to this group, and respectfully use this thread to discuss the advantages and disadvantages of switching Tomcat's build process within the 6.0.x or possibly 7.0 release schedule. Please use this thread to voice your opinion. Reply to this message with any comments, and/or simple votes for or against the migration to Maven 2. I don't think this work is acceptable in 6.0.x. Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
New tag
Hi, As I stated a couple weeks ago, I think a new tag is needed. As a result, I propose creating a 6.0.15 tag at the end of this week (late Friday). Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [Fwd: [Security] - **Updated** Important vulnerability disclosed in Apache Tomcat webdav servlet]
On Sat, 2007-10-20 at 23:04 -0400, Mark Thomas wrote: The mitigations available are: - - Disable write access until a fixed version is released - - Limit write access to trusted users - - Apply the following patch which will be included in the next releases of 6.0.x, 5.5.x and 4.1.x Since it's an obvious hacking attempt, I chose to use this method instead: documentBuilder.setEntityResolver (new EntityResolver() { public InputSource resolveEntity(String publicId, String systemId) throws SAXException, IOException { return new InputSource(new StringReader()); } }); - no logging, replace with blank text (I was using an ISE right before instead of an input source, but there's no real justification) Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: svn commit: r588000 - /tomcat/tc6.0.x/trunk/STATUS
On Wed, 2007-10-24 at 20:39 -0300, Lucas Galfaso wrote: Hi all, I do not know what each of you actually thinks about this bug, but I would hardly consider this a must have, nor a show stopper (thing that I thought was the case for the next tomcat 6.0.x tag.) I feel a bit like you. I was about to state that I would not consider any patch that was not already in the STATUS file for the 6.0.15 tag. If every known bug with a patch is been considered, then I would like to propose http://issues.apache.org/bugzilla/attachment.cgi?id=20975 that fixes http://issues.apache.org/bugzilla/show_bug.cgi?id=42565 This is a small bug in the jjtree EL grammar. Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
New tag
Hi, As the issue list seems relatively empty, I would like to tag 6.0.15 soon, probably late tomorrow. Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Time to organise svn - Take 3
On Thu, 2007-11-01 at 10:03 +, Mark Thomas wrote: jean-frederic clere wrote: Why Friday? Shouldn't we wait until 6.0.15 (or 6.0.15 + n) is voted stable? We can do if that is the preference. My motivation is that I am keen to get back to a CTR codebase asap as I find only having RTC a real pain. Yes, this is why I had proposed less stringent process, inspired from but not equivalent to the one from httpd (then my vote got hijacked). Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[VOTE] Release build 6.0.15
The candidates binaries are available here: http://people.apache.org/~remm/tomcat-6/v6.0.15/ According to the release process, the 6.0.15 tag is: [ ] Broken [ ] Alpha [ ] Beta [ ] Stable Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Time to organise svn - Take 4
On Sun, 2007-11-04 at 16:10 +, Mark Thomas wrote: Do the (unless there is a pressing need - eg a major security issue) final stable release of 6.0.x. Freeze development of the 6.0.x branch. -1. Branches should continue to be open as long as committers want to propose patches and make releases. Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Rough plan for Tomcat 8
2013/11/15 Mark Thomas ma...@apache.org The major refactorings planned for 8.0.x are complete. There are a few minor refactorings that can either done before the first stable release or (if they turn out to be a little bigger) left until 9.0.x. Cool. Since there was never a 7.1, I suppose there will be no 8.1 either, but instead feature additions here and there in the 8.0 branch ? Rémy
Re: [VOTE] Release Apache Tomcat 8.0.0-RC7
2013/12/13 Violeta Georgieva miles...@gmail.com I'm receiving NPE when testing an application with jsp that specifies tld location with relative path: java.lang.NullPointerException org.apache.jasper.compiler.TldCache.getTaglibXml(TldCache.java:97) org.apache.jasper.compiler.TagLibraryInfoImpl.init(TagLibraryInfoImpl.java:150) org.apache.jasper.compiler.Parser.parseTaglibDirective(Parser.java:411) org.apache.jasper.compiler.Parser.parseDirective(Parser.java:469) org.apache.jasper.compiler.Parser.parseElements(Parser.java:1455) org.apache.jasper.compiler.Parser.parse(Parser.java:139) org.apache.jasper.compiler.ParserController.doParse(ParserController.java:229) org.apache.jasper.compiler.ParserController.parse(ParserController.java:100) org.apache.jasper.compiler.Compiler.generateJava(Compiler.java:200) org.apache.jasper.compiler.Compiler.compile(Compiler.java:375) org.apache.jasper.compiler.Compiler.compile(Compiler.java:355) org.apache.jasper.compiler.Compiler.compile(Compiler.java:342) org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:557) There used to be some normalization code for the taglib uri, like uri = RequestUtil.normalize(uri); in TagLibraryInfoImpl.generateTLDLocation (now generateTldResourcePath), and since the uri is used to match the catch entry, I don't see how it can work without it. I'll add it back. Rémy
Re: [VOTE] Release Apache Tomcat 8.0.0-RC7
2013/12/13 Violeta Georgieva miles...@gmail.com I found another issue: If a tag file is placed in a jar file in WEB-INF/lib and then is used, the The change in the code is introduced with r1541960 Besides that the message is wrong because we check only for WEB-INF/tags [1], what should be done in addition? [1] org.apache.jasper.compiler.TagLibraryInfoImpl.createTagFileInfo(TagFileXml, Jar) 286} else if (!path.startsWith(/WEB-INF/tags)) { 287err.jspError(jsp.error.tagfile.illegalPath, path); 288} It looks like the condition was not refactored correctly. Rémy
Re: [VOTE] Release Apache Tomcat 8.0.0-RC7
2013/12/13 Violeta Georgieva miles...@gmail.com Unfortunately with that fix I'm receiving FNF: java.io.FileNotFoundException: /META-INF/tags/my.tag org.apache.jasper.servlet.JspServletWrapper.loadTagFile(JspServletWrapper.java:235) org.apache.jasper.compiler.TagFileProcessor.loadTagFile(TagFileProcessor.java:579) org.apache.jasper.compiler.TagFileProcessor.access$000(TagFileProcessor.java:50) org.apache.jasper.compiler.TagFileProcessor$TagFileLoaderVisitor.visit(TagFileProcessor.java:661) org.apache.jasper.compiler.Node$CustomTag.accept(Node.java:1521) org.apache.jasper.compiler.Node$Nodes.visit(Node.java:2359) org.apache.jasper.compiler.Node$Visitor.visitBody(Node.java:2411) org.apache.jasper.compiler.Node$Visitor.visit(Node.java:2417) org.apache.jasper.compiler.Node$Root.accept(Node.java:464) org.apache.jasper.compiler.Node$Nodes.visit(Node.java:2359) org.apache.jasper.compiler.TagFileProcessor.loadTagFiles(TagFileProcessor.java:679) org.apache.jasper.compiler.Compiler.generateJava(Compiler.java:231) org.apache.jasper.compiler.Compiler.compile(Compiler.java:375) org.apache.jasper.compiler.Compiler.compile(Compiler.java:355) org.apache.jasper.compiler.Compiler.compile(Compiler.java:342) org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:557) org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:357) I'll try to create a test case for that. The JspCompilationContext used to keep a map of the JarResource corresponding to the tags: public JarResource getTagFileJarResource(String tagFile) { return this.tagFileJarUrls.get(tagFile); } public void setTagFileJarResource(String tagFile, JarResource jarResource) { this.tagFileJarUrls.put(tagFile, jarResource); } This was set in TagLibraryInfoImpl.createTagFileInfo and then used by the TagFileProcessor.loadTagFile. This should be ok without it because the code keeps around the Jar instead and uses the entry name now, but apparently something is still missing and the removed flag gets set (this is what causes this exception). So I guess add a test, it can't hurt. Rémy
Re: [VOTE] Release Apache Tomcat 8.0.0-RC7
2013/12/13 Violeta Georgieva miles...@gmail.com Here [1] is the test [1] https://www.dropbox.com/s/3zd106lpqfb3ytv/test.zip Fixed. I didn't add the test yet since I suppose adding a binary should be avoided. Rémy
Re: [VOTE] Release Apache Tomcat 8.0.0-RC9
2013/12/18 Mark Thomas ma...@apache.org The proposed Apache Tomcat 8.0.0 release candidate 9 is now available for voting. Given this is a release candidate I am working on the basis that it is equivalent to an alpha. The main changes since RC5 are: - Better handling of generic types in the WebSocket 1.0 implementation - Refactor resource handling for the class loader - Add Cobertura support to the unit tests - Remove anti-Jar locking feature and replace it with open stream tracking - Update to a post Commons Pool 2.0 release snapshot - Complete refactoring of TLD handling including caching of parsed TLDs - More consistent handling of XML validation options - Much more detailed visibility of DBCP connections pools in JMX - Better organisation of JMX beans in the default JConsole view - Performance improvements - Numerous bug fixes It can be obtained from: https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.0.0-RC9/ The Maven staging repo is: https://repository.apache.org/content/repositories/orgapachetomcat-063/ The svn tag is: http://svn.apache.org/repos/asf/tomcat/tags/TOMCAT_8_0_0_RC9/ The proposed 8.0.0-RC9 release is: [ ] Broken - do not release [X] Alpha - go ahead and release as 8.0.0-RC9 alpha Rémy
Re: [VOTE] Release Apache Tomcat 8.0.0-RC10
2013/12/19 Mark Thomas ma...@apache.org The proposed 8.0.0-RC10 release is: [ ] Broken - do not release [X] Alpha - go ahead and release as 8.0.0-RC10 alpha Rémy
Re: svn commit: r1556071 - /tomcat/trunk/java/org/apache/tomcat/websocket/server/WsRemoteEndpointImplServer.java
2014/1/8 Mark Thomas ma...@apache.org Tomcat only does this if the WriteListener is set via the CoyoteOutputStream. It does not do it if HTTP upgrade is used (which uses a lighter-weight ServletOutputStream). That looks like a bug that needs fixing. Yes, I saw after my commit there was code to do it for HTTP, but upgrade uses the code in the AbstractServletOutputStream which only sets the listener and doesn't do anything with it until there's an actual write. Rémy
Context switch integration
Hi, In Tomcat, entering a context is done simply by setting the context classloader and sometimes firing some random events (like the ones from the ServletRequestListener). This isn't so good for integration and could use improvements, in addition to at least one place where this is not done at all (very recently a bug reported to me that indicated there was an issue with SSO session expiration, and indeed it expires sessions from other contexts just like that). So I would propose harmonizing this (and fix the SSO issue too, obviously), with a new dedicated ThreadBindingListener interface (two methods: bind/unbind) and a get/set for it also in Context, to be used for integration. Although usually generic listeners are used (and there can be multiple ones), this has worked well for me so far. Ideally I would have liked to also take over the privileged actions in a utility class, but this would make PA use less optimal (for example in the request dispatcher). Comments ? (or better ideas ?) Rémy
Re: Context switch integration
2014/1/15 Mark Thomas ma...@apache.org If folks integrating with Tomcat need to extend / replace what is currently in StandardContext.bindThread() and StandardContext.unbindThread() having a listener for this rather than having to extend StandardContext makes sense to me. Ok. I'm not sure I follow what you are proposing for PAs. Nothing, if a utility class was doing the PA itself to make it easier and do PA + setContextCL + call the listener, there would be more PAs (maybe a performance impact). Rémy
Re: svn commit: r1558930 - in /tomcat/trunk: java/org/apache/catalina/ java/org/apache/catalina/authenticator/ java/org/apache/catalina/core/ java/org/apache/catalina/security/ java/org/apache/catalin
2014/1/17 Mark Thomas ma...@apache.org +protected static final ThreadBindingListener DEFAULT_NAMING_LISTENER = (new ThreadBindingListener() { +public void bind() {} +public void unbind() {} +}); +protected ThreadBindingListener threadBindingListener = DEFAULT_NAMING_LISTENER; What is the purpose of the above code? Would it not be simpler just to leave threadBindingListener as null? Nothing special, just less null checks. Rémy
Re: Git
2014/1/21 Mark Thomas ma...@apache.org On the down side: - there is much more potential to mess things up Yes, I can help with that if you want :) - cleaning up is potentially more complex - the disruption of the move - particularly if we want to move to a single git repo - could be significant Thoughts? Risky, but it can be attempted. I'm not very familiar with the infrastructure used though. git.apache.org is there already, it mirrors svn, and it is then mirrored again in github ? The repo to do the commits would then be git.apache.org ? Since we're considering big changes, what about Maven too ? I'll spare you the pros and cons. Rémy
Re: Git
2014/1/21 Mark Thomas ma...@apache.org Since we're considering big changes, what about Maven too ? I'll spare you the pros and cons. I still see more cons than pros for moving to Maven. It (mostly) works now and allows behaving better with bigger projects. IMO it is unavoidable now. Rémy
Re: Git
2014/1/21 Yoav Shapira yo...@apache.org Also +1 to separating the git vs svn discussion from the build system discussion (Maven, Ant, whatever.) (Side note: been using gradle recently, fun.) +1 Never used that Gradle yet though. Rémy
Re: Context switch integration
2014/1/22 Mark Thomas ma...@apache.org I've been looking at this some more with an eye to reducing code duplication. I think the thread binding listener could be merged with the ContainerListener. I can't see a good reason to keep them apart at this point. With this in mind, the changes I'm planning are: - Add bindThread() and unbindThread() to the Context interface - Add optional PA (as in callers can opt to use it if they wish) support to bindThread() and unbindThread() - Switch all the components currently implementing their own bind/unbind methods to use these new context methods - Switch container event type to use an enum - Add bind and unbind events to this new enum - Remove ThreadBindingListener The main benefit of this is that all the actions required on bind and unbind will be in a single place rather than duplicated (sometimes inconsistently) across the code base. +1 Rémy
Re: Context switch integration
2014/1/22 Konstantin Kolinko knst.koli...@gmail.com Thus I think that the calls to ContextBindings.bindThread(...) / unbindThread(...) as performed by those StandardContext methods are needed only during some initial set up. Using them as general methods would be a waste. Those methods are overly synchronized and may be a nuisance. It is true these two are only needed during start/stop, and they do more than what is otherwise needed. On the overuse of container events, they are called for configuration changes and (for whatever reason) all session updates. The session updates indeed mean setting one listener is a rather big impact for meany applications (if they use lots of session attributes). Ultimately, I added a dedicated thing because it is a thing EE needs so it got graduated to something more visible and straightforward to use. Last [not directly related], looking at the code, I notice that session expiration already takes care of changing the context CL. I was not aware of that, and it means I need to drop/rework my SSO patch. Oops. And I'll also rework the session code since it switches context even if not needed (in most cases, it is not needed). I'll do that when Mark is done. Rémy
Re: Context switch integration
2014/1/22 Mark Thomas ma...@apache.org I thought about that. There is some coupling between the entry and exit methods (the class loader) that the caller would need to retain and some standard try/finally plumbing. There is also an optimisation not to try to change the class loader if there is no need to do so. That looks like a good summary ! In an effort to reduce this common code for every caller, I started down a different route. I do wonder if the entry/exit approach might end up being cleaner as even with my approach, you end up with some not so nice exception handling. I'll see what folks think of the patch so far and maybe explore the entry/exit option as well. The first patch however looks more complicated than the dumb way IMO. Rémy
Re: Context switch integration
2014/1/23 Mark Thomas ma...@apache.org http://people.apache.org/~markt/dev/2014-01-23-tccl-tc8-v2.patch It was easier to write, removes more code and the result is easier to read. Unless there are objections I plan to commit this along with some additional work to migrate other code that is currently doing this itself. Ok, let's try that. Don't forget to keep SecurityClassLoad in sync. Rémy
Re: svn commit: r1560702 - in /tomcat/trunk/java/org/apache/tomcat/websocket: WsRemoteEndpointImplBase.java WsSession.java pojo/PojoEndpointBase.java
2014/1/23 Mark Thomas ma...@apache.org - Call onClose before actually closing anything (sending a close message closes the endpoint). -1 (veto) for a specification violation Section 2.1.5 requires that: the implementation must attempt to send the websocket close frame prior to calling the onClose() method of the websocket endpoint. Ok, but then onClose has the endpoint already closed, and it cannot do anything (like send messages). - After using a blocking send, clear the client buffer in case it gets reused (I have no idea why this is needed ...). This seems wrong (although I don't see the harm). I'd really expect the client to take care of that. Well, yes, I agree ! Rémy
Re: svn commit: r1560702 - in /tomcat/trunk/java/org/apache/tomcat/websocket: WsRemoteEndpointImplBase.java WsSession.java pojo/PojoEndpointBase.java
2014/1/23 Mark Thomas ma...@apache.org I'd need to go back and check the archives but the I think the expectation from the EG was that onClose() would be used to clean up resources rather than anything else. It is always possible that I have misunderstood the intention of the spec here but it looks to be pretty clear. Let's assume you didn't misunderstand anything for now. I think the important fix is calling onError if onClose throws an exception, I cannot find any item indicating that sending a message is mandatory. Rémy
Re: [VOTE] Release Apache Tomcat 8.0.0
2014-01-28 Mark Thomas ma...@apache.org The proposed Apache Tomcat 8.0.0 release is now available for voting. This is the first non-RC release so I am leaving all the stability options open to get an idea of where folks think we currently stand. The main changes since RC10 are: - Fix broken sendfile support in NIO - Add a readOnly option for WebResourceSet - Fix the regression in handling ternary expressions - Separate out JARs for EL, JSP and WebSocket for embedded - Remove svn keywords from source - Numerous bug fixes and improvements It can be obtained from: https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.0.0/ The Maven staging repo is: https://repository.apache.org/content/repositories/orgapachetomcat-1002/ The svn tag is: http://svn.apache.org/repos/asf/tomcat/tags/TOMCAT_8_0_0/ The proposed 8.0.0 release is: [ ] Broken - do not release [ ] Alpha - go ahead and release as 8.0.0 (alpha) [X] Beta - go ahead and release as 8.0.0 (beta) [ ] Stable - go ahead and release as 8.0.0 (stable) Rémy
Re: [VOTE] Release Apache Tomcat 8.0.1
2014-01-30 Mark Thomas ma...@apache.org: The proposed 8.0.1 release is: [ ] Broken - do not release [ ] Alpha - go ahead and release as 8.0.1 (alpha) [X] Beta - go ahead and release as 8.0.1 (beta) [ ] Stable - go ahead and release as 8.0.1 (stable) Rémy
Re: [VOTE][RESULT] Release Apache Tomcat 6.0.39
2014-01-31 Mark Thomas ma...@apache.org: stable (binding) : markt, rjung, kfujino, kkolinko stable (non-binding) : Ognjen Blagojevic This release vote therefore passes. Thanks to everyone who voted. I'll move the bits to dist/release now and announce this once the mirrors catch up. Given that that will take this into the weekend, it might slip a little. IMO, there's something wrong here. It is unprecedented to jump straight from alpha to stable [if someone heard of any major piece of software which did that, let me know ;) ], it just shouldn't be possible since there's never going to be enough testing during the alpha stage. Rémy
Re: [VOTE][RESULT] Release Apache Tomcat 6.0.39
2014-01-31 Mark Thomas ma...@apache.org: Check the subject Remy. This is for 6.0.39, not 8.0.1. Hum. Ok :D Rémy
Re: [VOTE] Release Apache Tomcat 8.0.3
2014-02-07 19:16 GMT+01:00 Mark Thomas ma...@apache.org: The proposed 8.0.3 release is: [ ] Broken - do not release [ ] Alpha - go ahead and release as 8.0.3 (alpha) [X] Beta - go ahead and release as 8.0.3 (beta) [ ] Stable - go ahead and release as 8.0.3 (stable) Rémy
NIO 2 connector
Hi, I've been working on porting a NIO2 connector that was originally developed for JBoss AS by Nabil Benothman (an intern at Red Hat). Due to the very different connector structure in Tomcat and my preference for basing it on the existing NIO1 connector, it is mostly new code, though. So I would now like to contribute this in trunk as a new experimental connector. It should have feature parity with the NIO1 connectors. Of course, while the basics are in, it will need some time to pass the testsuite, fix issues, improve stability, etc. Coyote version number could be moved up to 2.0 with this addition along with all the connector refactorings that Mark did in trunk. The code is there (rebased to the current trunk): https://github.com/rmaucher/tomcat A quick NIO2 (the API) presentation. Pros: - Significantly faster, although the API looks slower by design - Resources friendly - Seemingly trivial to use - Polling is neatly hidden away - Thread pool is also neatly hidden away - Per operation timeouts - Read/Write is symmetric - Trivial blocking IO Cons: - No real non blocking IO - No concurrency allowed [for the socket impl of NIO2] although the API looks concurrent - Simplicity is sometimes misleading, the API doesn't provide some tools for the needed synchronization, check pending operations and their possible optimizations - SSL is not integrated any better than with NIO1, it is still SSLEngine which leads to creating the obligatory AsynchronousSSLSocketChannel wrapper class yourself - No real sendfile Comments ? Rémy
Re: NIO 2 connector
2014-03-04 19:26 GMT+01:00 Mark Thomas ma...@apache.org: On 04/03/2014 16:16, Rémy Maucherat wrote: Hi, I've been working on porting a NIO2 connector that was originally developed for JBoss AS by Nabil Benothman (an intern at Red Hat). Due to the very different connector structure in Tomcat and my preference for basing it on the existing NIO1 connector, it is mostly new code, though. There looks to be opportunities to share code with the current NIO connector. Don't know. I used the structure, but the algorithms are quite different. You'll have the opportunity to do it if you want of course ;) There are some changes to the existing code, such as java/org/apache/coyote/http11/upgrade/AbstractServletInputStream.java that I would like to understand. Nothing special, it needs a first read to start its polling, so that's the reason for adding isReady() (normally harmless). I added the third event method too for consistency, and that's it. So I would now like to contribute this in trunk as a new experimental connector. It should have feature parity with the NIO1 connectors. Of course, while the basics are in, it will need some time to pass the testsuite, fix issues, improve stability, etc. Can you wait until we split 8.0.x from trunk or did you want to get this into 8.0.x? Depends, if you want to branch soon or not. It would have to be experimental for a while anyway, but it will likely bring something useful. Coyote version number could be moved up to 2.0 with this addition along with all the connector refactorings that Mark did in trunk. I'd rather drop the version number entirely as it is pretty much meaningless. If we are going to do that, we may as well change the server header to Apache Tomcat. I'm neutral on whether to include the major or full version number. I think the spec says there should be a /0.0 version number, and I like the Coyote name, but you can change it. The code is there (rebased to the current trunk): https://github.com/rmaucher/tomcat A quick NIO2 (the API) presentation. Pros: - Significantly faster, although the API looks slower by design - Resources friendly - Seemingly trivial to use - Polling is neatly hidden away - Thread pool is also neatly hidden away - Per operation timeouts - Read/Write is symmetric - Trivial blocking IO Cons: - No real non blocking IO Hmm. I was considering dropping the BIO connector entirely for Tomcat 9 because of its inability to do non-blocking IO and the way the Servlet API was heading. Does it make sense to add a different non-blocking connector implementation or have I misunderstood your point? Well, nobody is going for non blocking IO, people are using async IO. But then obviously the read(ByteBuffer) call which used to return an int (possibly 0 with non blocking IO) now returns a FutureInteger. And if you want to know the result you have to wait for it and there's an IO operation pending the entire time. So that's the cons. It would be nice to have non blocking operations for peek scenarios. So instead to peek a read, you have to read with a completion handler, then see if it returns inline. And if it doesn't there's an async process now doing IO (possibly not the greatest thing that could happen). - No concurrency allowed [for the socket impl of NIO2] although the API looks concurrent Do you mean no concurrent read and writes? That would be a huge issue. No, don't worry, that works. It's concurrency on the same operation (like read, write, accept). The API looks relatively magic, so you'd think you could do it. - Simplicity is sometimes misleading, the API doesn't provide some tools for the needed synchronization, check pending operations and their possible optimizations - SSL is not integrated any better than with NIO1, it is still SSLEngine which leads to creating the obligatory AsynchronousSSLSocketChannel wrapper class yourself - No real sendfile Comments ? The pros look nice but I am worried about the cons. The summary is that in the worst case, it's much better. Rémy
Re: NIO 2 connector
2014-03-04 21:51 GMT+01:00 Mark Thomas ma...@apache.org: Can you wait until we split 8.0.x from trunk or did you want to get this into 8.0.x? Depends, if you want to branch soon or not. It would have to be experimental for a while anyway, but it will likely bring something useful. I would like to branch soon. On the other hand if we hold this back until 9.0.x then it will be a lot longer before folks can really use it. It's very early in 8.0, so there would be plenty of time to backport if it is not included before branching. I thought you backported everything anyway. Basically, I can commit it as soon as there's general agreement on it, it's very easy in the meantime to keep rebasing on trunk. I'll try to work on improving the testsuite status as much as possible [I see some websockets tests failing - not a big surprise -, although a lot of it is working], and also see how I can remove that isReady from the AbstractServletInputStream. I think the spec says there should be a /0.0 version number, and I like the Coyote name, but you can change it. I'll try to remember to take a look at it. Maybe just change it to match the Tomcat major/minor version numbers. Ok. Thanks for the clarifications. I'm +1 with the following caveats: - no @author tags - exclude the change that makes NIO2 the default - document that the NIO2 connector is currently experimental - They came from the NIO1 classes I used as the basis for the new classes, should I remove them anyway ? - I changed that already. - Yes, I copied from the current NIO documentation, but experimental is not mentioned anywhere, which is obviously a mistake. I'll add a log during the endpoint start, so that it can't be missed, it's the best way (nobody reads the docs ...). Rémy
Re: svn commit: r1574167 - /tomcat/trunk/java/org/apache/catalina/connector/CoyoteAdapter.java
2014-03-05 11:07 GMT+01:00 Konstantin Kolinko knst.koli...@gmail.com: In my opinion, this is wrong. One should not skip access logging. If context==null, it can be logged via CoyoteAdapter.log(...) that logs into (ROOT app or Host or Engine). Ok, I'll try to improve it. I see the call at the end of event is unsafe as well, since it would skip recycling if something bad happens. There are several other places in CoyoteAdapter where context.logAccess(..) is called. Some are with similar NPE checks, some are not. Rémy
Re: NIO 2 connector
2014-03-04 17:16 GMT+01:00 Rémy Maucherat r...@apache.org: The code is there (rebased to the current trunk): https://github.com/rmaucher/tomcat Updated commit here: https://github.com/rmaucher/tomcat/commit/614d8c43d8d1f3eeb4d5e4c2493ead589a72bf2c I have removed the two main hacks and the testsuite status is relatively clean (some failures though, but some of theses are tests which have a timing which looks a bit too adapted for the other connectors; I did look individually at all the problems and made some fixes). So far there are 3 people in favor of merging this in the current trunk (still unbranched 8.0). Any additional comments ? Rémy
Re: NIO 2 connector
2014-03-07 23:16 GMT+01:00 Konstantin Kolinko knst.koli...@gmail.com: 1. It is a month since release 8.0.3 and thus I think 8.0.4 is expected in a week or so.. I am -1 to destabilize 8.0.x now. As a new component, it is not supposed to destabilize the other components that are in 8.0. To this, as you are saying that some tests do not pass. I woudn't want to make buildbot fail this close to a release, as we may miss something important. It will cause failures only if the tests are enabled for this connector at build time, there's no reason to do so at the moment since it is experimental. So I think the time to merge this is not earlier than 8.0.4 is voted and released + some time to cool off. I think that is about 10 days from now. 2. How am I supposed to review this? Is there some viewable patch against trunk? You just gave the link for the commit below, it shows all possible diffs, and it is rebased against trunk. The comments below are from a quick review of https://github.com/rmaucher/tomcat/commit/614d8c43d8d1f3eeb4d5e4c2493ead589a72bf2c 1) There are a number of unrelated changes, including some comment / typo fixes. That does not help reviewing. Why aren't those in Tomcat trunk yet? There's only a very small amount of them (maybe 3), it shouldn't hinder reviewing in any way. Actually, that's a good idea, I'll go ahead and commit the changes to the existing classes, since I consider they make some sense on their own. I'm not asking for a full review anyway, just some general feedback on the component. 2). There is a lot of code that from the first glance seems similar to Nio1 one. Just a matter of preference, not a stopper. The shared code can be move to a superclass eventually. But it is very different, a lot of major pieces of code from the NIO1 connector have no equivalent. (In your opening e-mail you say there are many differences. This similarity is my first impression. Maybe there aren't any. I'd need some time to compare). 3). What are the selling points of this implementation? I described some of the pros and cons of the NIO2 API. If the pros end up in the implementation without being degraded, it means the connector should be faster, more resources friendly, simpler (no poller, no selector, etc), since that's what happened with a similar NIO2 connector in JBoss. But I can't make any big promises yet. The documentation part of the patch does not say anything in particular. Also my -1 that it does not say that this connector is an experimental one. On startup, it logs: WARNING [main] org.apache.tomcat.util.net.Nio2Endpoint.startInternal The NIO2 connector is currently EXPERIMENTAL and should not be used in production So it is annoyingly visible, but I can add another mention in the docs. Rémy
Re: Tomcat 8: Firing listeners from within a synchronized(writeLock) in onWritePossible()
2014-03-08 21:16 GMT+01:00 Konstantin Kolinko knst.koli...@gmail.com: My question is that these events are fired while holding a synchronized (writeLock) { lock. Is holding the writeLock lock is needed here. The on write possible event is processed by webapp's code. I think that in theory the web application can delegate processing to some other thread. That other thread will hang trying to obtain the writeLock to perform the actual writing. I'm not convinced these write locks are needed, there should be no write concurrency in the first place. If the consensus is to keep the lock to try to avoid the most serious corruption issues, then in the onWritePossible method only the call to writeInternal would need to be locked, not the rest. Rémy
Re: Tomcat 8: Firing listeners from within a synchronized(writeLock) in onWritePossible()
2014-03-10 10:36 GMT+01:00 Mark Thomas ma...@apache.org: I'm not convinced these write locks are needed, there should be no write concurrency in the first place. The explanation for why the lock is required is in the comments in the writeInternal() method. The lock is definitely required. The issue described in that comment was observed when running the Autobahn WebSocket test suite. Actually, it could also happen with NIO2, so the lock on writeInternal is justified then. If the consensus is to keep the lock to try to avoid the most serious corruption issues, then in the onWritePossible method only the call to writeInternal would need to be locked, not the rest. It does look like the lock could be narrowed but I'm not sure it can be quite that narrow. The lock is effectively protecting buffer from concurrent access (some comments to that effect would not hurt). I'll add some more comments to that class and see what can be done to narrow that scope of the writeLock. Ok. Rémy
Re: NIO 2 connector
2014-03-10 14:03 GMT+01:00 Konstantin Kolinko knst.koli...@gmail.com: There are 52 warning shown by Eclipse IDE. Most of them are boxing/unboxing ones. To remind, the setting are documented here: res\ide-support\eclipse\java-compiler-errors-warnings.txt Ahah, ok, NIO2 does a lof of boxing/unboxing indeed. I'll try to improve that. I would like execute.test.nio2 property to be added to build.properties.default BUILDING.txt but that can be postponed a bit. It depends on how well the tests run for NIO2. Don't do it, it will fail. Very minor issues, but not necessarily easy to fix. Rémy
Re: buildbot failure in ASF Buildbot on tomcat-trunk
2014-03-10 22:16 GMT+01:00 build...@apache.org: The Buildbot has detected a new failure on builder tomcat-trunk while building ASF Buildbot. Full details are available at: http://ci.apache.org/builders/tomcat-trunk/builds/5571 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: bb-vm_ubuntu Build Reason: scheduler Build Source Stamp: [branch tomcat/trunk] 1576014 Blamelist: remm BUILD FAILED: failed compile_1 There's one failure with NIO2 and TestWebSocketFrameClient. It's working for me, and seems to be timeout related. Overall, I'm skeptical about timeout handling in upgrade mode, since it's pretending not to use IO timeouts, but NIO still has a write timeout (sometimes, depending on the code that was run before the upgrade). This particular test may be taking just a bit too long (the timeout is 5s). It completes just below 5s when I run it, but takes over 5s on the test server. Weird ... Rémy
Re: buildbot failure in ASF Buildbot on tomcat-trunk
2014-03-11 13:14 GMT+01:00 Konstantin Kolinko knst.koli...@gmail.com: The only test that fails is org.apache.tomcat.util.net.TestSsl. Its failure repeats consistently (all 6 of 6 runs), with NIO2. BIO,NIO,APR connectors do not fail. [[[ Testsuite: org.apache.tomcat.util.net.TestSsl Tests run: 4, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 6,697 sec - (...) - Testcase: testSimpleSsl took 2,721 sec Testcase: testKeyPass took 0,499 sec Testcase: testRenegotiateWorks took 3,418 sec Caused an ERROR Connection reset java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:196) at java.net.SocketInputStream.read(SocketInputStream.java:122) at sun.security.ssl.InputRecord.readFully(InputRecord.java:442) at sun.security.ssl.InputRecord.read(InputRecord.java:480) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927) at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:884) at sun.security.ssl.AppInputStream.read(AppInputStream.java:102) at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177) at java.io.InputStreamReader.read(InputStreamReader.java:184) at java.io.BufferedReader.fill(BufferedReader.java:154) at java.io.BufferedReader.readLine(BufferedReader.java:317) at java.io.BufferedReader.readLine(BufferedReader.java:382) at org.apache.tomcat.util.net.TestSsl.testRenegotiateWorks(TestSsl.java:206) That test never failed for me actually. It is a bit suspect that I've had almost no SSL problems overall with the new code, although reusing the NIO secure channel structure probably helped. Can you post the server exception (if any) ? Rémy
Re: buildbot failure in ASF Buildbot on tomcat-trunk
2014-03-11 14:30 GMT+01:00 Konstantin Kolinko knst.koli...@gmail.com: That test never failed for me actually. It is a bit suspect that I've had almost no SSL problems overall with the new code, although reusing the NIO secure channel structure probably helped. Can you post the server exception (if any) ? Which one? Do you mean some code change for that? In the logs there are no exceptions. The only one is in JUnit output, cited above. Ok. I meant TestSsl never failed for me. So it looks like there are some timing issues involved. All those tests are a bit egdy sometimes, so it's probably not surprising. Should I disable again the tesuite for NIO2 ? Rémy
Re: buildbot failure in ASF Buildbot on tomcat-trunk
2014-03-11 15:21 GMT+01:00 Konstantin Kolinko knst.koli...@gmail.com: Looking at org.apache.tomcat.util.net.TestSsl.testRenegotiateWorks(TestSsl.java:206) more closely, it is a broken test. +1 on the test being broken as it is now, for me it works because I don't get an IOE, but it is null instead. Normally the test returns a meaningful answer (with NIO, it does) so there's something broken. Rémy
Re: [GUMP@vmgump]: Project tomcat-trunk-test-nio2 (in module tomcat-trunk) failed
2014-03-12 15:44 GMT+01:00 Bill Barker billbar...@apache.org: Full details are available at: http://vmgump.apache.org/gump/public/tomcat-trunk/tomcat-trunk-test-nio2/index.html So, after debugging it, I can confirm there's an issue with the test TestWebSocketFrameClient.testConnectToServerEndpointSSL It does seem to be caused by a characteristic of NIO2, where the data has to be encrypted inline (unlike the NIO1 connector). I don't see any reasonable workaround for this. The test uses a lot of resources (2GB+ of RAM for me, plus significant CPU), so this processing time could spike. This then seems to cause a deadlock in the websockets code, which I have been able to reproduce 100% of the time by adding a Thread.sleep(20) at the beginning of Nio(2)ServletOutputStream.doWrite (for both NIO and NIO2 connectors, the behavior then becomes identical). So this can apparently could only affect heavily loaded servers using SSL. The deadlock is later broken up by the FutureSendHandler timeout. Although this failure only occurs with the NIO2 connector, I don't consider its behavior to be incorrect, as non blocking is not supposed to block on any IO operation, but this doesn't mean all calls need to have zero processing time. I'll try to investigate more this likely deadlock sometime later if Mark doesn't have any ideas about it. Rémy
Re: [VOTE] Release Apache Tomcat 8.0.4
2014-03-19 22:14 GMT+01:00 Mark Thomas ma...@apache.org: [X] Beta - go ahead and release as 8.0.4 (beta) [ ] Stable - go ahead and release as 8.0.4 (stable) This looks good enough to me to be Stable, but with this many fixes included it is likely there will be a couple hotfixes needed. I won't complain if it is released as Stable though. Rémy
Re: [VOTE] Release Apache Tomcat 8.0.4
2014-03-20 14:58 GMT+01:00 Mark Thomas ma...@apache.org: Unit tests hang for NIO2 on Windows (not an issue since NIO2 is still experimental) I got curious so I installed Java on my Windows. This didn't hang for me, and there were no failures on the big IO tests. Platform: Java 8 + Win 8.1-64 + your 8.0.4 source bundle (88s overdose ah) Issues: - Some tests fails on i18n on HTTP status other than 200 (I'll try to fix that) - Some SSL tests fail - Some other fails elsewhere (EL, org.apache.jasper.tagplugins.jstl.core.TestForEach compilation error, a UTF8 fail) So all except the i18n look caused by using Java 8. Complete FAIL list (except status line i18n): TEST-javax.el.TestResourceBundleELResolver.NIO2.txt:FAILED TEST-org.apache.catalina.startup.TestTomcat.NIO2.txt:FAILED TEST-org.apache.jasper.tagplugins.jstl.core.TestForEach.NIO2.txt:FAILED TEST-org.apache.jasper.tagplugins.jstl.core.TestForEach.NIO2.txt:FAILED TEST-org.apache.tomcat.util.buf.TestUtf8.NIO2.txt:FAILED TEST-org.apache.tomcat.util.net.TestCustomSsl.NIO2.txt:FAILED TEST-org.apache.tomcat.util.net.TestSsl.NIO2.txt:FAILED TEST-org.apache.tomcat.util.net.TestSsl.NIO2.txt:FAILED Rémy
Re: [VOTE] Release Apache Tomcat 8.0.4
2014-03-21 10:42 GMT+01:00 Konstantin Kolinko knst.koli...@gmail.com: I am not voting yet. The test suite did pass successfully for BIO,NIO,APR,NIO2 with JDK 7u51 (32-bit) on Windows 7, except for the following test with NIO2: TEST-org.apache.catalina.connector.TestCoyoteOutputStream.NIO2.txt [[[ Testcase: testNonBlockingWriteTwiceBlockingWriteOnce took 300,096 sec Caused an ERROR Works for me (like that SSL test ...). There are a number of the following messages in the log files (in 3 files for NIO2, in 1 file for BIO): java.util.concurrent.RejectedExecutionException: Executor not running, can't force a command into the queue Yes, I get some of them too. There's a completion handler waiting on keepalive, that gets a failed when closing the server socket, which seems normal to me. But there are no checks for the running state there, it will try to process the close. So that should be cosmetic, I think I'll check the best option between toning down the thing to debug level, adding a check for the running flag, etc. Example stack from the TestCoyoteOutputStream logs: 20-Mar-2014 14:48:14.605 WARNING [http-nio2-127.0.0.1-auto-4-exec-7] org.apache.tomcat.util.net.Nio2Endpoint.processSocket0 Executor rejected socket [org.apache.tomcat.util.net.Nio2Endpoint$Nio2SocketWrapper@3e7747dc:null] for processing java.util.concurrent.RejectedExecutionException: Executor not running, can't force a command into the queue at org.apache.tomcat.util.threads.TaskQueue.force(TaskQueue.java:63) at org.apache.tomcat.util.threads.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:180) at org.apache.tomcat.util.threads.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:155) at org.apache.tomcat.util.net.Nio2Endpoint.processSocket0(Nio2Endpoint.java:561) at org.apache.tomcat.util.net.Nio2Endpoint$1.failed(Nio2Endpoint.java:932) at org.apache.tomcat.util.net.Nio2Endpoint$1.failed(Nio2Endpoint.java:919) Those are in TEST-org.apache.catalina.connector.TestCoyoteOutputStream.NIO2.txt TEST-org.apache.coyote.http11.upgrade.TestUpgrade.BIO.txt TEST-org.apache.coyote.http11.upgrade.TestUpgrade.NIO2.txt TEST-org.apache.tomcat.websocket.TestWsWebSocketContainer.NIO2.txt According to the logs when RejectedExecutionException happens the protocol handler have been stopped and destroyed several msecs before that event. With websockets, I get some additional shutdown errors too. BTW, the stability vote is not about NIO2, it is experimental (and will remain that way for an indefinite amount of time, maybe until its removal if not successful enough). Rémy
Re: [VOTE] Release Apache Tomcat 8.0.4
2014-03-21 19:02 GMT+01:00 Christopher Schultz ch...@christopherschultz.net : It appears that the test has completely-stalled in NIO2 on Linux 2.6.32 x86_64 under Oracle's 1.7.0_45 JVM. I haven't run into trouble with that particular test yet. Some tests could use some timeout and clean fail, though, for example, this one has a while (true) that is never going to exit if it doesn't get its exception, and that's wrong (and the test also doesn't check any assertion, so it needs improvements). Does it always fail for you ? (you can quickly check it by using test.name =org/apache/catalina/connector/TestCoyoteAdapter.java) Rémy
Re: [VOTE] Release Apache Tomcat 8.0.4
2014-03-21 20:26 GMT+01:00 Christopher Schultz ch...@christopherschultz.net : I'll give it a try. It looks like I have another hang, the first time I (re)tried. Here is the tail of the console log: [junit] 21-Mar-2014 15:21:41.229 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler [http-nio2-127.0.0.1-auto-4] [junit] 21-Mar-2014 15:21:41.230 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service Tomcat [junit] 21-Mar-2014 15:21:41.230 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet Engine: Apache Tomcat/8.0.4 [junit] HTTP/1.1 200 OK [junit] Server: Apache-Coyote/1.1 [junit] Content-Type: text/plain;charset=UTF-8 [junit] Transfer-Encoding: chunked [junit] Date: Fri, 21 Mar 2014 19:21:40 GMT [junit] [junit] 4 [junit] TEST [junit] 21-Mar-2014 15:21:41.239 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler [http-nio2-127.0.0.1-auto-4-59301] [junit] 21-Mar-2014 15:21:41.239 WARNING [main] org.apache.tomcat.util.net.Nio2Endpoint.startInternal The NIO2 connector is currently EXPERIMENTAL and should not be used in production [junit] 4 [junit] TEST Sending SIGQUIT shows that the main method is again stuck in testBug54928: [junit]java.lang.Thread.State: TIMED_WAITING (sleeping) [junit] at java.lang.Thread.sleep(Native Method) [junit] at org.apache.catalina.connector.TestCoyoteAdapter.testBug54928(TestCoyoteAdapter.java:305) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) I can post the entire thread dump if that would be helpful. For now, I'm going to re-test with nio2 disabled. The test should now be improved and it shouldn't hang. Rémy
Re: svn commit: r1580821 - in /tomcat/trunk/webapps/docs: changelog.xml web-socket-howto.xml
2014-03-24 13:18 GMT+01:00 ma...@apache.org: Author: markt Date: Mon Mar 24 12:18:57 2014 New Revision: 1580821 URL: http://svn.apache.org/r1580821 Log: Fix https://issues.apache.org/bugzilla/show_bug.cgi?id=56304 WebSocket + BIO HTTP + Production = bad idea Shouldn't the BIO connector be deprecated in 8.0 ? That would be simpler to understand. Rémy
Re: Connectors, blocking, and keepalive
2014-03-24 18:08 GMT+01:00 Mark Thomas ma...@apache.org: Given that this feature offers little/no benefit at the price of having to run through a whole pile of code only to end up back where you started, I'm tempted to hard-code the return value of breakKeepAliveLoop() to false for BIO HTTP. Yes please [that's how it used to be]. The rule for that connector is one thread - one connection, that's its only way of doing something useful for some users. Rémy
Re: [VOTE] Release Apache Tomcat 8.0.5
2014-03-24 19:47 GMT+01:00 Mark Thomas ma...@apache.org: The proposed 8.0.5 release is: [ ] Broken - do not release [ ] Alpha - go ahead and release as 8.0.5 (alpha) [X] Beta - go ahead and release as 8.0.5 (beta) [ ] Stable - go ahead and release as 8.0.5 (stable) I expect it will need some more fixes before stable. Rémy
Re: Connectors, blocking, and keepalive
2014-03-25 15:57 GMT+01:00 Christopher Schultz ch...@christopherschultz.net : What about when an Executor is used, where the number of threads can fluctuate (up to a maximum) but are (or can be) also shared with other connectors? This is not really related, the connector stops using a thread when the connection closes, so if there are two java.io connectors sharing one executor, the thread count is the current connection count between the two connectors. Blocking on all io is a characteristic of java.io, and it's on its way to deprecation for that reason. This limitation should be accepted and embraced, attempts to work around it are mostly counter productive: the connector doesn't become more efficient, but its performance goes down. Rémy
Re: [VOTE] Release Apache Tomcat 8.0.5
2014-03-24 19:47 GMT+01:00 Mark Thomas ma...@apache.org: - Various NIO2 fixes Along with some other fixes, I cleaned up some shutdown issues. It now looks ok in Linux, but there's one known issue on Windows: https://bugs.openjdk.java.net/browse/JDK-7056546 When testing on Windows traces like this can happen on shutdown (they do happen a lot when running the testsuite): Exception in thread Thread-3 java.nio.channels.ShutdownChannelGroupException at sun.nio.ch.Invoker.invokeIndirectly(Invoker.java:210) at sun.nio.ch.Invoker.invoke(Invoker.java:176) at sun.nio.ch.Invoker.invoke(Invoker.java:285) at sun.nio.ch.WindowsAsynchronousSocketChannelImpl$ReadTask.failed(WindowsAsynchronousSocketChannelImpl.java:600) at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:399) at java.lang.Thread.run(Thread.java:744) So it is expected (the pending reads for keepalive cannot be stopped), and it cannot be caught. Rémy
Re: [ANN] ApacheCon NA 2013 Tomcat presentation videos available
2014-03-26 12:18 GMT+01:00 Mark Thomas ma...@apache.org: All, Most (all?) of the presentations from ApacheCon NA 2013 are now available on YouTube. To help you to navigate to the Tomcat presentations we have created this page: http://tomcat.apache.org/presentations.html Nice, I just went and read them. In the most recent presentation, one quick comment on the problem areas of Websockets on Servlets. Actually, I found after the EG discussion that autoblocking works fine as a solution with no deadlock, if it is really an internal autoblocking mechanism (like the semaphore use in NIO2). It is not perfect though, the timeout is different. Rémy
Re: [ANN] ApacheCon NA 2013 Tomcat presentation videos available
2014-03-26 14:52 GMT+01:00 Mark Thomas ma...@apache.org: Yes, it is a solvable problem as long as you drop the self-imposed requirement to run on top of the Servlet API. Now the difficulties are understood, I'm beginning to think about dropping that requirement and using some of the Tomcat internals directly. The blocking/non-blocking gets a whole lot easier very quickly if you do that. Ok. I don't think it would make much difference in the container code since the upgrade mode is light already and all it needs is to pass down a block flag, but your websocket impl would be able to shed its simulated blocking. Rémy
Java 8 support
Hi, As I was looking at 55988, the testsuite seems to have two issues with Java 8 at the moment: - test.name=org/apache/tomcat/util/buf/TestUtf8.java UTF-8 experts wanted ! - test.name=org/apache/jasper/tagplugins/jstl/core/TestForEach.java This would look like a JDT configuration issue or bug, it doesn't care about the default keyword in the newly added Iterator interface method Rémy
Re: Java 8 support
2014-03-26 17:10 GMT+01:00 Konstantin Kolinko knst.koli...@gmail.com: 1. Can you cite what the errors are? An error occurred at line: [22] in the generated java file: [C:\Work\apache-tomcat-8.0.5-src\output\test-tmp\work\Tomcat\localhost\test\org\apache\jsp\bug5\bug54242_jsp.java] The type new Iterator(){} must implement the inherited abstract method Iterator.forEachRemaining(Consumer) As tested on Windows for the for foreach tag plugin and the Iterator interface. Not a showstopper since the plugin can be disabled. And the other: Testcase: testJvmDecoder took 0 sec FAILED Valid sequence padded from one byte to two expected:2 but was:1 junit.framework.AssertionFailedError: Valid sequence padded from one byte to two expected:2 but was:1 at org.apache.tomcat.util.buf.TestUtf8.doTest(TestUtf8.java:401) at org.apache.tomcat.util.buf.TestUtf8.testJvmDecoder(TestUtf8.java:367) 2. There is a number of known issues in JDK 1.8.0 itself http://www.oracle.com/technetwork/java/javase/8-known-issues-2157115.html Ok. 3. BTW, I usually use test.entry property to run a single test class (as documented in BUILDING.txt), though test.name is more capable. Well, it worked for me. Rémy
Re: [VOTE] Release Apache Tomcat Native 1.1.30
2014-04-10 13:50 GMT+02:00 Mladen Turk mt...@apache.org: The Apache Tomcat Native 1.1.30 is [X] Stable, go ahead and release [ ] Broken because of ... Rémy
NIO2 connector status
Hi, With some fixes in, I think the status is now better than what the welcome message says, which is: The NIO2 connector is currently EXPERIMENTAL and should not be used in production In preparation for the next build, I would like to update it to: The NIO2 connector is currently BETA and should not be used in production It is now supposed to be doing semi useful things, but with possible remaining bugs. At least it can be tested. The known issue is that (possible) testsuite failure: test.entry=org.apache.tomcat.websocket.TestWebSocketFrameClientSSL test.entry.methods=testConnectToServerEndpoint (after removing the assertion) But I haven't been able to reproduce it despite lots of hacks to skew the timings. The most current theory given the symptoms is it would be a missing onWritePossible event (but no idea why it is SSL specific, and I did some theorical tightening which didn't improve anything so I'm not sure there's an issue with that). So if someone has better luck and (hopefully) has an idea how to fix it, I'm interested. Other than this one, the testsuite now seems very reliable on NIO2, which cannot hurt. Rémy
CI
Hi, I'm a bit confused about the infra for CI. - ci.apache.org seems down - http://vmgump.apache.org/gump/public/tomcat-trunk/ does mostly the same, isn't down but apparently doesn't have a build status history; also it has a tomcat-trunk-test-nio2http://vmgump.apache.org/gump/public/tomcat-trunk/tomcat-trunk-test-nio2/index.htmlwhich was added by someone Thanks, Rémy
Re: CI
2014-04-17 14:40 GMT+02:00 Martin Grigorov mgrigo...@apache.org: Hi, There are failed disks with BuildBot's master server (hostname: aegis). pctony (infra guy) said that they have ordered new ones and they will replace them today (US time ?!). Thanks for the info, but it looks like it is still dead. Rémy
NIO buffering
Hi, I am not convinced by the NIO buffering that is used on output. Due to concurrent access issues I couldn't use it in NIO 2, but then I cannot see either what it does to justify using a more complex structure over a simpler array list. If the idea was to reuse buffers (which it doesn't), there is no option except using a static buffer pool. Was that the original general idea around this deque structure ? Then the upgrade buffering is even more basic, but is probably not a performance issue, the current code is likely fast enough since it is so lightweight. Rémy
Re: NIO buffering
2014-04-23 16:50 GMT+02:00 Filip Hanik fi...@hanik.com: I am not convinced by the NIO buffering that is used on output. what are you exactly referring to? Maybe I can shed some light on it. Ok, so more precisely I was talking about the AbstractOutputBuffer.bufferedWrites field. Rémy On Fri, Apr 18, 2014 at 1:30 PM, Rémy Maucherat r...@apache.org wrote: Hi, I am not convinced by the NIO buffering that is used on output. Due to concurrent access issues I couldn't use it in NIO 2, but then I cannot see either what it does to justify using a more complex structure over a simpler array list. If the idea was to reuse buffers (which it doesn't), there is no option except using a static buffer pool. Was that the original general idea around this deque structure ? Then the upgrade buffering is even more basic, but is probably not a performance issue, the current code is likely fast enough since it is so lightweight. Rémy
Re: NIO buffering
2014-04-23 18:30 GMT+02:00 Mark Thomas ma...@apache.org: I'd agree with that assessment. I do remember having to be very careful with some of that code to get things working correctly. If there is a cleaner solution then I'd be all for it as long as performance is no worse. Mark [1] http://svn.apache.org/viewvc?view=revisionrevision=1358055 Well, it's fine, but I didn't see the point for me since the buffers were throw away and I didn't need to track their flip state. So the plan next for me would be to try a buffer pool for NIO2 [this is appropriate for NIO2 since scatter/gather can be used to access them]. No idea if it would actually be really faster. Rémy
Re: svn commit: r1589672 - in /tomcat/tc7.0.x/trunk: ./ java/org/apache/coyote/ajp/ java/org/apache/coyote/http11/ webapps/docs/
2014-04-24 15:04 GMT+02:00 Konstantin Kolinko knst.koli...@gmail.com: 2. AbstractEndpoint,unlockAccept() s.setSoLinger(getSocketProperties().getSoLingerOn(),getSocketProperties().getSoLingerTime()); I expect the above call to be broken because of NPE. I don't see the NPE, but that would explain the accept exception I was getting since the accept would still be going on. 3. It seems that APR connector is broken. Firefox fails to connect to http;//localhost:8080/ I am pretty sure it was not a mistake to disable linger, hence the -1 that was set. Linger would likely be harmful for a web server (sockets staying around for some time after being closed = sounds like a big resources drain). Rémy
Re: Time for 8.0.6
2014-04-26 21:38 GMT+02:00 Mark Thomas ma...@apache.org: The remaining open 8.0.x bugs are related to cookie parsing and I don't plan on addressing them until at least after the next release. The remaining open 7.0.x issue doesn't affect 8.0.x The remaining open 6.0.x issues don't affect 8.0.x Therefore, I plan to tag 8.0.6 shortly. Before I do that I want to check that the unit tests are OK on the usual platforms. +1 It could be useful to fix http://ci.apache.org/builders/tomcat-trunk if possible before. Rémy
Re: Time for 8.0.6
2014-04-27 0:41 GMT+02:00 Mark Thomas ma...@apache.org: Agreed, but that it out of our control. However, I'm seeing some fairly consistent failures with NIO2 and TestCoyoteAdapter.testBug54928 that I'd like to get to the bottom of. It doesn't (usually) fail if I run the test on its own but if I run testPathParamExtRootNoParam before it it usually fails. I'm seeing this on Windows when running on the command line but not when running through Eclipse. I also appear to see the same failure on OSX but haven't dug into the failure on that platform yet. This test never fails for me. However, the behavior is not identical to NIO (you need more output before getting an IOE, for some reason) so I added more wait. Rémy
Re: Time for 8.0.6
2014-04-28 11:12 GMT+02:00 Mark Thomas ma...@apache.org: This test never fails for me. However, the behavior is not identical to NIO (you need more output before getting an IOE, for some reason) so I added more wait. Oddly, it only fails for me on a VM. I'm still trying to get to the bottom of this. There's room for a purely IO issue related to your VM there, since the test relies on an IO exception being triggered when a client disconnects. CI/gump never failed that test I think [after I increased a bit the delay]. Rémy
Re: Time for 8.0.6
2014-04-27 1:03 GMT+02:00 Konstantin Kolinko knst.koli...@gmail.com: If someone is able to edit Buildbot configuration, maybe it is possible to skip that RAT report publishing step? The build is now going further, but it seems a folder is missing. http://ci.apache.org/builders/tomcat-trunk/builds/39/steps/MasterShellCommand/logs/stdio Rémy
Re: Proposed patch for JDTCompiler
2014-05-06 19:25 GMT+02:00 Christopher Schultz ch...@christopherschultz.net : Since this is not really a bug per se, I decided not to create a BZ issue. If it would be easier for everyone if I attached the patch to a BZ issue I'm happy to do that as well. Ok, but this is not a review then commit branch, so you can commit it without asking. I think it is a good idea to ask for comments for features and significant refactorings, but this one looks like a regular enhancement. Rémy
Re: svn commit: r1593303 - /tomcat/trunk/java/org/apache/coyote/http11/AbstractHttp11Processor.java
2014-05-08 16:55 GMT+02:00 ma...@apache.org: Author: markt Date: Thu May 8 14:55:08 2014 New Revision: 1593303 URL: http://svn.apache.org/r1593303 Log: Fix test failure with NIO2 where additional, unexpected access log entry was being created during connector shutdown. I didn't run into any problem with the testsuite, and I don't see a direct relation with the connector shutdown, can you give me the trace and details ? Maybe that catch is the best place to do it, but I'd like to be sure. Thanks, Rémy
Re: Plans for 8.0.6
2014-05-12 14:33 GMT+02:00 Mark Thomas ma...@apache.org: Over the weekend I ran the unit tests for BIO, NIO and APR on OSX, Win64 and Linux64 and they all passed. My plans for 8.0.6 are therefore: - work through the backlog of messages that built up during the mail server outage; - fix any new issues that might emerge from the backlog - re-run the unit tests - tag 8.0.6 and start the release +1 (the changelog is now significant, but there doesn't seem to be any testsuite regression - I did run it often during the time buildbot was down) In the backlog, I had a question about read pending. Rémy
Re: svn commit: r1593303 - /tomcat/trunk/java/org/apache/coyote/http11/AbstractHttp11Processor.java
2014-05-12 16:25 GMT+02:00 Mark Thomas ma...@apache.org: In terms of whether there is a better place, I didn't look lower down the stack. We probably only want to swallow this if we haven't read any part of the request line. Ok, I'll add back the two try/catch I removed, I should have cleaned up the bad cases where the completion handler from the buffer is pending, but the processor is recycled. Hopefully ;) (if not, hey, it's still beta) Rémy
Re: Plans for 8.0.6
2014-05-12 16:39 GMT+02:00 Mark Thomas ma...@apache.org: I just got to that. I've answered your immediate question but I want to do a little more digging of my own. I'll respond in more detail on that thread. That said, I'm not expecting NIO2 to be declared stable in this release so I don't see this as a release blocker. Ok. I looked at it more, and I think the comment you made is correct: on shutdown it would simply try to read data (but the read for the keepalive is pending, so it gets an exception). So I'm not reverting anything, the code as is won't do anything that would cause real issues even if it's not perfect. The main idea was to avoid ignoring read pending exceptions. Rémy
Re: Plans for 8.0.6
2014-05-13 9:37 GMT+02:00 Mark Thomas ma...@apache.org: I'm running the unit tests now. I want to look at BZ 56516 some more and I don't want to hold up a 8.0.x release any longer. There's nothing to look at IMO, the guy is using scriptlets and expects things to work with the tag variable. Jasper has no idea to know there's flow control in his Java code fragments. Rémy From his test case (with a message variable info): % try { % hello:helloWorld id=message/ % } catch (Exception e) { //String message = null; % hello:helloWorld id=message/ % } %
Re: svn commit: r1594436 - in /tomcat/trunk: java/org/apache/catalina/connector/ java/org/apache/coyote/ java/org/apache/coyote/ajp/ java/org/apache/coyote/http11/ webapps/docs/
2014-05-14 2:44 GMT+02:00 kkoli...@apache.org: Author: kkolinko Date: Wed May 14 00:44:33 2014 New Revision: 1594436 URL: http://svn.apache.org/r1594436 Log: Fix https://issues.apache.org/bugzilla/show_bug.cgi?id=56399 When recycling a Coyote request, ensure that Catalina request have been recycled. This shouldn't cause regressions (cosmetic or performance). So this needs to be refined, and shouldn't be committed for now. I'll have to -1 this commit until it doesn't cause issues. The code can be left in instead, but the calls to getAdapter().checkRecycled(request, response) could be disabled with a system property flag. Rémy
Re: [VOTE] Release Apache Tomcat 8.0.6
2014-05-14 13:39 GMT+02:00 Mark Thomas ma...@apache.org: - Further NIO2 fixes which is now considered BETA Thanks for the testing and fixes. The proposed 8.0.6 release is: [ ] Broken - do not release [ ] Alpha - go ahead and release as 8.0.6 (alpha) [X] Beta - go ahead and release as 8.0.6 (beta) [ ] Stable - go ahead and release as 8.0.6 (stable) I would like to vote stable, but the changelog is quite big. Maybe another build following relatively quickly and focusing mostly on serious bugs found (if any) could do it. Rémy
Re: [VOTE] Release Apache Tomcat 8.0.6
2014-05-16 14:42 GMT+02:00 Jeanfrancois Arcand jfarcand@gmail.com: Seems https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.0.6/ is private as I do get a Not Found https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.0.7/ But I would have waited a bit for some more testing. Rémy
Re: svn commit: r1594436 - in /tomcat/trunk: java/org/apache/catalina/connector/ java/org/apache/coyote/ java/org/apache/coyote/ajp/ java/org/apache/coyote/http11/ webapps/docs/
2014-05-15 14:57 GMT+02:00 Mark Thomas ma...@apache.org: Konstantin, (bcc'd directly as list mail still seems to be delayed by 8-12 hours) It looks like I am going to need to tag a new 8.0.x release as DBCP is currently broken. Please can you revert this commit as per Remy's veto ASAP so I can tag 8.0.7 without this change present. If r1594436 isn't reverted by the time I come to tag 8.0.7 I will revert it myself (but I'd rather not have to do that). There's some effort to improve it, and it's not too visible (the classloader log is far more problematic), so I'm withdrawing the veto. Rémy
Re: [VOTE] Release Apache Tomcat 6.0.40
2014-05-15 16:14 GMT+02:00 Mark Thomas ma...@apache.org: The proposed 6.0.40 release is: [ ] Broken - do not release [X] Stable - go ahead and release as 6.0.40 Stable Great job on clearing up all the issues. Rémy
Re: [VOTE] Release Apache Tomcat 8.0.8
2014-05-16 22:55 GMT+02:00 Mark Thomas ma...@apache.org: The proposed 8.0.8 release is: [ ] Broken - do not release [ ] Alpha - go ahead and release as 8.0.8 (alpha) [X] Beta - go ahead and release as 8.0.8 (beta) [ ] Stable - go ahead and release as 8.0.8 (stable) Looking good. Rémy
Re: [VOTE] Release Apache Tomcat 8.0.8
2014-05-19 14:51 GMT+02:00 Mark Thomas ma...@apache.org: Unit tests all pass for BIO, NIO and APR/native on Linux, OSX and Windows with 64-bit Java. NIO2 included ? I've come the conclusion the the cookie work - if done carefully - can be done without destabilising 8.0.x therefore I am voting stable. That would be a first ;) Maybe with the zillion weird unit tests that were added it is now possible, if you are willing to ignore the deluge of bug reports about formerly working cookies. Rémy
Re: [VOTE] Release Apache Tomcat 6.0.41
2014-05-19 14:58 GMT+02:00 Mark Thomas ma...@apache.org: The proposed 6.0.41 release is: [ ] Broken - do not release [X] Stable - go ahead and release as 6.0.41 Stable Rémy
Re: [VOTE] Release Apache Tomcat 7.0.54
2014-05-20 12:04 GMT+02:00 Violeta Georgieva violet...@apache.org: The proposed 7.0.54 release is: [ ] Broken - do not release [X] Stable - go ahead and release as 7.0.54 Stable Rémy
Re: Error handling
2014-05-29 14:31 GMT+02:00 Mark Thomas ma...@apache.org: There are four variations: 1. Content length set, some content written, response not committed. 2. Content length set, some content written, response committed. 3. Content length not set, some content written, response not committed. 4. Content length not set, some content written, response committed. Variation 1 3 are easy. The response has not been committed so the ErrorReportingValve resets the response and writes an error response instead. In both cases it is clear to the client that there was an error. Variation 2 is not handled so well. The ErrorReportValve can't set the status code nor can it write content so an incomplete response is sent to the client. The client then waits for the rest of the response (since the content length header is set) and eventually times out (actually I think Tomcat times out the connection waiting for the next request). Variation 4 is also not handled well. The ErrorReportValve can't set the status code nor can it write content so an incomplete response is sent to the client. Because chunked encoding is being used, Tomcat ends the request with an end chunk so, to the client, it looks like a complete response. Depending on the content, it may or may not be clear to the client that an incomplete response has been received. In this particular case, I did it on purpose, I thought I had the opportunity to return a well formed response, but no possibility in the protocol to report an error, hence the behavior. Now what you say makes sense, so if you want to change it ... Rémy
Re: svn commit: r1598483 - in /tomcat/trunk/webapps: docs/changelog.xml examples/WEB-INF/classes/websocket/echo/EchoAsyncAnnotation.java examples/WEB-INF/classes/websocket/echo/EchoStreamAnnotation.ja
2014-05-31 0:47 GMT+02:00 Konstantin Kolinko knst.koli...@gmail.com: I fixed svn:eol-style and other issues in r1598763 Sorry for that problem. What I do not like here is that it is not clear how to use these classes. They are not referenced in public examples html pages, not used in unit tests, have no javadoc, Maybe add Javadoc to them? Well, I only just really found out about using that test, and you basically have to install it and run it with the given config. But it is complex to explain, and IMO it is the job of its documentation rather than Tomcat's doc. Rémy
Re: svn commit: r1600950 - in /tomcat/trunk: java/org/apache/tomcat/util/net/Nio2Endpoint.java webapps/docs/changelog.xml
2014-06-06 18:14 GMT+02:00 r...@apache.org: Author: remm Date: Fri Jun 6 16:14:44 2014 New Revision: 1600950 URL: http://svn.apache.org/r1600950 Log: Try removing the beta tag from the NIO2 connector to get more feedback, after the latest round of fixes. In practice since it is not used by default, it doesn't change much. So if someone wants to keep it as beta, let me know before the tag. Rémy
Re: svn commit: r1600950 - in /tomcat/trunk: java/org/apache/tomcat/util/net/Nio2Endpoint.java webapps/docs/changelog.xml
2014-06-06 18:39 GMT+02:00 Konstantin Kolinko knst.koli...@gmail.com: 1. Doesn't it feel too early? Maybe, but tests pass and no reports. 2. You only removed a startup warning. I think the documentation still labels it as beta. This could be a good middle ground. Rémy
Re: svn commit: r1602159 - in /tomcat/trunk/java/org/apache/catalina/loader: LocalStrings.properties LocalStrings_es.properties WebappClassLoader.java
2014-06-12 15:54 GMT+02:00 Mark Thomas ma...@apache.org: -1. These log messages indicate problems that users need to do something to fix. While I've no problem with reducing error - warning, logging them at info is not acceptable. Ok. This indicates a potential leak, about which the user may or may not care about. This is never an error however, and I think warn is also too high since the user may not even be able to fix it. Rémy
Re: svn commit: r1603274 - /tomcat/trunk/webapps/examples/WEB-INF/classes/websocket/echo/EchoAsyncAnnotation.java
2014-06-17 22:36 GMT+02:00 Mark Thomas ma...@apache.org: There are some large messages in the Autobahn tests (multi-MB). On reflection I think it will be easier to just disable it again. Good job on fixing the example, I couldn't see anything wrong with it ! Besides that IMO it should remain disabled, it is nice that it does something different (buffering) but then it can use an arbitrary amount of memory which looked like a dos to me. Rémy
Re: [VOTE] Release Apache Tomcat 8.0.9
2014-06-19 16:00 GMT+02:00 Mark Thomas ma...@apache.org: The proposed 8.0.9 release is: [ ] Broken - do not release [ ] Alpha - go ahead and release as 8.0.9 (alpha) [ ] Beta - go ahead and release as 8.0.9 (beta) [X] Stable - go ahead and release as 8.0.9 (stable) Tests are ok, including now autobahn as well. Rémy
Re: [PROPOSAL] Drop comet support in Tomcat 9 onwards
2014-06-18 23:40 GMT+02:00 Mark Thomas ma...@apache.org: I have been thinking about connector refactoring for Tomcat 9. Currently BIO is planned to be dropped due to the number of features in the servlet spec that rely on non-blocking IO. I would like to propose dropping support for Comet in Tomcat 9 onwards. The reasons are: - WebSocket is already far more popular (based on question volume on the users list) - Comet support adds a fair amount of complexity to the connector code Thoughts? Keep in mind that 8.0.x isn't stable yet and any 9.0.x is likely to be several years away. Yes, of course, that API is only a precursor to Servlet 3.1, and is now fully replaced. Most likely existing code can be ported over to Servlet 3.1 rather easily. Rémy
Re: Possible TomEE release news?
2014-06-24 20:02 GMT+02:00 David Blevins david.blev...@gmail.com: Wonder what people's thoughts are about possibly including TomEE releases in the Tomcat news feed on occasion? Having some mention of TomEE on tomcat.apache.org would be nice -- most users still do not know it exists. However, I've never mentioned it as I personally couldn't think of a good way to do that that didn't feel obtrusive or unnatural. This might be more palatable and less permanent. I remember Rémy did this some many years ago for an OpenEJB release once and it really helped. We have a TomEE 1.7.0 release coming out soon we could try it with. Thoughts? +1 for TomEE specifically, which looks a lot like Tomcat and is from the ASF. However, -1 for the others like Geronimo (as proposed by Konstantin) which are very different from Tomcat. There's little relationship besides some embedding, while the idea would be to point somewhere for Tomcat users asking for Tomcat + CDI, JSF, etc. Rémy
Re: svn commit: r1605723 - /tomcat/trunk/java/org/apache/catalina/webresources/StandardRoot.java
2014-06-26 13:32 GMT+02:00 Konstantin Kolinko knst.koli...@gmail.com: What are the circumstances when this NPE happens? StandardRoot.initInternal() throws an IllegalStateException if the context is null. How can it be without a context? Avoid NPE with storeconfig: storeconfig creates instances to determine default values for properties. Rémy