Re: AccessLogValve performance fixes
Yep! That's a full 10 seconds slower than the new version, making the new version about 44% faster. Great ! I'll commit the patch later today. This test is single-threaded, but I think it still shows noticeable performance improvement. Something else that I could do with this that I haven't yet is make the valve able to buffer log file output. From my tests, it looks to me like this make the test run about 4 seconds faster, a maximum of 1.6% gain (not real big, but it's something). If it's really 4s out of 13s, isn't it more than 1.6% ? If the gain is another 4s, then it's significant, and I think we should go for it. Remy
Re: AccessLogValve performance fixes
Remy Maucherat wrote: Yep! That's a full 10 seconds slower than the new version, making the new version about 44% faster. Great ! I'll commit the patch later today. This test is single-threaded, but I think it still shows noticeable performance improvement. Something else that I could do with this that I haven't yet is make the valve able to buffer log file output. From my tests, it looks to me like this make the test run about 4 seconds faster, a maximum of 1.6% gain (not real big, but it's something). If it's really 4s out of 13s, isn't it more than 1.6% ? If the gain is another 4s, then it's significant, and I think we should go for it. Remy Okay, that's one way to look at it. I was thinking 4s out of 24s, but now that you mention it, it's now 4s out of 13s. So, I'll work on that too. About buffering, I was thinking that not everyone would want the output buffered.. If you're debugging and you're trying individual requests and looking at the log information for each request, and the log lines don't show up in the log file after you make the request, that can be annoying/puzzling to the person trying to debug. I thought it might be a good idea if you could configure the valve to buffer or not buffer. Buffering should probably be turned on as the default. But, as long as the config says something like 'buffering="true"' then at least people will take note that buffering is turned on for that log file (it can serve as a sort of a warning). So, by default, it'll be fast, and the buffering could be turned off. That would be nice -- as long as this switchable feature does not really degrade performance. Ideas? -- Jason Brittain Software Engineer, Olliance Inc.http://www.Olliance.com Current Maintainer, Locomotive Project http://www.Locomotive.org
Re: AccessLogValve performance fixes
Quoting "Craig R. McClanahan" [EMAIL PROTECTED]: Logging messages in some other environments -- and even Tomcat 3.x IIRC -- are actually written out by a background thread, rather than the main request-processing thread (as it currently is with Catalina, or at least it was last time I looked :-). Have you thought about implementing such a thing? Actually, I was about to propose that we add asynchronous logging :) If we go that way, some thought might be spent on perhaps consolidating background writing for all Loggers into some number of threads less than the total number of loggers -- to improve scalability in large installations. Remy
Re: AccessLogValve performance fixes
Jason Brittain wrote: It sounds like a good idea. While I was working on this valve I was thinking that it would be great to queue up the log messages for another thread, but I first wanted to quickly get these lighter optimization changes into the codebase. Sounds like a reasonable plan. Yes, it would be great for Catalina to offer asynchronous logging. If we're going to do that, I'd propose we do it in some pluggable fashion. Which means we should also look at stuff like Log4J and so on. There was also a recent Java Specification Request for a standardized logging API -- I haven't followed where this is in the process, but it should be findable on the JCP web site http://java.sun.com/jcp. Really, what I'd like to see is for Avalon to offer asynchronous logging and for Catalina to plug into that. But, I'm sure lots of folks who use Catalina by itself (without Avalon) would like Catalina itself to have asynchronous logging code of its own. We could always do both (especially since Avalon and Catalina use the same license).. ideas? There has also been some work to put (optional) Avalon wrappers around Catalina. This is certainly a place we might be able to leverage Avalon more. So Craig, you're saying that Tomcat 3.2 does this now? Well, I remember that Alex Chaffee was working on some logging enhancements over the summer -- but it appears that they are *not* in the 3.2 code base. They are probably in the 3.3-dev branch, though, so we can still go harvest 'em. -- Jason Brittain Software Engineer, Olliance Inc.http://www.Olliance.com Current Maintainer, Locomotive Project http://www.Locomotive.org Craig
