RE: AccessLogValve enhancement
Ok, I have some more time to work on this. Though the tricky part is to find out what to delete. I was just using anything that started with the 'prefix'. This doesn't seem too bad to me since if you want to delete old files, you just need to pick a good prefix. In my test case, I set the file format to have the minute and second in the file name. The rolling worked perfectly. it is possible to use it to generate different directories for different dates. This does put a damper on things. The valve could easily remember previous files, but files from previous executions would be forgotten, which doesn't seem right to me. Reliably getting a list of files to delete in all situations seems like a pretty complex task. However, if we force the idea of a good prefix selection, something similar could be done to what is in the patch code already, it would just have to be recursive. Then again, this is starting to have a few too many caveats. I had wondered about removing old files for a while, and had seen other people ask this as well, so I thought a simple solution might be possible. However, now I am rethinking this. Coming back to allowing the use of a standard logger. In my mini-benchmarks, it didn't seems like it made much of a difference time wise; the time spent in the logger took between 1-2 millis on my server. This seems like it would be dwarfed by the time to actually create and deliver the response. Still, maybe this is a problem that most people have solved their own way already and people don't want to have solved again. I don't want to make anything more complicated or harder to use. I can continue to work on this, but I am losing confidence that I can come up with something that will be worth of adoption. Comments? -Original Message- From: Konstantin Kolinko [mailto:knst.koli...@gmail.com] Sent: Thursday, March 29, 2012 4:44 PM To: Tomcat Developers List Subject: Re: AccessLogValve enhancement 2012/3/29 Mark Claassen ma...@donnell.com: After thinking about this some more, and deliberating on the subtle issues that were coming up, I decided to reduce the scope of my enhancement. I still like the idea of using a standard logger for this logging, but for now I thought I would focus more on my specific problem. I wrote more about this in the issue. https://issues.apache.org/bugzilla/show_bug.cgi?id=52688 Is this something that could be adopted in a future 7.0 release? To delete or otherwise rotate (move, compress, e-mail, etc.) old files - I think I would write a Listener, or set up a shell script in cron. When logger starts to write a new file it has to do so quickly, because it happens in a synchronized block. Deleting is possible, because it is a quick operation as well. Though the tricky part is to find out what to delete. The files are named by date and they are not created unless necessary. The date format is configurable. E.g. it is possible to use it to generate different directories for different dates. IIRC JRE's java.util.logging.FileHandler can be configured to keep n old files, but it uses different naming scheme. Best regards, Konstantin Kolinko - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
RE: AccessLogValve enhancement
After thinking about this some more, and deliberating on the subtle issues that were coming up, I decided to reduce the scope of my enhancement. I still like the idea of using a standard logger for this logging, but for now I thought I would focus more on my specific problem. I wrote more about this in the issue. https://issues.apache.org/bugzilla/show_bug.cgi?id=52688 Is this something that could be adopted in a future 7.0 release? -Original Message- From: Mark Claassen [mailto:markclaass...@gmail.com] Sent: Tuesday, March 27, 2012 11:17 AM To: Tomcat Developers List Subject: Re: AccessLogValve enhancement That improved my timings considerably. Instead of between 300 - 1100, they are between 100 - 330. However, they are still slower than the current mechanism. (Although, I am not sure how much this benchmark is worth, anyway.) I would imagine that the Async logger has lots of optimizations in it to handle the threading aspects; I wonder if it would be better under load than the current mechanism? This info is not going to catalina.out. Other comments? Suggestions for direction: 1) I can add my outputLoggerName attribute, which will switch the valve to use the JULI logger 2) I can create some sort of AccessLogWriter that the AccessLogValves will delegate their writing to - With 2 implementations, one being the current method and one being a JULI implementation - Potentially removing one in the future if performance is not an issue. 3) I can just add an attribute to the AccessLogValves to automatically delete older files 4) I can just make my own private subclass to do what I want, and leave the tomcat code unaltered 5) Scrap the current file writing implementation in the AccessLogValve and go straight JULI I would welcome any advice. Mark On Tue, Mar 27, 2012 at 9:46 AM, Konstantin Kolinko knst.koli...@gmail.comwrote: 2012/3/27 Mark Claassen markclaass...@gmail.com: I did a quick performance test, and it is true that the default mechanism is quite a bit faster than the standard JULI logger. For this test, I just modified the source code to write each log message 1000 times. The first set of timings (in millis) is from the current AccessLogValve, and the second set is using JULI. AccessLogValve Elapsed Time: 8 AccessLogValve Elapsed Time: 19 AccessLogValve Elapsed Time: 63 AccessLogValve Elapsed Time: 6 AccessLogValve Elapsed Time: 7 AccessLogValve Elapsed Time: 8 INFO: AccessLogValve Elapsed Time: 830 INFO: AccessLogValve Elapsed Time: 1122 INFO: AccessLogValve Elapsed Time: 451 INFO: AccessLogValve Elapsed Time: 531 INFO: AccessLogValve Elapsed Time: 764 INFO: AccessLogValve Elapsed Time: 347 1. Try to configure JULI with org.apache.juli.AsyncFileHandler. It might show better numbers. (The usual FileHandler by default performs flush() after each log message). 2. Is the same printed to console? If yes then remove ConsoleHandler. Best regards, Konstantin Kolinko - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: AccessLogValve enhancement
2012/3/29 Mark Claassen ma...@donnell.com: After thinking about this some more, and deliberating on the subtle issues that were coming up, I decided to reduce the scope of my enhancement. I still like the idea of using a standard logger for this logging, but for now I thought I would focus more on my specific problem. I wrote more about this in the issue. https://issues.apache.org/bugzilla/show_bug.cgi?id=52688 Is this something that could be adopted in a future 7.0 release? To delete or otherwise rotate (move, compress, e-mail, etc.) old files - I think I would write a Listener, or set up a shell script in cron. When logger starts to write a new file it has to do so quickly, because it happens in a synchronized block. Deleting is possible, because it is a quick operation as well. Though the tricky part is to find out what to delete. The files are named by date and they are not created unless necessary. The date format is configurable. E.g. it is possible to use it to generate different directories for different dates. IIRC JRE's java.util.logging.FileHandler can be configured to keep n old files, but it uses different naming scheme. Best regards, Konstantin Kolinko - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: AccessLogValve enhancement
Ok. I am back working on this again. From the previous comments, I am not sure my efforts are going in the right direction. However, I have a working implementation that uses the standard logging mechanism now, while addressing some of the previous concerns. I did a quick performance test, and it is true that the default mechanism is quite a bit faster than the standard JULI logger. For this test, I just modified the source code to write each log message 1000 times. The first set of timings (in millis) is from the current AccessLogValve, and the second set is using JULI. AccessLogValve Elapsed Time: 8 AccessLogValve Elapsed Time: 19 AccessLogValve Elapsed Time: 63 AccessLogValve Elapsed Time: 6 AccessLogValve Elapsed Time: 7 AccessLogValve Elapsed Time: 8 INFO: AccessLogValve Elapsed Time: 830 INFO: AccessLogValve Elapsed Time: 1122 INFO: AccessLogValve Elapsed Time: 451 INFO: AccessLogValve Elapsed Time: 531 INFO: AccessLogValve Elapsed Time: 764 INFO: AccessLogValve Elapsed Time: 347 Granted, this certainly falls into the Contrived micro-benchmark realm. A real test would involve lots of threads each writing very little. Also, each thread would be doing a lot of other things, making the time spent in the access logger less significant. However, even in the worse case, it still only took a millis to write one message. Honing my test case to test the speed of 1 single message (not just the write, but the whole process of creating and then writing the message), I got that it takes between 0 - 1 millis in the first case, and 0 - 2 millis with JULI. Most messages took 1 millis for each. (I have a pretty old machine I am testing on now.) I didn't test the speed of the file rolling and such. Also, I have no idea which would perform better under a heavy load. I don't really have the facilities to do a realistic performance test. However, I think something can be gleaned from the above results. Does anyone have any advice or suggestions on what direction I should go? My main goal is to have automatic deletion of old log files. This is something I would imagine others would like as well, so I thought I would present here instead of just doing something myself. I certainly do not what to make anything more complicated to use / configure than it has to be. I am considering 5 options: 1) I can add my outputLoggerName attribute, which will switch the valve to use the JULI logger 2) I can create some sort of AccessLogWriter that the AccessLogValves will delegate their writing to - With 2 implementations, one being the current method and one being a JULI implementation - Potentially removing one in the future if performance is not an issue. 3) I can just add an attribute to the AccessLogValves to automatically delete older files 4) I can just make my own private subclass to do what I want, and leave the tomcat code unaltered 5) Scrap the current file writing implementation in the AccessLogValve and go straight JULI I would welcome any advice. Thanks, Mark
Re: AccessLogValve enhancement
That improved my timings considerably. Instead of between 300 - 1100, they are between 100 - 330. However, they are still slower than the current mechanism. (Although, I am not sure how much this benchmark is worth, anyway.) I would imagine that the Async logger has lots of optimizations in it to handle the threading aspects; I wonder if it would be better under load than the current mechanism? This info is not going to catalina.out. Other comments? Suggestions for direction: 1) I can add my outputLoggerName attribute, which will switch the valve to use the JULI logger 2) I can create some sort of AccessLogWriter that the AccessLogValves will delegate their writing to - With 2 implementations, one being the current method and one being a JULI implementation - Potentially removing one in the future if performance is not an issue. 3) I can just add an attribute to the AccessLogValves to automatically delete older files 4) I can just make my own private subclass to do what I want, and leave the tomcat code unaltered 5) Scrap the current file writing implementation in the AccessLogValve and go straight JULI I would welcome any advice. Mark On Tue, Mar 27, 2012 at 9:46 AM, Konstantin Kolinko knst.koli...@gmail.comwrote: 2012/3/27 Mark Claassen markclaass...@gmail.com: I did a quick performance test, and it is true that the default mechanism is quite a bit faster than the standard JULI logger. For this test, I just modified the source code to write each log message 1000 times. The first set of timings (in millis) is from the current AccessLogValve, and the second set is using JULI. AccessLogValve Elapsed Time: 8 AccessLogValve Elapsed Time: 19 AccessLogValve Elapsed Time: 63 AccessLogValve Elapsed Time: 6 AccessLogValve Elapsed Time: 7 AccessLogValve Elapsed Time: 8 INFO: AccessLogValve Elapsed Time: 830 INFO: AccessLogValve Elapsed Time: 1122 INFO: AccessLogValve Elapsed Time: 451 INFO: AccessLogValve Elapsed Time: 531 INFO: AccessLogValve Elapsed Time: 764 INFO: AccessLogValve Elapsed Time: 347 1. Try to configure JULI with org.apache.juli.AsyncFileHandler. It might show better numbers. (The usual FileHandler by default performs flush() after each log message). 2. Is the same printed to console? If yes then remove ConsoleHandler. Best regards, Konstantin Kolinko - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: AccessLogValve enhancement
I haven't had too much time to work on this in the past week, but will try to take another crack at something the weekend. Thanks for all the comments. I will try to find a way that address these concerns, and does not get too big or complicated. Another route to take, would be to just add more configuration options to the existing access logger (like how many versions to keep). This would directly solve my immediate problem. However, it seems the vibe is leaning a bit toward having an option to use the JULI logger. On Tue, Feb 21, 2012 at 11:37 AM, Christopher Schultz ch...@christopherschultz.net wrote: Mark, On 2/19/12 12:49 PM, Mark Thomas wrote: On 19/02/2012 17:20, Christopher Schultz wrote: Philosophically, I'm not really sure why the flexibility of a full-featured logging system (JULI, log4j, etc.) is required for access logging: there's not much opportunity to use all the features they provide (log level filtering, log-message aggregation to log files and log-file multiplexing/splitting). Access logs typically log one thing (accesses), log it all the time, and always write to the same log file. I'm not adverse to the patch, I just don't really see a need for it. I was coming from the I'd rather not maintain a bunch of code to solve a problem that the logging frameworks have already solved (threading, roll-over, output to multiple destinations) angle. That's something I hadn't really thought about, honestly. It's one of the best reasons to use a logging framework IMO. I'd like to see some performance metrics, though, since putting a fully-fledged logging framework in the middle here has a distinct possibility of slowing everything down (for every request, of course). I kind of like Konstantin's idea of building extension-points into the AccessLogValve to allow a particular implementation of the Valve to handle the physical logging aspects while the AccessLogValve itself takes care of actually generating and emitting the log messages. That way, users can get the best of both worlds. You know me, always happy to delete some code from the Tomcat code base. Amen, brother. Multiple implementations certainly doesn't lead to that end, but we'll see how things shake out. My main point is that it is something worthy of discussion. And that is what we are doing. Absolutely. -chris
Re: AccessLogValve enhancement
Mark, On 2/19/12 12:49 PM, Mark Thomas wrote: On 19/02/2012 17:20, Christopher Schultz wrote: Philosophically, I'm not really sure why the flexibility of a full-featured logging system (JULI, log4j, etc.) is required for access logging: there's not much opportunity to use all the features they provide (log level filtering, log-message aggregation to log files and log-file multiplexing/splitting). Access logs typically log one thing (accesses), log it all the time, and always write to the same log file. I'm not adverse to the patch, I just don't really see a need for it. I was coming from the I'd rather not maintain a bunch of code to solve a problem that the logging frameworks have already solved (threading, roll-over, output to multiple destinations) angle. That's something I hadn't really thought about, honestly. It's one of the best reasons to use a logging framework IMO. I'd like to see some performance metrics, though, since putting a fully-fledged logging framework in the middle here has a distinct possibility of slowing everything down (for every request, of course). I kind of like Konstantin's idea of building extension-points into the AccessLogValve to allow a particular implementation of the Valve to handle the physical logging aspects while the AccessLogValve itself takes care of actually generating and emitting the log messages. That way, users can get the best of both worlds. You know me, always happy to delete some code from the Tomcat code base. Amen, brother. Multiple implementations certainly doesn't lead to that end, but we'll see how things shake out. My main point is that it is something worthy of discussion. And that is what we are doing. Absolutely. -chris signature.asc Description: OpenPGP digital signature
Re: AccessLogValve enhancement
Mark, On 2/15/12 4:33 PM, Mark Thomas wrote: I also be +1 to considering making this the sole way AccessLogValve logging may be output. The only possible reason why we wouldn't want to do this is that lots of users simply cannot figure out how to configure the loggers. Yes, it's really not all that hard and it's fairly well-documented, but at least with the output file name in the Valve attributes, there's no question where the log messages are going. Philosophically, I'm not really sure why the flexibility of a full-featured logging system (JULI, log4j, etc.) is required for access logging: there's not much opportunity to use all the features they provide (log level filtering, log-message aggregation to log files and log-file multiplexing/splitting). Access logs typically log one thing (accesses), log it all the time, and always write to the same log file. I'm not adverse to the patch, I just don't really see a need for it. -chris signature.asc Description: OpenPGP digital signature
Re: AccessLogValve enhancement
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 19/02/2012 17:20, Christopher Schultz wrote: Mark, On 2/15/12 4:33 PM, Mark Thomas wrote: I also be +1 to considering making this the sole way AccessLogValve logging may be output. The only possible reason why we wouldn't want to do this is that lots of users simply cannot figure out how to configure the loggers. Yes, it's really not all that hard and it's fairly well-documented, but at least with the output file name in the Valve attributes, there's no question where the log messages are going. Fair point. Philosophically, I'm not really sure why the flexibility of a full-featured logging system (JULI, log4j, etc.) is required for access logging: there's not much opportunity to use all the features they provide (log level filtering, log-message aggregation to log files and log-file multiplexing/splitting). Access logs typically log one thing (accesses), log it all the time, and always write to the same log file. I'm not adverse to the patch, I just don't really see a need for it. I was coming from the I'd rather not maintain a bunch of code to solve a problem that the logging frameworks have already solved (threading, roll-over, output to multiple destinations) angle. You know me, always happy to delete some code from the Tomcat code base. My main point is that it is something worthy of discussion. And that is what we are doing. Mark -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBAgAGBQJPQTZBAAoJEBDAHFovYFnneXEQAJucbwt4zm/gTv093FuwfC0Y m6bJDiObPzaxuZNnWvaKKwM1XY4ZCGCkKV7H5xoZQIuMOnzMVm1Z/9qcuMiiuml+ UoUxAH1mZoKeyDmRxAgl7DhYYwVf3XPAnL+oI2K/XuxNRuJvehUQRm3ouxZeO81a gTKs3GYA/ZxVL0p7fJ1pPnh44xqg4ly7dDLaUx5dVZU6WJ/14pwuIwQCCLDu9C+r FvC2k/Bc13fXxrAzYuTiat83DztEMohRPXmyxVUnM4XG+eM1jMub31/4+6S9TrSt 2TX6lRmzw36khrYUKYWsmPJxsS+5rSVaaOfO5ekSUzLnuhshoAEUYe1pPzVTHd02 1a7njJuqzdKNjU4B1sfWivLXPspmhmK6pnUtmoIF98NRr1XsMe9O/4AhG7NUIpoX Ci7Nd1WygWUdn9evMxzcUhtrmkAbvcMuQ+71737Xn8OJKDzvdc0b6p6ZEgm8cHnB QHM+whU38Cb6Vy+vlyFQpwmJ8qolRrT2Ob76h22pZxm9PDQF18OWU6tqi3EabjU+ 9YV1fPPJNjrvNk9H27vaQIurrXMhX5yTZGb7PsVSinjXbq4zNEWZlcQSI5t+Pf5K 5AaRnxcptDIFcv+y9YFZv4ecqOPFHCUj03XSjRLBZd1V2vUdbkuTmpToLLo4Mwwp Rzy9If3xAw9+lSqza8hS =t+lP -END PGP SIGNATURE- - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: AccessLogValve enhancement
My initial motivation for doing this was in trying to figure out a way to have the logs automatically delete old ones eventually. I couldn't find a way to do it, looked at the source, and got a bit sidetracked. The rest is history On Feb 19, 2012 12:50 PM, Mark Thomas ma...@apache.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 19/02/2012 17:20, Christopher Schultz wrote: Mark, On 2/15/12 4:33 PM, Mark Thomas wrote: I also be +1 to considering making this the sole way AccessLogValve logging may be output. The only possible reason why we wouldn't want to do this is that lots of users simply cannot figure out how to configure the loggers. Yes, it's really not all that hard and it's fairly well-documented, but at least with the output file name in the Valve attributes, there's no question where the log messages are going. Fair point. Philosophically, I'm not really sure why the flexibility of a full-featured logging system (JULI, log4j, etc.) is required for access logging: there's not much opportunity to use all the features they provide (log level filtering, log-message aggregation to log files and log-file multiplexing/splitting). Access logs typically log one thing (accesses), log it all the time, and always write to the same log file. I'm not adverse to the patch, I just don't really see a need for it. I was coming from the I'd rather not maintain a bunch of code to solve a problem that the logging frameworks have already solved (threading, roll-over, output to multiple destinations) angle. You know me, always happy to delete some code from the Tomcat code base. My main point is that it is something worthy of discussion. And that is what we are doing. Mark -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBAgAGBQJPQTZBAAoJEBDAHFovYFnneXEQAJucbwt4zm/gTv093FuwfC0Y m6bJDiObPzaxuZNnWvaKKwM1XY4ZCGCkKV7H5xoZQIuMOnzMVm1Z/9qcuMiiuml+ UoUxAH1mZoKeyDmRxAgl7DhYYwVf3XPAnL+oI2K/XuxNRuJvehUQRm3ouxZeO81a gTKs3GYA/ZxVL0p7fJ1pPnh44xqg4ly7dDLaUx5dVZU6WJ/14pwuIwQCCLDu9C+r FvC2k/Bc13fXxrAzYuTiat83DztEMohRPXmyxVUnM4XG+eM1jMub31/4+6S9TrSt 2TX6lRmzw36khrYUKYWsmPJxsS+5rSVaaOfO5ekSUzLnuhshoAEUYe1pPzVTHd02 1a7njJuqzdKNjU4B1sfWivLXPspmhmK6pnUtmoIF98NRr1XsMe9O/4AhG7NUIpoX Ci7Nd1WygWUdn9evMxzcUhtrmkAbvcMuQ+71737Xn8OJKDzvdc0b6p6ZEgm8cHnB QHM+whU38Cb6Vy+vlyFQpwmJ8qolRrT2Ob76h22pZxm9PDQF18OWU6tqi3EabjU+ 9YV1fPPJNjrvNk9H27vaQIurrXMhX5yTZGb7PsVSinjXbq4zNEWZlcQSI5t+Pf5K 5AaRnxcptDIFcv+y9YFZv4ecqOPFHCUj03XSjRLBZd1V2vUdbkuTmpToLLo4Mwwp Rzy9If3xAw9+lSqza8hS =t+lP -END PGP SIGNATURE- - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: AccessLogValve enhancement
On 2/16/2012 2:12 PM, Konstantin Kolinko wrote: Do not send attachments to this mailing list ever! The world could implode!! - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: AccessLogValve enhancement
While AccessLogValve is the subject... Any chance we can do away with the default, unparseable timestamp format? With Apache, it is allowed to specify the timestamp using strftime(3)-like format, I'd like to be able to do the same with Tomcat's AccessLogValve using SimpleDateFormat patterns... -- Francis Galiegue, fgalie...@gmail.com It seems obvious [...] that at least some 'business intelligence' tools invest so much intelligence on the business side that they have nothing left for generating SQL queries (Stéphane Faroult, in The Art of SQL, ISBN 0-596-00894-5) - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: AccessLogValve enhancement
On 2/17/2012 8:55 AM, Francis Galiegue wrote: While AccessLogValve is the subject... Any chance we can do away with the default, unparseable timestamp format? With Apache, it is allowed to specify the timestamp using strftime(3)-like format, I'd like to be able to do the same with Tomcat's AccessLogValve using SimpleDateFormat patterns... Submit a patch to bugzilla at issues.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: AccessLogValve enhancement
On 17.02.2012 17:06, Filip Hanik - Dev Lists wrote: On 2/17/2012 8:55 AM, Francis Galiegue wrote: While AccessLogValve is the subject... Any chance we can do away with the default, unparseable timestamp format? With Apache, it is allowed to specify the timestamp using strftime(3)-like format, I'd like to be able to do the same with Tomcat's AccessLogValve using SimpleDateFormat patterns... Submit a patch to bugzilla at issues.apache.org It's already part of TC 7. Go ahead an configure :) Regards, Rainer - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: AccessLogValve enhancement
Looking at the docs, it looks like the asynchronous parts of the logging mechanism are controlled through the logging configuration in either logging.properties or log4j.properties. I don't see any way to ensure that this is the case or to check to make sure the output is not going to the console. That being said, this is just an option, and would hopefully be acceptable in many situations. I have not done tests to see the performance difference. Perhaps a strongly worded warning in the documentation would be sufficient to ensure people are smart about this? Unless someone thinks that this change is unacceptable, I will continue my efforts and see what happens with it. Thanks, Mark On Wed, Feb 15, 2012 at 5:07 PM, Konstantin Kolinko knst.koli...@gmail.comwrote: 2012/2/16 Mark Claassen markclaass...@gmail.com: We would like to use the flexibility of the standard logging mechanism for the AccessLogValue. I could find no way to do this. Looking at the AccessLogValve code in Tomcat 7, it seems it would really would not be hard to add. There already is standard logger in the AccessLogValve class, which is where error in the logger itself get routed. private static final Log log = LogFactory.getLog(AccessLogValve.class); This still makes a lot of sense, and should probably be kept separate. What I would propose is that a new attribute be added, called maybe systemLoggerName. If this is set, then the logger by that name would be used instead of the current hard-coded file logger. (Errors would still go to the statically defined Logger.) I already tried this with the XML listed below and a slightly modified AccessLogValve and it seems to work perfectly. If this is something that would be considered a positive change, I would be happy to submit a patch and propose some documentation updates. I am not exactly sure how this all works, especially since this is more of an enhancement rather than a bug fix. I sure there are guidelines for when in a release cycle these types of changes are appropriate. Mark Valve className=org.apache.catalina.valves.AccessLogValve directory=log prefix=localhost_access_log. systemLoggerName=mylogger pattern=common resolveHosts=false/ I would say that the performance requirements for the two are different. Access log should be fast and simple. The error log does not have to be as fast, as errors should be more rare. The default JULI FileHandler is not suitable here. You have to configure buffers. You have to perform flushing. AccessLogValve wires to Tomcat's background processing, but IIRC you cann do flush via commons-logging wrapper. The only viable alternative here is AsyncFileHandler. You should also make sure that the events wouldn't be printed to Console. The ideas to extract some common code to elsewhere (e.g. time formatting cache) were spoken several times. They are feasible. Regarding this systemLoggerName (or logCategory as I would name it) feature, I think it does not belong to the current AccessLogValve but you might plug it there after some refactoring. Anyway I would like to see the whole picture, which includes the logging configuration that you would use with AccessLogValve. Best regards, Konstantin Kolinko - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: AccessLogValve enhancement
2012/2/17 Mark Claassen markclaass...@gmail.com: I attached the two new valves in a zip file, Do not send attachments to this mailing list ever! The proper procedure is documented here: http://tomcat.apache.org/bugreport.html#How_to_submit_patches_and_enhancement_requests Best regards, Konstantin Kolinko - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: AccessLogValve enhancement
Apologies. Won't happen again. I saw an attachment on an email from the 15th, but I see this was just junk. I created at issue for this (52688) and attached the diffs. On Thu, Feb 16, 2012 at 4:12 PM, Konstantin Kolinko knst.koli...@gmail.comwrote: 2012/2/17 Mark Claassen markclaass...@gmail.com: I attached the two new valves in a zip file, Do not send attachments to this mailing list ever! The proper procedure is documented here: http://tomcat.apache.org/bugreport.html#How_to_submit_patches_and_enhancement_requests Best regards, Konstantin Kolinko - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
AccessLogValve enhancement
We would like to use the flexibility of the standard logging mechanism for the AccessLogValue. I could find no way to do this. Looking at the AccessLogValve code in Tomcat 7, it seems it would really would not be hard to add. There already is standard logger in the AccessLogValve class, which is where error in the logger itself get routed. private static final Log log = LogFactory.getLog(AccessLogValve.class); This still makes a lot of sense, and should probably be kept separate. What I would propose is that a new attribute be added, called maybe systemLoggerName. If this is set, then the logger by that name would be used instead of the current hard-coded file logger. (Errors would still go to the statically defined Logger.) I already tried this with the XML listed below and a slightly modified AccessLogValve and it seems to work perfectly. If this is something that would be considered a positive change, I would be happy to submit a patch and propose some documentation updates. I am not exactly sure how this all works, especially since this is more of an enhancement rather than a bug fix. I sure there are guidelines for when in a release cycle these types of changes are appropriate. Mark Valve className=org.apache.catalina.valves.AccessLogValve directory=log prefix=localhost_access_log. systemLoggerName=mylogger pattern=common resolveHosts=false/
Re: AccessLogValve enhancement
On 15/02/2012 21:23, Mark Claassen wrote: We would like to use the flexibility of the standard logging mechanism for the AccessLogValue. I could find no way to do this. Looking at the AccessLogValve code in Tomcat 7, it seems it would really would not be hard to add. I added org.apache.juli.VerbatimFormatter some time ago with this in mind but never got around to implementing it. There already is standard logger in the AccessLogValve class, which is where error in the logger itself get routed. private static final Log log = LogFactory.getLog(AccessLogValve.class); This still makes a lot of sense, and should probably be kept separate. +1 What I would propose is that a new attribute be added, called maybe systemLoggerName. If this is set, then the logger by that name would be used instead of the current hard-coded file logger. (Errors would still go to the statically defined Logger.) aside Woot! A bike shed to paint. :) /aside outputLoggerName? I already tried this with the XML listed below and a slightly modified AccessLogValve and it seems to work perfectly. If this is something that would be considered a positive change, I would be happy to submit a patch and propose some documentation updates. I assume that the directory and prefix attributes below are unnecessary. I am not exactly sure how this all works, especially since this is more of an enhancement rather than a bug fix. I sure there are guidelines for when in a release cycle these types of changes are appropriate. There is a very detailed 143 page process document for approving changes like this. Fortunately it may be summed up as If you can convince a committer to make the change and no other committer vetos the change it will be in the next release.. As long as backwards compatibility is retained, I can't imagine why someone might complain (unless the patch is a mess). I also be +1 to considering making this the sole way AccessLogValve logging may be output. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: AccessLogValve enhancement
Awesome. Thanks for the immediate response! I see that the ExtendedAccessLogValve would need to be tweaked as well, and that one would loose the repeating header bit when using the system logger (or outputLogger) I guess th header could still be written on init (and maybe when the pattern changed?) writer.println(#Fields: + pattern); writer.println(#Version: 2.0); writer.println(#Software: + ServerInfo.getServerInfo()); I also be +1 to considering making this the sole way AccessLogValve logging may be output. +1 I assume that the directory and prefix attributes below are unnecessary. Yep. And a few others Based on your comments, I will go ahead and modify those to classes and then see where it goes from there. It will be completely backwards compatible. On Wed, Feb 15, 2012 at 4:33 PM, Mark Thomas ma...@apache.org wrote: On 15/02/2012 21:23, Mark Claassen wrote: We would like to use the flexibility of the standard logging mechanism for the AccessLogValue. I could find no way to do this. Looking at the AccessLogValve code in Tomcat 7, it seems it would really would not be hard to add. I added org.apache.juli.VerbatimFormatter some time ago with this in mind but never got around to implementing it. There already is standard logger in the AccessLogValve class, which is where error in the logger itself get routed. private static final Log log = LogFactory.getLog(AccessLogValve.class); This still makes a lot of sense, and should probably be kept separate. +1 What I would propose is that a new attribute be added, called maybe systemLoggerName. If this is set, then the logger by that name would be used instead of the current hard-coded file logger. (Errors would still go to the statically defined Logger.) aside Woot! A bike shed to paint. :) /aside outputLoggerName? I already tried this with the XML listed below and a slightly modified AccessLogValve and it seems to work perfectly. If this is something that would be considered a positive change, I would be happy to submit a patch and propose some documentation updates. I assume that the directory and prefix attributes below are unnecessary. I am not exactly sure how this all works, especially since this is more of an enhancement rather than a bug fix. I sure there are guidelines for when in a release cycle these types of changes are appropriate. There is a very detailed 143 page process document for approving changes like this. Fortunately it may be summed up as If you can convince a committer to make the change and no other committer vetos the change it will be in the next release.. As long as backwards compatibility is retained, I can't imagine why someone might complain (unless the patch is a mess). I also be +1 to considering making this the sole way AccessLogValve logging may be output. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: AccessLogValve enhancement
2012/2/16 Mark Claassen markclaass...@gmail.com: We would like to use the flexibility of the standard logging mechanism for the AccessLogValue. I could find no way to do this. Looking at the AccessLogValve code in Tomcat 7, it seems it would really would not be hard to add. There already is standard logger in the AccessLogValve class, which is where error in the logger itself get routed. private static final Log log = LogFactory.getLog(AccessLogValve.class); This still makes a lot of sense, and should probably be kept separate. What I would propose is that a new attribute be added, called maybe systemLoggerName. If this is set, then the logger by that name would be used instead of the current hard-coded file logger. (Errors would still go to the statically defined Logger.) I already tried this with the XML listed below and a slightly modified AccessLogValve and it seems to work perfectly. If this is something that would be considered a positive change, I would be happy to submit a patch and propose some documentation updates. I am not exactly sure how this all works, especially since this is more of an enhancement rather than a bug fix. I sure there are guidelines for when in a release cycle these types of changes are appropriate. Mark Valve className=org.apache.catalina.valves.AccessLogValve directory=log prefix=localhost_access_log. systemLoggerName=mylogger pattern=common resolveHosts=false/ I would say that the performance requirements for the two are different. Access log should be fast and simple. The error log does not have to be as fast, as errors should be more rare. The default JULI FileHandler is not suitable here. You have to configure buffers. You have to perform flushing. AccessLogValve wires to Tomcat's background processing, but IIRC you cann do flush via commons-logging wrapper. The only viable alternative here is AsyncFileHandler. You should also make sure that the events wouldn't be printed to Console. The ideas to extract some common code to elsewhere (e.g. time formatting cache) were spoken several times. They are feasible. Regarding this systemLoggerName (or logCategory as I would name it) feature, I think it does not belong to the current AccessLogValve but you might plug it there after some refactoring. Anyway I would like to see the whole picture, which includes the logging configuration that you would use with AccessLogValve. Best regards, Konstantin Kolinko - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
