Re: [Zope-dev] Re: ConflictError's worthwhile to note?
Jean-Marc Orliaguet wrote: Absolutely, this is called debugging an application (= DEBUG, TRACE). Well, I call BS on this ;-) Things like ConflictErrors and slow-running requests are not something you often find out about during application development. They emerge once the load ramps up and are something any responsible admin of a large production system will want to keep a continuing eye on. cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ConflictError's worthwhile to note?
Dieter Maurer wrote: Jean-Marc Orliaguet wrote at 2005-12-4 22:28 +0100: ... In the log flle I'd like to be informed about events that are unexpected. Conflict errors of this kind occur by design. This argument is not convincing: In a similar way, I could argue that MemoryErrors are there by design. While it is true that *occational* ConflictErrors are nothing to worry about, a higher rate indicates that you soon may come into trouble -- because the risk increases that the automatic retries will not be able to recover and your users will see errors. Obviously, a ConflictError is far less important than a MemoryError, but a higher rate of conflicts should be analysed and if possible avoided. The conflict rate is related to the quality of your application design. I like quality related information in my logfiles -- to be able to take action before it is too late. But aren't you looking for some sort of application profiler? or some sort of benchmarker? One could argue that slowly rendered pages are sign that the application is badly designed too, still you wouldn't want to see in the log file: INFO: the request took more than 2 seconds to be fullfilled (this has occured 10 times already) Similarly if the number of cache objects is too low, the application will run slowly, do you want to see: INFO: 1000 objects loaded into the cache in the last 5 minutes So why not instead create a product that logs occasional conflict errors? there is already in the ZMI in the database management screen some information about cached objects- Why not add the information that you are looking for there instead? /JM ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: ConflictError's worthwhile to note?
Jean-Marc Orliaguet wrote: But aren't you looking for some sort of application profiler? The point is that you need information to build a profile... sort of benchmarker? One could argue that slowly rendered pages are sign that the application is badly designed too, Indeed! still you wouldn't want to see in the log file: INFO: the request took more than 2 seconds to be fullfilled (this has occured 10 times already) Well, yes, you do, but maybe not so verbosely ;-) We have Apache log the time to serve in microseconds for every single request, for later analysis. If you like, I'm happy to add a binary column to Z2.log ;-) Similarly if the number of cache objects is too low, the application will run slowly, Well, I don't know if you or I understand zodb's caching to know what to log... do you want to see: INFO: 1000 objects loaded into the cache in the last 5 minutes ...but yeah, if you could quantify that having 1000 objects loaded made a significant difference to the perfomance of the request where that was logged, then I'd be all for adding it... So why not instead create a product that logs occasional conflict errors? Because it's a lot more work than leaving it like it is but with improved inforamtion, which, from the vote, is going to keep the most people happy anyway ;-) there is already in the ZMI in the database management screen some information about cached objects- Why not add the information that you are looking for there instead? That's another possibility, are you volunteering to code it though? ;-) Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ConflictError's worthwhile to note?
Jean-Marc Orliaguet wrote at 2005-12-6 10:59 +0100: ... But aren't you looking for some sort of application profiler? or some sort of benchmarker? One could argue that slowly rendered pages are sign that the application is badly designed too, still you wouldn't want to see in the log file: You err: I do want to see them in the logfile and I stole Florents idea to use threadframe (in his DeadlockDebugger) to include a traceback for long running requests in the logfile. This way, it is easy not only to identify long running requests but also to get hints where they are spending their time... You will not believe how fast this has eliminated our remaining long running requests and especially to track down a non deterministic rarely occuring apparent deadlock situation... Thus, I can *very* strongly recommend to include such logging entries :-) INFO: the request took more than 2 seconds to be fullfilled (this has occured 10 times already) I agree with you about such stupid log entries. And your argument also holds for the stupid ConflictError log entries that Zope used to emit. But, we use our own Zope. And our Zope learned very early to emit informative ConflictError log entries -- those that show *which* object caused the ConflictError. These log entries gave valuable hints on how to improve our application. It is this experience that lets me argue for INFO level ConflictError log entries. By the way, I can be quite calm: whatever the Zope community might decide: our Zope will continue to log ConflictError messages at INFO level because we are interested to get early indications that potential problems crept into our application... -- Dieter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ConflictError's worthwhile to note?
Jean-Marc Orliaguet wrote at 2005-12-6 21:45 +0100: ... You err: I do want to see them in the logfile and I stole Florents idea to use threadframe (in his DeadlockDebugger) to include a traceback for long running requests in the logfile. This way, it is easy not only to identify long running requests but also to get hints where they are spending their time... You will not believe how fast this has eliminated our remaining long running requests and especially to track down a non deterministic rarely occuring apparent deadlock situation... Thus, I can *very* strongly recommend to include such logging entries :-) Absolutely, this is called debugging an application (= DEBUG, TRACE). Our sysadms are not really interested in this kind of info. Ours are -- especially to hunt down non-deterministic problems that actually are mail server problems (the above mentioned apparent deadlock was caused by a mail server occationally not responding for many minutes). They want to know if some action should be taken or not and messages such as these are misleading. Long running requests (and a high rate of ConflictErrors) *ARE* instances when some action should be taken -- probably to contact the vendor/developper. ... But, we use our own Zope. And our Zope learned very early to emit informative ConflictError log entries -- those that show *which* object caused the ConflictError. These log entries gave valuable hints on how to improve our application. Sure, but I think that you are both developer and sysadm at the same time. This is not the case everywhere. True. But ConflictErrors occur non-deterministically. Whenever, you are faced with non-deterministical problems, you are quite happy when the true operational system provides enough information to guide the analysis -- because it is so difficult to reproduce the behaviour in a development environment. -- Dieter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
