Re: [Zope-dev] Log rotation on Windows
On Jan 24, 2005, at 8:12 PM, Mark Hammond wrote: A number of Enfold's customers have requested a reasonable logfile rotation scheme for Zope on Windows. Enfold would like to work on this and contribute it back to Zope. The intention of this mail is to find a consensus on the general solution we should adopt, so we can supply patches with the greatest chance of getting into the core. Since ZConfig and zLOG are both designed to be extensible. They are built so that new loggers can be created without any changes to the Zope core. It seems to me that most of this could be handled by an add-on package. The ZConfig documentation http://svn.zope.org/ZConfig/trunk/doc/zconfig.pdf?rev=440view=log has as its extension example how to add a new logger (a log to a pager logger) to the system. (Of course once this is built, this add-on package could migrate to the core if there is a compelling need, either packaging convenience (Windows users don't need to grab an extra package to manage their systems well.) or so that enhancements to zLOG's loghandlers can keep the win32 loggers in mind.) As a rough cut, this will create a logger with a logrotate behavior similar to what you are looking for. win32logger/__init__.py: # empty file to signify a package. win32logger/component.xml: component prefix=win32logger.LogHandlers !-- extend the logging subsystem with a new file logger -- import package=zLOG/ sectiontype name=win32-logfile datatype=.Win32FileHandlerFactory implements=zLOG.loghandler extends=logfile key name=rotate-path required=yes/ /sectiontype /component win32logger/LogHandlers.py: import zLOG import os class Win32FileHandler(zLOG.LogHandlers.FileHandler): A file based loghandler that renames on rotate def __init__(self, path, rotate_path): zLOG.LogHandlers.FileHandler.__init__(self,path) self.rotateFilename = rotate_path def reopen(self): self.close() error = None try: os.rename(self.baseFilename, self.rotateFilename) except OSError, err: error = err self.stream = open(self.baseFilename, self.mode) if error: zLOG.LOG(Win32Logger, zLOG.ERROR, Rotate Error, error) class Win32FileHandlerFactory(zLOG.datatypes.FileHandlerFactory): def create_loghandler(self): return Win32FileHandler(self.section.path,self.section.rotate_path) and then adjust the zope.conf with: %import win32logger eventlog level all win32-logfile level DEBUG path $INSTANCE/log/-event.log rotate-path $INSTANCE/log/event.log.closed /win32-logfile /eventlog logger access level WARN win32-logfile path $INSTANCE/log/Z2.log rotate-path $INSTANCE/log/Z2.log.closed /win32-logfile /logger Of course, it needs to be fleshed out with better error handling, etc. Although time consuming, maybe the better behavior would be to have the reopen method copy the current log to the backup file and then seek() and truncate() the original file. That would take longer to rotate, but would protect against log files whose close() succeed but open() fails. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] Re: Log rotation on Windows
[Duncan] +1 on fixing the problem of log rotation on windows. I don't like a solution that requires you to wait an arbitrary period for the snapshot to appear. Yes, I see your concern. Assuming we stick to a rename operation, the external process could simply wait for the expected file to appear (using, say, FindFirstChangeNotification or ReadDirectoryChanges). Even a loop with 1 second sleeps before checking the file should work. Especially given that windows has a tendency to swap processes out to disc just because they haven't done anything recently. This means that if you run the logrotate at midnight you might easily spend the 5 seconds just swapping Zope back into memory. Given that you don't have a system 'kill' command so you'll need to run something zope specific to generate the equivalent, you might as well make it two-way to begin with and signal when the snapshot has been created. Certainly that is doable, and it touches on a subject I (indirectly) brought up here quite some time ago - a general notification mechanism so Zope can let its parent know its status (eg, starting, running, etc). If we can get a more generic notification scheme off the ground, it may make sense to reuse that. OTOH, it adds a level complexity where a slightly smarter wait for file to appear loop in the external process may be all that is needed. I guess that the most obvious Windows equivalent of the kill command would be to use win32event.CreateEvent() to create a named event. That can then be easily signalled from outside zope, but a thread inside zope would have to explicitly wait on the event (unless RegisterWaitForSingleObject could be used, but it isn't currently exposed in win32event). Yep - that is exactly what I was thinking. Adding a thread to handle windows events to Zope could also be used to improve the communication when run as a service. Currently the zope process is terminated using TerminateProcess and it would be nicer if the service could generate a termination signal to give Zope a chance to shutdown cleanly and only resort to TerminateProcess if the clean shutdown was ignored. Collector issues 1527 and 1533 provide startup/shutdown for Windows as it exists on Linux, hopefully making that moot. As mentioned above, it has been identified that Zope really could do with a generic notification mechanism to improve the situation on all platforms. I believe that if we keep the task small (ie, just log rotation), we can come up with a fairly minimal patch which does the job perfectly. I'm concerned that if we try and expand it into any kind of generic mechanism, it simply will not happen. I note that the only comments you made were on the mechanics of the operation - which is good! I expected the scheme itself (ie, rename to .snapshot if possible) to be contentious. Assuming no other serious objections come up, we will steam ahead! Thanks, Mark. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
