On Thu, 2 Apr 2015, Risto Vaarandi wrote: > 2015-04-02 1:22 GMT+03:00 David Lang <da...@lang.hm>: > >> On Wed, 1 Apr 2015, Risto Vaarandi wrote: >> >> hi David, >>> >>> is my understanding correct that you would like to have pre_restart event, >>> in order to execute the following steps: >>> 1) when HUP is received, match pre_restart synthetic event and save event >>> correlation state to disk, >>> 2) let the HUP handling procedure to drop all event correlation state from >>> memory, >>> 3) match sec_restart synthetic event and load event correlation state from >>> disk back to memory >>> >>> If my understanding is correct and you want to use the above sequence for >>> making HUP a log rotation signal, there would be a number of non-trivial >>> issues to handle. Firstly, while some state like contexts can be stored >>> and >>> restored to/from disk in a straightforward way, things get a bit more >>> complex with storing/restoring event correlation operations. Secondly, >>> receiving HUP also means that all inputs will be closed and reopened, and >>> at reopen sec always jumps to the very end of each input file. This means >>> that any lines appended to input files between close and reopen will be >>> missed. In short, I would never recommend using the HUP signal just for >>> rotating log files, since HUP's true purpose is doing full restart (in >>> fact, it is almost identical to taking sec process down and starting a new >>> instance). >>> >> >> In my use case, the input is a pipe, so unless sec is falling behind, >> little, if anything would be lost. >> >> I agree it's not the best thing to do, for rotation, but here's another >> case. >> >> I have an instance of SEC that's keeping running totals of things in logs, >> and I have it writing the totals out every minute. I also have it write out >> the totals when it shuts down. >> >> But if it gets a HUP, it looses all it's current data and has no chance to >> write it out or do anything else with it. >> > > Different treatment of HUP is actually one of the differences between > stateful and stateless unix daemons. It is a common practice to use HUP for > reinitializing a daemon which normally means reopening inputs/outputs, > reloading configuration and starting with a clean state. Daemons like > rsyslog are largely stateless -- they don't keep any state about processed > events in memory, since their main purpose is fast forwarding of events > from inputs to outputs. For this reason, sending a HUP to rsyslog largely > means reopening input/outputs and reloading your configuration. Thus, if > you haven't modified your configuration, you can easily use HUP for log > rotation, since you have no state to lose in memory.
not actually true with rsyslog. If it did a re-read of it's config, lots of things can change and there's a lot of state that can be maintained. But because HUP is so commonly used for log rotation, and a full startup takes so long, HUP was redefined to just be a log rotation event (close inputs and outputs) rather than a full reparse of the config. Too many people were loosing too much data with the old way of doing things. We could have gone down the route of defining a different signal for the rotation and kept HUP as the full restart, but the result would have been more lost logs for people over the years. Did we do the wrong thing? arguable. From a purist point of view we did, from a pragmatic point of view probably not. > With stateful daemons like sec and snort the HUP signal is not just about > reopening input/output channels and reloading rules/signatures, but you > always lose state which has been stored in memory. As a result, with > stateful daemons the HUP signal can not have a dual role of > reinitialization and log rotation. That is why sec uses a different signal > (USR2) designed solely for log rotation. That's all well and good, but why do you loose more state when you do a HUP than when you do a full shutdown? I'm not even talking about restoring state after the HUP, I'm talking about the ability to output running counters. If I have a counter that's accumulating data that I output every hour (via a calendar command) and it's 1 sec before the end of the hour. If I do a full kill/restart I have the chance to output the data before I shutdown But if I do a HUP instead, that data is irrevokably lost. why should a full kill be more data friendly than a HUP? David Lang ------------------------------------------------------------------------------ Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Simple-evcorr-users mailing list Simple-evcorr-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
