On Wed, 1 Apr 2015, Risto Vaarandi wrote: > hi David, > > is my understanding correct that you would like to have pre_restart event, > in order to execute the following steps: > 1) when HUP is received, match pre_restart synthetic event and save event > correlation state to disk, > 2) let the HUP handling procedure to drop all event correlation state from > memory, > 3) match sec_restart synthetic event and load event correlation state from > disk back to memory > > If my understanding is correct and you want to use the above sequence for > making HUP a log rotation signal, there would be a number of non-trivial > issues to handle. Firstly, while some state like contexts can be stored and > restored to/from disk in a straightforward way, things get a bit more > complex with storing/restoring event correlation operations. Secondly, > receiving HUP also means that all inputs will be closed and reopened, and > at reopen sec always jumps to the very end of each input file. This means > that any lines appended to input files between close and reopen will be > missed. In short, I would never recommend using the HUP signal just for > rotating log files, since HUP's true purpose is doing full restart (in > fact, it is almost identical to taking sec process down and starting a new > instance).
In my use case, the input is a pipe, so unless sec is falling behind, little, if anything would be lost. I agree it's not the best thing to do, for rotation, but here's another case. I have an instance of SEC that's keeping running totals of things in logs, and I have it writing the totals out every minute. I also have it write out the totals when it shuts down. But if it gets a HUP, it looses all it's current data and has no chance to write it out or do anything else with it. > Since you mentioned a discussion in rsyslog mailing list about forwarding > HUP signal from rsyslog to its child processes, and possibly replacing HUP > with another signal, there was a somewhat similar discussion in the sec > mailing list few years ago. The main idea of the discussion was the > limitation of child process runtime, and a relevant recipe was posted to > the SEC FAQ: http://simple-evcorr.sourceforge.net/FAQ.html#20 I was starting from #15 > The above recipe can be easily reused for handling your problem by rolling > out a similar Perl wrapper: > > #!/usr/bin/perl -w > # > # wrapper.pl -- this wrapper starts sec and if HUP is sent to > # the wrapper, it sends USR2 signal to the sec process > # > if (scalar(@ARGV) < 1) { exit(1); } > $pid = fork(); > if ($pid == -1) { > exit(1); > } elsif ($pid == 0) { > exec("@ARGV"); > } else { > $SIG{HUP} = sub { kill USR2, $pid; }; > $SIG{TERM} = sub { kill TERM, $pid; exit(0); }; > waitpid($pid, 0); > exit($? >> 8); > } > > > If you start the following command line from rsyslog: > wrapper.pl /usr/bin/sec --conf=/etc/sec/sec.conf --notail --input=- > every HUP signal that is forwarded to wrapper.pl will be replaced with USR2 > and sent to sec. Also, since the sec process inherits its standard input > from the wrapper, it can receive input from the pipe that rsyslog has > created. While using a wrapper might initially seem a bit complex, rsyslog > v5 has to use wrapper script anyway for programs with command line options > (one example can be found at http://simple-evcorr.sourceforge.net/FAQ.html#2). > With v5 you can easily modify the above wrapper for handling both tasks, so > no additional complexity will be added to your setup. I'm using v8.8, and 7.x is in current distros, so v5 stuff is very quickly becoming distant legacy. It's good to have options at both ends to deal with signals like this. But I'm not sure that this completely avoids the uses of being able to do something before data is lost when receiving something other than a full shutdown. > Hope this helps, It does, thanks. David Lang > risto > > > 2015-04-01 9:54 GMT+03:00 David Lang <da...@lang.hm>: > >> On Tue, 31 Mar 2015, John P. Rouillard wrote: >> >>> Hi David: >>> >>> In message <alpine.deb.2.02.1503311452250.26...@nftneq.ynat.uz>, >>> David Lang writes: >>>> before I realized that I really only needed USR2, I was sending sec >>>> a HUP to get it to close all it's output files (for log rotation), >>>> but I was running into the problem that the HUP restart looses all >> context. >>> >>> Not just context but all correlation operations and variables (however >>> variables set explicitly via perl functions/expressions IIRC aren't >>> flushed). >>> >>>> There is the SEC_RESTART event that shows up after the HUP finishes and >>>> all state is cleared, but it doesn't look like SEC_SHUTDOWN takes place >>>> prior to the HUP. >>> >>> Yeah I can see the agrument for that, but I would really expect a >>> SEC_SHUTDOWN only when it's really being shut down. >>> >>>> Am I missing something? or is there no way to save state when a HUP >>>> is received? >>> >>> I don't think you are missing anything. I am not sure if it makes sense >> to >>> create events for: >>> >>> PRE_SEC_RESTART - this event will be the last event that SEC sees before >>> reloading its configuration (HUP) >>> >>> PRE_SEC_SOFTRESTART - this event will be the last event that SEC sees >>> before reloading its configuration (ABRT) >>> >>> PRE_SEC_LOGROTATE - this event will be the last event that SEC sees >>> before reopening its log file and closing its outputs (USR2) >>> >>> just for completeness. >>> >>> I can't think of an occasion where I would send a HUP to a SEC and >>> want to retain the state. (Then again I don't find keeping state and >>> reloading it after a shutdown/restart (SIGTERM/ service restart) to be >>> very useful given all the caveats associated with it.) >>> >>> I am interested in what your use case would be for these theoretical >>> events. >> >> This started with me running sec from rsyslog and hitting the file rotation >> problem because when rsyslog is sent a HUP to have it close it's outputs, >> sec >> doesn't receive anything. So I slapped a quick HUP for sec in my rotate >> script >> and reported this on the rsyslog mailing list. A day later, there's a >> feature in >> rsyslog git so that when a HUP is sent to rsyslog, it will send a HUP on to >> programs that it's started. >> >> Thinks seemed to be working, until I setup another ruleset in sec to alert >> for >> logs disappearing on my (if the last log is >3 min ago, send one e-mail), >> but >> this wasn't firing reliably because the logs were rotating every 5 min and >> the >> HUP erased the sec history. >> >> so I changed my rotate script to send USR2 and reported this to the >> rsyslog list >> to see about making it so that when rsyslog receives a HUP it can send a >> different signal to it's child processes. >> >> That's pending, but in the meantime, as I'm working on various sec >> rulesets, I >> may have a bunch of different copies of sec running, each receiving a >> different >> feed (different subsets of traffic, and different contents in what is being >> sent). It's FAR easier to do killall -HUP sec than to figure out which of >> the >> several copies of sec I need to send the signal to. >> >> So, eventually, I will be able to work around the issues that triggered my >> desire for a pre_restart event. But as the system gets more complex and >> grows >> more instances of sec running, working around the lack is going to keep >> getting >> more complex. >> >> David Lang >> >> >> ------------------------------------------------------------------------------ >> Dive into the World of Parallel Programming The Go Parallel Website, >> sponsored >> by Intel and developed in partnership with Slashdot Media, is your hub for >> all >> things parallel software development, from weekly thought leadership blogs >> to >> news, videos, case studies, tutorials and more. Take a look and join the >> conversation now. http://goparallel.sourceforge.net/ >> _______________________________________________ >> Simple-evcorr-users mailing list >> Simple-evcorr-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users >> > ------------------------------------------------------------------------------ Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Simple-evcorr-users mailing list Simple-evcorr-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
