I have been trying to come up with a simple plan to make log rotation more robust, particularly in not splitting log lines across files. Greg Smith complained that lines could be split if logs are rotated on size, but AFAICS the danger also exists for time based rotation.
My thought is to have the logger mark each true log line line end with a null byte. The syslogger would have a state variable called, say, safe_to_rotate, which would be set true if and only if the last thing written to the log file was a "\n" from the input stream which was followed by a null byte. The null bytes would of course never actually be sent to the files. Log rotation would be delayed until variable this was set true.
There is a small danger that the delay in log rotation might be substantial - with a series of partial lines read. I suspect that in practice this danger would be vanishingly small, and we might reasonably expect that the condition would come true within a few cycles.
An alternative scheme would involve keeping some extra buffer(s) of data so that we ensure we never write out a partial line to the file. But that seems to me to involve a lot more processing and so I'm wary of it.
I don't think we should contemplate providing for CSV logs until we have this problem solved, so it's rather important.
Thoughts? cheers andrew ---------------------------(end of broadcast)--------------------------- TIP 5: don't forget to increase your free space map settings
