Hi Risto:
In message
<CAGfjSCM2V4WLsGgdDprJbW+qO=4T4udxZRSMoQN64CHG=D=k...@mail.gmail.com> ,
Risto Vaarandi writes:
>hi John,
>in the 2.7 version there is an alternative for measuring the lag. I've
>added few lines of code to the dump file creation function which
>record the size and current read offset for the input file (this is
>done only for regular files, not FIFOs). As a result, the recorded
>statistics looks like this:
>
>Input sources:
>============================================================
>/var/log/messages (status: Open, type: regular file, read offset:
>1214299, file size: 1214299, device/inode: 2050/12519, received data:
>13 lines, no context set)
>
>From the example above, it is evident that there is no lag, since all
>available bytes have been successfully read. Nevertheless, if inputs
>are updated at a high rate, a difference of several kilobytes could be
>observed. That's because creating the dump file requires extra
>processing time, and during this time few additional KBs could arrive.
I assume the first thing in the dump handler is a stat of the input
files, so looping over all input files may take a little time but I
wouldn't expect that much of a change.
>In my own environment, I've seen accidental differences of 8-10KB
>which disappear when a new dump file is created in the next second.
>
>However, it is important to know that the read offset is not always
>reflecting the amount of already processed input data. Since reading
>from file is done by up to 1KB chunks, these chunks could easily
>contain several lines. As a consequence, only the first line is
>matched against rules immediately, while the rest will reside in a
>buffer for a very short amount of time.
I wonder if the "read offset" could be quickly turned into a
"processed offset" (which is what I attempt to capture) using:
read offset - (size of buffer contents - end of current line in buffer)
>Nevertheless, any real lag can be certainly detected by comparing
>recorded read offsets with file sizes.
Yup. I think that would work to throw into an rrd for monitoring.
One thing I was considering was writing a perl function that I could
call from a calender rule that would allow me to dump the stats on a
regular basis into and rrd or other database. How easily could I reuse
your changes in such a senario?
--
-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.
------------------------------------------------------------------------------
Master HTML5, CSS3, ASP.NET, MVC, AJAX, Knockout.js, Web API and
much more. Get web development skills now with LearnDevNow -
350+ hours of step-by-step video tutorials by Microsoft MVPs and experts.
SALE $99.99 this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122812
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
