Thanks Peter. That clarifies what I saw and (by the values he specified)
seems to confirm what Tore saw.
To serve as a reminder for users, would it be possible to mention the
set value in the 'stats' profile pages? If so, IMHO, the following would
be order of preference, with the top one being highest:
1) show date/time of earliest profile data (most relevant for date-based
expiry)
2) show most profile size (in MB/GB/etc) and time last checked (most
relevant for size-based expiry)
3) show low-water-mark as globally configured
Also, it would be useful to have a short mention (in the profile stats
page) that low-water-mark is globally configured (in nfsen.conf) as a
reminder.
For (1) and (2) I am assuming they are regularly tested for expiry (so
should not require additional processing overhead?), and (3) can just be
read from the config file.
Thanks
Ivan
On 10/01/2016 13:11, Peter Haag wrote:
> Hi Ivan, Иван,
>
> For expiring files:
> As Ivan explained, there is a size and and time limit. Whichever limit
> is reached first, triggers the expire process for that profile. What is
> important to note, is the fact, that it not only does expire one single file
> but a sequence of files. Think of the example of a water basin which has
> two water marks - a max water mark and min water mark. The expire process
> makes sure, that the water in the basin remains within the two marks. If max
> water is reached, water is dropped down to min water. The basin then fills
> up again. Your water level moves up and down between the two marks.
> max water mark is the time/size limit. If it's reached files are dropped to
> min water mark. This min water mark is max water mark * $low_water in the
> config file. $low_water is in % of max. time/size. So files are dropped until
> your size/time value falls down to this level - by default 90% and then fills
> up again. The larger your storage is the more is 10% in files or space of
> course
> and you may want to adjust this value to 95 or 98%. However, it means the
> expire
> process runs more often but shorter, but never longer than the time interval.
>
> Hope, this help
>
> - Peter
>
>
>
> On 08.01.16 15:33, Ivan Beveridge wrote:
>> Although Tore's original question remains unanswered ("profile expiry
>> based on date" does not seem to work as expected) ...
>>
>>
>> In my experience, getting close to (or breaching) the profile "max size"
>> causes a job to run to delete a block of old data (a number of nfcapd
>> files). It doesn't just delete the minimum amount (eg 5 minutes or 1
>> hour) worth, but several GB in my case.
>>
>> There have been occasions where I double-checked the disk utilisation as
>> I could not understand why it was this much lower than the "max size".
>> To this end, it appears to echo what is happening in Tore's case
>> (date-based expiry). However I would be more concerned about the
>> experience in Tore's case.
>>
>> nfcapd cannot just "overwrite the oldest record" as that will be in a
>> different file (in a different directory), so realistically it needs to
>> delete the oldest file/files in the profile.
>>
>> Ivan
>>
>> On 07/01/2016 12:09, Иван Стрельников wrote:
>>> Maybe I didn't get the last sentence, but
>>> when max size is reached, it will just overwrite the oldest record, not
>>> just stop everything.
>>>
>>> 07.01.2016 14:45 пользователь "Tore Anderson" <t...@fud.no
>>> <mailto:t...@fud.no>> написал:
>>>
>>> * Ivan Beveridge <i...@livejournalinc.com
>>> <mailto:i...@livejournalinc.com>>
>>>
>>> > The thing I would suggest checking is that you don't _also_ have
>>> > profile expiry based on "max size".
>>>
>>> I don't, and I had a much larger profile (about three years of data). I
>>> then configured the two years retention, and a few minutes later the
>>> data from January-March 2014 had unexpectedly vanished. So it seems to
>>> me to it just ends up counting the days wrong somehow.
>>>
>>> I didn't know that "Max. Size" also causes expiry to happen. Thanks for
>>> making me aware of that fact! The on-line help doesn't say anything
>>> about it, so I was under the impression that it would just stop
>>> gathering further data when the max size was reached.
>>>
>>> Tore
>>
>>
>
--
Ivan Beveridge VP Operations, LiveJournal
e: <i...@livejournalinc.com> ; w: http://livejournal.com/
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
