On 10.08.21 16:14, Jens Hektor wrote:
> Am 10.08.21 um 15:55 schrieb Brian Candler:
>> On 10/08/2021 14:30, nfsen-discuss-requ...@lists.sourceforge.net wrote:
>>> Particularly I try to look at top talkers of these files, especially in the
>>> "inet6" domain:
>>> -rw-r--r--. 1 apache apache 3,5G 9. Aug 08:00
>>> /usr/local/nfsen/profiles-data/live/ixia-poc/2021/08/09/nfcapd.202108090755
>>> -rw-r--r--. 1 apache apache 2,3G 9. Aug 08:04
>>> /usr/local/nfsen/profiles-data/live/ixia-poc/2021/08/09/nfcapd.202108090800
>>> -rw-r--r--. 1 apache apache 891M 9. Aug 08:10
>>> /usr/local/nfsen/profiles-data/live/ixia-poc/2021/08/09/nfcapd.202108090805
>>> -rw-r--r--. 1 apache apache 702M 9. Aug 08:15
>>> /usr/local/nfsen/profiles-data/live/ixia-poc/2021/08/09/nfcapd.202108090810
>>> -rw-r--r--. 1 apache apache 674M 9. Aug 08:20
>>> /usr/local/nfsen/profiles-data/live/ixia-poc/2021/08/09/nfcapd.202108090815
>>> -rw-r--r--. 1 apache apache 737M 9. Aug 08:25
>>> /usr/local/nfsen/profiles-data/live/ixia-poc/2021/08/09/nfcapd.202108090820
>>>
>>> 2021/08/09/nfcapd.202108090820: Sys: 36.063s
>>> 2021/08/09/nfcapd.202108090815: Sys: 38.893s
>>> 2021/08/09/nfcapd.202108090810: Sys: 35.795s
>>> 2021/08/09/nfcapd.202108090805: Sys: 6141.546s
>>> 2021/08/09/nfcapd.202108090800: - still waiting (started 3 hours ago)
>
> This is still running (now for 20+ h).
>
>> I would be inclined to look at the RSS of the nfdump process, and overall
>> RAM utilisation of your system, while those queries are going on. (e.g. "top
>> -o RES", "watch free")
>
> The process does not need much RAM (0.5%)
>
>> My guess is that the RAM usage is going so high that it's sending your
>> system heavily into swap.
>
> The system has 128G mostly used for buffers.
>
>> You may find that having a *small* swap partition (e.g. 1GB) is better than
>> a large one; or just turn off swap entirely. If nfdump runs out of RAM it
>> will be killed by the OOM killer, but at least your system won't turn into
>> treacle and
>> freeze.
>
> RAM is not my trouble.
>
> My guess is that IPv6 calculations for top talker are CPU wise "expensive".
Actually there should be no difference of IPv4 and v6 - calculation wise. I
rather suspect the internal hash does not scale well enough, if IPv6 is
used heavily. See my previous mail regarding testing nfdump 1.7 best, unicorn
branch.
- peter
>
>> Can you show the actual nfdump query you were using? What were you grouping
>> on?
>
> /usr/local/bin/nfdump -M /usr/local/nfsen/profiles-data/live/ixia-poc -r
> 2021/08/09/nfcapd.202108090800 -n 10 -s ip/flows -6 "(( ident ixia-poc) and (
> ( inet6 and src net 2a00:8a60::/32 ) ) or ( ident ixia-poc) and ( ( inet6 and
> dst net
> 2a00:8a60::/32 ) ))"
>
>
>
> _______________________________________________
> Nfsen-discuss mailing list
> Nfsen-discuss@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
>
--
Be nice to your netflow data. Use NfSen and nfdump :)
_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
