You can also check the volume of received and processed flows in the system logs (/var/log/messages):
Oct 23 12:40:00 localhost nfcapd[13249]: Ident: 'Router' Flows: 926058, Packets: 2554980000, Bytes: 1850960251500, Sequence Errors: 23, Bad Packets: 0 Compare the values here from sections where you get "holes" with adjacent sections and see if there are differences. It's possible you are losing packets on the network. Or that your flow-expire timers are not set to 300s. If you install a system monitoring tool (e.g. munin does a decent job), you can correlate the graphs you get with the ethernet activity of your ethernet adapter. If you're missing packets, you should see holes in your ethernet traffic as well. One more idea. If you're restarting nfsen you should expect such holes. Maybe you're not the one restarting it, but maybe it is restarted by an external trigger (e.g. logrotate). Reviewing logs from that period will tell you if the flows arrive (and the box doesn't have time to process them), or if the flows simply never arrive. One last idea. You can check your routers that they have enough resources to export the flows. Depending on your router type/software version you should be able to see if it dropped netflow packets due to insufficient buffers in the router (if it happens infrequently, it could cause drops in the graph. If it happens all the time, then you won't notice it in the graphs). Good luck On Fri, Oct 23, 2015 at 11:02 AM, nathalie el haber < nathalie.elha...@gmail.com> wrote: > >Make sure they finish before the 5 minute mark. There's no easy way to do > it. Sometimes you just can't, but at least you know what causes the issue > > > > All the holes in my graphs takes 5 minutes or 10 minutes (for example the > hole is between 12:00 and 12:05 or 12:00 and 12:10 a multiple of 5) and > nfcapd take 5 minutes to generate a file what this could be meant ? > > > > > Another option would be to reduce the volume of data coming into nfsen > (e.g. either split it into multiple machines sending data from different > routers to different servers, or use netflow sampling to reduce the amount > of data). Less data should mean less processing time. If your backend > plugins need to do filtering before doing processing consider creating > profiles that do the filtering for you and run the plugin only on that > profile. > > > > I don’t think this is a problem of amount of data. I have holes in “pick > time” where I have a lot of data and also I have holes where I don’t have > so much traffic like in this picture. > > > > [image: Inline image 1] > > On Thu, Oct 22, 2015 at 4:17 PM, Borja Marcos <bor...@sarenet.es> wrote: > >> >> On Oct 22, 2015, at 1:35 PM, nathalie el haber wrote: >> >> > >By the way, is it a virtual machine or a real one? >> > >> > Yes it is a virtual machine can you provide a solution >> >> We still don't have enough data for a diagnosis. In these cases, a "more >> is better" policy can help. But >> I would recommend you to set up some monitoring software so that you can >> see if that data loss is related >> to CPU, memory or I/O shortages. >> >> Maybe just giving it more CPU resources can help. >> >> Looking at your graph, you are experiencing issues at random times, not >> just high traffic hours. You say that it was working "normally" >> before. So, what can cause this? >> >> - Have you added any plugins recently? Any of them depending on a >> database, maybe serving other applications? >> >> - Do you have other virtual machines on the same infrastructure consuming >> lots of resources? >> >> - Can you move the installation to a physical machine? >> >> Hard to give you a simple solution at this point, but I insist on the >> need to monitor resoure consumption. >> >> >> >> >> >> Borja. >> >> > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Nfsen-discuss mailing list > Nfsen-discuss@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/nfsen-discuss > >
------------------------------------------------------------------------------
_______________________________________________ Nfsen-discuss mailing list Nfsen-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
