Arianna If I’m on the dashboard looking at the tcp://127.0.0.1:5556 interface, the gauge bottom middle flips from far left to far right, spending more time in the red (far right) than anywhere else. To the right the data count goes from 0b anywhere upto 12 or more Gbps, tells me I have 1500+ hosts and ~450 flows. I have the refresh frequency set to 1 minute. Above the guage I have the flow talking graphic. I see everything you suggest I should see except “alerts” - I have the bps/pps count, uptime, hosts and flows. And a little blue graph in between the gauge and the data in the bottom right corner.
Every tab appears to be working 100%, except perhaps the protocols tab that only lists DNS and then says it has no data. That seems peculiar. Because on the dashboard I see a lot of elasticsearch, mysql and http traffic. From time to time and independent of browser, I get a “warning, unresponsive script message ( … js/sankey.js:118 ) and this stops the UI from working. This usually gets in to a cycle wherre I stop the script then the UI works for a while, then get the error again. The cause of this error seems indeterminate, but the longer ntopng has been up, the more likely it is to happen. It doesn’t seem related at all to my historical problem, I just mention it for completeness. When I move to the historical interface and specify a time range, I get the well done message but nothing further. Even if I leave this for an hour, the overview tab still says 0 bytes. I’m picking up about 500M of sqlite files a day in /var/tmp/0/flows/2015, owned by nobody:nobody. I also have a directory in /var/tmp/0 containing a directory entry for each host, and a number of protocol rrds. I’m currently running RPM’s I picked up yesterday. Thanks! Barnaby On Feb 10, 2015, at 12:28 AM, Arianna Avanzini <[email protected]> wrote: > On Tue, Feb 10, 2015 at 3:56 AM, barnaby cockcroft <[email protected]> wrote: > Arianna > > Hi Barnaby, > > So I seem to be stuck here, with no access to my historical data even though > it is piling up in sqlite files in /var/tmp. Updating to the latest versions > in SVN has not solved the problem. Reinstalling everything and deleting the > previous historical data has not fixed anything. Searching the archives lead > me to a number of other Centos users over the last few months complaining > they can’t get historical data, but nothing pointing to a solution. I tried > looking at debug level output for ntopng, and even straced it. But I can see > nothing to point to the reason this is not working :( > If there’s anything more you can suggest I can do I’d be happy to try. > Inability to access historical sflow data is a bit of a showstopper for me. > > I'll try to look into it on a CentOS system (I seem to have missed this piece > of information before). Just to be sure I understood everything well - when > you look at the bottom of ntopng's web interface, the gauge shows traffic? > And do you see the current throughput and uptime, along with number of > alerts, flows and hosts, in the bottom right corner of the web interface? And > however when you try to load historical data you see no animation and no > notification about loaded files, right? > > Thanks, > Arianna > > > Barnaby > > nprobe.conf: > --collector-port=6343 > --zmq=tcp://127.0.0.1:5556 > -G= > -I=nprobe > -g=/var/run/nprobe.pid > > ntopng.conf > -i=tcp://127.0.0.1:5556 > -d=/var/tmp > -w=3000 > -m=“ … " > -e= > -F=db > -p=/etc/ntopng/protos.txt > -A=2 > -C= > -D=all > -E=all > -S=all > -G=/var/run/ntopng/ntopng.pid > > This is all I ever see in the ntopng log: > > 09/Feb/2015 18:54:11 [NetworkInterface.cpp:958] Cleanup interface Historical > 09/Feb/2015 18:54:11 [NetworkInterface.cpp:932] Started packet polling on > interface Historical [id: 1]... > > > >> >> On Feb 5, 2015, at 5:45 AM, Luca Deri <[email protected]> wrote: >> >>> Correct >>> >>> Sent from my iPad >>> >>> On 04 Feb 2015, at 21:29, barnaby cockcroft <[email protected]> wrote: >>> >>>> >>>> As in the *8914* set of ntopng/pfring related rpms dated 2/4/15? >>>> >>>> On Feb 4, 2015, at 12:07 AM, Luca Deri <[email protected]> wrote: >>>> >>>>> Barnaby >>>>> this bug should have been fixed over the week-end. Can you please make >>>>> sure you are running the latest ntopng code present in SVN? >>>>> >>>>> Regards Luca >>>>> >>>>> On 02/03/2015 10:40 PM, barnaby cockcroft wrote: >>>>>> >>>>>> >>>>>> Arianna >>>>>> >>>>>> My reply was blocked because I included a small screenshot of the >>>>>> overview tab. Anyhow it’s removed from this email, and I’m resending. >>>>>> >>>>>> After the “data loading process started successfully” I see nothing - no >>>>>> animation, no notifications. I do not believe any process is running to >>>>>> load data - the overview tab has no new tabs. >>>>>> >>>>>> I do see a number of these messages in the log: >>>>>> >>>>>> Feb 3 13:15:51 mgmt10 ntopng: [Lua.cpp:3651] WARNING: Script failure >>>>>> [/usr/share/ntopng/scripts/callbacks/second.lua][/usr/share/ntopng/scripts/callbacks/second.lua:41: >>>>>> /var/tmp/0/rrd/bytes.rrd: not a simple integer: '1.8446744072412e+19'] >>>>>> >>>>>> However, I was having this problem before seeing this error message, >>>>>> when I had less historical data than I do now. >>>>>> >>>>>> Data is certainly piling up: >>>>>> >>>>>> 3.3G 0/flows >>>>>> 951M 0/rrd >>>>>> 37M 0/top_talkers >>>>>> >>>>>> I start the process on the command line using a config file called >>>>>> /etc/ntopng.conf - it’s shown below in the email thread. >>>>>> >>>>>> I appreciate your help, >>>>>> >>>>>> Barnaby >>>>>> On Jan 30, 2015, at 2:00 PM, Arianna Avanzini <[email protected]> wrote: >>>>>> >>>>>>> Hi Barnaby, >>>>>>> >>>>>>> On 30/01/2015 22:49, barnaby cockcroft wrote: >>>>>>>> >>>>>>>> Upgraded to 8884 this morning. No difference. >>>>>>>> >>>>>>>> Can you tell me what I should expect to happen after choosing a >>>>>>>> historical period and the screen going green and saying “data started >>>>>>>> loading successfully”? What happens is that I move over to the >>>>>>>> overview tab there’s no data at all, and when I go back to the >>>>>>>> configuration page there’s no feedback regarding the supposed loading >>>>>>>> of the historical data. No other screens show any data either. There >>>>>>>> are definitely sqlite files in /var/tmp/0/flows/2015 that cover the >>>>>>>> time periods I tried to load. >>>>>>>> >>>>>>> >>>>>>> After seeing the "data loading process started successfully" message >>>>>>> you should see an animation showing the load in progress in the right >>>>>>> part of the webpage footer. In that same position (bottom right corner >>>>>>> of the screen) you should see notifications with the format "XX Loaded >>>>>>> Files", "XX Missing Files" and "XX Query Error". Do you see them? Do >>>>>>> they show that anything was loaded, or was seen to be missing? >>>>>>> >>>>>>> If anything is successfully loaded, as soon as you select the >>>>>>> "Overview" tab, two more tabs should appear ("Packets" and >>>>>>> "Protocols"). The tabs should show historical stats regarding the >>>>>>> interface in the time interval you selected. >>>>>>> >>>>>>> Also, if you don't mind me asking, are you starting ntopng from command >>>>>>> line? Do you see any warning or error message after you click on "Load >>>>>>> Historical Data"? >>>>>>> >>>>>>> Thank you, >>>>>>> Arianna >>>>>>> >>>>>>> >>>>>>>> But I have no clear idea of what I should be seeing, so it’s hard for >>>>>>>> me to even define what functionality is broken and how. >>>>>>>> >>>>>>>> On Jan 29, 2015, at 2:22 PM, Arianna Avanzini <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Barnaby, >>>>>>>>> >>>>>>>>> On 29/01/2015 19:04, barnaby cockcroft wrote: >>>>>>>>>> 1.2.2 >>>>>>>>>> >>>>>>>>>> Speficially these rpms, on Centos 6: >>>>>>>>>> ntopng-data-1.2.2-8774.noarch >>>>>>>>>> ntopng-1.2.2-8774.x86_64 >>>>>>>>>> >>>>>>>>> >>>>>>>>> Thank you for the information. Could you please try the latest SVN >>>>>>>>> (r8884)? The development repo has just been updated with a few fixes >>>>>>>>> related to the Historical interface. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Arianna >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Jan 29, 2015, at 7:43 AM, Arianna Avanzini <[email protected] >>>>>>>>>> <mailto:[email protected]>> wrote: >>>>>>>>>> >>>>>>>>>>> On 29/01/2015 00:17, barnaby cockcroft wrote: >>>>>>>>>>>> >>>>>>>>>>>> I am having a hard time getting any historical data out of ntopng. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi Barnaby, >>>>>>>>>>> >>>>>>>>>>>> I think I’m collecting data: >>>>>>>>>>>> >>>>>>>>>>>> [[email protected] <mailto:[email protected]> tmp]# du -sh /var/tmp/0/* >>>>>>>>>>>> 1.6G /var/tmp/0/flows >>>>>>>>>>>> 882M /var/tmp/0/rrd >>>>>>>>>>>> 25M /var/tmp/0/top_talkers >>>>>>>>>>>> >>>>>>>>>>>> The flows directory has a sqlite db for each 5 minute period, eg: >>>>>>>>>>>> >>>>>>>>>>>> ls -l 0/flows/2015/01/22/11/ >>>>>>>>>>>> total 15800 >>>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1418240 Jan 22 11:05 00.sqlite >>>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1333248 Jan 22 11:10 05.sqlite >>>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1362944 Jan 22 11:15 10.sqlite >>>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1313792 Jan 22 11:20 15.sqlite >>>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1283072 Jan 22 11:25 20.sqlite >>>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1417216 Jan 22 11:30 25.sqlite >>>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1336320 Jan 22 11:35 30.sqlite >>>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1294336 Jan 22 11:40 35.sqlite >>>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1395712 Jan 22 11:45 40.sqlite >>>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1328128 Jan 22 11:50 45.sqlite >>>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1349632 Jan 22 11:55 50.sqlite >>>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1321984 Jan 22 12:00 55.sqlite >>>>>>>>>>>> >>>>>>>>>>>> The rrd directory has rrd files for individual protocols, and for >>>>>>>>>>>> each host >>>>>>>>>>>> on the network a directory of similar protocol rrd files. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> My configuration file is as follows: >>>>>>>>>>>> >>>>>>>>>>>> [[email protected] <mailto:[email protected]> tmp]# cat /etc/ntopng.conf >>>>>>>>>>>> -i=tcp://127.0.0.1:5556 >>>>>>>>>>>> -d=/var/tmp >>>>>>>>>>>> -w=3000 >>>>>>>>>>>> -m=“XXXXXXXXX" >>>>>>>>>>>> -e= >>>>>>>>>>>> -F=db >>>>>>>>>>>> -p=/etc/ntopng/protos.txt >>>>>>>>>>>> -A=2 >>>>>>>>>>>> -C= >>>>>>>>>>>> -D=all >>>>>>>>>>>> -E=all >>>>>>>>>>>> -S=all >>>>>>>>>>>> -G=/var/run/ntopng/ntopng.pid >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> When I choose “historical” from interfaces rather than >>>>>>>>>>>> “tcp://127.0.0.1:5556”, I am redirected to specify a time period. >>>>>>>>>>>> When I do, >>>>>>>>>>>> it tells me “Well done! Data loading process started successfully”. >>>>>>>>>>>> >>>>>>>>>>>> However, the overview tab says I have no data, and any tab where >>>>>>>>>>>> I’d imagine >>>>>>>>>>>> I’d be able to see flows or hosts I get “No results found” in a >>>>>>>>>>>> light red bar >>>>>>>>>>>> across the screen. >>>>>>>>>>>> >>>>>>>>>>>> Even in regular mode, I notice I never have more than an hour’s >>>>>>>>>>>> worth of data >>>>>>>>>>>> in the “activity map”. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I assume I’m making some rookie mistake here, but I haven’t been >>>>>>>>>>>> able to >>>>>>>>>>>> figure out what. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Which ntopng version are you using? >>>>>>>>>>> >>>>>>>>>>> Thank you, >>>>>>>>>>> Arianna >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> Ntop-misc mailing list >>>>>>>>>>>> [email protected] >>>>>>>>>>>> <mailto:[email protected]> >>>>>>>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> /* >>>>>>>>>>> * Arianna Avanzini >>>>>>>>>>> *[email protected] <mailto:[email protected]> >>>>>>>>>>> *http://ava.webhop.me <http://ava.webhop.me/> >>>>>>>>>>> */ >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Ntop-misc mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> <mailto:[email protected]> >>>>>>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> /* >>>>>>>>> * Arianna Avanzini >>>>>>>>> * [email protected] >>>>>>>>> * http://ava.webhop.me >>>>>>>>> */ >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> /* >>>>>>> * Arianna Avanzini >>>>>>> * [email protected] >>>>>>> * http://ava.webhop.me >>>>>>> */ >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Ntop-misc mailing list >>>>>> [email protected] >>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>>>> >>>>> _______________________________________________ >>>>> Ntop-misc mailing list >>>>> [email protected] >>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>>> >>>> _______________________________________________ >>>> Ntop-misc mailing list >>>> [email protected] >>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>> _______________________________________________ >>> Ntop-misc mailing list >>> [email protected] >>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >> > > > _______________________________________________ > Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > > > > -- > /* > * Arianna Avanzini > * [email protected] > * http://ava.webhop.me > */ > _______________________________________________ > Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc
_______________________________________________ Ntop-misc mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-misc
