Arianna So I seem to be stuck here, with no access to my historical data even though it is piling up in sqlite files in /var/tmp. Updating to the latest versions in SVN has not solved the problem. Reinstalling everything and deleting the previous historical data has not fixed anything. Searching the archives lead me to a number of other Centos users over the last few months complaining they can’t get historical data, but nothing pointing to a solution. I tried looking at debug level output for ntopng, and even straced it. But I can see nothing to point to the reason this is not working :( If there’s anything more you can suggest I can do I’d be happy to try. Inability to access historical sflow data is a bit of a showstopper for me. Barnaby
nprobe.conf: --collector-port=6343 --zmq=tcp://127.0.0.1:5556 -G= -I=nprobe -g=/var/run/nprobe.pid ntopng.conf -i=tcp://127.0.0.1:5556 -d=/var/tmp -w=3000 -m=“ … " -e= -F=db -p=/etc/ntopng/protos.txt -A=2 -C= -D=all -E=all -S=all -G=/var/run/ntopng/ntopng.pid This is all I ever see in the ntopng log: 09/Feb/2015 18:54:11 [NetworkInterface.cpp:958] Cleanup interface Historical 09/Feb/2015 18:54:11 [NetworkInterface.cpp:932] Started packet polling on interface Historical [id: 1]... > > On Feb 5, 2015, at 5:45 AM, Luca Deri <[email protected]> wrote: > >> Correct >> >> Sent from my iPad >> >> On 04 Feb 2015, at 21:29, barnaby cockcroft <[email protected]> wrote: >> >>> >>> As in the *8914* set of ntopng/pfring related rpms dated 2/4/15? >>> >>> On Feb 4, 2015, at 12:07 AM, Luca Deri <[email protected]> wrote: >>> >>>> Barnaby >>>> this bug should have been fixed over the week-end. Can you please make >>>> sure you are running the latest ntopng code present in SVN? >>>> >>>> Regards Luca >>>> >>>> On 02/03/2015 10:40 PM, barnaby cockcroft wrote: >>>>> >>>>> >>>>> Arianna >>>>> >>>>> My reply was blocked because I included a small screenshot of the >>>>> overview tab. Anyhow it’s removed from this email, and I’m resending. >>>>> >>>>> After the “data loading process started successfully” I see nothing - no >>>>> animation, no notifications. I do not believe any process is running to >>>>> load data - the overview tab has no new tabs. >>>>> >>>>> I do see a number of these messages in the log: >>>>> >>>>> Feb 3 13:15:51 mgmt10 ntopng: [Lua.cpp:3651] WARNING: Script failure >>>>> [/usr/share/ntopng/scripts/callbacks/second.lua][/usr/share/ntopng/scripts/callbacks/second.lua:41: >>>>> /var/tmp/0/rrd/bytes.rrd: not a simple integer: '1.8446744072412e+19'] >>>>> >>>>> However, I was having this problem before seeing this error message, when >>>>> I had less historical data than I do now. >>>>> >>>>> Data is certainly piling up: >>>>> >>>>> 3.3G 0/flows >>>>> 951M 0/rrd >>>>> 37M 0/top_talkers >>>>> >>>>> I start the process on the command line using a config file called >>>>> /etc/ntopng.conf - it’s shown below in the email thread. >>>>> >>>>> I appreciate your help, >>>>> >>>>> Barnaby >>>>> On Jan 30, 2015, at 2:00 PM, Arianna Avanzini <[email protected]> wrote: >>>>> >>>>>> Hi Barnaby, >>>>>> >>>>>> On 30/01/2015 22:49, barnaby cockcroft wrote: >>>>>>> >>>>>>> Upgraded to 8884 this morning. No difference. >>>>>>> >>>>>>> Can you tell me what I should expect to happen after choosing a >>>>>>> historical period and the screen going green and saying “data started >>>>>>> loading successfully”? What happens is that I move over to the overview >>>>>>> tab there’s no data at all, and when I go back to the configuration >>>>>>> page there’s no feedback regarding the supposed loading of the >>>>>>> historical data. No other screens show any data either. There are >>>>>>> definitely sqlite files in /var/tmp/0/flows/2015 that cover the time >>>>>>> periods I tried to load. >>>>>>> >>>>>> >>>>>> After seeing the "data loading process started successfully" message you >>>>>> should see an animation showing the load in progress in the right part >>>>>> of the webpage footer. In that same position (bottom right corner of the >>>>>> screen) you should see notifications with the format "XX Loaded Files", >>>>>> "XX Missing Files" and "XX Query Error". Do you see them? Do they show >>>>>> that anything was loaded, or was seen to be missing? >>>>>> >>>>>> If anything is successfully loaded, as soon as you select the "Overview" >>>>>> tab, two more tabs should appear ("Packets" and "Protocols"). The tabs >>>>>> should show historical stats regarding the interface in the time >>>>>> interval you selected. >>>>>> >>>>>> Also, if you don't mind me asking, are you starting ntopng from command >>>>>> line? Do you see any warning or error message after you click on "Load >>>>>> Historical Data"? >>>>>> >>>>>> Thank you, >>>>>> Arianna >>>>>> >>>>>> >>>>>>> But I have no clear idea of what I should be seeing, so it’s hard for >>>>>>> me to even define what functionality is broken and how. >>>>>>> >>>>>>> On Jan 29, 2015, at 2:22 PM, Arianna Avanzini <[email protected]> wrote: >>>>>>> >>>>>>>> Hi Barnaby, >>>>>>>> >>>>>>>> On 29/01/2015 19:04, barnaby cockcroft wrote: >>>>>>>>> 1.2.2 >>>>>>>>> >>>>>>>>> Speficially these rpms, on Centos 6: >>>>>>>>> ntopng-data-1.2.2-8774.noarch >>>>>>>>> ntopng-1.2.2-8774.x86_64 >>>>>>>>> >>>>>>>> >>>>>>>> Thank you for the information. Could you please try the latest SVN >>>>>>>> (r8884)? The development repo has just been updated with a few fixes >>>>>>>> related to the Historical interface. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Arianna >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> On Jan 29, 2015, at 7:43 AM, Arianna Avanzini <[email protected] >>>>>>>>> <mailto:[email protected]>> wrote: >>>>>>>>> >>>>>>>>>> On 29/01/2015 00:17, barnaby cockcroft wrote: >>>>>>>>>>> >>>>>>>>>>> I am having a hard time getting any historical data out of ntopng. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi Barnaby, >>>>>>>>>> >>>>>>>>>>> I think I’m collecting data: >>>>>>>>>>> >>>>>>>>>>> [[email protected] <mailto:[email protected]> tmp]# du -sh /var/tmp/0/* >>>>>>>>>>> 1.6G /var/tmp/0/flows >>>>>>>>>>> 882M /var/tmp/0/rrd >>>>>>>>>>> 25M /var/tmp/0/top_talkers >>>>>>>>>>> >>>>>>>>>>> The flows directory has a sqlite db for each 5 minute period, eg: >>>>>>>>>>> >>>>>>>>>>> ls -l 0/flows/2015/01/22/11/ >>>>>>>>>>> total 15800 >>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1418240 Jan 22 11:05 00.sqlite >>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1333248 Jan 22 11:10 05.sqlite >>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1362944 Jan 22 11:15 10.sqlite >>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1313792 Jan 22 11:20 15.sqlite >>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1283072 Jan 22 11:25 20.sqlite >>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1417216 Jan 22 11:30 25.sqlite >>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1336320 Jan 22 11:35 30.sqlite >>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1294336 Jan 22 11:40 35.sqlite >>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1395712 Jan 22 11:45 40.sqlite >>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1328128 Jan 22 11:50 45.sqlite >>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1349632 Jan 22 11:55 50.sqlite >>>>>>>>>>> -rw-r--r-- 1 nobody nobody 1321984 Jan 22 12:00 55.sqlite >>>>>>>>>>> >>>>>>>>>>> The rrd directory has rrd files for individual protocols, and for >>>>>>>>>>> each host >>>>>>>>>>> on the network a directory of similar protocol rrd files. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> My configuration file is as follows: >>>>>>>>>>> >>>>>>>>>>> [[email protected] <mailto:[email protected]> tmp]# cat /etc/ntopng.conf >>>>>>>>>>> -i=tcp://127.0.0.1:5556 >>>>>>>>>>> -d=/var/tmp >>>>>>>>>>> -w=3000 >>>>>>>>>>> -m=“XXXXXXXXX" >>>>>>>>>>> -e= >>>>>>>>>>> -F=db >>>>>>>>>>> -p=/etc/ntopng/protos.txt >>>>>>>>>>> -A=2 >>>>>>>>>>> -C= >>>>>>>>>>> -D=all >>>>>>>>>>> -E=all >>>>>>>>>>> -S=all >>>>>>>>>>> -G=/var/run/ntopng/ntopng.pid >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> When I choose “historical” from interfaces rather than >>>>>>>>>>> “tcp://127.0.0.1:5556”, I am redirected to specify a time period. >>>>>>>>>>> When I do, >>>>>>>>>>> it tells me “Well done! Data loading process started successfully”. >>>>>>>>>>> >>>>>>>>>>> However, the overview tab says I have no data, and any tab where >>>>>>>>>>> I’d imagine >>>>>>>>>>> I’d be able to see flows or hosts I get “No results found” in a >>>>>>>>>>> light red bar >>>>>>>>>>> across the screen. >>>>>>>>>>> >>>>>>>>>>> Even in regular mode, I notice I never have more than an hour’s >>>>>>>>>>> worth of data >>>>>>>>>>> in the “activity map”. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I assume I’m making some rookie mistake here, but I haven’t been >>>>>>>>>>> able to >>>>>>>>>>> figure out what. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Which ntopng version are you using? >>>>>>>>>> >>>>>>>>>> Thank you, >>>>>>>>>> Arianna >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Ntop-misc mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> <mailto:[email protected]> >>>>>>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> /* >>>>>>>>>> * Arianna Avanzini >>>>>>>>>> *[email protected] <mailto:[email protected]> >>>>>>>>>> *http://ava.webhop.me <http://ava.webhop.me/> >>>>>>>>>> */ >>>>>>>>>> _______________________________________________ >>>>>>>>>> Ntop-misc mailing list >>>>>>>>>> [email protected] >>>>>>>>>> <mailto:[email protected]> >>>>>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> /* >>>>>>>> * Arianna Avanzini >>>>>>>> * [email protected] >>>>>>>> * http://ava.webhop.me >>>>>>>> */ >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> /* >>>>>> * Arianna Avanzini >>>>>> * [email protected] >>>>>> * http://ava.webhop.me >>>>>> */ >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Ntop-misc mailing list >>>>> [email protected] >>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>>> >>>> _______________________________________________ >>>> Ntop-misc mailing list >>>> [email protected] >>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>> >>> _______________________________________________ >>> Ntop-misc mailing list >>> [email protected] >>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >> _______________________________________________ >> Ntop-misc mailing list >> [email protected] >> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >
_______________________________________________ Ntop-misc mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-misc
