Hi, RRD files have by construction a constant size so you don't have to worry about that.
I am still not sure you upgraded to the latest version. Could you please make sure you are running the latest dev version? Also, only if you can afford losing historical traffic data recorded, may I suggest you to wipe contents of /storage/ntopng before restarting ntopng. Then, if you still get errors, run ntopng with option -vv and attach output. Simone On Sun, Mar 27, 2016 at 5:07 PM, Peter Ruta <[email protected]> wrote: > Hi Simone, > > Even after I added writing rights to storage I still get the same errors > but I think the issue is not only with Amazon or Google traffic but with > all the discovered protocols as my charts are really strange(See below). > Here are the files from /storage.ntopng/1/rrd directory. I see that > majority of the rrd files are 80720 bytes. Any other suggestions for > troubleshooting? > BTW, I am looking to purchase three more licenses for the customer but I > need to solve this issue first. > > Thanks, > peter > > /usr/share/ntopng/scripts/callbacks# ls -al /storage/ntopng/1/rrd/ > total 11372 > drwxrwxrwx 4 nobody nogroup 4096 Mar 18 08:45 . > drwxrwxrwx 7 root root 4096 Nov 9 18:24 .. > -rwxrwxrwx 1 nobody nogroup 0 Sep 25 2015 Amazon.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Oct 14 14:20 AppleiTunes.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Dec 21 00:55 Apple.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Nov 12 12:50 BitTorrent.rrd > -rwxrwxrwx 1 nobody nogroup 1057000 Mar 27 11:04 bytes.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 25 23:00 Citrix_Online.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 25 23:00 CNN.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Oct 13 16:20 DCE_RPC.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Sep 15 2015 DHCP.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Oct 3 06:10 DirectConnect.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 25 23:00 DNS.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 27 11:00 DropBox.rrd > -rwxrwxrwx 1 nobody nogroup 1057000 Mar 27 11:04 drops.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Oct 14 14:30 eBay.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 25 23:00 eDonkey.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 25 23:00 Facebook.rrd > drwxrwxrwx 3 nobody nogroup 4096 Sep 11 2015 fe80__250_56ff_fea2_179a > -rwxrwxrwx 1 nobody nogroup 80720 Mar 26 11:05 FTP_DATA.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 22 08:43 GloboTV.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 25 23:00 GMail.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 23 09:00 GoogleMaps.rrd > -rwxrwxrwx 1 nobody nogroup 0 Sep 25 2015 Google.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Oct 13 16:20 H323.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 19 08:36 HTTP_Connect.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 25 23:00 HTTP.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 26 11:55 ICMP.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Sep 28 08:50 ICMPV6.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 23 09:50 IGMP.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 18 09:05 IMAPS.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 24 16:30 Instagram.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 25 23:00 IPsec.rrd > -rwxrwxrwx 1 nobody nogroup 0 Oct 13 17:40 IRC.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Oct 13 16:20 Kerberos.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Oct 13 16:20 LDAP.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 27 11:00 LLMNR.rrd > drwxrwxrwx 2 nobody nogroup 4096 Sep 11 2015 localstats > -rwxrwxrwx 1 nobody nogroup 80720 Oct 13 16:20 Microsoft.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 16 00:05 MSN.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Oct 7 14:20 MsSQL.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 25 23:00 NetBIOS.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 24 19:40 NetFlix.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 27 11:00 NetFlow.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Oct 13 16:25 NFS.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 26 13:15 NTP.rrd > -rwxrwxrwx 1 nobody nogroup 711192 Mar 27 11:04 num_flows.rrd > -rwxrwxrwx 1 nobody nogroup 711192 Mar 27 11:04 num_hosts.rrd > -rwxrwxrwx 1 nobody nogroup 711192 Mar 27 11:04 num_http_hosts.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Oct 13 16:20 Oscar.rrd > -rwxrwxrwx 1 nobody nogroup 1057000 Mar 27 11:04 packets.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 25 23:00 Pandora.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 27 11:00 POP3.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Nov 12 12:50 POPS.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 25 23:00 Quic.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Oct 13 16:20 RDP.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 24 16:30 Redis.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Oct 13 16:20 RTCP.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 26 11:55 RTMP.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Oct 13 16:20 RTP.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Nov 6 08:30 RTSP.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 25 23:00 SIP.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 25 23:00 Skype.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Oct 13 16:20 SMB.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Oct 11 23:45 SMTP.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Oct 14 14:40 SMTPS.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 27 11:00 SNMP.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 26 13:15 SSDP.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 26 13:15 SSH.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 25 23:00 SSL_No_Cert.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 25 23:00 SSL.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 27 10:40 STUN.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Oct 13 16:20 TeamSpeak.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Dec 21 01:05 TeamViewer.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Nov 5 23:05 Telnet.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Oct 13 16:25 Teredo.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Nov 6 08:50 Tor.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 24 19:40 Twitch.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 16 00:05 Twitter.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Oct 20 00:35 UbuntuONE.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 25 23:00 Unknown.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Oct 14 14:40 Viber.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Oct 13 16:20 VMware.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 16 09:25 WhatsApp.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 26 13:15 Whois-DAS.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 24 16:20 Wikipedia.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 27 11:00 WindowsUpdate.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 14 15:32 WinMX.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 25 23:00 Yahoo.rrd > -rwxrwxrwx 1 nobody nogroup 80720 Mar 24 19:40 YouTube.rrd > > > > On Mar 27, 2016, at 6:00 AM, [email protected] wrote: > > Send Ntop mailing list submissions to > [email protected] > > To subscribe or unsubscribe via the World Wide Web, visit > http://listgateway.unipi.it/mailman/listinfo/ntop > or, via email, send a message with subject or body 'help' to > [email protected] > > You can reach the person managing the list at > [email protected] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Ntop digest..." > > > Today's Topics: > > 1. Re: Ntop Digest, Vol 142, Issue 39 (Simone Mainardi) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sat, 26 Mar 2016 20:03:11 +0100 > From: Simone Mainardi <[email protected]> > To: [email protected] > Subject: Re: [Ntop] Ntop Digest, Vol 142, Issue 39 > Message-ID: > <CAJcXkCCLdBvMyGQWv=7eqorud8f0mfxnoo2gt00bc3mrtge...@mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi, I can see the following warning at the end of the logs attached: > > 25/Mar/2016 23:00:04 [Lua.cpp:4419] WARNING: Script failure > > [/usr/share/ntopng/scripts/callbacks/minute.lua][/usr/share/ntopng/scripts/callbacks/minute.lua:100: > mmaping file '/storage/ntopng/1/rrd/Google.rrd': Invalid argument] > > could you please make sure ntopng has sufficient privileges to write to > /storage/ntopng/ > > I guess these warning are the root cause of your issue: ntopng tries to > write l7 protocol information but if fails and hence it shows you empty > charts. > > thank you > > simone > > > On Sat, Mar 26, 2016 at 4:04 AM, Petrica Ruta <[email protected]> wrote: > > Hi Simone, > > 1. I am currently using : 2.1.151117 > < > https://github.com/ntop/ntopng/commit/7d1f1b95fdc382fd1ae2bb435f2cd73fdc68173f> > - > Professional Edition > 2. Here is the config and the ntopng.log after start. > > I am upgrading now to the latest version. I will let you know if the same > problem occurs. > > Thanks, > Peter > > root@rxr-phonsys1:~# more /etc/ntopng/ntopng.conf > > -n=3 > > -w=3000 > > -W=0 > > -g=-1 > > -m="10.0.0.0/255.0.0.0,192.168.0.0/255.255.0.0,172.16.0.0/255.240.0.0" > > -A=2 > > -E=all > > -S=all > > -F=es;flows;ntopng-rxr-%Y.%m.%d;http://10.100.100.86:9200/_bulk; > > -d=/storage/ntopng > > -C > > -G=/var/tmp/ntopng.pid > > -i=eth0 > > -i=eth1 > > -i=eth2 > > -i=eth3 > > and the logs after restart: > > 25/Mar/2016 22:57:05 [Prefs.cpp:659] Using ElasticSearch for data dump > [flows][ntopng-rxr-%Y.%m.%d][http://10.100.100.86:9200/_bulk] > > 25/Mar/2016 22:57:05 [Ntop.cpp:933] Setting local networks to > 10.0.0.0/255.0.0.0,192.168.0.0/255.255.0.0,172.16.0.0/255.240.0.0 > > 25/Mar/2016 22:57:05 [Redis.cpp:106] Successfully connected to redis > 127.0.0.1:6379@0 > > 25/Mar/2016 22:57:05 [NtopPro.cpp:117] [LICENSE] Read license from Redis > [229166344F5741AD0A6E400AC93041281473453628F8AA6238] > > 25/Mar/2016 22:57:05 [PcapInterface.cpp:86] Reading packets from interface > eth1... > > 25/Mar/2016 22:57:05 [Ntop.cpp:1152] Registered interface eth1 [id: 0] > > 25/Mar/2016 22:57:05 [PcapInterface.cpp:86] Reading packets from interface > eth2... > > 25/Mar/2016 22:57:05 [Ntop.cpp:1152] Registered interface eth2 [id: 1] > > 25/Mar/2016 22:57:05 [PcapInterface.cpp:86] Reading packets from interface > eth0... > > 25/Mar/2016 22:57:05 [Ntop.cpp:1152] Registered interface eth0 [id: 2] > > 25/Mar/2016 22:57:05 [PcapInterface.cpp:86] Reading packets from interface > eth3... > > 25/Mar/2016 22:57:05 [Ntop.cpp:1152] Registered interface eth3 [id: 3] > > 25/Mar/2016 22:57:05 [Ntop.cpp:1165] Registered interface view eth1 [id: 0] > > 25/Mar/2016 22:57:05 [Ntop.cpp:1165] Registered interface view eth2 [id: 1] > > 25/Mar/2016 22:57:05 [Ntop.cpp:1165] Registered interface view eth0 [id: 2] > > 25/Mar/2016 22:57:05 [Ntop.cpp:1165] Registered interface view eth3 [id: 3] > > 25/Mar/2016 22:57:05 [Utils.cpp:304] User changed to nobody > > 25/Mar/2016 22:57:05 [main.cpp:240] PID stored in file /var/tmp/ntopng.pid > > 25/Mar/2016 22:57:05 [HTTPserver.cpp:458] Please read > https://github.com/ntop/ntopng/blob/dev/doc/README.SSL if you want to > enable SSL. > > 25/Mar/2016 22:57:05 [HTTPserver.cpp:501] Web server dirs > [/usr/share/ntopng/httpdocs][/usr/share/ntopng/scripts] > > 25/Mar/2016 22:57:05 [HTTPserver.cpp:504] HTTP server listening on port > 3000 > > 25/Mar/2016 22:57:05 [main.cpp:290] Working directory: /storage/ntopng > > 25/Mar/2016 22:57:05 [main.cpp:292] Scripts/HTML pages directory: > /usr/share/ntopng > > 25/Mar/2016 22:57:05 [Ntop.cpp:260] Welcome to ntopng x86_64 v.2.1.151117 > - (C) 1998-15 ntop.org > > 25/Mar/2016 22:57:05 [Ntop.cpp:265] Built on Ubuntu 12.04.5 LTS > > 25/Mar/2016 22:57:05 [PeriodicActivities.cpp:53] Started periodic > activities loop... > > 25/Mar/2016 22:57:05 [RuntimePrefs.cpp:32] Dumping alerts into syslog > > 25/Mar/2016 22:57:05 [NtopPro.cpp:234] [LICENSE] ntopng systemId: > 68A5360682072B23 > > 25/Mar/2016 22:57:05 [NtopPro.cpp:245] [LICENSE] ntopng license: > 229166344F5741AD0A6E400AC93041281473453628F8AA6238 > > 25/Mar/2016 22:57:05 [NtopPro.cpp:266] [LICENSE] Maintenance is available > until Fri Sep 9 16:40:28 2016 [167 days left] > > 25/Mar/2016 22:57:05 [Ntop.cpp:516] Adding 10.11.150.0/24 as IPv4 local > network for eth0 > > 25/Mar/2016 22:57:05 [Ntop.cpp:546] Adding fe80::250:56ff:fea2:3004/64 as > IPv6 local network for eth0 > > 25/Mar/2016 22:57:05 [Ntop.cpp:546] Adding fe80::250:56ff:fea2:179a/64 as > IPv6 local network for eth1 > > 25/Mar/2016 22:57:05 [Ntop.cpp:546] Adding fe80::250:56ff:fea2:4a2a/64 as > IPv6 local network for eth2 > > 25/Mar/2016 22:57:05 [Ntop.cpp:546] Adding fe80::250:56ff:fea2:4d44/64 as > IPv6 local network for eth3 > > 25/Mar/2016 22:57:05 [NetworkInterface.cpp:1427] Started packet polling on > interface eth1 [id: 0]... > > 25/Mar/2016 22:57:05 [NetworkInterface.cpp:1422] Setting affinity of > interface eth2 to core 1 > > 25/Mar/2016 22:57:05 [NetworkInterface.cpp:1427] Started packet polling on > interface eth2 [id: 1]... > > 25/Mar/2016 22:57:05 [NetworkInterface.cpp:1422] Setting affinity of > interface eth0 to core 2 > > 25/Mar/2016 22:57:05 [NetworkInterface.cpp:1427] Started packet polling on > interface eth0 [id: 2]... > > 25/Mar/2016 22:57:05 [NetworkInterface.cpp:1422] Setting affinity of > interface eth3 to core 3 > > 25/Mar/2016 22:57:05 [NetworkInterface.cpp:1427] Started packet polling on > interface eth3 [id: 3]... > > 25/Mar/2016 22:57:06 [NetworkInterface.cpp:950] Invalid packet received > [len: 4110][MTU: 1518]. > > 25/Mar/2016 22:57:06 [NetworkInterface.cpp:951] WARNING: If you have > TSO/GRO enabled, please disable it > > 25/Mar/2016 22:57:06 [NetworkInterface.cpp:953] WARNING: Use: sudo ethtool > -K eth0 gro off gso off tso off > > > 25/Mar/2016 23:00:04 [Lua.cpp:4419] WARNING: Script failure > > [/usr/share/ntopng/scripts/callbacks/minute.lua][/usr/share/ntopng/scripts/callbacks/minute.lua:100: > mmaping file '/storage/ntopng/1/rrd/Google.rrd': Invalid argument] > > > On Fri, Mar 25, 2016 at 5:07 AM <[email protected]> wrote: > > Send Ntop mailing list submissions to > [email protected] > > To subscribe or unsubscribe via the World Wide Web, visit > http://listgateway.unipi.it/mailman/listinfo/ntop > or, via email, send a message with subject or body 'help' to > [email protected] > > You can reach the person managing the list at > [email protected] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Ntop digest..." > > > Today's Topics: > > 1. Re: ntopng:application breakdown usage reporting issue > (Simone Mainardi) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 25 Mar 2016 10:06:44 +0100 > From: Simone Mainardi <[email protected]> > To: [email protected] > Subject: Re: [Ntop] ntopng:application breakdown usage reporting issue > Message-ID: > <CAJcXkCAnP7U=GNcjGrRFTCU5rgi8q= > [email protected]> > Content-Type: text/plain; charset="utf-8" > > Peter, > > Could you please try and > > 1. update to the latest dev version (which version are you using?) > 2. Inspect the ntopng log file -- or console output -- and post here any > error together with the full configuration used > > Also note that reports may take some time to generate. > > > On Fri, Mar 25, 2016 at 3:37 AM, Peter Ruta <[email protected]> wrote: > > Hi, > > I am using the commercial edition for a while now mainly troubleshooting > real-time security and traffic events. > Recently, my customer asked for a usage report by application/protocol > and tried with no success to generate a report all the applications > supported by ntopng. > > None of the options available work: > > -tried reporting tool and selected last 3-6h time window. I have seeing > the traffic graph(total) for any of the interfaces (that receive SPAN > traffic) as you can see below but the application protocol breakdown > doesn?t work for any of the interfaces (one shows no data and the other > shows weird traffic graph only for less than 10 applications. > > - i also tried to generate a historical report with the reporting > > tool(as > > I use the commercial version) but I have no data in the report for the > Application Breakdown. > > See below some screenshots and any advise would be welcome. > > Thanks in advance, > Peter > > > > > > > > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > > http://listgateway.unipi.it/pipermail/ntop/attachments/20160325/1bac168f/attachment.htm > > > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: PastedGraphic-2.tiff > Type: image/tiff > Size: 409864 bytes > Desc: not available > URL: < > > http://listgateway.unipi.it/pipermail/ntop/attachments/20160325/1bac168f/attachment.tiff > > > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: PastedGraphic-3.tiff > Type: image/tiff > Size: 322554 bytes > Desc: not available > URL: < > > http://listgateway.unipi.it/pipermail/ntop/attachments/20160325/1bac168f/attachment-0001.tiff > > > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: PastedGraphic-1.tiff > Type: image/tiff > Size: 295480 bytes > Desc: not available > URL: < > > http://listgateway.unipi.it/pipermail/ntop/attachments/20160325/1bac168f/attachment-0002.tiff > > > > ------------------------------ > > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop > > End of Ntop Digest, Vol 142, Issue 39 > ************************************* > > > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > http://listgateway.unipi.it/pipermail/ntop/attachments/20160326/8c410fd5/attachment-0001.htm > > > > ------------------------------ > > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop > > End of Ntop Digest, Vol 142, Issue 41 > ************************************* > > > > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop >
_______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop
