Hi Garret Sorry for the late reply, have used www-data for the permissions
root@flows:/data/netflow-data/profiles-data# ls -la total 20 drwxrwxr-x 5 www-data www-data 4096 Aug 7 21:34 . drwxr-xr-x 9 www-data www-data 4096 Aug 7 16:13 .. drwxrwxr-x 4 www-data www-data 4096 Aug 7 16:14 live drwxrwxr-x 3 www-data www-data 4096 Aug 7 21:40 lyndonteracojhb_gp drwxrwxr-x 6 www-data www-data 4096 Aug 7 21:40 mtdbnc1_kzn root@flows:/data/netflow-data/profiles-data# ls -la /usr/bin/nf* -rwxr-xr-x 1 www-data www-data 113632 Aug 7 19:48 /usr/bin/nfanon -rwxr-xr-x 1 www-data www-data 645752 Aug 7 19:48 /usr/bin/nfcapd -rwxr-xr-x 1 www-data www-data 364128 Aug 7 19:48 /usr/bin/nfdump -rwxr-xr-x 1 www-data www-data 127304 Aug 7 19:48 /usr/bin/nfexpire -rwxr-xr-x 1 www-data www-data 138784 Aug 7 19:48 /usr/bin/nfprofile -rwxr-xr-x 1 www-data www-data 514640 Aug 7 19:48 /usr/bin/nfreplay -rwxr-xr-x 1 www-data www-data 111880 Aug 7 19:48 /usr/bin/nftrack root@flows:/data/netflow-data/profiles-data# new files under the live directory are reared with the below permissions -rw-r--r-- 1 www-data www-data 13519 Aug 8 06:40 nfcapd.201608080635 -rw-r--r-- 1 www-data www-data 276 Aug 8 06:40 nfcapd.current.28699 nothing is created under the other profiles > On 07 Aug 2016, at 22:00, Garrett Burke <gbu...@egenera.com> wrote: > > Which user is nfcapd running as? > > My config has nfcapd running as user Netflow. > > drwxrwxr-x 5 netflow apache 4096 Jun 22 07:30 live > drwxrwxr-x 8 netflow apache 4096 Jun 2 15:44 PARTNER1 > > -----Original Message----- > From: Lyndon Labuschagne [mailto:lynd...@mobiletorque.co.za] > Sent: Sunday, August 7, 2016 8:57 PM > To: Garrett Burke <gbu...@egenera.com> > Subject: Re: [Nfsen-discuss] Error reading channel stat information. Missing > key 'first' > > I have tried differing permissions but the user and group is www-data the > apache user > > no errors regarding permissions that I can see > > >> On 07 Aug 2016, at 21:52, Garrett Burke <gbu...@egenera.com> wrote: >> >> Are the permissions correct on the other profile directory? Check the logs >> that it's not readable. I've seen an issue where you need to set the group >> to allow the hierarchy to be traversed. >> >> -----Original Message----- >> From: Lyndon Labuschagne [mailto:lynd...@mobiletorque.co.za] >> Sent: Sunday, August 7, 2016 8:49 PM >> To: Garrett Burke <gbu...@egenera.com> >> Cc: nfsen-discuss@lists.sourceforge.net >> Subject: Re: [Nfsen-discuss] Error reading channel stat information. Missing >> key 'first' >> >> Hi >> >> Yes they have been committed and the the status is ok >> >> >> with more digging around I see that the nfcapd.YYYYMMDDHHMM files are only >> created under the live profile and not under the other profiles # >> name lyndonteracojhb_gp >> group (nogroup) >> tcreate Sun Aug 7 21:34:19 2016 >> tstart Sun Aug 7 21:47:18 2016 >> tend Sun Aug 7 21:47:18 2016 >> updated Sun Aug 7 21:47:18 2016 >> expire 60 days 0 hours >> size 0 >> maxsize 10.0 GB >> type continuous >> locked 0 >> status OK >> version 130 >> channel lyndonteracojhb_gp sign: + colour: #0099FF order: 1 >> sourcelist: lyndonteracojhb_gp ERR Error reading channel stat information. >> Missing key 'first' >> Files: 0 Size: 0 >> >> >> >>> On 07 Aug 2016, at 21:44, Garrett Burke <gbu...@egenera.com> wrote: >>> >>> Lyndon, >>> >>> Did you commit the new profile? >>> >>> GB >>> >>> -----Original Message----- >>> From: Lyndon Labuschagne [mailto:lynd...@mobiletorque.co.za] >>> Sent: Sunday, August 7, 2016 7:24 PM >>> To: nfsen-discuss@lists.sourceforge.net >>> Subject: Re: [Nfsen-discuss] Error reading channel stat information. >>> Missing key 'first' >>> >>> OK went one further scraped the VM >>> clean install of Ubuntu 16.04 >>> Running nfsen (1.3.7) and the latest nfdump from github >>> >>> rrdtool 1.5.5 >>> and still the same issue, RRDs are only generated on the live profile >>> >>> any advise would be much appreciated >>> >>>> On 07 Aug 2016, at 13:00, Lyndon Labuschagne <lynd...@mobiletorque.co.za> >>>> wrote: >>>> >>>> Hi All >>>> >>>> I really hope someone can assist >>>> >>>> It seems a few people have run into this issue, and the resolution >>>> seems to be related to either a change in rrdtool (upgrade or >>>> downgrade) or having meaningful expiry information >>>> >>>> I have tried this on Ubuntu 16 and 14, rrdtool 1.5.5 rrdtool 1.5.6 >>>> and now rrdtool 1.4.7 >>>> >>>> This is the error i get >>>> Aug 7 12:55:15 flows nfsen[1868]: Update profile mtdbnc1_kzn in >>>> group Routers Aug 7 12:55:15 flows nfsen[1868]: Error reading channel >>>> stat information. Missing key 'first' >>>> Aug 7 12:55:15 flows nfsen[1868]: message repeated 3 times: [ Error >>>> reading channel stat information. Missing key 'first'] >>>> >>>> the Live profile works fine, I have created a Routers profile and set the >>>> expiry to 5 G / 30 Days. >>>> The Routers profile is Continous Profile individual channels, and a >>>> Real Profile >>>> >>>> >>>> Here is a log extract >>>> >>>> Aug 7 12:55:00 flows nfcapd[1577]: Ident: 'lyndonteracojhb_gp' Flows: >>>> 750, Packets: 52492, Bytes: 72559620, Sequence Errors: 0, Bad Packets: >>>> 0 Aug 7 12:55:00 flows nfcapd[1577]: Ident: 'mtdbnc1_kzn' Flows: >>>> 27930, Packets: 248372, Bytes: 64251361, Sequence Errors: 0, Bad >>>> Packets: 0 Aug 7 12:55:00 flows nfcapd[1577]: Total ignored packets: >>>> 0 Aug 7 12:55:15 flows nfsen[2467]: connection on UNIX socket Aug >>>> 7 >>>> 12:55:15 flows nfsen[2467]: comm server started: 20929 Aug 7 >>>> 12:55:15 flows nfsen[20929]: Cmd Decode: signal Aug 7 12:55:15 flows >>>> nfsen[20929]: Cmd Decode: quit Aug 7 12:55:15 flows nfsen[2466]: Signal >>>> 'start-periodic' >>>> Aug 7 12:55:15 flows nfsen[2466]: Run periodic at Sun Aug 7 >>>> 12:55:00 >>>> 2016 Aug 7 12:55:15 flows nfsen[2466]: Prepare profiling './live' >>>> Aug 7 12:55:15 flows nfsen[2466]: Prepare profiling 'Routers/mtdbnc1_kzn' >>>> Aug 7 12:55:15 flows nfsen[2466]: 4 channels/alerts to profile Aug >>>> 7 >>>> 12:55:15 flows nfsen[2466]: Limit profilers: 4 Aug 7 12:55:15 flows >>>> nfsen[2467]: comm child[20929] terminated with no exit value Aug 7 >>>> 12:55:15 flows nfsen[20932]: profile opts: >>>> Routers#mtdbnc1_kzn#2#www#mtdbnc1_kzn for profiler 1 Aug 7 12:55:15 >>>> flows nfsen[20930]: profile opts: >>>> Routers#mtdbnc1_kzn#2#https#mtdbnc1_kzn for profiler 0 Aug 7 >>>> 12:55:15 flows nfsen[20930]: profiler 0 started Aug 7 12:55:15 >>>> flows >>>> nfsen[20932]: profiler 1 started Aug 7 12:55:15 flows nfsen[20934]: >>>> profile opts: Routers#mtdbnc1_kzn#2#ftp#mtdbnc1_kzn for profiler 2 >>>> Aug >>>> 7 12:55:15 flows nfsen[20934]: profiler 2 started Aug 7 12:55:15 >>>> flows nfsen[20930]: profiler 0 finished Aug 7 12:55:15 flows >>>> nfsen[20932]: profiler 1 finished Aug 7 12:55:15 flows nfsen[20934]: >>>> profiler 2 finished Aug 7 12:55:15 flows nfsen[20936]: profile opts: >>>> Routers#mtdbnc1_kzn#2#dns#mtdbnc1_kzn for profiler 3 Aug 7 12:55:15 >>>> flows nfsen[20936]: profiler 3 started Aug 7 12:55:15 flows nfsen[20936]: >>>> profiler 3 finished Aug 7 12:55:15 flows nfsen[2466]: Update profile live >>>> in group . >>>> Aug 7 12:55:15 flows nfsen[2466]: Add channel size 86016 Aug 7 >>>> 12:55:15 flows nfsen[2466]: Add channel size 1257472 Aug 7 12:55:15 >>>> flows nfsen[2466]: Set new profile size: 1343488 Aug 7 12:55:15 >>>> flows >>>> nfsen[2466]: Add .:live:201608071250 for plugin processing Aug 7 >>>> 12:55:15 flows nfsen[2467]: connection on UNIX socket Aug 7 >>>> 12:55:15 flows nfsen[2467]: comm server started: 20940 Aug 7 >>>> 12:55:15 flows >>>> nfsen[20940]: Cmd Decode: signal Aug 7 12:55:15 flows nfsen[20940]: >>>> Cmd Decode: quit Aug 7 12:55:15 flows nfsen[2467]: comm >>>> child[20940] terminated with no exit value Aug 7 12:55:15 flows >>>> nfsen[1868]: >>>> Signal 'start-periodic' >>>> Aug 7 12:55:15 flows nfsen[1868]: Run periodic at Sun Aug 7 >>>> 12:55:00 >>>> 2016 Aug 7 12:55:15 flows nfsen[1868]: Prepare profiling './live' >>>> Aug 7 12:55:15 flows nfsen[1868]: Prepare profiling 'Routers/mtdbnc1_kzn' >>>> Aug 7 12:55:15 flows nfsen[1868]: 4 channels/alerts to profile Aug >>>> 7 >>>> 12:55:15 flows nfsen[1868]: Limit profilers: 4 Aug 7 12:55:15 flows >>>> nfsen[2467]: connection on UNIX socket Aug 7 12:55:15 flows >>>> nfsen[2467]: comm server started: 20946 Aug 7 12:55:15 flows >>>> nfsen[20946]: Cmd Decode: signal Aug 7 12:55:15 flows nfsen[20946]: >>>> Cmd Decode: quit Aug 7 12:55:15 flows nfsen[2233]: Signal >>>> 'start-periodic' >>>> Aug 7 12:55:15 flows nfsen[2233]: Run periodic at Sun Aug 7 >>>> 12:55:00 >>>> 2016 Aug 7 12:55:15 flows nfsen[20941]: profile opts: >>>> Routers#mtdbnc1_kzn#2#dns#mtdbnc1_kzn for profiler 0 Aug 7 12:55:15 >>>> flows nfsen[20941]: profiler 0 started Aug 7 12:55:15 flows nfsen[2233]: >>>> Prepare profiling './live' >>>> Aug 7 12:55:15 flows nfsen[20945]: profile opts: >>>> Routers#mtdbnc1_kzn#2#ftp#mtdbnc1_kzn for profiler 2 Aug 7 12:55:15 >>>> flows nfsen[20945]: profiler 2 started Aug 7 12:55:15 flows nfsen[2233]: >>>> Prepare profiling 'Routers/mtdbnc1_kzn' >>>> Aug 7 12:55:15 flows nfsen[20941]: profiler 0 finished Aug 7 >>>> 12:55:15 flows nfsen[2233]: 4 channels/alerts to profile Aug 7 >>>> 12:55:15 flows nfsen[2233]: Limit profilers: 4 Aug 7 12:55:15 flows >>>> nfsen[20945]: profiler 2 finished Aug 7 12:55:15 flows nfsen[20948]: >>>> profile opts: Routers#mtdbnc1_kzn#2#www#mtdbnc1_kzn for profiler 3 >>>> Aug >>>> 7 12:55:15 flows nfsen[20948]: profiler 3 started Aug 7 12:55:15 >>>> flows nfsen[2467]: comm child[20946] terminated with no exit value >>>> Aug >>>> 7 12:55:15 flows nfsen[20943]: nfsend: exit child[20944] Exit: 255, >>>> Signal: 0, Core: 0 Aug 7 12:55:15 flows nfsen[20943]: profile opts: >>>> Routers#mtdbnc1_kzn#2#https#mtdbnc1_kzn for profiler 1 Aug 7 >>>> 12:55:15 flows nfsen[20943]: profiler 1 started Aug 7 12:55:15 >>>> flows >>>> nfsen[1868]: nfsend: exit child[20943] Exit: 0, Signal: 13, Core: 0 >>>> Aug 7 12:55:15 flows nfsen[20948]: profiler 3 finished Aug 7 >>>> 12:55:15 flows nfsen[20952]: profile opts: >>>> Routers#mtdbnc1_kzn#2#https#mtdbnc1_kzn for profiler 1 Aug 7 >>>> 12:55:15 flows nfsen[20955]: profile opts: >>>> Routers#mtdbnc1_kzn#2#www#mtdbnc1_kzn for profiler 3 Aug 7 12:55:15 >>>> flows nfsen[20952]: profiler 1 started Aug 7 12:55:15 flows >>>> nfsen[20950]: profile opts: Routers#mtdbnc1_kzn#2#dns#mtdbnc1_kzn >>>> for profiler 0 Aug 7 12:55:15 flows nfsen[20950]: profiler 0 >>>> started Aug >>>> 7 12:55:15 flows nfsen[20955]: nfsend: exit child[20956] Exit: 255, >>>> Signal: 0, Core: 0 Aug 7 12:55:15 flows nfsen[20955]: profiler 3 >>>> started Aug 7 12:55:15 flows nfsen[20953]: profile opts: >>>> Routers#mtdbnc1_kzn#2#ftp#mtdbnc1_kzn for profiler 2 Aug 7 12:55:15 >>>> flows nfsen[20953]: profiler 2 started Aug 7 12:55:15 flows >>>> nfsen[20952]: profiler 1 finished Aug 7 12:55:15 flows nfsen[2233]: >>>> nfsend: exit child[20955] Exit: 0, Signal: 13, Core: 0 Aug 7 12:55:15 >>>> flows nfsen[20950]: profiler 0 finished Aug 7 12:55:15 flows nfsen[1868]: >>>> Update profile mtdbnc1_kzn in group Routers Aug 7 12:55:15 flows >>>> nfsen[1868]: Error reading channel stat information. Missing key 'first' >>>> Aug 7 12:55:15 flows nfsen[1868]: message repeated 3 times: [ Error >>>> reading channel stat information. Missing key 'first'] Aug 7 >>>> 12:55:15 flows nfsen[1868]: Set new profile size: 0 Aug 7 12:55:15 >>>> flows >>>> nfsen[1868]: Add Routers:mtdbnc1_kzn:201608071250 for plugin >>>> processing Aug 7 12:55:15 flows nfsen[20953]: profiler 2 finished >>>> Aug >>>> 7 12:55:15 flows nfsen[2233]: Run plugins for 201608071250 Aug 7 12:55:15 >>>> flows nfsen[2233]: Run plugins done. >>>> Aug 7 12:55:15 flows nfsen[2233]: Check alerts for Sun Aug 7 >>>> 12:50:00 2016 Aug 7 12:55:15 flows nfsen[2233]: Check alerts done. >>>> Aug 7 12:55:15 flows nfsen[2233]: Run expire at Sun Aug 7 12:55:00 >>>> 2016 Aug 7 12:55:15 flows nfsen[2233]: End expire at Sun Aug 7 >>>> 12:55:00 2016 Aug 7 12:55:15 flows nfsen[2467]: connection on UNIX >>>> socket Aug 7 12:55:15 flows nfsen[2467]: comm server started: 20958 >>>> Aug 7 12:55:15 flows nfsen[20958]: Cmd Decode: signal Aug 7 >>>> 12:55:15 flows nfsen[20958]: Cmd Decode: quit Aug 7 12:55:15 flows >>>> nfsen[20958]: Cleanup Routine Aug 7 12:55:15 flows nfsen[2233]: >>>> Signal 'end-periodic' >>>> Aug 7 12:55:15 flows nfsen[2467]: comm child[20958] terminated with >>>> no exit value Aug 7 12:55:15 flows nfsen[2466]: Run plugins for >>>> 201608071250 Aug 7 12:55:15 flows nfsen[2467]: connection on UNIX >>>> socket Aug 7 12:55:15 flows nfsen[2467]: comm server started: 20959 >>>> Aug 7 12:55:15 flows nfsen[20959]: Cmd Decode: run-plugins Aug 7 >>>> 12:55:15 flows nfsen[20959]: Plugin Cycle: ., live, 201608071250 Aug >>>> 7 12:55:15 flows nfsen[20959]: Cmd Decode: quit Aug 7 12:55:15 >>>> flows >>>> nfsen[2466]: Run plugins done. >>>> Aug 7 12:55:15 flows nfsen[2466]: Check alerts for Sun Aug 7 >>>> 12:50:00 2016 Aug 7 12:55:15 flows nfsen[2466]: Check alerts done. >>>> Aug 7 12:55:15 flows nfsen[2466]: Run expire at Sun Aug 7 12:55:00 >>>> 2016 Aug 7 12:55:15 flows nfsen[2466]: Expire profile live group . low >>>> water mark: 90%% >>>> Aug 7 12:55:15 flows nfsen[2466]: nfexpire: Expired files: 0 >>>> Aug 7 12:55:15 flows nfsen[2466]: nfexpire: Expired file size: 0 B >>>> Aug 7 12:55:15 flows nfsen[2466]: nfexpire: Expired time range: 0 >>>> sec Aug 7 12:55:15 flows nfsen[2466]: nfexpire: >>>> Aug 7 12:55:15 flows nfsen[2466]: End expire at Sun Aug 7 12:55:00 >>>> 2016 Aug 7 12:55:15 flows nfsen[2467]: connection on UNIX socket >>>> Aug >>>> 7 12:55:15 flows nfsen[2467]: comm server started: 20961 Aug 7 >>>> 12:55:15 flows nfsen[2467]: comm child[20959] terminated with no >>>> exit value Aug 7 12:55:15 flows nfsen[20961]: Cmd Decode: signal >>>> Aug 7 >>>> 12:55:15 flows nfsen[20961]: Cmd Decode: quit Aug 7 12:55:15 flows >>>> nfsen[20961]: Cleanup Routine Aug 7 12:55:15 flows nfsen[2466]: >>>> Signal 'end-periodic' >>>> Aug 7 12:55:15 flows nfsen[2467]: comm child[20961] terminated with >>>> no exit value Aug 7 12:55:15 flows nfsen[1868]: Run plugins for >>>> 201608071250 Aug 7 12:55:15 flows nfsen[2467]: connection on UNIX >>>> socket Aug 7 12:55:15 flows nfsen[2467]: comm server started: 20962 >>>> Aug 7 12:55:15 flows nfsen[20962]: Cmd Decode: run-plugins Aug 7 >>>> 12:55:15 flows nfsen[20962]: Plugin Cycle: Routers, mtdbnc1_kzn, >>>> 201608071250 Aug 7 12:55:15 flows nfsen[20962]: Cmd Decode: quit >>>> Aug >>>> 7 12:55:15 flows nfsen[1868]: Run plugins done. >>>> Aug 7 12:55:15 flows nfsen[1868]: Check alerts for Sun Aug 7 >>>> 12:50:00 2016 Aug 7 12:55:15 flows nfsen[1868]: Check alerts done. >>>> Aug 7 12:55:15 flows nfsen[1868]: Run expire at Sun Aug 7 12:55:00 >>>> 2016 Aug 7 12:55:15 flows nfsen[1868]: Expire profile live group . >>>> low water mark: 90%% Aug 7 12:55:15 flows nfsen[2467]: comm child[20962] >>>> terminated with no exit value >>>> Aug 7 12:55:15 flows nfsen[1868]: nfexpire: Expired files: 0 >>>> Aug 7 12:55:15 flows nfsen[1868]: nfexpire: Expired file size: 0 B >>>> Aug 7 12:55:15 flows nfsen[1868]: nfexpire: Expired time range: 0 >>>> sec Aug 7 12:55:15 flows nfsen[1868]: nfexpire: >>>> Aug 7 12:55:15 flows nfsen[1868]: Expire profile mtdbnc1_kzn group >>>> Routers low water mark: 90%% Aug 7 12:55:15 flows nfsen[1868]: >>>> nfexpire: Force rebuild requested by stat record in >>>> /data/netflow-data/profiles-data/Routers/mtdbnc1_kzn/dns >>>> Aug 7 12:55:15 flows nfsen[1868]: nfexpire: Force rebuild requested >>>> by stat record in >>>> /data/netflow-data/profiles-data/Routers/mtdbnc1_kzn/https >>>> Aug 7 12:55:15 flows nfsen[1868]: nfexpire: Force rebuild requested >>>> by stat record in >>>> /data/netflow-data/profiles-data/Routers/mtdbnc1_kzn/www >>>> Aug 7 12:55:15 flows nfsen[1868]: nfexpire: Force rebuild requested >>>> by stat record in >>>> /data/netflow-data/profiles-data/Routers/mtdbnc1_kzn/ftp >>>> Aug 7 12:55:15 flows nfsen[1868]: nfexpire: Scanning files in >>>> /data/netflow-data/profiles-data/Routers/mtdbnc1_kzn/dns .. done. >>>> Aug 7 12:55:15 flows nfsen[1868]: nfexpire: Scanning files in >>>> /data/netflow-data/profiles-data/Routers/mtdbnc1_kzn/https .. done. >>>> Aug 7 12:55:15 flows nfsen[1868]: nfexpire: Scanning files in >>>> /data/netflow-data/profiles-data/Routers/mtdbnc1_kzn/www .. done. >>>> Aug 7 12:55:15 flows nfsen[1868]: nfexpire: Scanning files in >>>> /data/netflow-data/profiles-data/Routers/mtdbnc1_kzn/ftp .. done. >>>> Aug 7 12:55:15 flows nfsen[1868]: nfexpire: Expired files: 0 >>>> Aug 7 12:55:15 flows nfsen[1868]: nfexpire: Expired file size: 0 B >>>> Aug 7 12:55:15 flows nfsen[1868]: nfexpire: Expired time range: 0 >>>> sec Aug 7 12:55:15 flows nfsen[1868]: nfexpire: >>>> Aug 7 12:55:15 flows nfsen[1868]: End expire at Sun Aug 7 12:55:00 >>>> 2016 >>>> -------------------------------------------------------------------- >>>> - >>>> - >>>> -------- _______________________________________________ >>>> Nfsen-discuss mailing list >>>> Nfsen-discuss@lists.sourceforge.net >>>> https://lists.sourceforge.net/lists/listinfo/nfsen-discuss >>> >>> >>> --------------------------------------------------------------------- >>> - >>> -------- _______________________________________________ >>> Nfsen-discuss mailing list >>> Nfsen-discuss@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/nfsen-discuss >> > ------------------------------------------------------------------------------ _______________________________________________ Nfsen-discuss mailing list Nfsen-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
