Hi Paolo, Thanks for your assistance.
Capturing method is NetFlow, and in this case I'm using a Netflow simulator (http://www.paessler.com/tools/netflowgenerator), however I have nfacctd_time_new set to true. I compiled using --enable-rabbitmq and --enable-jansson flags. The first cache purge looks to be correct, it's the subsequent ones that seem to have the trouble. Below is the JSON debug output from the console. I started the process at approximately 2013-10-09 01:47:00 GMT: {"port_dst": 80, "peer_ip_src": "192.168.24.100", "port_src": 1000, "ip_dst": "10.0.2.254", "ip_src": "10.0.1.1", "ip_proto": "tcp", "stamp_updated": "2013-10-09 01:50:01", "stamp_inserted": "2013-10-09 01:45:00", "packets": 0, "bytes": 137} {"port_dst": 80, "peer_ip_src": "192.168.24.100", "port_src": 1000, "ip_dst": "10.0.2.254", "ip_src": "10.0.1.1", "ip_proto": "tcp", "stamp_updated": "2013-10-09 01:55:01", "stamp_inserted": "2013-10-09 01:45:00", "packets": 0, "bytes": 300} {"port_dst": 80, "peer_ip_src": "192.168.24.100", "port_src": 1000, "ip_dst": "10.0.2.254", "ip_src": "10.0.1.1", "ip_proto": "tcp", "stamp_updated": "2013-10-09 02:00:01", "stamp_inserted": "2013-10-09 01:50:00", "packets": 0, "bytes": 300} {"port_dst": 80, "peer_ip_src": "192.168.24.100", "port_src": 1000, "ip_dst": "10.0.2.254", "ip_src": "10.0.1.1", "ip_proto": "tcp", "stamp_updated": "2013-10-09 02:05:01", "stamp_inserted": "2013-10-09 01:55:00", "packets": 0, "bytes": 300} Complete config is as follows: debug: true daemonize: false nfacctd_port: 9996 nfacctd_disable_checks: true nfacctd_time_new: true ! AMQP connection details amqp_host: 127.0.0.1 amqp_user: guest amqp_passwd: guest amqp_exchange: pmacct.direct amqp_exchange_type: direct plugins: amqp[5m_ipip] ! 5 minutely IP to IP aggregate[5m_ipip]: src_host, dst_host, src_port, dst_port, proto, peer_src_ip amqp_routing_key[5m_ipip]: 5m_ipip amqp_history[5m_ipip]: 5m amqp_time_roundoff[5m_ipip]: m amqp_refresh_time[5m_ipip]: 300 Thanks, Nathan. -----Original Message----- From: pmacct-discussion [mailto:pmacct-discussion-boun...@pmacct.net] On Behalf Of Paolo Lucente Sent: Wednesday, 9 October 2013 6:13 a.m. To: pmacct-discussion@pmacct.net Subject: Re: [pmacct-discussion] amqp_plugin and the stamp_inserted timestamp Hi Nathan, I did try to reproduce your issue but with no luck. Can you say what capturing method are you using (libpcap, NetFlow, sFlow, or ..?). Is it that all data is mistakenly placed into the previous timeslot or it is possible you might be effectively receiving only bits of data belonging to the previous timeslot and bulk of the data belongs to the current timeslot? I can imagine this can happen a lot with NetFlow if nfacctd_time_new is set to false, which is default, or with libpcap and/or sFlow in case buffering is enabled. Let me know. Cheers, Paolo On Fri, Oct 04, 2013 at 12:45:49AM +0000, Nathan Kennedy wrote: > Hi all, > > I've managed to compile with rabbitmq enabled, and publish data to an > exchange - works great! > > I have noticed some strange behaviour with the stamp_inserted value. > When amqp_history is set, the stamp_inserted value is being set to the > previous slot rather than the current slot. > For example, stamp_updated may be 2013-10-04 00:37:11 but stamp_inserted is > 2013-10-04 00:30:00. > > My config for this aggregation is as follows: > > ! 5 minutely IP to IP > aggregate[5m_ipip]: src_host, dst_host > amqp_routing_key[5m_ipip]: 5m_ipip > amqp_history[5m_ipip]: 5m > amqp_time_roundoff[5m_ipip]: m > amqp_refresh_time[5m_ipip]: 10 > > I managed to get the expected results by switching lines 194 and 195 of > amqp_plugin.c, but would like to confirm that this is the correct fix and > that it won't affect anything else? > > Thanks again! > Nathan. > _______________________________________________ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
