Hi Stanislaw, I would suggest a few tests to drill this issue down. You are aggregating src_as, dst_as but to better compare against SNMP i would suggest an aggregation peer_src_ip, in_iface - this way you know for sure you are comparing apples with apples without relying on the knowledge of one or multiple ASNs being beyond a specific port. Moreover this way you can also easily compare different ports so to spot whether such deltas apply to all of them - or maybe only to a specific hardware platform and/or software version.
How much sFlow data is hitting the collector? How is CPU doing on the collector box? Have you used the same sFlow data before so that you can trust the switches export is OK (or maybe you are collecting on some other collector in parallel to pmacct)? If not doing that, this is anyway a valid test at some stage: you can take sflowtool, focus on a specific switch/port for a time window and see whether pmacct output and sum of sflowtool collected data is (roughly) matching. Any suspicious message in the logs (both pmacct and mysql)? Cheers, Paolo On Thu, Oct 10, 2013 at 03:35:57PM +0300, Stanislaw Datskevich (Giganet) wrote: > Hello, list. > I'm using sfacctd to account traffic between autonomous systems at > Internet Exchange Point (IXP). > IXP runs on Extreme Networks x670 and BlackDiamond x8 switches with > enabled sFlow accounting only on customer ports ingress traffic. > I've noticed that sfacctd count much lesser traffic than SNMP port > statistics. To compare sflow AS traffic and port statistics I've > chosen one customer which announces only one AS, so theoretically AS > traffic and port traffic must be almost identical. > > I looked into port graph with 1-hour interval: during hour bandwidth > is raised from 10G to 12G (period from 14:00 to 15:00 on the > attached graph) > > I ran this SQL query: > select SUM(bytes) from flow_daily where as_src = 59613 and > stamp_inserted = 1381402800; > +--------------+ > | SUM(bytes) | > +--------------+ > | 677594784256 | > +--------------+ > Where as_src is AS of that customer who announces only one AS, > stamp_inserted is interval I measure. So this shows summary > transmitted by that AS traffic during hour from 14:00 to 15:00. > > Converting transmitted bytes to average speed: > > 677594784256 * 8 = 5420758274048 bits > 5420758274048 / 3600 = 1505766187 average bits per second > 1505766187 / 1024 / 1024 / 1024 = 1.4Gbit per second. > > > > Details of my pmacct system: > pmacct version 0.14.3 > sfacctd config: > syslog: local6 > daemonize: true > sfacctd_port: 6343 > sfacctd_renormalize: true > sfacctd_net: bgp > sfacctd_as_new: bgp > > ! BGP settings > bgp_daemon: true > bgp_daemon_msglog: false > bgp_peer_src_as_type: bgp > bgp_daemon_ip: 91.245.221.251 > bgp_daemon_port: 179 > bgp_agent_map: /srv/pmacct/etc/bgp_agent_map > > sfacctd_ip: 172.17.172.3 > plugins: mysql[daily], mysql[monthly], mysql[tmp] > plugin_pipe_size[daily]: 68812800 > plugin_pipe_size[monthly]: 68812800 > plugin_pipe_size[tmp]: 68812800 > > plugin_buffer_size[daily]: 22400 > plugin_buffer_size[monthly]: 22400 > plugin_buffer_size[tmp]: 22400 > > aggregate[daily]: src_as, dst_as > aggregate[monthly]: src_as, dst_as > aggregate[tmp]: src_as, dst_as > > sql_table[daily]: flow_daily > sql_table[monthly]: flow_monthly > sql_table[tmp]: flow_tmp > > sql_host[daily]: localhost > sql_host[monthly]: localhost > sql_host[tmp]: localhost > > sql_user[daily]: pmacct > sql_user[monthly]: pmacct > sql_user[tmp]: pmacct > > sql_passwd[daily]: 1 > sql_passwd[monthly]: 1 > sql_passwd[tmp]: 1 > > sql_db[daily]: pmacct > sql_db[monthly]: pmacct > sql_db[tmp]: pmacct > > sql_table_version[daily]: 6 > sql_table_version[monthly]: 6 > sql_table_version[tmp]: 6 > > sql_dont_try_update[daily]: true > sql_dont_try_update[monthly]: true > sql_dont_try_update[tmp]: true > > sql_multi_values[daily]: 140000000 > sql_multi_values[monthly]: 140000000 > sql_multi_values[tmp]: 140000000 > > sql_cache_entries[daily]: 14000003 > sql_cache_entries[monthly]: 14000003 > sql_cache_entries[tmp]: 14000003 > > sql_locking_style[daily]: row > sql_locking_style[monthly]: row > sql_locking_style[tmp]: row > > sfacctd_disable_checks: true > sql_history_since_epoch[daily]: true > sql_history_since_epoch[monthly]: true > sql_history_since_epoch[tmp]: true > > sql_history_roundoff[daily]: h > sql_history_roundoff[monthly]: d > sql_history_roundoff[tmp]: h > > sql_history[daily]: 1h > sql_history[monthly]: 1d > sql_history[tmp]: 1h > > sql_refresh_time[daily]: 3600 > sql_refresh_time[monthly]: 86400 > sql_refresh_time[tmp]: 3600 > > > My MySQL table structure: > | flow_daily | CREATE TABLE `flow_daily` ( > `agent_id` int(4) unsigned NOT NULL, > `class_id` char(16) NOT NULL, > `mac_src` char(17) NOT NULL, > `mac_dst` char(17) NOT NULL, > `vlan` int(2) unsigned NOT NULL, > `as_src` int(4) unsigned NOT NULL, > `as_dst` int(4) unsigned NOT NULL, > `ip_src` char(15) NOT NULL, > `ip_dst` char(15) NOT NULL, > `src_port` int(2) unsigned NOT NULL, > `dst_port` int(2) unsigned NOT NULL, > `ip_proto` char(6) NOT NULL, > `tos` int(4) unsigned NOT NULL, > `packets` int(10) unsigned NOT NULL, > `bytes` bigint(20) unsigned NOT NULL, > `flows` int(10) unsigned NOT NULL, > `stamp_inserted` int(8) NOT NULL DEFAULT '0', > `stamp_updated` int(8) DEFAULT NULL, > PRIMARY KEY (`stamp_inserted`,`as_src`,`as_dst`) > ) ENGINE=InnoDB DEFAULT CHARSET=utf8 > /*!50100 PARTITION BY RANGE (stamp_inserted) > (PARTITION p2013_09_08 VALUES LESS THAN (1378674000) ENGINE = InnoDB, > PARTITION p2013_09_09 VALUES LESS THAN (1378760400) ENGINE = InnoDB, > PARTITION p2013_09_10 VALUES LESS THAN (1378846800) ENGINE = InnoDB, > PARTITION p2013_09_11 VALUES LESS THAN (1378933200) ENGINE = InnoDB, > PARTITION p2013_09_12 VALUES LESS THAN (1379019600) ENGINE = InnoDB, > PARTITION p2013_09_13 VALUES LESS THAN (1379106000) ENGINE = InnoDB, > PARTITION p2013_09_14 VALUES LESS THAN (1379192400) ENGINE = InnoDB, > PARTITION p2013_09_15 VALUES LESS THAN (1379278800) ENGINE = InnoDB, > PARTITION p2013_09_16 VALUES LESS THAN (1379365200) ENGINE = InnoDB, > PARTITION p2013_09_17 VALUES LESS THAN (1379451600) ENGINE = InnoDB, > PARTITION p2013_09_18 VALUES LESS THAN (1379538000) ENGINE = InnoDB, > PARTITION p2013_09_19 VALUES LESS THAN (1379624400) ENGINE = InnoDB, > PARTITION p2013_09_20 VALUES LESS THAN (1379710800) ENGINE = InnoDB, > PARTITION p2013_09_21 VALUES LESS THAN (1379797200) ENGINE = InnoDB, > PARTITION p2013_09_22 VALUES LESS THAN (1379883600) ENGINE = InnoDB, > PARTITION p2013_09_23 VALUES LESS THAN (1379970000) ENGINE = InnoDB, > PARTITION p2013_09_24 VALUES LESS THAN (1380056400) ENGINE = InnoDB, > ... and other partitions up to today. > > Can you please help me with this issue? > > > _______________________________________________ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
