Re: [pmacct-discussion] hsflowd & sfacctd - flow generation & analysis
Hi Nicolas, Support for sFlow counters was introduced in 1.5.2 and made more robust in 1.5.3. However consider this is interface counter stats; the host sFlow structs is currently not supported - we can think about it if there is interest around it. Same applies to the agent side of the things, which was your second question. Cheers, Paolo On Tue, Feb 23, 2016 at 05:47:57PM +0100, nicolas prochazka wrote: > Hello, > it seems that sfacctd discards the counter sample and works only on flow > sample, > so host sflow ( http://www.sflow.net/ ) cannot be used with pmacct . > > However, does it exist a solution to use pmacct with the host sflow agent ? > > > Regards, > Nicolas Prochazka > ___ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
[pmacct-discussion] hsflowd & sfacctd - flow generation & analysis
Hello, it seems that sfacctd discards the counter sample and works only on flow sample, so host sflow ( http://www.sflow.net/ ) cannot be used with pmacct . However, does it exist a solution to use pmacct with the host sflow agent ? Regards, Nicolas Prochazka ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] hsflowd sfacctd - flow generation analysis
Hi David, The other possibility that comes to mind is that you are only exporting counter samples (disregarded by pmacct) and not flow samples (used by pmacct). You can confirm this by reading your sFlow packets with sflowtool. Alternatively you can follow-up privately and send me a brief capture of your sflow packets for some investigation (needless to say i'm happy to support). Cheers, Paolo On Thu, Nov 06, 2014 at 04:23:05PM +0100, David Winterstein wrote: Hi Paolo, I changed the sfacct config as you suggested so it looks like this now for the time of testing: debug: true daemonize:false aggregate: src_host,dst_host,proto,src_port,dst_port logfile: /var/log/sfacctd.log pidfile: /var/run/sfacctd.pid interface:eth0 interface_wait: true promisc: false sfacctd_ip: 10.41.16.155 sfacctd_port: 6343 sfacctd_net: sflow sfacctd_disable_checks: false plugins: print print_markers:true print_output: formatted print_num_protos: true print_refresh_time: 60 print_history:10m print_output_file:/tmp/sfacctd_print.txt print_output_file_append: true While the logs show that sfacct is doing *something*, but the file indicates that the flow packets are empty if I'm interpreting it right: root@domU:~# cat /var/log/sfacctd.log Nov 06 11:58:27 INFO ( default/print ): 229376 bytes are available to address shared memory segment; buffer size is 200 bytes. Nov 06 11:58:27 INFO ( default/print ): Trying to allocate a shared memory segment of 5734400 bytes. Nov 06 11:58:27 INFO ( default/core ): waiting for sFlow data on 10.41.16.155:6343 Nov 06 11:59:01 INFO ( default/print ): *** Purging cache - START *** Nov 06 11:59:01 INFO ( default/print ): *** Purging cache - END (QN: 0, ET: 0) *** Nov 06 12:00:01 INFO ( default/print ): *** Purging cache - START *** Nov 06 12:00:01 INFO ( default/print ): *** Purging cache - END (QN: 0, ET: 0) *** Nov 06 12:01:01 INFO ( default/print ): *** Purging cache - START *** Nov 06 12:01:01 INFO ( default/print ): *** Purging cache - END (QN: 0, ET: 0) *** root@domU:~# cat /tmp/sfacctd_print.txt SRC_IP DST_IP SRC_PORT DST_PORT PROTOCOL PACKETS BYTES --START (1415271480+60)-- --END-- --START (1415271540+60)-- --END-- --START (1415271600+60)-- --END-- So what does this mean? The flow packets are empty? iptables shows nothing - neither on dom0 nor on domU: root@dom0:~# iptables -t nat -L -n -v Chain PREROUTING (policy ACCEPT 3725M packets, 6310G bytes) pkts bytes target prot opt in out source destination Chain INPUT (policy ACCEPT 1610 packets, 251K bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 453 packets, 35984 bytes) pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 3363K packets, 214M bytes) pkts bytes target prot opt in out source destination root@domU:~# iptables -t nat -L -n -v Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain INPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination There is actually nothing else between dom0 and domU because they are host and virtual machine, or am I guessing wrong? Am 06.11.2014 um 02:14 schrieb Paolo Lucente: Hi David, Two things to try: 1) simplify your config by printing to stdout or flat-files so to remove the possibility issues are with the schema; 2) make sure no firewall, ie. iptables, is blocking packets: tcpdump socket is served before packet filtering, sfacctd indeed after that. Keep me posted how these go and let's get it from there. Let me also confirm you the wireshark dissector for sflow is not of the best quality - so exceptions like the one you ran into are normal. Cheers, Paolo On Wed, Nov 05, 2014 at 09:20:49AM +0100, David Winterstein wrote: Hi, I'm currently having trouble setting up an sflow generating and receiving / interpreting environment and do not really know where to ask for help, I hope this is the right place. The current configuration includes a xenserver connected to the monitoring port of a HP switch - in detail: *Switch:* HP Procurve 2848 [Mirror Port]42 [Monitoring sources] Ports 01 - 41 43 - 48 *Server:* [BOARD] Supermicro X9SCI/X9SCA 1.01 [CPU] Intel Xeon E31230 4x @3.2 GHz [RAM] 16GB
Re: [pmacct-discussion] hsflowd sfacctd - flow generation analysis
Hi David, Two things to try: 1) simplify your config by printing to stdout or flat-files so to remove the possibility issues are with the schema; 2) make sure no firewall, ie. iptables, is blocking packets: tcpdump socket is served before packet filtering, sfacctd indeed after that. Keep me posted how these go and let's get it from there. Let me also confirm you the wireshark dissector for sflow is not of the best quality - so exceptions like the one you ran into are normal. Cheers, Paolo On Wed, Nov 05, 2014 at 09:20:49AM +0100, David Winterstein wrote: Hi, I'm currently having trouble setting up an sflow generating and receiving / interpreting environment and do not really know where to ask for help, I hope this is the right place. The current configuration includes a xenserver connected to the monitoring port of a HP switch - in detail: *Switch:* HP Procurve 2848 [Mirror Port]42 [Monitoring sources] Ports 01 - 41 43 - 48 *Server:* [BOARD] Supermicro X9SCI/X9SCA 1.01 [CPU] Intel Xeon E31230 4x @3.2 GHz [RAM] 16GB @1333 MHz The server has a xenserver dom0 with Debian Wheezy, running hsflowd version 1.26.2 with a configuration as simple as the following: sflow { agent = eth0 DNSSD = off polling = 20 sampling = 512 collector { ip = 10.41.16.155 udpport = 6343 } } The network on the dom0 is configured as follows: # eth0 allow-hotplug eth0 auto eth0 iface eth0 inet static address 10.41.16.152 netmask 255.255.255.0 gateway 10.41.16.1 # eth1 bridge allow-hotplug xenbr1 auto xenbr1 iface xenbr1 inet static address 10.41.16.153 netmask 255.255.255.0 bridge-ports eth1 with the domU network part of the config file being vif = [ 'ip=10.41.1.155,mac=xx:xx:xx:xx:xx:xx,bridge=xenbr1' ] which results in the domU network configuration as follows: # eth0 auto eth0 allow-hotplug eth0 iface eth0 inet static address 10.41.16.155 netmask 255.255.255.0 gateway 10.41.16.1 With this configuration flows should be generated from the traffic arriving on dom0 eth0 by hsflowd and be sent to domU eth0, where sfacctd is listening. All interfaces are working the way they should and tcpdump shows a lot of traffic going through on dom0 eth0. tcpdump with a filter for UDP packets on port 6343 shows packets like these: root@dom0:~# tcpdump -n udp port 6343 17:43:44.030582 IP 10.41.16.153.36533 10.41.16.155.6343: sFlowv5, IPv4 agent 10.41.16.152, agent-id 1, length 468 root@domU:~# tcpdump -n udp port 6343 17:43:44.188244 IP 10.41.16.153.36533 10.41.16.155.6343: sFlowv5, IPv4 agent 10.41.16.152, agent-id 1, length 468 The sfacctd configuration is held quite simple, too: debug: true daemonize:true aggregate: src_host,dst_host,proto,src_port,dst_port logfile: /var/log/sfacctd.log pidfile: /var/run/sfacctd.pid interface:eth0 interface_wait: true promisc: false sfacctd_ip: 10.41.16.155 sfacctd_port: 6343 sfacctd_net: sflow sfacctd_disable_checks: false plugins: mysql sql_max_writers: 11 sql_cache_entries:4 sql_dont_try_update: true sql_use_copy: true sql_multi_values: 1024000 sql_locking_style:table sql_trigger_exec: true sql_trigger_time: 300 sql_host: localhost sql_user: pmacct sql_passwd: sql_db: pmacct sql_table:v4_%Y%m%d sql_table_schema: /usr/local/pmacct/conf/mysql_schema_v4.cnf sql_table_version:4 sql_optimize_clauses: true sql_refresh_time: 60 sql_history: 10m sql_history_roundoff: m sql_recovery_logfile: /var/log/pmacct/recovery_log with the sql_table_schema being defined as this: CREATE TABLE v4_%Y%m%d ( ip_src CHAR(15) NOT NULL, ip_dst CHAR(15) NOT NULL, src_port INT(2) UNSIGNED NOT NULL, dst_port INT(2) UNSIGNED NOT NULL, ip_proto CHAR(6) NOT NULL, packets INT UNSIGNED NOT NULL, bytes BIGINT UNSIGNED NOT NULL, stamp_inserted DATETIME NOT NULL, stamp_updated DATETIME, PRIMARY KEY (ip_src, ip_dst, src_port, dst_port, ip_proto, stamp_inserted) ); sfacctd is definitely up and running, as ps and netstat suggest: root@domU:~# ps auxf |sed '1p;/sfacct/!d' USER PID %CPU %MEMVSZ RSS TTY STAT START TIME COMMAND root 19739 0.0 0.0 8412 552 pts/2S+ 08:38 0:00 | \_ sed 1p;/sfacct/!d root 5352 0.0 0.0 75748 7404 ?Ss Nov04 0:00 sfacctd: Core Process [default] root 5353 0.0 0.1 89380 15592 ?SNov04 0:03 \_ sfacctd: MySQL Plugin [default] root@domU:~# netstat -tulnapee |sed '2p;/sfacct/!d'