Hi Paolo, Thanks for the help and suggestion , Custom primitives works for me with the latest code:) . I'm able to read a field from packet and export as custom template and I can see those templates in netflow messages also. I'll continue with viewing the flows in a collector, I'm using Elastiflow to view reports.
I have a small doubt. I was reading a field of each packet and map it to a custom template.(just trying to read ethertype for testing the custom templates eventhough etype is available) The value shown in pmacct seems to be right. I tried verifying this data with nfacctd, but the value shown in nfacctd seems to be different. I have pasted the conf files and output. *Conf files:* *primitives.lst file* *name=dummy_byte packet_ptr=packet:+12 len=2 semantics=raw field_type=41234:100* *pmacct.conf* *daemonize: false interface: wlp1s0 !pcap_interfaces_map: pcap_interfaces.map aggregate_primitives: primitives.lst aggregate: src_host, dst_host, src_port, dst_port, proto, tos, dummy_byte plugins: nfprobe, print nfprobe_version: 10 nfprobe_engine: 100 nfprobe_receiver: 192.168.1.6:2100 <http://192.168.1.6:2100> !nfprobe_receiver: 10.40.6.6:16367 <http://10.40.6.6:16367>* *nfacct.conf* *daemonize: false nfacctd_ip: 192.168.1.6 nfacctd_port: 2100 aggregate_primitives: primitives.lst aggregate: src_host, dst_host, src_port, dst_port, proto, tos, dummy_byte plugins: print nfacctd_disable_checks: true* $ sudo /usr/local/sbin/pmacctd -f pmacct.conf INFO ( default/core ): Promiscuous Mode Accounting Daemon, pmacctd 1.7.3-git (20181217-00) INFO ( default/core ): '--enable-l2' '--enable-64bit' '--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins' '--enable-st-bins' INFO ( default/core ): Reading configuration file '/home/certes-rajesh/pmacct/pmacct/pmacct.conf'. INFO ( default/core ): [primitives.lst] (re)loading map. INFO ( default/core ): [primitives.lst] map successfully (re)loaded. INFO ( default_nfprobe/nfprobe ): NetFlow probe plugin is originally based on softflowd 0.9.7 software, Copyright 2002 Damien Miller <d...@mindrot.org> All rights reserved. INFO ( default_nfprobe/nfprobe ): TCP timeout: 3600s INFO ( default_nfprobe/nfprobe ): TCP post-RST timeout: 120s INFO ( default_nfprobe/nfprobe ): TCP post-FIN timeout: 300s INFO ( default_nfprobe/nfprobe ): UDP timeout: 300s INFO ( default_nfprobe/nfprobe ): ICMP timeout: 300s INFO ( default_nfprobe/nfprobe ): General timeout: 3600s INFO ( default_nfprobe/nfprobe ): Maximum lifetime: 604800s INFO ( default_nfprobe/nfprobe ): Expiry interval: 60s INFO ( default_nfprobe/nfprobe ): Exporting flows to [172.24.1.219]:2100 INFO ( default_print/print ): cache entries=16411 base cache memory=54878384 bytes WARN ( default_print/print ): no print_output_file and no print_output_lock_file defined. INFO ( default/core ): [enp0s31f6,0] link type is: 1 INFO ( default_print/print ): *** Purging cache - START (PID: 4301) *** SRC_IP DST_IP SRC_PORT DST_PORT PROTOCOL TOS *dummy_byte* udp_len PACKETS BYTES 172.24.1.197 239.255.255.250 56940 1900 udp 0 *08-00* 180 4 800 fe80::3436:5d8f:abd7:6f0c f2::fb 5353 5353 udp 0 *86-DD* 48 3 264 $ sudo nfacctd -f nfacct.conf INFO ( default/core ): NetFlow Accounting Daemon, nfacctd 1.7.3-git (20181217-00) INFO ( default/core ): '--enable-l2' '--enable-64bit' '--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins' '--enable-st-bins' INFO ( default/core ): Reading configuration file '/home/certes-rajesh/pmacct/pmacct/nfacct.conf'. INFO ( default/core ): [primitives.lst] (re)loading map. INFO ( default/core ): [primitives.lst] map successfully (re)loaded. INFO ( default/core ): waiting for NetFlow/IPFIX data on 172.24.1.219:2100 INFO ( default_print/print ): cache entries=16411 base cache memory=54878384 bytes WARN ( default_print/print ): no print_output_file and no print_output_lock_file defined. INFO ( default_print/print ): *** Purging cache - START (PID: 4356) *** INFO ( default_print/print ): *** Purging cache - END (PID: 4356, QN: 0/0, ET: X) *** INFO ( default_print/print ): *** Purging cache - START (PID: 4379) *** INFO ( default_print/print ): *** Purging cache - END (PID: 4379, QN: 0/0, ET: X) *** INFO ( default_print/print ): *** Purging cache - START (PID: 4410) *** INFO ( default_print/print ): *** Purging cache - END (PID: 4410, QN: 0/0, ET: X) *** INFO ( default_print/print ): *** Purging cache - START (PID: 4443) *** SRC_IP DST_IP SRC_PORT DST_PORT PROTOCOL TOS *dummy_byte* PACKETS BYTES 172.24.1.197 239.255.255.250 56940 1900 udp 0 *30-38* 4 800 On Mon, Dec 17, 2018 at 6:47 AM Paolo Lucente <pa...@pmacct.net> wrote: > > Hi Rajesh, > > Thanks for pointing this out. I've committed some code to unlock > field_type also for uacctd/pmacctd daemons precisely for the use case > you mentioned. Here the details: > > > https://github.com/pmacct/pmacct/commit/87ebf3a9f907c331f752c96a76ea247e77f99107 > > You can back port this patch to latest stable release or use master > code. Keep me posted if it works for you - it did work for me in lab > using your config as a base. > > One recommendation: use IPFIX instead of NetFlow v9 if possible. IPFIX > allows to define the field type as <PEN>:<field_type>, where pmacct PEN > is documented here: > > https://github.com/pmacct/pmacct/blob/master/docs/IPFIX > > So you could use, say, 43874:100 as field type instead of squatting the > public code points. > > Paolo > > On Sat, Dec 15, 2018 at 12:04:54AM +0530, RAJESH KUMAR S.R wrote: > > Hi, > > > > I need some understanding in exporting the custom defined primitives in > > netflow v9 messages, if that is possible, as I want to define custom > fields > > and send out to netflow collector and visualize using graphs (if the > > collector supports custom templates) > > > > As a first step, I am trying to use the custom aggregate primitive used > in > > examples/primitives.lst.example. > > > > " Defines a primitive called 'udp_len': base pointer is set to the UDP > > header > > (l4:17) plus 4 bytes offset, reads for 2 byte and will present it as > > unsigned > > int. > > > > name=udp_len packet_ptr=l4:17+4 len=2 semantics=u_int > > " > > > > I used to classify flows after defining "udp_len" as mentioned above. > > My conf file for pmacctd is > > > > > > > > > > > > > > > > > > *" daemonize:false interface: wlp1s0 aggregate_primitives: > > primitives.lst aggregate: etype, proto, src_host, dst_host, src_port, > > dst_port, udp_len plugins: nfprobe, print nfprobe_receiver: > > 172.24.1.123:9996 <http://172.24.1.123:9996> nfprobe_version: 9* > > *"* > > My primitives.lst file defines custom primitive as follows > > > > *"name=udp_len packet_ptr=l4:17+4 len=2 semantics=u_int"* > > > > When I run the pmacct "sudo pmacctd -f pmacct.conf", I'm able to see the > > flows that has udp_len column displayed in the console using print > plugin. > > > > Output of > > "sudo pmacctd -f pmacct.conf" > > > > INFO ( default/core ): Promiscuous Mode Accounting Daemon, pmacctd > > 1.7.2-git (20180701-01) > > INFO ( default/core ): '--enable-l2' '--enable-ipv6' '--enable-64bit' > > '--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins' > > '--enable-st-bins' > > INFO ( default/core ): Reading configuration file > > '/home/certes-rajesh/pmacct/pmacct/pmacct.conf'. > > INFO ( default/core ): [primitives.lst] (re)loading map. > > INFO ( default/core ): [primitives.lst] map successfully (re)loaded. > > INFO ( default_nfprobe/nfprobe ): NetFlow probe plugin is originally > based > > on softflowd 0.9.7 software, Copyright 2002 Damien Miller < > d...@mindrot.org> > > All rights reserved. > > INFO ( default_nfprobe/nfprobe ): TCP timeout: 3600s > > INFO ( default_nfprobe/nfprobe ): TCP post-RST timeout: 120s > > INFO ( default_nfprobe/nfprobe ): TCP post-FIN timeout: 300s > > INFO ( default_nfprobe/nfprobe ): UDP timeout: 300s > > INFO ( default_nfprobe/nfprobe ): ICMP timeout: 300s > > INFO ( default_nfprobe/nfprobe ): General timeout: 3600s > > INFO ( default_nfprobe/nfprobe ): Maximum lifetime: 604800s > > INFO ( default_nfprobe/nfprobe ): Expiry interval: 60s > > INFO ( default_nfprobe/nfprobe ): Exporting flows to [192.168.122.1]:9996 > > *ERROR ( default_nfprobe/nfprobe ): custom primitive 'udp_len' has null > > field_type* > > INFO ( default_print/print ): cache entries=16411 base cache > > memory=54878384 bytes > > WARN ( default_print/print ): no print_output_file and no > > print_output_lock_file defined. > > INFO ( default/core ): [wlp1s0,0] link type is: 1 > > *WARN ( default/core ): connection lost to 'default_nfprobe-nfprobe'; > > closing connection.* > > INFO ( default_print/print ): *** Purging cache - START (PID: 2837) *** > > ETYPE SRC_IP > > DST_IP SRC_PORT DST_PORT > > PROTOCOL udp_len PACKETS BYTES > > 86dd fd50:1d9:a341:f100:8ae:86f3:123d:3654 > > ff02::fb 5353 5353 > > udp 41 3 243 > > ....... > > > > When I try to give a dummy field type, it throws > > "WARN ( default/core ): [primitives.lst] field_type is only supported in > > nfacctd.". > > > > I need help in figuring out whether I'm doing the right thing for > exporting > > custom fields as part netflow messages as I will need to send out more > > custom fields that are read from the packet. > > > _______________________________________________ > > pmacct-discussion mailing list > > http://www.pmacct.net/#mailinglists > > > _______________________________________________ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists >
_______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
