Hi Rajesh,

Thanks for bringing this further point up. While all was working with
hex and u_int semantics, there was an issue with the raw one - which was
important to fix since it's a very generic transport for any kind of
info. Here is the commit:

https://github.com/pmacct/pmacct/commit/e1206a530af64c46748b5f891a9034dec89e9b7a

Let me know if this works for you.

Paolo

On Mon, Dec 17, 2018 at 10:25:27PM +0530, RAJESH KUMAR S.R wrote:
> Hi Paolo,
> 
> Thanks for the help and suggestion ,
> Custom primitives works for me with the latest code:) . I'm able to read a
> field from packet and export as custom template and I can see those
> templates in netflow messages also.
> I'll continue with viewing the flows in a collector, I'm using Elastiflow
> to view reports.
> 
> 
> I have a small doubt. I was reading a field of each packet and map it to a
> custom template.(just trying to read ethertype for testing the custom
> templates eventhough etype is available)
> The value shown in pmacct seems to be right.  I tried verifying this data
> with nfacctd, but the value shown in nfacctd seems to be different. I have
> pasted the conf files and output.
> 
> *Conf files:*
> *primitives.lst file*
> *name=dummy_byte packet_ptr=packet:+12 len=2 semantics=raw
> field_type=41234:100*
> 
> *pmacct.conf*
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> *daemonize: false   interface: wlp1s0   !pcap_interfaces_map:
> pcap_interfaces.map   aggregate_primitives: primitives.lst   aggregate:
> src_host, dst_host, src_port, dst_port, proto, tos, dummy_byte   plugins:
> nfprobe, print   nfprobe_version: 10   nfprobe_engine: 100
> nfprobe_receiver: 192.168.1.6:2100 <http://192.168.1.6:2100>
> !nfprobe_receiver: 10.40.6.6:16367 <http://10.40.6.6:16367>*
> 
> 
> *nfacct.conf*
> 
> 
> 
> 
> 
> 
> 
> *daemonize: false   nfacctd_ip: 192.168.1.6   nfacctd_port: 2100
> aggregate_primitives: primitives.lst   aggregate: src_host, dst_host,
> src_port, dst_port, proto, tos, dummy_byte   plugins: print
> nfacctd_disable_checks: true*
> 
> $   sudo /usr/local/sbin/pmacctd -f pmacct.conf
> INFO ( default/core ): Promiscuous Mode Accounting Daemon, pmacctd
> 1.7.3-git (20181217-00)
> INFO ( default/core ):  '--enable-l2' '--enable-64bit'
> '--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins'
> '--enable-st-bins'
> INFO ( default/core ): Reading configuration file
> '/home/certes-rajesh/pmacct/pmacct/pmacct.conf'.
> INFO ( default/core ): [primitives.lst] (re)loading map.
> INFO ( default/core ): [primitives.lst] map successfully (re)loaded.
> INFO ( default_nfprobe/nfprobe ): NetFlow probe plugin is originally based
> on softflowd 0.9.7 software, Copyright 2002 Damien Miller <d...@mindrot.org>
> All rights reserved.
> INFO ( default_nfprobe/nfprobe ):           TCP timeout: 3600s
> INFO ( default_nfprobe/nfprobe ):  TCP post-RST timeout: 120s
> INFO ( default_nfprobe/nfprobe ):  TCP post-FIN timeout: 300s
> INFO ( default_nfprobe/nfprobe ):           UDP timeout: 300s
> INFO ( default_nfprobe/nfprobe ):          ICMP timeout: 300s
> INFO ( default_nfprobe/nfprobe ):       General timeout: 3600s
> INFO ( default_nfprobe/nfprobe ):      Maximum lifetime: 604800s
> INFO ( default_nfprobe/nfprobe ):       Expiry interval: 60s
> INFO ( default_nfprobe/nfprobe ): Exporting flows to [172.24.1.219]:2100
> INFO ( default_print/print ): cache entries=16411 base cache
> memory=54878384 bytes
> WARN ( default_print/print ): no print_output_file and no
> print_output_lock_file defined.
> INFO ( default/core ): [enp0s31f6,0] link type is: 1
> INFO ( default_print/print ): *** Purging cache - START (PID: 4301) ***
> SRC_IP                         DST_IP                       SRC_PORT
> DST_PORT  PROTOCOL    TOS    *dummy_byte*  udp_len  PACKETS
> BYTES
> 172.24.1.197            239.255.255.250                 56940
> 1900               udp                 0       *08-00*           180
> 4                     800
> fe80::3436:5d8f:abd7:6f0c      f2::fb              5353
> 5353         udp                 0            *86-DD*
> 48       3                     264
> 
> 
> $   sudo nfacctd  -f nfacct.conf
> INFO ( default/core ): NetFlow Accounting Daemon, nfacctd 1.7.3-git
> (20181217-00)
> INFO ( default/core ):  '--enable-l2' '--enable-64bit'
> '--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins'
> '--enable-st-bins'
> INFO ( default/core ): Reading configuration file
> '/home/certes-rajesh/pmacct/pmacct/nfacct.conf'.
> INFO ( default/core ): [primitives.lst] (re)loading map.
> INFO ( default/core ): [primitives.lst] map successfully (re)loaded.
> INFO ( default/core ): waiting for NetFlow/IPFIX data on 172.24.1.219:2100
> INFO ( default_print/print ): cache entries=16411 base cache
> memory=54878384 bytes
> WARN ( default_print/print ): no print_output_file and no
> print_output_lock_file defined.
> INFO ( default_print/print ): *** Purging cache - START (PID: 4356) ***
> INFO ( default_print/print ): *** Purging cache - END (PID: 4356, QN: 0/0,
> ET: X) ***
> INFO ( default_print/print ): *** Purging cache - START (PID: 4379) ***
> INFO ( default_print/print ): *** Purging cache - END (PID: 4379, QN: 0/0,
> ET: X) ***
> INFO ( default_print/print ): *** Purging cache - START (PID: 4410) ***
> INFO ( default_print/print ): *** Purging cache - END (PID: 4410, QN: 0/0,
> ET: X) ***
> INFO ( default_print/print ): *** Purging cache - START (PID: 4443) ***
> SRC_IP                          DST_IP                 SRC_PORT  DST_PORT
> PROTOCOL    TOS    *dummy_byte*  PACKETS               BYTES
> 172.24.1.197          239.255.255.250            56940
> 1900          udp                 0            *30-38*
> 4                     800
> 
> 
> 
> 
> 
> On Mon, Dec 17, 2018 at 6:47 AM Paolo Lucente <pa...@pmacct.net> wrote:
> 
> >
> > Hi Rajesh,
> >
> > Thanks for pointing this out. I've committed some code to unlock
> > field_type also for uacctd/pmacctd daemons precisely for the use case
> > you mentioned. Here the details:
> >
> >
> > https://github.com/pmacct/pmacct/commit/87ebf3a9f907c331f752c96a76ea247e77f99107
> >
> > You can back port this patch to latest stable release or use master
> > code. Keep me posted if it works for you - it did work for me in lab
> > using your config as a base.
> >
> > One recommendation: use IPFIX instead of NetFlow v9 if possible. IPFIX
> > allows to define the field type as <PEN>:<field_type>, where pmacct PEN
> > is documented here:
> >
> > https://github.com/pmacct/pmacct/blob/master/docs/IPFIX
> >
> > So you could use, say, 43874:100 as field type instead of squatting the
> > public code points.
> >
> > Paolo
> >
> > On Sat, Dec 15, 2018 at 12:04:54AM +0530, RAJESH KUMAR S.R wrote:
> > > Hi,
> > >
> > > I need some understanding in exporting the custom defined primitives in
> > > netflow v9 messages, if that is possible, as I want to define custom
> > fields
> > > and send out to netflow collector and visualize using graphs (if the
> > > collector supports custom templates)
> > >
> > > As a first step, I am trying to use the custom aggregate primitive  used
> > in
> > > examples/primitives.lst.example.
> > >
> > > " Defines a primitive called 'udp_len': base pointer is set to the UDP
> > > header
> > >  (l4:17) plus 4 bytes offset, reads for 2 byte and will present it as
> > > unsigned
> > >  int.
> > >
> > > name=udp_len    packet_ptr=l4:17+4      len=2   semantics=u_int
> > > "
> > >
> > > I used to classify flows after defining "udp_len" as mentioned above.
> > > My conf file for pmacctd is
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > *"   daemonize:false   interface: wlp1s0   aggregate_primitives:
> > > primitives.lst   aggregate: etype, proto, src_host, dst_host, src_port,
> > > dst_port, udp_len   plugins: nfprobe, print   nfprobe_receiver:
> > > 172.24.1.123:9996 <http://172.24.1.123:9996>   nfprobe_version: 9*
> > > *"*
> > > My primitives.lst file defines custom primitive as follows
> > >
> > > *"name=udp_len    packet_ptr=l4:17+4      len=2   semantics=u_int"*
> > >
> > > When I run the pmacct "sudo pmacctd -f pmacct.conf", I'm able to see the
> > > flows that has udp_len column displayed in the console using print
> > plugin.
> > >
> > > Output of
> > > "sudo pmacctd -f pmacct.conf"
> > >
> > > INFO ( default/core ): Promiscuous Mode Accounting Daemon, pmacctd
> > > 1.7.2-git (20180701-01)
> > > INFO ( default/core ):  '--enable-l2' '--enable-ipv6' '--enable-64bit'
> > > '--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins'
> > > '--enable-st-bins'
> > > INFO ( default/core ): Reading configuration file
> > > '/home/certes-rajesh/pmacct/pmacct/pmacct.conf'.
> > > INFO ( default/core ): [primitives.lst] (re)loading map.
> > > INFO ( default/core ): [primitives.lst] map successfully (re)loaded.
> > > INFO ( default_nfprobe/nfprobe ): NetFlow probe plugin is originally
> > based
> > > on softflowd 0.9.7 software, Copyright 2002 Damien Miller <
> > d...@mindrot.org>
> > > All rights reserved.
> > > INFO ( default_nfprobe/nfprobe ):           TCP timeout: 3600s
> > > INFO ( default_nfprobe/nfprobe ):  TCP post-RST timeout: 120s
> > > INFO ( default_nfprobe/nfprobe ):  TCP post-FIN timeout: 300s
> > > INFO ( default_nfprobe/nfprobe ):           UDP timeout: 300s
> > > INFO ( default_nfprobe/nfprobe ):          ICMP timeout: 300s
> > > INFO ( default_nfprobe/nfprobe ):       General timeout: 3600s
> > > INFO ( default_nfprobe/nfprobe ):      Maximum lifetime: 604800s
> > > INFO ( default_nfprobe/nfprobe ):       Expiry interval: 60s
> > > INFO ( default_nfprobe/nfprobe ): Exporting flows to [192.168.122.1]:9996
> > > *ERROR ( default_nfprobe/nfprobe ): custom primitive 'udp_len' has null
> > > field_type*
> > > INFO ( default_print/print ): cache entries=16411 base cache
> > > memory=54878384 bytes
> > > WARN ( default_print/print ): no print_output_file and no
> > > print_output_lock_file defined.
> > > INFO ( default/core ): [wlp1s0,0] link type is: 1
> > > *WARN ( default/core ): connection lost to 'default_nfprobe-nfprobe';
> > > closing connection.*
> > > INFO ( default_print/print ): *** Purging cache - START (PID: 2837) ***
> > > ETYPE  SRC_IP
> > > DST_IP                                         SRC_PORT  DST_PORT
> > > PROTOCOL    udp_len  PACKETS               BYTES
> > > 86dd   fd50:1d9:a341:f100:8ae:86f3:123d:3654
> > > ff02::fb                                       5353      5353
> > > udp         41       3                     243
> > > .......
> > >
> > > When I try to give a dummy field type, it throws
> > > "WARN ( default/core ): [primitives.lst] field_type is only supported in
> > > nfacctd.".
> > >
> > > I need help in figuring out whether I'm doing the right thing for
> > exporting
> > > custom fields as part netflow messages as I will need to send out more
> > > custom fields that are read from the packet.
> >
> > > _______________________________________________
> > > pmacct-discussion mailing list
> > > http://www.pmacct.net/#mailinglists
> >
> >
> > _______________________________________________
> > pmacct-discussion mailing list
> > http://www.pmacct.net/#mailinglists
> >

_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists

Reply via email to