See man nfdump for output format options. You should add something like
%srcas %dstas to display AS numbers.
On Tue, Feb 20, 2018 at 10:17 PM, Leandro <ingrog...@gmail.com> wrote:
> Dear Alan ... I would like to share my results.
> So far can not get it work.
>
> What I did:
> 1 Downloaded and installed "Net-NfDump-1.25" using cpan. ok
> 2 Copy a nfdump generated file (this file is being used by nfsen for
> graphics) ok
> 3 Verify file content: ok
> [root@AR-LXNF01 bin]$ /usr/local/bin/nfdump -r /usr/local/nfsen/profiles-
> data/asn_updated/nfcapd.201802201310 | more
> Date first seen Duration Proto Src IP Addr:Port Dst
> IP Addr:Port Packets Bytes Flows
> 2018-02-20 12:54:35.741 0.000 TCP 187.118.18.149:46938 ->
> 169.60.79.184:5222 500 155500 1
> 2018-02-20 12:54:35.741 0.000 TCP 186.183.22.205:443 ->
> 187.118.11.18:54343 500 750000 1
> 2018-02-20 12:54:35.742 0.000 UDP 187.174.42.129:3658 ->
> 187.12.187.134:3658 500 33500 1
> 2018-02-20 12:54:35.745 0.000 UDP 187.118.10.47:55796 ->
> 186.183.22.206:443 500 32000 1
> .....trunked
>
> 4 Apply perl script for asn update: ok
>
> /usr/src/Net-NfDump-1.25/bin/nfasnupd nfcapd.201802201310
> Subroutine main::pack_sockaddr_in6 redefined at
> /usr/share/perl5/vendor_perl/Exporter.pm line 66.
> at /usr/src/Net-NfDump-1.25/bin/nfasnupd line 8.
> Subroutine main::unpack_sockaddr_in6 redefined at
> /usr/share/perl5/vendor_perl/Exporter.pm line 66.
> at /usr/src/Net-NfDump-1.25/bin/nfasnupd line 8.
> Subroutine main::sockaddr_in6 redefined at
> /usr/share/perl5/vendor_perl/Exporter.pm
> line 66.
> at /usr/src/Net-NfDump-1.25/bin/nfasnupd line 8.
> 2018-02-20.13:23:28[18171]: Checking BGP database for new version.
> 2018-02-20.13:23:28[18171]: Loading AS database.
> 2018-02-20.13:23:34[18171]: Updating records.
> 2018-02-20.13:23:39[18171]: Processed 394412 flows in 5 secs.
>
> 4 Verify converted file content NOK
> [root@AR-LXNF01 asn_updated]$ /usr/local/bin/nfdump -r
> nfcapd.201802201310
>
> Skip unknown record type 10
>
> Skip unknown record type 10
>
> Skip unknown record type 10
> ...trunked
>
> ############################After that I did:
> 1 compiled nfdump ver 1.6.16
> 2 Read the file again:
>
> [leo@arch nfdump]$ nfdump -r nfcapd.201802201310 | more
> Date first seen Duration Proto Src IP Addr:Port Dst
> IP Addr:Port Packets Bytes Flows
> 2018-02-20 13:09:03.614 27.624 UDP 187.183.47.57:56084 ->
> 187.183.22.205:443 1500 105500 1
> 2018-02-20 13:09:10.518 28.752 TCP 198.38.124.206:80 ->
> 181.118.2.92:50632 4500 6.6 M 1
> 2018-02-20 13:09:31.239 0.000 TCP 187.183.37.208:49822 ->
> 172.217.28.163:80 500 26000 1
> 2018-02-20 13:09:21.275 19.923 TCP 187.183.22.204:443 ->
> 187.183.21.174:6036 1500 2.2 M 1
>
> As you can see with this newer version of nfdump I can read the file but
> don't see the asn number.
>
> ############################ some debug
> While reading the nfasnupd script found the following important paths:
>
> my $DBDIR = "/var/db/flowtools";
> my $CASNSDB = "/usr/local/etc/asns.txt"; # cuscom ASN DB
> #This file was not here, I copied it from
> /usr/src/Net-NfDump-1.25/bin/asns.txt
> my $BGPDB = "/var/tmp/asns-$>.gz";
> #this file does not exist even while script is
> working.
> my $BGPDB_MD5 = "/var/tmp/asns-$>.gz.md5";
> #this file does not exist even while script is working.
> my $GEO_DB4 = $DBDIR.'/geo/GeoIP.dat';
> #this file does not exist even while script is
> working.
> my $GEO_DB6 = $DBDIR.'/geo/GeoIPv6.dat';
> #this file does not exist even while script is working.
>
> my $FNAME_TMP = ".nffile_$$.tmp";
> my $DEBUG = 1;
> my $FNAME;
> my $VERSION = "1.10";
>
> ###############So ...
> I think those files are very important for the script to work propperly.
> You you confirm if you can see those files ?
> Also can you share a few lines of the nfdump output when displays properly
> the asn number ?
> btw ... my generated file is bigger than original.
>
>
> Ok .... thats all ... thanks for the help , any idea would be preciated.
> Leandro.
>
>
>
>
>
>
>
>
>
> On 09/02/18 03:58, Alan Whinery wrote:
>
> On 1/26/2018 4:52 AM, Leandro wrote:
>
>
> Hi, Alan , thanks for the response.
> It think the most difficult part of this is how to insert asn number
> on the flow data structure.
> First I need to read the data file , then iterate over all flows
> (perhaps using pcap library ?) then take the ips involved in the
> flow and insert their asn.
> The idea is that this modification allow us to keep using nfsen
> graphics and filter for further analisys.
> I think it is necessary to have a very deep knowledge of the necessary
> development tools.
> Do you know some tool / api to analize and edit the flow data file at
> a higher level ?
>
> After an uncanny sequence of events, I do, in fact have an answer to this.
>
> We have a flow analysis project going and not long after your question,
> I was tasked with doing the very same thing here. It turns out that two
> of our border routers are Juniper QFX, which appear only to export
> SFlow, and they don't write ASNs in the records, even though they have
> BGP tables.
>
> I found a Perl module https://metacpan.org/pod/Net::NfDump which has the
> necessary capabilities. While I was in the process of getting something
> working to insert AS numbers into flow files, I found that the example
> program, "nfasnupd" not only inserts AS numbers, but also geo info,
> which I had thought about as a next step!
>
>
> Command updates nfdump file and adds AS and geoIP information
>
> Usage:
>
> nfasnupd [ -d <level> ] -b -g [ -a -5 -4 -6 ] [ -c <ASN_db_file> ]
> <nfdump_file>
>
> Options:
>
> -d <level> : debug level (dafault: 1)
>
> -B do NOT update AS numbers (srcas, dstas)
> -g update country code (*xsrcport, *xdstport)
>
> -a <file> : path to BGPDB file (default: /var/tmp/asns-0.gz)
> -5 <file> : path to BGPDB MD5 (default: /var/tmp/asns-0.gz.md5)
> -c <file> : path to additional textfile with ASN mapping (default:
> /usr/local/etc/asns.txt)
> -4 <file> : path to IPv4 GeoIP database (default:
> /var/db/flowtools/geo/GeoIP.dat)
> -6 <file> : path to IPv6 GeoIP database (default:
> /var/db/flowtools/geo/GeoIPv6.dat)
>
> Part of libnf.net project, version: 1.10
>
>
> I am only at the stage of running it on sample files, not yet committing
> to bulk operations, but it is promising.
>
> It does make the flow file smaller. I haven't yet figured out why.
>
> If you hand it a flow file, it effectively rewrites it in place
> (probably writes a temp and then moves it to the original).
>
> the Geo stuff doesn't happen by default, so one only has to complete
> dependencies and run
>
> ./nfasnupd flowfile.nfdump
>
> If you're not Perl-proficient, send me an off list message. I'm going to
> try to move toward bulk conversion, with appropriate backups first.
>
> -Alan
>
>
> Thanks,
> Leo.
>
>
>
>
>
> On 26/01/18 00:08, Alan Whinery wrote:
>
> Of course, generally when you export flows from a BGP router with a full
> table, it should already have ASNs populated.
>
> If you have flow data with no ASN, probably the easiest way to fill it
> in would be to script something with MaxMind's open source ASN data:
> https://www.maxmind.com/en/open-source-data-and-api-for-ip-geolocation
>
> I don't know off-hand of software that updates fields in nfdump files,
> but there must be something out there, or some Perl or Python modules to
> do so.
>
> In the past, I've rolled my own ASN-to-prefix cross-ref by grabbing the
> global routing table from a BGP router and then annotating it with the
> asn lists from cidr-report.org:
>
> http://www.cidr-report.org/as2.0/autnums.html
>
> which is linked from:http://www.cidr-report.org/as2.0/
>
>
> On 1/25/2018 5:37 AM, Leandro wrote:
>
> Hi guys , Im trying to analyze incoming traffic from an specific asn ,
> I can not filter this using source ip since this operator uses a lot
> of subnets (about 7k).
> My idea is to grab a flow file and insert the asn for further
> analysis. Is there something about this ?
> Any idea would help ,
> Regards ,
> Leo.
>
>
> ------------------------------------------------------------------------------
>
>
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Nfsen-discuss mailing
> listNfsen-discuss@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/nfsen-discuss
>
> ------------------------------------------------------------------------------
>
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Nfsen-discuss mailing
> listNfsen-discuss@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/nfsen-discuss
>
>
> ------------------------------------------------------------------------------
>
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Nfsen-discuss mailing
> listNfsen-discuss@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/nfsen-discuss
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Nfsen-discuss mailing
> listNfsen-discuss@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/nfsen-discuss
>
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Nfsen-discuss mailing list
> Nfsen-discuss@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
>
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Nfsen-discuss mailing list
Nfsen-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
