Re: [EXTERNAL] Re: Flow collection and analysis

[Marcel Mitsuto]

My colleagues developers started adding valid let's encrypt certs
everywhere. Now I have multiple NAT entry points for build-servers in my
VPC because of the renewal frequency.

I feel less secure with them adding valid SSL certs everywhere that runs on
a PRIVATE NETWORK.

It's just dumb reasoning, and the CTO agreed with them. They are all gone
by now, but their legacy remains.

Now I have to find all those certs and replace them with 10 year
self-signed, and add --no-check-certificates flags in their http client
requests.

All NAT entrypoints are gone.

I'm feeling safe now.

On Fri, 28 Jan 2022 at 13:26, Jean St-Laurent via NANOG 
wrote:

> Why DNS are still travelling in clear text?
>
> The software running the DNS services worldwide are probably written in C
> or any languages you mentioned below.
>
> Why don't they just strap a libressl on DNS or NanoSSL?
>
> Okay, there is DNS over https. I don't know the stats, but I doubt it's
> close to 100% adoption worldwide.
>
> I don't understand what is the issue about SSL, zero trust has anything to
> do about collecting flows. Do I need ssl to run shell commands in my
> terminal to read flows? Not really. Do I need to strap ssl on grep, notepad
> and excel? I'm not sure how could one do that.
>
> When you see the flows of your customers, you have access to how many
> times did they use Netflix, facebook and anything you could think of
> because these people are querying DNS to reach these... in clear text. They
> are also hitting servers that are well known.
>
> I would worry more about who is reading the flows of my business'
> customers than these flows being  not protected by SSL. They are anyway in
> a highly secure environment with zero trust.
>
> So if you don't like elastiflow or any software that are not being
> protected by SSL, then maybe switch off your computer. Protonmail won't
> help you to keep your digital life secure.
>
> This email was sent by a secure infrastructure using TLS 1.2 and clear
> text dns.
>
> Thank you
>
> Jean
>
> -Original Message-
> From: NANOG  On Behalf Of Laura
> Smith via NANOG
> Sent: January 28, 2022 5:15 AM
> To: Mel Beckman 
> Cc: nanog@nanog.org list 
> Subject: Re: [EXTERNAL] Re: Flow collection and analysis
>
> ‐‐‐ Original Message ‐‐‐
>
> On Friday, January 28th, 2022 at 03:55, Mel Beckman 
> wrote:
>
> > But nobody asked for anything from scratch Eric. Open SSL is it complete
> ready to integrate package. Any developer worth his salt should be able to
> put it on any web application. In addition to OpenSSL, there are very
> compact commercial SSL libraries such as Mocana NanoSSL and wolfSSL, if you
> want to really simplify the process.
> >
>
> Yup. Every single modern programming language out there has a crypto
> library.
>
> The high-level languages (e.g. Go) have crypto built into the standard
> library.
>
> The low-level languages (e.g C or Rust) all have at least one or more well
> supported third party crypto libraries (e.g. for C there's OpenSSL, GnuTLS,
> LibreSSL, Boring SSL, Mbed TLS ... and those are the ones that I can think
> of off the top of my head).
>
> There's no need to do any crypto "from scratch", and indeed you SHOULD NOT.
>
>

RE: [EXTERNAL] Re: Flow collection and analysis

[Laura Smith via NANOG]

‐‐‐ Original Message ‐‐‐

On Friday, January 28th, 2022 at 11:52, Jean St-Laurent  
wrote:

> Why DNS are still travelling in clear text?
>

It doesn't have to.  In 2022 there are many encryption options for DNS. There 
are also things like DNSSEC and DANE for ensuring authenticity over cleartext.

In addition, if the latest US Federal guidance is anything to go by, we may be 
witnessing the first big nail being put into the cleartext DNS coffin. 
(https://www.bastionzero.com/blog/i-read-the-federal-governments-zero-trust-memo-so-you-dont-have-to)

RE: [EXTERNAL] Re: Flow collection and analysis

[Jean St-Laurent via NANOG]

Why DNS are still travelling in clear text?

The software running the DNS services worldwide are probably written in C or 
any languages you mentioned below.

Why don't they just strap a libressl on DNS or NanoSSL?

Okay, there is DNS over https. I don't know the stats, but I doubt it's close 
to 100% adoption worldwide.

I don't understand what is the issue about SSL, zero trust has anything to do 
about collecting flows. Do I need ssl to run shell commands in my terminal to 
read flows? Not really. Do I need to strap ssl on grep, notepad and excel? I'm 
not sure how could one do that.

When you see the flows of your customers, you have access to how many times did 
they use Netflix, facebook and anything you could think of because these people 
are querying DNS to reach these... in clear text. They are also hitting servers 
that are well known.

I would worry more about who is reading the flows of my business' customers 
than these flows being  not protected by SSL. They are anyway in a highly 
secure environment with zero trust.

So if you don't like elastiflow or any software that are not being protected by 
SSL, then maybe switch off your computer. Protonmail won't help you to keep 
your digital life secure.

This email was sent by a secure infrastructure using TLS 1.2 and clear text dns.

Thank you

Jean

-Original Message-
From: NANOG  On Behalf Of Laura Smith 
via NANOG
Sent: January 28, 2022 5:15 AM
To: Mel Beckman 
Cc: nanog@nanog.org list 
Subject: Re: [EXTERNAL] Re: Flow collection and analysis

‐‐‐ Original Message ‐‐‐

On Friday, January 28th, 2022 at 03:55, Mel Beckman  wrote:

> But nobody asked for anything from scratch Eric. Open SSL is it complete 
> ready to integrate package. Any developer worth his salt should be able to 
> put it on any web application. In addition to OpenSSL, there are very compact 
> commercial SSL libraries such as Mocana NanoSSL and wolfSSL, if you want to 
> really simplify the process.
>

Yup. Every single modern programming language out there has a crypto library.

The high-level languages (e.g. Go) have crypto built into the standard library.

The low-level languages (e.g C or Rust) all have at least one or more well 
supported third party crypto libraries (e.g. for C there's OpenSSL, GnuTLS, 
LibreSSL, Boring SSL, Mbed TLS ... and those are the ones that I can think of 
off the top of my head).

There's no need to do any crypto "from scratch", and indeed you SHOULD NOT.

Re: [EXTERNAL] Re: Flow collection and analysis

[Laura Smith via NANOG]

‐‐‐ Original Message ‐‐‐

On Friday, January 28th, 2022 at 03:55, Mel Beckman  wrote:

> But nobody asked for anything from scratch Eric. Open SSL is it complete 
> ready to integrate package. Any developer worth his salt should be able to 
> put it on any web application. In addition to OpenSSL, there are very compact 
> commercial SSL libraries such as Mocana NanoSSL and wolfSSL, if you want to 
> really simplify the process.
>

Yup. Every single modern programming language out there has a crypto library.

The high-level languages (e.g. Go) have crypto built into the standard library.

The low-level languages (e.g C or Rust) all have at least one or more well 
supported third party crypto libraries (e.g. for C there's OpenSSL, GnuTLS, 
LibreSSL, Boring SSL, Mbed TLS ... and those are the ones that I can think of 
off the top of my head).

There's no need to do any crypto "from scratch", and indeed you SHOULD NOT.

Re: [EXTERNAL] Re: Flow collection and analysis

[Laura Smith via NANOG]

‐‐‐ Original Message ‐‐‐

On Wednesday, January 26th, 2022 at 14:49, heasley  wrote:
>
> confidentiality and integrity, even if you do not care about authentication.
>
> I am surprised that question is asked.
>

Indeed.

And to add the obvious to the obvious observation above, in certain industries 
and/or jurisdictions its effectively mandatory to encrypt the whole stack.

And that's before we start talking about the "modern" "Zero Trust" mentality 
(which incidentally is nothing new and has been around since at least 2004 with 
the Jericho Forum... but I guess "Zero Trust" sounds cooler).

Re: [EXTERNAL] Re: Flow collection and analysis

[Mel Beckman]

But nobody asked for anything from scratch Eric. Open SSL is it complete ready 
to integrate package. Any developer worth his salt should be able to put it on 
any web application. In addition to OpenSSL, there are very compact commercial 
SSL libraries such as Mocana NanoSSL and wolfSSL, if you want to really 
simplify the process.

Nobody need write any crypto software at all, and the extensive manhours you 
claim are not real.

 -mel

On Jan 27, 2022, at 6:26 PM, Eric Kuhnke  wrote:


Not at all, what I'm recommending is that people who develop something that is 
specialized (like netflow analysis software) don't need to expend the 
person-hours and extensive development time to implement something that has 
already been better implemented by people who are httpd specialists.

The amount of possible design complexities and security risks that go into 
shipping a 'stable' versio of apache2 or nginx are beyond the scope of any 
small to medium sized non-httpd-related opens source software project. Let the 
apache2 or nginx developers handle that.

It's like saying that because a piece of software communicates with something 
externally by SMTP, either inbound or outbound email or both, some software 
developer should take the time to re-implemnt and write from scratch their own 
SMTP, rather than relaying mail via a postfix daemon running on the same server.

Or because you have a piece of software that queries something over SNMP, don't 
use the perfectly good ISC SNMP packages that exist for centos or debian to 
issue snmpgets, but write from scratch your own snmp poller.








On Wed, 26 Jan 2022 at 07:34, Mel Beckman 
mailto:m...@beckman.org>> wrote:
People who advocate TLS lash-ups like nginx front ends remind me of Mr. Beans 
DIY automobile security, which started with a screwed-on metal hasp and 
padlock, and then continued to a range of additional “layers”. Not 
“defense-in-depth”, merely unwarranted “complexity-in-depth”:

https://youtu.be/CCl_KxGLgOA

TLS is a standardized, fully open-source package that can be integrated into 
even tiny IoT devices (witness this $10 WiFi module 
https://www.adafruit.com/product/4201). 
The argument that people who want intrinsically secure products can just 
bolt-on their own security are missing the point entirely. Every web-enabled 
product should be required to implement TLS and then let custiners decide when 
they want to enable it. Vendors who are so weak that they can’t should have 
their products go straight into /dev/null.

-mel via cell

On Jan 26, 2022, at 6:51 AM, heasley 
mailto:h...@shrubbery.net>> wrote:

Wed, Jan 26, 2022 at 07:21:19AM -0600, Mike Hammett:
Why is it [TLS] even necessary for such a function?

confidentiality and integrity, even if you do not care about authentication.
I am surprised that question is asked.

The fewer things that are left unprotected, the better for everyone.  those
with concern about erosion of their privacy and human rights benefit from
everything being protected, everywhere for everyone.

Re: [EXTERNAL] Re: Flow collection and analysis

[Eric Kuhnke]

Not at all, what I'm recommending is that people who develop something that
is specialized (like netflow analysis software) don't need to expend the
person-hours and extensive development time to implement something that has
already been better implemented by people who are httpd specialists.

The amount of possible design complexities and security risks that go into
shipping a 'stable' versio of apache2 or nginx are beyond the scope of any
small to medium sized non-httpd-related opens source software project. Let
the apache2 or nginx developers handle that.

It's like saying that because a piece of software communicates with
something externally by SMTP, either inbound or outbound email or both,
some software developer should take the time to re-implemnt and write from
scratch their own SMTP, rather than relaying mail via a postfix daemon
running on the same server.

Or because you have a piece of software that queries something over SNMP,
don't use the perfectly good ISC SNMP packages that exist for centos or
debian to issue snmpgets, but write from scratch your own snmp poller.








On Wed, 26 Jan 2022 at 07:34, Mel Beckman  wrote:

> People who advocate TLS lash-ups like nginx front ends remind me of Mr.
> Beans DIY automobile security, which started with a screwed-on metal hasp
> and padlock, and then continued to a range of additional “layers”. Not
> “defense-in-depth”, merely unwarranted “complexity-in-depth”:
>
> https://youtu.be/CCl_KxGLgOA
>
> TLS is a standardized, fully open-source package that can be integrated
> into even tiny IoT devices (witness this $10 WiFi module
>  https://www.adafruit.com/product/4201
> ). The argument that people who
> want intrinsically secure products can just bolt-on their own security are
> missing the point entirely. Every web-enabled product should be required to
> implement TLS and then let custiners decide when they want to enable it.
> Vendors who are so weak that they can’t should have their products go
> straight into /dev/null.
>
> -mel via cell
>
> On Jan 26, 2022, at 6:51 AM, heasley  wrote:
>
> Wed, Jan 26, 2022 at 07:21:19AM -0600, Mike Hammett:
>
> Why is it [TLS] even necessary for such a function?
>
>
> confidentiality and integrity, even if you do not care about
> authentication.
> I am surprised that question is asked.
>
> The fewer things that are left unprotected, the better for everyone.  those
> with concern about erosion of their privacy and human rights benefit from
> everything being protected, everywhere for everyone.
>
>

Re: [EXTERNAL] Re: Flow collection and analysis

[Eric Kuhnke]

If the purpose of the software is not to be a dedicated purpose http
daemon, use something that already exists with a deep feature set that can
be configured as needed for the purpose, such as apache2 with openssl or
nginx.

It's not reasonable to expect that the developers of elastiflow reinvent
the wheel and write their own httpd with TLS support, if it can be easily
put "behind" apache2 or nginx. The risks of having people who aren't full
time httpd specialists write their own http daemon and mitigate every
possible security risk in a TLS setup are greater than using what already
exists.

It's a one page size configuration file in nginx.




On Wed, 26 Jan 2022 at 05:17, Laura Smith via NANOG  wrote:

> ‐‐‐ Original Message ‐‐‐
>
> On Wednesday, January 26th, 2022 at 11:08, Eric Kuhnke <
> eric.kuh...@gmail.com> wrote:
>
> > elastiflow is extremely easy to run on an httpd listening only on
> localhost and proxy behind a simple nginx TLS1.2/1.3 only configuration
> listening on port 443.
> >
>
> I don't know about anyone else here, but frankly in 2022 TLS support
> should be a first class citizen.
>
> If I have to mess around with running something else as a proxy in front
> of it then that's the end of my software evaluation.
>
> Crypto is no longer "nice to have" option these days.
>

Re: Flow collection and analysis

[Joel Esler via NANOG]

Are you asking for commercial solutions?  Free solutions?  Open Source?

> On Jan 25, 2022, at 10:46 AM, David Bass  wrote:
> 
> Wondering what others in the small to medium sized networks out there are 
> using these days for netflow data collection, and your opinion on the tool?
> 
> Thanks!

Re: [EXTERNAL] Re: Flow collection and analysis

[Mel Beckman]

Nick,

you can always choose to use nginx if you like, but there’s no reason anyone 
else should be forced to.

 -mel

On Jan 26, 2022, at 7:55 AM, Nick Suan via NANOG  wrote:


While I agree that, yes everything SHOULD support TLS, there's a perfectly good 
reason for terminating TLS in something like (nginx/caddy/apache/etc):  X 
number of things supporting TLS on their web interface means X number of ways 
of configuring TLS.   If I terminate it on nginx, there's only a single way: 
the nginx config, which is then farily easily leveraged into having a single 
set allowed protocols and  ciphers.

On Wed, Jan 26, 2022, at 9:33 AM, Mel Beckman wrote:
People who advocate TLS lash-ups like nginx front ends remind me of Mr. Beans 
DIY automobile security, which started with a screwed-on metal hasp and 
padlock, and then continued to a range of additional “layers”. Not 
“defense-in-depth”, merely unwarranted “complexity-in-depth”:

https://youtu.be/CCl_KxGLgOA


TLS is a standardized, fully open-source package that can be integrated into 
even tiny IoT devices (witness this $10 WiFi module 
https://www.adafruit.com/product/4201). 
The argument that people who want intrinsically secure products can just 
bolt-on their own security are missing the point entirely. Every web-enabled 
product should be required to implement TLS and then let custiners decide when 
they want to enable it. Vendors who are so weak that they can’t should have 
their products go straight into /dev/null.

-mel via cell

On Jan 26, 2022, at 6:51 AM, heasley  wrote:

Wed, Jan 26, 2022 at 07:21:19AM -0600, Mike Hammett:

Why is it [TLS] even necessary for such a function?

confidentiality and integrity, even if you do not care about authentication.
I am surprised that question is asked.

The fewer things that are left unprotected, the better for everyone.  those
with concern about erosion of their privacy and human rights benefit from
everything being protected, everywhere for everyone.

Re: [EXTERNAL] Re: Flow collection and analysis

[Nick Suan via NANOG]

While I agree that, yes everything SHOULD support TLS, there's a perfectly good 
reason for terminating TLS in something like (nginx/caddy/apache/etc):  X 
number of things supporting TLS on their web interface means X number of ways 
of configuring TLS.   If I terminate it on nginx, there's only a single way: 
the nginx config, which is then farily easily leveraged into having a single 
set allowed protocols and  ciphers. 

On Wed, Jan 26, 2022, at 9:33 AM, Mel Beckman wrote:
> People who advocate TLS lash-ups like nginx front ends remind me of Mr. Beans 
> DIY automobile security, which started with a screwed-on metal hasp and 
> padlock, and then continued to a range of additional “layers”. Not 
> “defense-in-depth”, merely unwarranted “complexity-in-depth”: 
> 
> https://youtu.be/CCl_KxGLgOA
> 
> 
> TLS is a standardized, fully open-source package that can be integrated into 
> even tiny IoT devices (witness this $10 WiFi module 
> https://www.adafruit.com/product/4201). The argument that people who want 
> intrinsically secure products can just bolt-on their own security are missing 
> the point entirely. Every web-enabled product should be required to implement 
> TLS and then let custiners decide when they want to enable it. Vendors who 
> are so weak that they can’t should have their products go straight into 
> /dev/null. 
> 
> -mel via cell
> 
>> On Jan 26, 2022, at 6:51 AM, heasley  wrote:
>> 
>> Wed, Jan 26, 2022 at 07:21:19AM -0600, Mike Hammett:
>> 
>>> Why is it [TLS] even necessary for such a function? 
>> 
>> confidentiality and integrity, even if you do not care about authentication.
>> I am surprised that question is asked.
>> 
>> The fewer things that are left unprotected, the better for everyone.  those
>> with concern about erosion of their privacy and human rights benefit from
>> everything being protected, everywhere for everyone.

Re: [EXTERNAL] Re: Flow collection and analysis

[Mel Beckman]

People who advocate TLS lash-ups like nginx front ends remind me of Mr. Beans 
DIY automobile security, which started with a screwed-on metal hasp and 
padlock, and then continued to a range of additional “layers”. Not 
“defense-in-depth”, merely unwarranted “complexity-in-depth”:

https://youtu.be/CCl_KxGLgOA

TLS is a standardized, fully open-source package that can be integrated into 
even tiny IoT devices (witness this $10 WiFi module 
https://www.adafruit.com/product/4201). 
The argument that people who want intrinsically secure products can just 
bolt-on their own security are missing the point entirely. Every web-enabled 
product should be required to implement TLS and then let custiners decide when 
they want to enable it. Vendors who are so weak that they can’t should have 
their products go straight into /dev/null.

-mel via cell

On Jan 26, 2022, at 6:51 AM, heasley  wrote:

Wed, Jan 26, 2022 at 07:21:19AM -0600, Mike Hammett:
Why is it [TLS] even necessary for such a function?

confidentiality and integrity, even if you do not care about authentication.
I am surprised that question is asked.

The fewer things that are left unprotected, the better for everyone.  those
with concern about erosion of their privacy and human rights benefit from
everything being protected, everywhere for everyone.

Re: [EXTERNAL] Re: Flow collection and analysis

[heasley]

Wed, Jan 26, 2022 at 07:21:19AM -0600, Mike Hammett:
> Why is it [TLS] even necessary for such a function? 

confidentiality and integrity, even if you do not care about authentication.
I am surprised that question is asked.

The fewer things that are left unprotected, the better for everyone.  those
with concern about erosion of their privacy and human rights benefit from
everything being protected, everywhere for everyone.

Re: [EXTERNAL] Re: Flow collection and analysis

[Chris Adams]

Once upon a time, Laura Smith  said:
> I don't know about anyone else here, but frankly in 2022 TLS support should 
> be a first class citizen.
> 
> If I have to mess around with running something else as a proxy in front of 
> it then that's the end of my software evaluation.
> 
> Crypto is no longer "nice to have" option these days.

Having every thing under the sun trying to implement the complexities of
TLS leads to lots of failures and security issues... so lots of web
things are designed to be simple and only implement HTTP, listen on
localhost, and let a well-optimized front-end (e.g. nginx) handle the
crypto side (as well as all the weird things browsers do).

It also makes it easier from an system admin point of view, because
handling cert updates in nginx is easy and well-known, so you don't have
to figure out 27 different ways alternate software handles certs and
updates.

-- 
Chris Adams 

Re: [EXTERNAL] Re: Flow collection and analysis

[Mike Hammett]

Why is it even necessary for such a function? 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 

Midwest-IX 
http://www.midwest-ix.com 

- Original Message -

From: "Laura Smith via NANOG"  
To: "nanog@nanog.org list"  
Sent: Wednesday, January 26, 2022 7:17:09 AM 
Subject: Re: [EXTERNAL] Re: Flow collection and analysis 

‐‐‐ Original Message ‐‐‐ 

On Wednesday, January 26th, 2022 at 11:08, Eric Kuhnke  
wrote: 

> elastiflow is extremely easy to run on an httpd listening only on localhost 
> and proxy behind a simple nginx TLS1.2/1.3 only configuration listening on 
> port 443. 
> 

I don't know about anyone else here, but frankly in 2022 TLS support should be 
a first class citizen. 

If I have to mess around with running something else as a proxy in front of it 
then that's the end of my software evaluation. 

Crypto is no longer "nice to have" option these days. 

Re: [EXTERNAL] Re: Flow collection and analysis

[Laura Smith via NANOG]

‐‐‐ Original Message ‐‐‐

On Wednesday, January 26th, 2022 at 11:08, Eric Kuhnke  
wrote:

> elastiflow is extremely easy to run on an httpd listening only on localhost 
> and proxy behind a simple nginx TLS1.2/1.3 only configuration listening on 
> port 443.
>

I don't know about anyone else here, but frankly in 2022 TLS support should be 
a first class citizen.

If I have to mess around with running something else as a proxy in front of it 
then that's the end of my software evaluation.

Crypto is no longer "nice to have" option these days.

Re: [EXTERNAL] Re: Flow collection and analysis

[Eric Kuhnke]

elastiflow is extremely easy to run on an httpd listening only on localhost
and proxy behind a simple nginx TLS1.2/1.3 only configuration listening on
port 443.

as are a number of other tools.



On Tue, 25 Jan 2022 at 16:06, Laura Smith via NANOG  wrote:

> On Tuesday, January 25th, 2022 at 23:50, Compton, Rich A <
> rich.comp...@charter.com> wrote:
>
> > You can pretty much do the same thing with Elastic’s filebeat (
> https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-netflow.html).
>
> >
>
> Has Elastic decided to join the rest of the world in the 21st century yet ?
>
> Last time I looked at it (not too many years ago) they had no TLS
> support.  Bit of a show-stopper in today's security environment.
>

Re: [EXTERNAL] Re: Flow collection and analysis

[John Schiel]

Samplicator is a nifty tool.

--John

On 1/25/22 16:50, Compton, Rich A wrote:


Elastiflow is pretty cool. https://www.elastiflow.com  or the old open 
source version: https://github.com/robcowart/elastiflow


You can pretty much do the same thing with Elastic’s filebeat 
(https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-netflow.html). 



Pmacct is also good for grabbing netflow http://www.pmacct.net and 
sending it somewhere (file, database, kafka, etc.) You can also grab 
BMP and streaming telemetry with it.


If you’re looking for open source DDoS detection using netflow, check 
out https://github.com/pavel-odintsov/fastnetmon


Shameless plug, check out my tool to look for spoofed UDP 
amplification request traffic coming into your network 
https://github.com/racompton/tattle-tale


FYI, you can send netflow to multiple collectors with 
https://github.com/sleinen/samplicator


-Rich

*From: *NANOG  on 
behalf of David Bass 

*Date: *Tuesday, January 25, 2022 at 11:06 AM
*To: *Christopher Morrow 
*Cc: *NANOG list 
*Subject: *[EXTERNAL] Re: Flow collection and analysis

*CAUTION:*The e-mail below is from an external source. Please exercise 
caution before opening attachments, clicking links, or following 
guidance.


Most of these things, yes.

Add:

Troubleshooting/operational support

Customer reporting

On Tue, Jan 25, 2022 at 1:38 PM Christopher Morrow 
 wrote:


On Tue, Jan 25, 2022 at 10:53 AM David Bass
 wrote:

Wondering what others in the small to medium sized networks
out there are using these days for netflow data collection,
and your opinion on the tool?

a question not asked, and answer not provided here, is:
  "What are you actually trying to do with the netflow?"

Answers of the form:
  "Dos detection and mitigation planning"
  "Discover peering options/opportunities"
  "billing customers"

  "traffic analysis for future network planning"

  "abuse monitoring/management/investigations"

  "pretty noc graphs"

are helpful.. I'm sure other answers would as well.. but: "how do
you collect?" is "with a collector" and isn't super helpful if the
collector can't feed into the tooling / infrastructure / long-term
goal you have.

The contents of this e-mail message and
any attachments are intended solely for the
addressee(s) and may contain confidential
and/or legally privileged information. If you
are not the intended recipient of this message
or if this message has been addressed to you
in error, please immediately alert the sender
by reply e-mail and then delete this message
and any attachments. If you are not the
intended recipient, you are notified that
any use, dissemination, distribution, copying,
or storage of this message or any attachment
is strictly prohibited. 

Re: [EXTERNAL] Re: Flow collection and analysis

[Laura Smith via NANOG]

On Tuesday, January 25th, 2022 at 23:50, Compton, Rich A 
 wrote:

> You can pretty much do the same thing with Elastic’s filebeat 
> (https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-netflow.html).
>   
>

Has Elastic decided to join the rest of the world in the 21st century yet ?

Last time I looked at it (not too many years ago) they had no TLS support.  Bit 
of a show-stopper in today's security environment.

Re: [EXTERNAL] Re: Flow collection and analysis

[Compton, Rich A]

Elastiflow is pretty cool.  https://www.elastiflow.com  or the old open source 
version: https://github.com/robcowart/elastiflow
You can pretty much do the same thing with Elastic’s filebeat 
(https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-netflow.html).
Pmacct is also good for grabbing netflow http://www.pmacct.net  and sending it 
somewhere (file, database, kafka, etc.) You can also grab BMP and streaming 
telemetry with it.
If you’re looking for open source DDoS detection using netflow, check out 
https://github.com/pavel-odintsov/fastnetmon
Shameless plug, check out my tool to look for spoofed UDP amplification request 
traffic coming into your network https://github.com/racompton/tattle-tale
FYI, you can send netflow to multiple collectors with 
https://github.com/sleinen/samplicator

-Rich

From: NANOG  on behalf of 
David Bass 
Date: Tuesday, January 25, 2022 at 11:06 AM
To: Christopher Morrow 
Cc: NANOG list 
Subject: [EXTERNAL] Re: Flow collection and analysis

CAUTION: The e-mail below is from an external source. Please exercise caution 
before opening attachments, clicking links, or following guidance.
Most of these things, yes.

Add:
Troubleshooting/operational support
Customer reporting




On Tue, Jan 25, 2022 at 1:38 PM Christopher Morrow 
mailto:morrowc.li...@gmail.com>> wrote:


On Tue, Jan 25, 2022 at 10:53 AM David Bass 
mailto:davidbass...@gmail.com>> wrote:
Wondering what others in the small to medium sized networks out there are using 
these days for netflow data collection, and your opinion on the tool?

a question not asked, and answer not provided here, is:
  "What are you actually trying to do with the netflow?"

Answers of the form:
  "Dos detection and mitigation planning"
  "Discover peering options/opportunities"
  "billing customers"
  "traffic analysis for future network planning"
  "abuse monitoring/management/investigations"
  "pretty noc graphs"

are helpful.. I'm sure other answers would as well.. but: "how do you collect?" 
is "with a collector" and isn't super helpful if the collector can't feed into 
the tooling / infrastructure / long-term goal you have.
E-MAIL CONFIDENTIALITY NOTICE: 
The contents of this e-mail message and any attachments are intended solely for 
the addressee(s) and may contain confidential and/or legally privileged 
information. If you are not the intended recipient of this message or if this 
message has been addressed to you in error, please immediately alert the sender 
by reply e-mail and then delete this message and any attachments. If you are 
not the intended recipient, you are notified that any use, dissemination, 
distribution, copying, or storage of this message or any attachment is strictly 
prohibited.

RE: Flow collection and analysis

[Jean St-Laurent via NANOG]

I agree with you.

 

The tool doesn’t really matter. Windows, linux, cloud or not.

 

It’s really important to first understand what are you trying to solve or 
improve?

 

If this step is forgotten, then it will just be another tool to support to add 
in your long list of useless tools.

 

My personal favorites are a mix of:

 

*   Ntop with PF_RING enabled.
*   Nfdump
*   Elasticsearch 

 

I’m sure all the other tools are also very good. Csv in excel or grep/awk could 
also work if you know exactly what you’re looking for. 

 

Jean

 

 

From: NANOG  On Behalf Of Christopher 
Morrow
Sent: January 25, 2022 12:38 PM
To: David Bass 
Cc:  
Subject: Re: Flow collection and analysis

 

 

 

On Tue, Jan 25, 2022 at 10:53 AM David Bass mailto:davidbass...@gmail.com> > wrote:

Wondering what others in the small to medium sized networks out there are using 
these days for netflow data collection, and your opinion on the tool?

 

a question not asked, and answer not provided here, is:
  "What are you actually trying to do with the netflow?"

 

Answers of the form:
  "Dos detection and mitigation planning"
  "Discover peering options/opportunities"
  "billing customers"

  "traffic analysis for future network planning"

  "abuse monitoring/management/investigations"

  "pretty noc graphs"

 

are helpful.. I'm sure other answers would as well.. but: "how do you collect?" 
is "with a collector" and isn't super helpful if the collector can't feed into 
the tooling / infrastructure / long-term goal you have.

Re: Flow collection and analysis

[Laura Smith via NANOG]

‐‐‐ Original Message ‐‐‐

On Tuesday, January 25th, 2022 at 16:44, Mel Beckman  wrote:

> We use, depending on the situation, Intermapper, PRTG, and NTop.
>
> PRTG includes its web-based flow collector and viewer for free, and there is 
> even a free 100-sensor edition of the product that lets you use just the flow 
> stuff fir free forever.
>

Once upon a time we used to use PRTG.

Nothing bad to say about it as a product, apart from the fact it only runs on 
Windows.

It is an unfortunate fact in today's world with Microsoft's desire to push 
everyone to Azure and make on-prem licensing increasingly unattractive.

Re: Flow collection and analysis

[David Bass]

Most of these things, yes.

Add:
Troubleshooting/operational support
Customer reporting




On Tue, Jan 25, 2022 at 1:38 PM Christopher Morrow 
wrote:

>
>
> On Tue, Jan 25, 2022 at 10:53 AM David Bass 
> wrote:
>
>> Wondering what others in the small to medium sized networks out there are
>> using these days for netflow data collection, and your opinion on the tool?
>
>
> a question not asked, and answer not provided here, is:
>   "What are you actually trying to do with the netflow?"
>
> Answers of the form:
>   "Dos detection and mitigation planning"
>   "Discover peering options/opportunities"
>   "billing customers"
>   "traffic analysis for future network planning"
>   "abuse monitoring/management/investigations"
>   "pretty noc graphs"
>
> are helpful.. I'm sure other answers would as well.. but: "how do you
> collect?" is "with a collector" and isn't super helpful if the collector
> can't feed into the tooling / infrastructure / long-term goal you have.
>

Re: Flow collection and analysis

[Pierre LANCASTRE]

Hi,

There is also Elastiflow https://docs.elastiflow.com/docs/
https://github.com/robcowart/elastiflow.

Cordialement / Best regards

Pierre Lancastre



Le mar. 25 janv. 2022 à 17:45, Mel Beckman  a écrit :

> We use, depending on the situation, Intermapper, PRTG, and NTop.
>
> Intermapper has the most powerful viewing app, but it’s expensive in that
> you have to pay for each flow collector. It’s an actual app (Windows, Mac,
> and Linux), rather than a web-based interface, so they can do more tricks
> with the GUI, like drill down and sorting.
>
> PRTG includes its web-based flow collector and viewer for free, and there
> is even a free 100-sensor edition of the product that lets you use just the
> flow stuff fir free forever.
>
> NTop is an open source web-based flow sensor and viewer, with a combo paid
> license model specifically for the flow collection. It connects directly to
> a mirror port and sucks up the flow info, where is the other products
> required to have some hardware device that exports flows. But you can get
> dirt cheap used Cisco routers that do this, such as the 1941, which
> although bulky do the job for a few hundred bucks. Once you get into 10 Gb
> speeds though you need dedicated hardware sensors in newer gear, which is
> pretty pricey. But if you have 10 Gb traffic you can afford it :-)
>
> Ntop also has a commercial arm called Nbox, Which has a range of hardware
> based ready to go solutions. However it’s all supported out of Italy, and
> pretty much by one guy, so we’ve had uneven results with customers that
> purchased it.
>
> -mel
>
> > On Jan 25, 2022, at 8:30 AM, Laura Smith via NANOG 
> wrote:
> >
> > On Tuesday, January 25th, 2022 at 15:46, David Bass <
> davidbass...@gmail.com> wrote:
> >
> >> Wondering what others in the small to medium sized networks out there
> are using these days for netflow data collection, and your opinion on the
> tool?
> >>
> >> Thanks!
> >
> >
> > Not a suggestion, but a question 
> >
> > Curious to know if anyone (apart from Cloudflare, obvs !) is using
> Goflow ? (https://github.com/cloudflare/goflow)
>

Re: Flow collection and analysis

[Christopher Morrow]

On Tue, Jan 25, 2022 at 10:53 AM David Bass  wrote:

> Wondering what others in the small to medium sized networks out there are
> using these days for netflow data collection, and your opinion on the tool?


a question not asked, and answer not provided here, is:
  "What are you actually trying to do with the netflow?"

Answers of the form:
  "Dos detection and mitigation planning"
  "Discover peering options/opportunities"
  "billing customers"
  "traffic analysis for future network planning"
  "abuse monitoring/management/investigations"
  "pretty noc graphs"

are helpful.. I'm sure other answers would as well.. but: "how do you
collect?" is "with a collector" and isn't super helpful if the collector
can't feed into the tooling / infrastructure / long-term goal you have.

Re: Flow collection and analysis

[Mel Beckman]

We use, depending on the situation, Intermapper, PRTG, and NTop. 

Intermapper has the most powerful viewing app, but it’s expensive in that you 
have to pay for each flow collector. It’s an actual app (Windows, Mac, and 
Linux), rather than a web-based interface, so they can do more tricks with the 
GUI, like drill down and sorting.

PRTG includes its web-based flow collector and viewer for free, and there is 
even a free 100-sensor edition of the product that lets you use just the flow 
stuff fir free forever.

NTop is an open source web-based flow sensor and viewer, with a combo paid 
license model specifically for the flow collection. It connects directly to a 
mirror port and sucks up the flow info, where is the other products required to 
have some hardware device that exports flows. But you can get dirt cheap used 
Cisco routers that do this, such as the 1941, which although bulky do the job 
for a few hundred bucks. Once you get into 10 Gb speeds though you need 
dedicated hardware sensors in newer gear, which is pretty pricey. But if you 
have 10 Gb traffic you can afford it :-)

Ntop also has a commercial arm called Nbox, Which has a range of hardware based 
ready to go solutions. However it’s all supported out of Italy, and pretty much 
by one guy, so we’ve had uneven results with customers that purchased it.

-mel

> On Jan 25, 2022, at 8:30 AM, Laura Smith via NANOG  wrote:
> 
> On Tuesday, January 25th, 2022 at 15:46, David Bass  
> wrote:
> 
>> Wondering what others in the small to medium sized networks out there are 
>> using these days for netflow data collection, and your opinion on the tool?
>> 
>> Thanks!
> 
> 
> Not a suggestion, but a question 
> 
> Curious to know if anyone (apart from Cloudflare, obvs !) is using Goflow ? 
> (https://github.com/cloudflare/goflow)

Re: Flow collection and analysis

[Laura Smith via NANOG]

On Tuesday, January 25th, 2022 at 15:46, David Bass  
wrote:

> Wondering what others in the small to medium sized networks out there are 
> using these days for netflow data collection, and your opinion on the tool?
>
> Thanks!


Not a suggestion, but a question 

Curious to know if anyone (apart from Cloudflare, obvs !) is using Goflow ? 
(https://github.com/cloudflare/goflow)

Re: Flow collection and analysis

[Kevin Glass via NANOG]

My company uses Auvik. It's easy to setup but needs some tuning and is 
furiously expensive. The traffic analysis is pretty good and can be export for 
you to use in other tools. If netflow is all you are using it for look 
elsewhere regardless of what a sales person sales.

Kevin

On Tue, Jan 25, 2022, at 10:46 AM, David Bass wrote:
> Wondering what others in the small to medium sized networks out there are 
> using these days for netflow data collection, and your opinion on the tool?
> 
> Thanks!

Re: Flow collection and analysis

[John Kristoff]

On Tue, 25 Jan 2022 11:46:14 -0400
David Bass  wrote:

> Wondering what others in the small to medium sized networks out there
> are using these days for netflow data collection, and your opinion on
> the tool?

Two open source tools you might consider:

  nfdump 
  pmacct 

John

Re: Flow collection and analysis

[Joe Loiacono]

If your looking to go low-cost (free) try:

1) Carnegie/Mellon's very robust, flexible and efficient collector 
analyzer (command line): SiLK - https://tools.netsa.cert.org/silk


2) FlowViewer - A comprehensive web-based user interface for SiLK which 
provides textual, graphical analysis, long term tracking and dashboard: 
http:flowviewer.net or https://sourceforge.net/projects/flowviewer


Best!

Joe


On 1/25/2022 10:46 AM, David Bass wrote:
Wondering what others in the small to medium sized networks out there 
are using these days for netflow data collection, and your opinion on 
the tool?


Thanks!

Re: Flow collection and analysis

[Mark Tinka]

On 1/25/22 17:46, David Bass wrote:

Wondering what others in the small to medium sized networks out there 
are using these days for netflow data collection, and your opinion on 
the tool?


Kentik.

Happy.

Mark.

Re: Flow collection and analysis

[Dave]

I’ll be the minority voice here - I have been very happy with Argus  
(qosient.com) but it does not have a GUI and that seems to be a factor for some

Dave

> On Jan 25, 2022, at 10:46 AM, David Bass  wrote:
> 
> Wondering what others in the small to medium sized networks out there are 
> using these days for netflow data collection, and your opinion on the tool?
> 
> Thanks!

Flow collection and analysis

[David Bass]

Wondering what others in the small to medium sized networks out there are
using these days for netflow data collection, and your opinion on the tool?

Thanks!