Re: Open source Netflow analysis for monitoring AS-to-AS traffic

On 2024-03-27 09:09, Marinos Dimolianis wrote: My only "concern" was that it did not provide an API for consuming data externally. This is very high on my todo list, notably because I don't want to reimplement Grafana. The API already exists (the current web interface uses it) but it is not

Re: Open source Netflow analysis for monitoring AS-to-AS traffic

Hi Peter, Thanks for that link. I did read the spec, and while the definition itself is clear, the escape clause gives a lot of wiggle room: "Hardware limitations may prevent an exact reporting of the underlying frame length, but an agent should attempt to be as accurate as possible." I read

Re: Open source Netflow analysis for monitoring AS-to-AS traffic

On Fri, 29 Mar 2024 at 20:10, Steven Bakker wrote: > To top it off, both the sFlow and IPFIX specs are sufficiently vague about > the meaning of the "frame size", so vendors can implement whatever they want > (include/exclude padding, include/exclude FCS). This implies that you > shouldn't

Re: Open source Netflow analysis for monitoring AS-to-AS traffic

The sFlow frame_length field isn't intended to be vague. If you are seeing non-conforming sFlow implementations, please raise the issue with the vendor so they can fix the issue. Verifying that the frame_length and stripped fields are correctly implemented is one of the tests performed by the

Re: Open source Netflow analysis for monitoring AS-to-AS traffic

On Fri, 2024-03-29 at 00:15 +, Nick Hilliard wrote: > Overall, sflow has one major advantage over netflow/ipfix, namely > that > it's a stateless sampling mechanism. Precisely. From my corner of the industry, my use case for flow data is extremely limited: I need (sampled) frame information:

Re: Open source Netflow analysis for monitoring AS-to-AS traffic

On Fri, 29 Mar 2024 at 02:15, Nick Hilliard wrote: > Overall, sflow has one major advantage over netflow/ipfix, namely that > it's a stateless sampling mechanism. Once you have hardware that can > Obviously, not all netflow/ipfix implementations implement flow state, > but most do; some

Re: Open source Netflow analysis for monitoring AS-to-AS traffic

On Thu, 28 Mar 2024 at 20:36, Peter Phaal wrote: > The documentation for IOS-XR suggests that enabling extended-router in the > sFlow configuration should export "Autonomous system path to the > destination", at least on the 8000 series routers: >

Re: Open source Netflow analysis for monitoring AS-to-AS traffic

Tom Beecher wrote on 28/03/2024 18:35: Fundamentally I've always disagreed with how sFlow aggregates flow data with network state data. "can aggregate" rather than "aggregates" - this is implementation dependent and most implementations don't bother with it. Overall, sflow has one major

Re: Open source Netflow analysis for monitoring AS-to-AS traffic

Thanks to all who took the time to comment and make suggestions. To summarize the private messages, one respondent suggested Argus as a collector. Another mentioned that they are still using AS-Stats. I'm drawn to Akvorado. I like the self-contained nature of the application. NF collector,

Re: Open source Netflow analysis for monitoring AS-to-AS traffic

The documentation for IOS-XR suggests that enabling extended-router in the sFlow configuration should export "Autonomous system path to the destination", at least on the 8000 series routers:

Re: Open source Netflow analysis for monitoring AS-to-AS traffic

Yeah, cost to implement dst_as_path lookups far outweighs the usefulness IMO. If you really want that it's much better to get it via BMP. ( Same with communities and localpref in the extended gateway definition of sflow. ) Fundamentally I've always disagreed with how sFlow aggregates flow data

Re: Open source Netflow analysis for monitoring AS-to-AS traffic

Hey, On Thu, 28 Mar 2024 at 17:49, Peter Phaal wrote: > sFlow was mentioned because I believe Brian's routers support the feature and > may well export the as-path data directly via sFlow (I am not aware that it > is a feature widely supported in vendor NetFlow/IPFIX implementations?).

Re: Open source Netflow analysis for monitoring AS-to-AS traffic

I hope my comments were useful. I was trying to raise awareness that bgp as-path information is an option and might be helpful in addressing Brian's requirements, "I want to see with which ASes I am exchanging the most traffic across my transits and IX links. I want to look for opportunities to

Re: Open source Netflow analysis for monitoring AS-to-AS traffic

In the same vein, if you can get your devices exporting sFlow, or for others reading that do have sFlow capable devices: the sFlow-RT team has built ready to deploy, all in one docker containers using Grafana and Prometheus that you can stand up within minutes to start visualizing and easily

Re: Open source Netflow analysis for monitoring AS-to-AS traffic

On 27/03/24 01:04, Brian Knight via NANOG wrote: What's presently the most commonly used open source toolset for monitoring AS-to-AS traffic? I want to see with which ASes I am exchanging the most traffic across my transits and IX links. I want to look for opportunities to peer so I can

Re: Open source Netflow analysis for monitoring AS-to-AS traffic

On Wed, 27 Mar 2024 at 21:02, Peter Phaal wrote: > Brian, you may want to see if your routers support sFlow (vendors have added > the feature over the last few years). Why is this a solution, what does it solve for OP? Why is it meaningful what the wire-format of the records are? I read OP's

Re: Open source Netflow analysis for monitoring AS-to-AS traffic

Brian, you may want to see if your routers support sFlow (vendors have added the feature over the last few years). In particular, see if it includes support for the sFlow extended_gateway structure: /* Extended Gateway Data */ /* opaque = flow_data; enterprise = 0; format = 1003 */ struct

Re: Open source Netflow analysis for monitoring AS-to-AS traffic

Try FlowViewer http://flowviewer.net Free, complete, graphical netflow analysis tool. Developed for NASA. Runs on top of SiLK, a powerful open-source netflow capture and analysis tool developed by Carnegie-Mellon for DoD. Supports IPFIX, netflow v5, sflow, IPv6. Text reports, graphing and

Re: Open source Netflow analysis for monitoring AS-to-AS traffic

Brian, I have used Akvorado in an environment with ~80G of traffic and I was super happy. It can be easily set via a docker-compose file and amongst its key benefits is the user-friendly UI that allows you to gain insight into your network traffic. There is also a demo instance available

Re: Open source Netflow analysis for monitoring AS-to-AS traffic

I’m using Alvarado for netflow and I’m pretty happy with it. Seeing it recommended more frequently on Reddit and elsewhere lately too. [akvorado.png] akvorado/akvorado: Flow collector, enricher and visualizer

Re: Open source Netflow analysis for monitoring AS-to-AS traffic

Interested in responses to this as well. Perhaps something informative that I can also adopt for zero $$ would be amazing. In case you do get pointers off-list kindly share- we can walk the journey together and compare notes :) On Wed, 27 Mar 2024 at 03:06, Brian Knight via NANOG wrote: >

Re: Open source Netflow analysis for monitoring AS-to-AS traffic

Brian, Take a peek at Akvorado - https://github.com/akvorado/akvorado We recently set up a lab instance, and seems to check the boxes below. > On Mar 26, 2024, at 19:04, Brian Knight via NANOG wrote: > > What's presently the most commonly used open source toolset for monitoring > AS-to-AS