[
https://issues.apache.org/jira/browse/NIFI-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16550814#comment-16550814
]
Joseph Witt commented on NIFI-5327:
-----------------------------------
There are some really important design elements and tradeoffs to implementing a
powerful feature such as this.
* I'd strongly advocate avoiding use of FlowFile attributes to hold any of the
netflow content. It is an anti-pattern to leverage the attributes for holding
content.
* The design needs to account for the fact that templates arrive dynamically
and they need to be processed to understand how to parse subsequent messages.
* For a use case where NiFi is just a netflow capture/bundle/forward system the
existing processors are likely sufficient or a very narrowly scoped
NetflowCapture processor would do that job just fine. For listening to netflow
in such a way that interesting things can happen with it in NiFi and make
follow-on consumption easier for others a ListenNetflowRecord processor would
likely be quite nice.
A ListenNetflowRecord processor would know each item could either be a template
or a message. If a template it would update an internal cache or a cache held
in a controller service for template. If is a message it would look up the
template associated with that message and write it out using the configured
record writer. It is possible/likely a new schema access strategy would need
to be provided/supported for this but the existing ones might do the trick.
Anyway, it can be tough to get into the details of the design once there is
already code for a current path... Can we reset on this?
Thanks
> NetFlow Processors
> ------------------
>
> Key: NIFI-5327
> URL: https://issues.apache.org/jira/browse/NIFI-5327
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Core Framework
> Affects Versions: 1.6.0
> Reporter: Prashanth Venkatesan
> Assignee: Prashanth Venkatesan
> Priority: Major
>
> As network traffic data scopes for the big data use case, would like NiFi to
> have processors to support parsing of those protocols.
> Netflow is a protocol introduced by Cisco that provides the ability to
> collect IP network traffic as it enters or exits an interface and is
> described in detail in here:
> [https://www.cisco.com/c/en/us/td/docs/net_mgmt/netflow_collection_engine/3-6/user/guide/format.html]
>
> Currently, I have created the following processor:
> *ParseNetflowv5*: Parses the ingress netflowv5 bytes and ingest as either
> NiFi flowfile attributes or as a JSON content. This also sends
> one-time-template.
>
> Further ahead, we can add many processor specific to network protocols in
> this nar bundle.
> I will create a pull request.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
