[
https://issues.apache.org/jira/browse/METRON-477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536438#comment-15536438
]
Nick Allen commented on METRON-477:
-----------------------------------
I have seen a number of tools that do different things with raw packet data;
for example performance monitoring, fraud analytics, etc. I can save a lot of
money if I only store that pcap data once and can make it available to multiple
consumers.
This JIRA is just one piece of a really efficient, useful, scalable packet
capture service.
1. I need to get pcap data in quickly. This can be done with commodity NICs
and Fastcapa.
2. I need a way to store, archive, and manage all that data; HDFS + METRON-477
3. I need a way to query that data; METRON-477
4. I need a way to feed this data to 3rd party tools and platforms; TBD.
Replay to a virtual NIC? Direct access in HDFS? Other?
> Support lower fidelity retention of network traffic over time
> -------------------------------------------------------------
>
> Key: METRON-477
> URL: https://issues.apache.org/jira/browse/METRON-477
> Project: Metron
> Issue Type: Improvement
> Reporter: Jon Zeolla
>
> Currently fastcapa supports full pcap capture. I would like to see the
> ability to retain network traffic for longer periods of time but at
> increasing less fidelity.
> For instance:
> - Full PCAP is ingested and stored in bucket 1
> - Transition "Full PCAP" to "Truncated PCAP" after bucket 1 hits X size,
> stored in bucket 2
> - Transform the truncated PCAP into flows or daily summaries after bucket 2
> hits X size, stored in bucket 3
> This system should be setup so that the transition jobs are highly
> configurable (as in sizes for each bucket, truncation cutoffs length,
> transition ordering, etc.). In addition, both the full pcap and truncated
> pcap should be able to be retrieved using the same method (CLI, UI, etc.).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
