[
https://issues.apache.org/jira/browse/METRON-477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16283544#comment-16283544
]
Jon Zeolla commented on METRON-477:
-----------------------------------
I wouldn't be against that as a start - my biggest concern is to have both full
pcap and truncated pcap for a period of time would be a pretty big waste of
space, but it lays an important foundation. Then any follow-on bucketing work
would be both more simple to implement and easier to review.
> Support lower fidelity retention of network traffic over time
> -------------------------------------------------------------
>
> Key: METRON-477
> URL: https://issues.apache.org/jira/browse/METRON-477
> Project: Metron
> Issue Type: Improvement
> Reporter: Jon Zeolla
>
> Currently fastcapa supports full pcap capture. I would like to see the
> ability to retain network traffic for longer periods of time but at
> increasing less fidelity.
> For instance:
> - Full PCAP is ingested and stored in bucket 1
> - Transition "Full PCAP" to "Truncated PCAP" after bucket 1 hits X size,
> stored in bucket 2
> - Transform the truncated PCAP into flows or daily summaries after bucket 2
> hits X size, stored in bucket 3
> This system should be setup so that the transition jobs are highly
> configurable (as in sizes for each bucket, truncation cutoffs length,
> transition ordering, etc.). In addition, both the full pcap and truncated
> pcap should be able to be retrieved using the same method (CLI, UI, etc.).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
