Thanks for response, but how often metron purges old data? pt., 17.06.2016 o 04:08 użytkownik Nick Allen <[email protected]> napisał:
> I don't think we have the functionality that you are looking for yet. Feel > free to open a JIRA to help define exactly what that might look like. > > I think providing some means of automatically managing data as it ages > would be interesting. One approach I've been thinking about is trading > some fidelity for space as it ages. For example, once I hit a ceiling on > the amount of raw pcap that I can store, the system could potentially > reprocess that raw pcap data to convert it to header-only pcap, flow > records or some other form that consumes less storage, but loses some > fidelity. > > > On Thu, Jun 16, 2016 at 2:50 PM, Kuba Sienkiewicz <[email protected]> > wrote: > > > What Merton thinks is old? (I mean when Merton purges data) About backup > > data I mean having some space for hdfs and having separate partition (or > > machine) for historical data. > > On 14 Jun 2016 22:10, "Nick Allen" <[email protected]> wrote: > > > > > The standard deployment does setup some cleanup tasks that purges 'old' > > > data. This seems to be different from what you're asking though. What > > do > > > you imagine using as your "backup space" if not /dev/null? > > > > > > > > > > > > > > > On Tue, Jun 14, 2016 at 7:45 AM, Kuba Sienkiewicz < > [email protected]> > > > wrote: > > > > > > > PS. I'm saying about automatically moving data away from hdfs to some > > > sort > > > > of backup space. Sorry for inaccuracy in previous email (removing). > > > > > > > > wt., 14.06.2016 o 13:32 użytkownik Kuba Sienkiewicz < > > > [email protected]> > > > > napisał: > > > > > > > > > Hi all, > > > > > I've tested metron for 2 days and I already have over 20GB of data > > (w/o > > > > > any special network traffic, server just stood untouched). > > > > > What are good practices with such big amounts of data that metron > > > stores? > > > > > Also do metron support removing historical data automatically? > > > > > > > > > > Best Regards, > > > > > Jakub Sienkiewicz > > > > > > > > > > > > > > > > > > > > > -- > > > Nick Allen <[email protected]> > > > > > > > > > -- > Nick Allen <[email protected]> >
