I didn't find that obvious. Thanks Dario. I agree that a 90 day data retention window would be long enough to deal with any corrupt/invalid data.
On Tue, Nov 4, 2014 at 11:12 PM, Dario Taraborelli < [email protected]> wrote: > On Nov 4, 2014, at 2:08 PM, Nuria <[email protected]> wrote: > > > No, database records are not affected. It should not impact your work with > EL in any way as your findings come from the records in the database. > The logs are used mainly for operational purposes by the dev team as > maintainers of the system. > > > this obviously means that any missing, corrupted or invalid data from the > log DB can only be recovered within the data retention window, which seems > reasonable to me. > > On Nov 4, 2014, at 11:55 AM, Aaron Halfaker <[email protected]> > wrote: > > Hey guys, > > Sorry for the late response, but I'm still not sure what lives in > */a/eventlogging/archive/** > > Will deleting from there affect what logs we have stored in the DB? Is > this an intermediate log storage place, a canonical one, etc.? > > What will we no longer be able to do after it is pruned? > > -Aaron > > On Thu, Oct 30, 2014 at 2:35 PM, Nuria Ruiz <[email protected]> wrote: > >> >Also, I'm not clear on the significance of the EL archive directory. >> Can you remind me/direct me to documentation? >> Well, the logs just record the incoming pipeline of events, we have used >> them to troubleshoot operational issues in the past but the bulk of data >> analysis in EL happens from data stored on database. >> >> Some info here: >> https://wikitech.wikimedia.org/wiki/EventLogging#Data_storage >> >> >> On Thu, Oct 30, 2014 at 12:22 PM, Aaron Halfaker <[email protected] >> > wrote: >> >>> Nuria, can you specify which logs will be trimmed. >>> >>> Also, I'm not clear on the significance of the EL archive directory. >>> Can you remind me/direct me to documentation? >>> >>> On Thu, Oct 30, 2014 at 12:36 PM, Nuria Ruiz <[email protected]> >>> wrote: >>> >>>> >>>> Hello, >>>> >>>> To comply with our privacy policy we are going to purge logs in 1002 >>>> that are older than 90 days. Please let us know whether this is an issue. >>>> We hope to have these changes done by the end of next week. >>>> >>>> A concrete example: >>>> >>>> Logs in, for example, the eventlogging archiving directory: >>>> >>>> @stat1002:/a/eventlogging/archive$ >>>> >>>> >>>> will be restricted to the last 90 days. >>>> >>>> Thanks, >>>> >>>> Nuria >>>> >>>> >>>> >>>> _______________________________________________ >>>> Analytics mailing list >>>> [email protected] >>>> https://lists.wikimedia.org/mailman/listinfo/analytics >>>> >>>> >>> >>> _______________________________________________ >>> Analytics mailing list >>> [email protected] >>> https://lists.wikimedia.org/mailman/listinfo/analytics >>> >>> >> >> _______________________________________________ >> Analytics mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/analytics >> >> > _______________________________________________ > Analytics mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/analytics > > _______________________________________________ > Analytics mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/analytics > > > > _______________________________________________ > Analytics mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/analytics > >
_______________________________________________ Analytics mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/analytics
