Mmm, the only issue with that is that our logs are not allowed to leave their secure environment - so unless Amazon open an IL3 level service, we don't get to use them ...
----- Original Message ----- From: "David Lang" <[email protected]> To: "rsyslog-users" <[email protected]> Sent: Friday, 12 September, 2014 7:57:13 PM Subject: Re: [rsyslog] Logging to central server / data loss .... On Fri, 12 Sep 2014, Gareth Bult wrote: > Hi David, > > Many thanks, that's very insightful. > > (do Amazon really claim 99.999999999% ? my impression is they fall short of > this ??) note that Glacier has more reliability than S3 storage http://aws.amazon.com/glacier/ Durable Amazon Glacier is designed to provide average annual durability of 99.999999999% for an archive. The service redundantly stores data in multiple facilities and on multiple devices within each facility. To increase durability, Amazon Glacier synchronously stores your data across multiple facilities before returning SUCCESS on uploading archives. Unlike traditional systems that can require laborious data verification and manual repair, Glacier performs regular, systematic data integrity checks and is built to be automatically self-healing. David Lang > Regards, > Gareth. > > > > ----- Original Message ----- > From: "David Lang" <[email protected]> > To: "rsyslog-users" <[email protected]> > Sent: Thursday, 11 September, 2014 9:48:02 PM > Subject: Re: [rsyslog] Logging to central server / data loss .... > > On Thu, 11 Sep 2014, Gareth Bult wrote: > >> Hi David, >> >>> If you can show what the normal operation of the system is, even having the >>> logging system modify the logs as they flow through the system is >>> acceptable (as >>> long as you can show the changes are done consistantly) >> >> Mmm, quite possibly. However my experience of being in court and facing an >> expert >> witness is that as soon as you accept a [potential] flaw in your mechanism >> for capturing >> information to be used as evidence, in real terms that evidence becomes >> worthless. So if >> we start with a view that we can't achieve 100% in terms of integrity, I >> have to wonder >> if it's worth worrying about signing at all .. (?) > > The question is what you are trying to defend against. > > If you are trying to prove that an event never happened, then you want to try > for perfect logging (and your app needs intent logging) > > If you are trying to prove that an event did happen, then the fact that your > logs show it should be enough, the fact that your logs may not show other > times > it happened should not be significant. > > Yes, the opposing lawyer can try to spin theories about wierd scenarios that > could explain away the existing logs, if you only had another log that said X, > but juries tend not to trust speculation that gets too complicated. > > Also, if you have multiple occasions that something has taken place (and how > many lawsuits are over a single access?), they whould have to speculate that > the > exact same got lost every time. If you show a dozen events, where 11 have > identical logs and the 12th is missing one line, it's really stretching things > to claim that this means that all 12 are missing some other line. > > > As for making logs tamper resistant. > > When you go into court (or do discovery), you don't give the other side every > log entry you have, you search your logs to give them relevant information. > This > breaks most anti-tamper methods in that you can't analyze the extracted logs > and > prove that there was no tampering. > > There are three scenarios for log tampering. > > 1. the logs are tampered with in real-time, as they are generated. > > This is _really_ hard to perfectly defend agains, but unless you have an > active attacker inside your network, it's not really a significant threat (and > if the opposing lawyer tries to claim that the company has systematically been > tinkering with it's logs to hide stuff for years, just for this case, that > gets > back to wild theories, but the company could have been, in spite of any > policies, code reviews, audits, etc that it does, because they could all be > faked) > > 2. the logs got to the central system intact and were stored, but later on, > when > someone knew that this case could be an issue, they went in and tampered with > them to cover up the evidence. > > This is actually the biggest threat, but also the easiest to defend against. > > grab each day's (or week's) worth of logs, compress them, encrypt them, and > send something offsite, a checksum e-mailed to an externally hosted account > that > records all access to that account, or is read-only works. Storing a copy on > Amazon Glaciar storage ($0.01/GB/Month, or less if they've dropped the price > in > the last year) is cheap enough to be roughly unlimited. > > but note, even Amazon doesn't claim to be perfect, they only claim about 11 > 9's of reliability (99.999999999%) > > 3. the logs are there in your storage, but when you searched them to extract > them for your opposition, you tinkered with the result. > > This is really a hard thing for a lawyer to claim, because it implies that > the > opposition lawyer is a bad actor and trying to fool the court. The bar for > doing > this is _really_ high (go do some reading about Prenda Law, they are a > horribly > bad set of lawyers, and it took years for them to get classed as such) Lawyers > act as Officers of the Court, and are morally required not to decieve the > court > about things like this. > > The defense against this is process, and if needed, bringing in the > opposition > to watch the process. Not anything technical (because if you are trying to > fake > the logs at this point, any technical protections don't matter as you have the > ability to bypass them. > >> I would agree that signing each message is too expensive, what I was >> suggesting was a tool >> to verify the integrity of a message. (if the physical process involves >> verifying the integrity >> of "more" than the message, i.e. a block of data) so be it. > > but while you are holding the messages to fill a block they could be tampered > with... > > This is why "100% reliable" is a bad goal to set > > You get in more trouble by clamining to be perfect and failing than by claming > to be _really_ good, but falling just short of perfection. > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
