On Fri, 12 Sep 2014, Gareth Bult wrote:
Hi David,
Many thanks, that's very insightful.
(do Amazon really claim 99.999999999% ? my impression is they fall short of
this ??)
note that Glacier has more reliability than S3 storage
http://aws.amazon.com/glacier/
Durable
Amazon Glacier is designed to provide average annual durability of 99.999999999%
for an archive. The service redundantly stores data in multiple facilities and
on multiple devices within each facility. To increase durability, Amazon Glacier
synchronously stores your data across multiple facilities before returning
SUCCESS on uploading archives. Unlike traditional systems that can require
laborious data verification and manual repair, Glacier performs regular,
systematic data integrity checks and is built to be automatically self-healing.
David Lang
Regards,
Gareth.
----- Original Message -----
From: "David Lang" <[email protected]>
To: "rsyslog-users" <[email protected]>
Sent: Thursday, 11 September, 2014 9:48:02 PM
Subject: Re: [rsyslog] Logging to central server / data loss ....
On Thu, 11 Sep 2014, Gareth Bult wrote:
Hi David,
If you can show what the normal operation of the system is, even having the
logging system modify the logs as they flow through the system is acceptable (as
long as you can show the changes are done consistantly)
Mmm, quite possibly. However my experience of being in court and facing an
expert
witness is that as soon as you accept a [potential] flaw in your mechanism for
capturing
information to be used as evidence, in real terms that evidence becomes
worthless. So if
we start with a view that we can't achieve 100% in terms of integrity, I have
to wonder
if it's worth worrying about signing at all .. (?)
The question is what you are trying to defend against.
If you are trying to prove that an event never happened, then you want to try
for perfect logging (and your app needs intent logging)
If you are trying to prove that an event did happen, then the fact that your
logs show it should be enough, the fact that your logs may not show other times
it happened should not be significant.
Yes, the opposing lawyer can try to spin theories about wierd scenarios that
could explain away the existing logs, if you only had another log that said X,
but juries tend not to trust speculation that gets too complicated.
Also, if you have multiple occasions that something has taken place (and how
many lawsuits are over a single access?), they whould have to speculate that the
exact same got lost every time. If you show a dozen events, where 11 have
identical logs and the 12th is missing one line, it's really stretching things
to claim that this means that all 12 are missing some other line.
As for making logs tamper resistant.
When you go into court (or do discovery), you don't give the other side every
log entry you have, you search your logs to give them relevant information. This
breaks most anti-tamper methods in that you can't analyze the extracted logs and
prove that there was no tampering.
There are three scenarios for log tampering.
1. the logs are tampered with in real-time, as they are generated.
This is _really_ hard to perfectly defend agains, but unless you have an
active attacker inside your network, it's not really a significant threat (and
if the opposing lawyer tries to claim that the company has systematically been
tinkering with it's logs to hide stuff for years, just for this case, that gets
back to wild theories, but the company could have been, in spite of any
policies, code reviews, audits, etc that it does, because they could all be
faked)
2. the logs got to the central system intact and were stored, but later on, when
someone knew that this case could be an issue, they went in and tampered with
them to cover up the evidence.
This is actually the biggest threat, but also the easiest to defend against.
grab each day's (or week's) worth of logs, compress them, encrypt them, and
send something offsite, a checksum e-mailed to an externally hosted account that
records all access to that account, or is read-only works. Storing a copy on
Amazon Glaciar storage ($0.01/GB/Month, or less if they've dropped the price in
the last year) is cheap enough to be roughly unlimited.
but note, even Amazon doesn't claim to be perfect, they only claim about 11
9's of reliability (99.999999999%)
3. the logs are there in your storage, but when you searched them to extract
them for your opposition, you tinkered with the result.
This is really a hard thing for a lawyer to claim, because it implies that the
opposition lawyer is a bad actor and trying to fool the court. The bar for doing
this is _really_ high (go do some reading about Prenda Law, they are a horribly
bad set of lawyers, and it took years for them to get classed as such) Lawyers
act as Officers of the Court, and are morally required not to decieve the court
about things like this.
The defense against this is process, and if needed, bringing in the opposition
to watch the process. Not anything technical (because if you are trying to fake
the logs at this point, any technical protections don't matter as you have the
ability to bypass them.
I would agree that signing each message is too expensive, what I was suggesting
was a tool
to verify the integrity of a message. (if the physical process involves
verifying the integrity
of "more" than the message, i.e. a block of data) so be it.
but while you are holding the messages to fill a block they could be tampered
with...
This is why "100% reliable" is a bad goal to set
You get in more trouble by clamining to be perfect and failing than by claming
to be _really_ good, but falling just short of perfection.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
