Re: Antwort: Re: ... Re: dangerous implementation of rep-sharing cache for fsfs

2010-07-01 Thread Greg Hudson
On Thu, 2010-07-01 at 08:56 -0400, michael.fe...@evonik.com wrote: > I better already start to run for it, > when I ever approve the use of the current implementation of the > representation cache. Here's what this says to me: it doesn't matter what the real risks are; it only matters that the q

Re: Antwort: Re: ... Re: dangerous implementation of rep-sharing cache for fsfs

2010-07-01 Thread Daniel Shahaf
Mark Mielke wrote on Thu, 1 Jul 2010 at 10:40 -0400: > I read that article several years ago. Correct me if I am wrong - but the > article does not describe a "real world collision". It describes how it is > technically possible to find a collision in fewer than previous thought > sample. Yeah. A

Re: Antwort: Re: ... Re: dangerous implementation of rep-sharing cache for fsfs

2010-07-01 Thread Mark Mielke
On 07/01/2010 08:00 AM, michael.fe...@evonik.com wrote: Thanks for your wishes, but it seems that I will never be famous: "Hash Function Update Due to Potential Weakness Found in SHA-1" http://www.rsa.com/rsalabs/node.asp?id=2834 I read that article several years ago. Correct me if I am wrong

Antwort: Re: ... Re: dangerous implementation of rep-sharing cache for fsfs

2010-07-01 Thread michael . felke
sion.apache.org" , michael.fe...@evonik.com Thema: Re: Antwort: Re: ... Re: dangerous implementation of rep-sharing cache for fsfs On Wed, Jun 30, 2010 at 10:13 PM, Daniel Shahaf wrote: > [ trim CC ] > > Mark Mielke wrote on Wed, 30 Jun 2010 at 21:37 -: >> On 0

Antwort: Re: ... Re: dangerous implementation of rep-sharing cache for fsfs

2010-07-01 Thread michael . felke
t;dev@subversion.apache.org" Kopie: michael.fe...@evonik.com Thema: Re: Antwort: Re: ... Re: dangerous implementation of rep-sharing cache for fsfs [ trim CC ] Mark Mielke wrote on Wed, 30 Jun 2010 at 21:37 -: > On 06/30/2010 05:57 AM, michael.fe...@evonik.com wrote: > >

Antwort: Re: ... Re: dangerous implementation of rep-sharing cache for fsfs

2010-07-01 Thread michael . felke
dangerous implementation of rep-sharing cache for fsfs I think if you could find a real life collision - you might be able to get some sort of award. Good luck. :-) Cheers, mark On 06/30/2010 05:57 AM, michael.fe...@evonik.com wrote: > Hello, > > O.K., it seems there is really a need to

Re: Antwort: Re: ... Re: dangerous implementation of rep-sharing cache for fsfs

2010-06-30 Thread Erik Huelsmann
On Wed, Jun 30, 2010 at 10:13 PM, Daniel Shahaf wrote: > [ trim CC ] > > Mark Mielke wrote on Wed, 30 Jun 2010 at 21:37 -: >> On 06/30/2010 05:57 AM, michael.fe...@evonik.com wrote: >> > P.S. Thanks for the warning; we are not going to use 1.7. > > Did you check what is the probability of dyin

Re: Antwort: Re: ... Re: dangerous implementation of rep-sharing cache for fsfs

2010-06-30 Thread Daniel Shahaf
[ trim CC ] Mark Mielke wrote on Wed, 30 Jun 2010 at 21:37 -: > On 06/30/2010 05:57 AM, michael.fe...@evonik.com wrote: > > P.S. Thanks for the warning; we are not going to use 1.7. Did you check what is the probability of dying in a car accident? > > At the Moment we are not using 1.6

Re: Antwort: Re: ... Re: dangerous implementation of rep-sharing cache for fsfs

2010-06-30 Thread Mark Mielke
I think if you could find a real life collision - you might be able to get some sort of award. Good luck. :-) Cheers, mark On 06/30/2010 05:57 AM, michael.fe...@evonik.com wrote: Hello, O.K., it seems there is really a need to discuss the problem of SHA-1 collisions more deeply. ... But one

Antwort: Re: ... Re: dangerous implementation of rep-sharing cache for fsfs

2010-06-30 Thread michael . felke
edu, Mark Phippard Thema: Re: Antwort: Re: Re: dangerous implementation of rep-sharing cache for fsfs On 06/25/2010 03:34 PM, Daniel Shahaf wrote: > [1] apparently, no SHA-1 collisions have been found to date. (see > #svn-dev log today) > We know SHA-1 collisions must exist, howe

Re: Antwort: Re: Re: dangerous implementation of rep-sharing cache for fsfs

2010-06-25 Thread Mark Mielke
On 06/25/2010 03:34 PM, Daniel Shahaf wrote: [1] apparently, no SHA-1 collisions have been found to date. (see #svn-dev log today) We know SHA-1 collisions must exist, however - they are also likely to take unlikely form. The algorithms were specifically chosen so that small changes in b

Re: Antwort: Re: Re: dangerous implementation of rep-sharing cache for fsfs

2010-06-25 Thread C. Michael Pilato
Daniel Shahaf wrote: > Daniel Shahaf wrote on Fri, 25 Jun 2010 at 22:34 -: >> If you have specific questions about FSFS internals, you can ask them on >> this list. > > Why don't you just use BDB? Or use FSFS with rep-sharing disabled? BDB does rep-sharing, too, and doesn't allow you to disa

Re: Antwort: Re: Re: dangerous implementation of rep-sharing cache for fsfs

2010-06-25 Thread Daniel Shahaf
Daniel Shahaf wrote on Fri, 25 Jun 2010 at 22:34 -: > If you have specific questions about FSFS internals, you can ask them on > this list. Why don't you just use BDB? Or use FSFS with rep-sharing disabled?

Re: Antwort: Re: Re: dangerous implementation of rep-sharing cache for fsfs

2010-06-25 Thread Daniel Shahaf
michael.fe...@evonik.com wrote on Fri, 25 Jun 2010 at 19:33 -: > Hello, > > Martin got my point: > >> It's not the probability which concerns me, it's what happens when > >> a file collides. If I understood the current algorithm right the > >> new file will be silently replaced by an unrelated

Antwort: Re: Re: dangerous implementation of rep-sharing cache for fsfs

2010-06-25 Thread michael . felke
Hello, Martin got my point: >> It's not the probability which concerns me, it's what happens when a file collides. If I understood the current algorithm right the new file will be silently replaced by an unrelated one and there will be no error and no warning at all. If it's some kind of mach

Re: Antwort: Re: dangerous implementation of rep-sharing cache for fsfs

2010-06-25 Thread Greg Hudson
On Fri, 2010-06-25 at 08:45 -0400, michael.fe...@evonik.com wrote: > I am actually more interested in finding reliable solution > instead of discussing mathematics and probabilities. The discussion of probabilities affects whether it would be justifiable to change Subversion to address hash colli

Re: dangerous implementation of rep-sharing cache for fsfs and libsvn_wc

2010-06-25 Thread Daniel Shahaf
By the way, wc-ng (in Subversion 1.7) will use sha1's for storing pristine texts too. (It will checksum the same contents that fsfs checksums, but currently it doesn't compare filesizes.) So the risk of sha1 collisions (as theoretical or not as it may be) will apply to working copies, too, no

Re: Re: dangerous implementation of rep-sharing cache for fsfs

2010-06-25 Thread Martin Furter
On Fri, 25 Jun 2010, Mark Phippard wrote: On Fri, Jun 25, 2010 at 8:45 AM, wrote: 4. you under estimate the error done by misusing math. methods.   As I already said in my first e-mail. SHA-1 is developed   to detected random and willful data manipulation.   It's a cryptographic hash, so t

Re: dangerous implementation of rep-sharing cache for fsfs

2010-06-25 Thread Daniel Shahaf
Mark Mielke wrote on Fri, 25 Jun 2010 at 17:15 -: > There are many widely used systems that rely on statistical improbability. Public-key encryption is another example. (You always assume that if someone else runs 'gpg --gen-key' they won't get *your* secret key by chance.) > Michael: Feel f

Re: dangerous implementation of rep-sharing cache for fsfs

2010-06-25 Thread Mark Mielke
The rep sharing collisions, even if possible due to collisions around the world (has this happened in practice?), still have no effect on the single repository which does not experience the collision. There are many widely used systems that rely on statistical improbability. Disk drive hardwar

Re: Re: dangerous implementation of rep-sharing cache for fsfs

2010-06-25 Thread Hyrum K. Wright
On Fri, Jun 25, 2010 at 1:45 PM, wrote: > Hello, > > I am actually more interested in finding reliable solution > instead of discussing mathematics and probabilities. You can just disable rep-sharing. You won't have the space savings, but you'll feel better inside, knowing that the near-zero pr

Re: Re: dangerous implementation of rep-sharing cache for fsfs

2010-06-25 Thread Mark Phippard
On Fri, Jun 25, 2010 at 8:45 AM, wrote: > 4. you under estimate the error done by misusing math. methods. > >   As I already said in my first e-mail. SHA-1 is developed >   to detected random and willful data manipulation. >   It's a cryptographic hash, so that there is a low chance of >   guessi

Antwort: Re: dangerous implementation of rep-sharing cache for fsfs

2010-06-25 Thread michael . felke
An: "michael.fe...@evonik.com" Kopie: "dev@subversion.apache.org" Thema: Re: Antwort: Re: dangerous implementation of rep-sharing cache for fsfs On Thu, 2010-06-24 at 11:29 -0400, michael.fe...@evonik.com wrote: > We must ensure that the d

Re: Antwort: Re: dangerous implementation of rep-sharing cache for fsfs

2010-06-24 Thread Greg Hudson
On Thu, 2010-06-24 at 11:29 -0400, michael.fe...@evonik.com wrote: > We must ensure that the data in the repository is, without any concerns, > the data we have once measured or written. You do realize that the probability of data corruption due to faulty hardware is much, much more likely than

Antwort: Re: dangerous implementation of rep-sharing cache for fsfs

2010-06-24 Thread michael . felke
Thema: Re: dangerous implementation of rep-sharing cache for fsfs Julian Foad wrote on Thu, 24 Jun 2010 at 17:21 -: > I am not sure whether the "representation" whose SHA-1 sum is stored is > ever an exact copy of the user's file. If it is - if it does not >

Re: dangerous implementation of rep-sharing cache for fsfs

2010-06-24 Thread Daniel Shahaf
Julian Foad wrote on Thu, 24 Jun 2010 at 17:21 -: > I am not sure whether the "representation" whose SHA-1 sum is stored is > ever an exact copy of the user's file. If it is - if it does not > include an extra header and is not stored in a delta format - then the That is not the case: [[

Re: dangerous implementation of rep-sharing cache for fsfs

2010-06-24 Thread Julian Foad
michael.fe...@evonik.com wrote: > [The new representation caching in 1.6] could save us a lot of disk space > on the server [...]. > > But unfortunately it seems we could not use it :-( > Because after what the source code of rep.cache.c and fs_fs.c in > libsvn_fs_fs looks to me, the mechanism to

dangerous implementation of rep-sharing cache for fsfs

2010-06-24 Thread michael . felke
Excuse me, but i original wrote the following E-Mail to Hyrum K. Wright directly, because I wasn't used to the guidelines of the subversion project. - Weitergeleitet von Michael Felke/AN/Stockhausen/DE am 24.06.2010 11:09 - Michael Felke 23.06.2010 14:07 An: hwri...@tigri