So, here my final report on the issue which has now an entry in our tracker:
http://subversion.tigris.org/issues/show_bug.cgi?id=4577
Patches for /trunk, 1.9 and 1.8 are available; earlier versions are not
affected.
I'll add a more approachable description of the trigger conditions at the
end
of this post.
On Tue, Apr 28, 2015 at 6:18 PM, Roderich Schupp <roderich.sch...@gmail.com>
wrote:
>
> Notes:
> - "svnadmin verify" does NOT report any errors
>
This is a mere caching effect. Because the data is being read incrementally,
the code happens to not run into the the complex reconstruction from disk
data condition.
If you verify only the offending revision, you will see the same error.
Also,
large repos may see the error even in during the full run.
> - indeed the output of "svnadmin dump" looks correct
>
Same as for 'svnadmin verify'.
> - enable-dir-deltification is definitely required to hit the bug
>
Not strictly but the plausible cases require either dir or prop
deltification.
> - also the length of the name of the directory seems to matter,
> at least ~12 characters are required
>
> The above script produces no error for Subversion 1.9.0 beta1, so FWIIW
> I ran svn-bisect on trunk from the branching-off point for 1.8.x ("good",
> ie. has the bug)
> to the branching-off point of 1.9.x ("bad", ie. it's fixed) and the
> "culprit" came out
> as r1554942 which supposedly isn't a fix at all (and modifies a file that
> doesn't exist in 1.8.x).
>
1.9 simply raised the bar for triggering this bug. I simply won't deltify
against very short reps such as the r0 root dir.
Now here is what triggers the bug. Technically, it may happen with any
type of representation - file text, dir contents or properties. Because we
never had PLAIN file text reps, the problem can only be triggered for
files that share a rep with either properties or directories - which is very
unlikely and even then requires various other circumstances.
For directories, at least the following conditions need to be met under
standard FSFS configuration:
1. The directory been created or touched while dir-deltification was
disabled.
Call the last revision in which that has been the case rP.
2. rP is either not packed or close to the end of the shard range, i.e. >
xzy990.
3. Within the next 512 modifications to the respective sub-tree, the number
of dir entries has grown by at least N. If D is the depth of the directory
(D=0 for the root), E the sum of the entries in the parent directories @rP
and C the number of changes in rP, then N must typically be > 3 + 2C + 5D
+ E.
4. Within the next 256 modification of the respective sub-tree (after 3.),
either
4.a) virtually all entries most have been touched, or
4.b) the directory must have grown by >1000 entries.
After 1024 commits to the respective sub-tree, the bug cannot be triggered.
So, rapidly growing directories are vulnerable and so are directories at or
close to
root in young repositories where there are only few entries and no cold
ones, yet.
Property representations have similar trigger conditions:
1. The node properties have been created or touched while
prop-deltification was
disabled. Call the last revision in which that has been the case rP.
2. rP is either not packed or close to the end of the shard range, i.e. >
xzy990.
3. Within the next 512 modifications to the properties, the list is grown
significantly,
several kB of extra property data is typically required.
4. Within the next 256 modification of the respective sub-tree (after 3.),
either
4.a) virtually all property data must be removed / replaced, or
4.b) at least 100kB of property data be inserted.
After 1024 commits to the respective sub-tree, the bug cannot be triggered.
Apart from abusing properties to store large quantities of data, a sequence
of
massive merges (hundreds or revision ranges each) might grow svn:mergeinfo
quickly enough. Depending on merge policies and branching depth, that might
be a realistic scenario.
-- Stefan^2.
