FYI - forwarding from users@
Stefan Fuhrmann wrote on Sun, 30 Jul 2017 19:14 +0200: > Hi David, > > thanks for the report; the issue turned out to be quite serious. > The underlying bug has been present since at least as early as > 1.8.0 but only surfaced with the added SHA1-collision detection > code in 1.8.18. > > Under certain circumstances, yet-to-be-committed data would be > cached under the wrong key. The impact ranges from bogus errors > as you experienced them to silent repository corruption that only > a 'svnadmin verify' run would detect. So far, no such corruption > has been demonstrated but it remains a plausible scenario. > > Luckily, the problem seems to be mostly restricted to 'svnadmin > load', with the "rep sharing" enable for the respective repo. > 'svnadmin load -M 0' will also not have that issue. > > I fixed the code and the fix should be shipped with the upcoming > 1.8.19 release. > > -- Stefan^2. > > On 29.07.2017 14:37, Bert Huijben wrote: > > This specific error message was added in the last release, so yes it is new > > in > > the last versions. The last 1.9.x version also added it and I’m surprised > > that > > you see the error on 1.8 and not 1.9 (or vice versa). > > > > It tries to tell you that you have two files with an identical SHA-1 hash, > > but > > different contents. A case that we accidentally didn’t catch before, that > > might > > have caused some data loss under extreme circumstances. > > > > This scenario is pretty unlikely to trigger, unless you specifically try to > > do > > that. I’ll try to verify what exactly is in your dump file to see if you > > really > > hit this scenario, or that we have a bug in our code that tries to catch > > this > > scenario. Assuming you properly verified with 1.9.6 (and not 1.9.5 or > > earlier) I > > assume that this is a bug in 1.8.18, as other scenarios are really > > extremely > > unlikely. > > > > I’m not sure I’ll have time to look into this before Monday though, so > > perhaps > > one of the other developers beats me to it. > > > > Bert > > > > *From:*David Engel [mailto:den...@magnitude.com] > > *Sent:* zaterdag 29 juli 2017 00:01 > > *To:* us...@subversion.apache.org > > *Subject:* Checksum mismatch bug in 1.8.18 > > > > Hi, > > > > I think I’m running into a bug in svnadmin in the latest 1.8 release > > (1.8.18). > > It is not present in 1.8.17 nor 1.9.5 nor 1.9.6. I’m guessing it’s related > > to > > the recent changes around strict repsharing. I’m not subscribed to the > > mailing > > list, so please CC me on any responses. > > > > Here are the relevant details: > > > > OS: Windows 10 and Windows 2012 R2 > > > > Subversion release: 1.8.18 > > > > Both SlikSVN release: > > > > svnadmin, version 1.8.18-SlikSvn-1.8.18-X64 (SlikSvn/1.8.18) X64 > > > > compiled Jul 17 2017, 13:03:37 on x86_64-microsoft-windows6.2.9200 > > > > And Collabnet release: > > > > svnadmin, version 1.8.18 (r1800620) > > > > compiled Jul 7 2017, 05:51:59 on x86_64-microsoft-windows6.2.9200 > > > > Performing an svnadmin load results in the following error: > > > > ... > > > > <<< Started new transaction, based on original revision 650 > > > > * editing path : > > Dev/Common/Utils/External/StampVerData/MetaBuilderVersion.inf ... done. > > > > * editing path : > > Dev/Common/Utils/External/StampVerData/MetaBuilderVersionDBG.inf ... done. > > > > * editing path : > > Dev/Common/Utils/External/StampVerData/OracleVersion.inf > > ...svnadmin: E160000: SHA1 of reps '-1 0 45 132 > > a6ea37d29094deece56250ebe167ce46 > > 6a2d6bc4c7e58c0d2627a1b0dd5b23e542aec4d1 650-i2/_8' and '-1 136 45 132 > > a6ea37d29094deece56250ebe167ce46 6a2d6bc4c7e58c0d2627a1b0dd5b23e542aec4d1 > > 650-i2/_8' matches (6a2d6bc4c7e58c0d2627a1b0dd5b23e542aec4d1) but contents > > differ > > > > svnadmin: E160004: Filesystem is corrupt > > > > svnadmin: E200014: Checksum mismatch while reading representation: > > > > expected: a6ea37d29094deece56250ebe167ce46 > > > > actual: 5f696f5d0755f3bcb5e927bdfba4bfa8 > > > > In order to reproduce the error, you can use the attached VersionStamps3 > > repository dump file. It was taken with the same version of svnadmin from > > a > > multi-GB repository that’s been around for over a decade and pruned > > (svndumpfilter) to a small set of files and revisions in one folder while > > still > > encountering the error. I first encountered the error another place very > > quickly in an incremental load on top of a repo that was loaded from > > 1.8.14, but > > that one was on a revision north of 120,000+. This is way easier to > > reproduce. > > I tried filtering the dump further and dropping empty revisions, but the > > error > > did not occur in those cases. The error also did not occur using 1.8.17, > > 1.9.5., and 1.9.6. It also did not occur in 1.8.18 if I set > > enable-rep-sharing > > = false before performing the load. > > > > Repro.bat script: > > > > @echo off > > > > :defineCommands > > > > rem You might need to adjust these lines to point to your > > > > rem compiled-from-source Subversion binaries, if using those: > > > > for %%i in (svn.exe) do set SVN="%%~$PATH:i" > > > > for %%i in (svnadmin.exe) do set SVNADMIN="%%~$PATH:i" > > > > :defineUrls > > > > rem Only supported access method: file:// <file:///>. If http:// or svn://, > > then > > > > rem you'll have to configure it yourself first. > > > > set URL=file:///%CD%/repos > > > > set URL=%URL:\=/% > > > > echo Base url for repo: %URL% > > > > :cleanAllDirsAndCreateRepo > > > > if exist repos rmdir /s /q repos > > > > if exist import-me rmdir /s /q import-me > > > > if exist wc rmdir /s /q wc > > > > %SVNADMIN% create repos > > > > pause > > > > :prepareGreekTree > > > > echo Making a Tree for import... > > > > mkdir import-me\Dev\Common\Utils\External > > > > echo Importing it... > > > > cd import-me > > > > %SVN% import -q -m "Initial import." %URL% > > > > cd .. > > > > rem This is where your reproduction recipe goes. > > > > svnadmin --version > > > > svnadmin load repos < VersionStamps3 > > > > goto :eof > > > > Thanks, > > > > David > >
