Re: Restoring svn database
Jean-Baptiste DUBOIS wrote on Tue, Mar 09, 2021 at 12:02:51 +:
> I have one last question regarding how svn tags are managed internally in
> fsfs.
> Is svn tag considered as a PLAIN data independant of previous revs or not ?
As far as FSFS is concerned, there's no such thing as a tag. What FSFS
deals with is a zero-indexed array of interrelated directory trees.
By "tag" people normally refer to a child of a well-known directory;
specifically, to its contents and copyfrom information. About these:
- The contents of any given file are stored in that file's `Last Changed
Revision` rev file or in an older rev file. The latter case is only
possible when rep-sharing is enabled, which in your case it's not,
because you use format 2 ("f2"). Watch:
.
1 % svn info --show-item=last-changed-revision file://$PWD/iota{0,1}@86
2 86 file://$PWD/iota0
3 85 file://$PWD/iota1
4 % svnlook tree --show-ids -r 86 ./
5 / <0.0.r86/428>
6 iota0 <2.0.r86/210>
7 iota1 <1.0.r85/210>
8 % xxd -s 210 -l 4096 -p db/revs/85 | xxd -r -p | sed -e '/^$/q'
9 id: 1.0.r85/210
10 type: file
11 pred: 1.0.r83/111
12 count: 42
13 text: 85 0 188 2340 b6cff4dd559998419b5210d8de7c7130
14 cpath: /iota1
15 copyroot: 0 /
16
iota1@86's node-rev header is in r85 (line 7). Its rep is also in r85
(line 13), but in a rep-sharing situation, it might've been in an
older rev file.
Line 8 was written for f2. For a variant supporting f3, f4, and f6, see
.
[For the curious, the bug manifesting in that post is #4129. It was
finally reproduced the following March:
.]
- The copyfrom information is stored in the node-rev header, so it'll be
stored in the rev file of the revision the tag was created in.
Under f2, which doesn't deltify directory reps (see
SVN_FS_FS__MIN_DELTIFICATION_FORMAT), the node-rev itself will be
accessible via the API, as discussed upthread.
Finally, note that "data independent of previous revs" and "PLAIN data"
aren't synonyms, since self-deltas are a thing.
Cheers,
Daniel
Re: How to Prevent SVN commits if the Valid number is not entered
Nico Kadel-Garcia wrote: >He's asking for a "pre-commit" hook. Piping little wrappers into your >local environment doesn't scale, doesn't automate, doesn't prevent >someone else using "TortoiseSVN" from overriding the desired filter, >etc., etc. > >There are many Google accessible answers, but perhaps our hero can >start with a classic example at >https://stackoverflow.com/questions/16751653/unable-to-generate-output-from-svn-pre-commit-hook >. why do you assume the OT is asking for a client (TSVN) side hook? To me his batch script looks like a server side pre-commit hook -- Lorenz
RE: Restoring svn database
Hi Daniel,
Thank you for you explanation.
I wrote a data extractor python script that call 'svnlook changed' and 'svnlook
cat' on valid revs files.
I sucessfully get back all data that do not depends on previous commits lost.
Thank you for your help.
I have one last question regarding how svn tags are managed internally in fsfs.
Is svn tag considered as a PLAIN data independant of previous revs or not ?
BR,
Jean-Baptiste.
-Message d'origine-
De : Daniel Shahaf
Envoyé : vendredi, mars 5, 2021 17:34
À : Jean-Baptiste DUBOIS
Cc : users@subversion.apache.org; Philippe DEMOUSTIER
Objet : Re: Restoring svn database
Be careful: This message was sent from a sender outside of Groupe Atlantic.
Please do not click links or open attachments unless you recognize the source
of this email and know the content is safe.
Jean-Baptiste DUBOIS wrote on Fri, Mar 05, 2021 at 11:04:01 +:
> Inside the 'db' folder we have 'rev', 'revprops' 'transactions' folder
> contains files.
revs/ is where file content lives. See subversion/libsvn_fs_fs/structure.
(You can read it in trunk too: it covers all FSFS formats, not only the newest
one.)
> Some files are missing but the company told us that all files
> recovered are fully recovered (ie: file integrity is OK).
That's good. It's also a very different question than the one Philippe asked.
>
> I know that we could not restore the entire database, but can we
> extract some 'SVN plain data' from file in 'revs' folder
>
> Hereunder a view of thoses files (ordered by Size) ...
>
> [revs]
For future reference, text is preferable to images. Copying the output of
`dir` would have been easier for us to consume.
> Is it possible to detect plain data with no dependency on previous revs
> inside these 'revs' files and extract them ?
Yes. In your case it'll actually be a four-liner loop in the scripting
language of your choice, but I'll give the full answer.
Let's take an example format-2 revision file:
[[[
% rm -rf r
% svnadmin create r --compatible-version=1.4 % cat r/db/format
2
% for i in {1..$(wc -l < ${fn::=~/src/svn/trunk/README})} ; do svnmucc -mm -U
file://$(pwd)/r put =(head -$i $fn) iota$((i % 2)) ; done > /dev/null %
cat_the_youngest_revision_file() { < r/db/revs/*(.om[1]) LC_ALL=C sed -e 's/[^
-~]/X/g' } # translate any octets other than printable ASCII to "X"
% cat_the_youngest_revision_file | nl -ba
1 DELTA 82 0 331
2 SVNX%XX(XXX&X& Finally, be sure to see Appendix B in the
Subversion Book. It
3contains a very quick overview of the major differences between
4CVS and Subversion.
5
6 ENDREP
7 id: 2.0.r86/210
8 type: file
9 pred: 2.0.r84/183
10 count: 42
11 text: 86 0 188 2341 c09759b8da81bf3e23647f4c517abbe0
12 cpath: /iota0
13 copyroot: 0 /
14
15 PLAIN
16 K 5
17 iota0
18 V 16
19 file 2.0.r86/210
20 K 5
21 iota1
22 V 16
23 file 1.0.r85/210
24 END
25 ENDREP
26 id: 0.0.r86/428
27 type: dir
28 pred: 0.0.r85/428
29 count: 86
30 text: 86 347 68 68 d3f8fb2002e019614ec0e47c79e2ac4c
31 cpath: /
32 copyroot: 0 /
33
34 2.0.t85-1 modify true false /iota0
35
36
37 428 558
%
]]]
(Aside: As an f8 repository, the svndiff delta would contain only the last line
of README, rather than the last three lines as in this
example.)
The parts you're interested in are:
- "DELTA %ld %ld %ld" lines (e.g., the «82» on line 1)
- "text:" and "props:" lines (e.g., the «86» on line 11)
- "DELTA\n" lines (without numbers)
- "PLAIN\n" lines
In the first two cases, the first number on the line identifies the revision
number in which depended-on data is found. See the aforementioned «structure»
file for details. The last two cases identify data that's present inline.
Using this information, you could build a DAG of reachable reps (a "rep"
is the thing between the "DELTA" or "PLAIN" line and the "ENDREP" line) and
extract them. However, since you're on format 2, there's an easier way.
Format 2 doesn't support rep-sharing and doesn't deltify directory reps, so
simply running `svnlook changed -r 86` and then `svnlook cat -r 86` against
each file printed thereby should extract everything extractable.
Any given `svn cat` invocation might fail if a DELTA line refers to a revision
whose rev files has been lost. ("text:" and "props:" lines will always point
into the rev file they themselves are in.)
Use `svnlook propget` in addition to `svn cat` to extract versioned properties.
svn_hash_read2() will parse the format. (It's a public API and likely
available via the bindings as well, but if needed, I happen to have posted a
pure Python implementation of that last week:
.)
Note: that's `svnlook cat -r`, not `sv
Re: How to Prevent SVN commits if the Valid number is not entered
On Tue, Mar 9, 2021 at 2:57 AM Kenneth Porter wrote: > > --On Monday, March 08, 2021 8:21 PM -0400 Phani Teja > wrote: > > > Hello i need help regarding SVN , how can i prevent if people do not enter > > with Letter DE followed by 3 or 4 number (e.g DE123 or DE1234) in svn log > > message while doing commit.below is the configurating i have in my > > pre-commit bat file.Any help or lead would be highly appreciated. > > I suggest writing a small script in Perl, Python, Powershell, or whatever > you're comfortable with, and invoking that from your batch file. > And if you want to do something more extensive (or foresee it growing into that), there are some useful utilities / bundles floating around on the net, such as this one: https://github.com/qazwart/SVN-Precommit-Kitchen-Sink-Hook Quite powerful (including functionality to validate commit messages against a regex), though perhaps a bit too much if the only thing you want to do is validate the commit message for the presence of a ticket number. In our environment we use an earlier version of that perl script (with config file), that was once circulated on this very mailinglist. -- Johan
Re: How to Prevent SVN commits if the Valid number is not entered
On Mon, Mar 8, 2021 at 10:03 PM Kenneth Porter wrote: > > --On Monday, March 08, 2021 9:29 PM -0500 Nico Kadel-Garcia > wrote: > > > He's asking for a "pre-commit" hook. Piping little wrappers into your > > local environment doesn't scale, doesn't automate, doesn't prevent > > someone else using "TortoiseSVN" from overriding the desired filter, > > etc., etc. > > I assumed he was running a server on Windows and that's why his pre-commit > hook was a Windows batch file rather than a shell script. (My own server is > on Linux+Apache so the pre-commit hook is based on the sample shell script.) Ahh. I thought you were suggesting an SVN wrapper to use instead of the Subversion command in his local environment. My apologies for misunderstanding your suggestion.
