Re: Is it possible to export multiple files in one command?
Bo Berglund wrote on Sat, 21 Jan 2023 09:30 +00:00: > What I would like to know is if there is an svn export command switch of some > kind that can be used to export a set of files in one go if they reside in the > same subversion directory? svn checkout --depth=empty https://svn.apache.org/repos/asf/subversion/trunk/notes cd notes svn up --set-depth=infinity -- svnsync.txt sasl.txt
Re: filename encodings and conversion failure
Daniel Sahlberg wrote on Fri, 23 Dec 2022 08:58 +00:00: > Example: Commit a file with ? (questionmark) in the filename on Linux and > checkout the file on Windows. Or case-colliding files: url=`svn info --show-item=url` svn mkdir -- $url/foo $url/FOO svn up > This is a case where a conversion might /be/ necessary (although I don't > have a concrete idea of what the conversion should be). Or else these files > should just be ignored on checkout. > > I'm just mentioning this in case someone looks at the code and decides make > changes to the conversions. Ditto. Cheers, Daniel
Re: filename encodings and conversion failure
Karl Berry wrote on Thu, 22 Dec 2022 22:40 +00:00:
> A file with a name that has some "eight-bit" UTF-8 bytes (fn...-utf8.tex)
> was committed to one of my repositories. When I try to check it out in
> the C locale, svn complains:
>
> $ echo $LC_ALL
> C
> $ svn update
> svn: E22: Can't convert string from 'UTF-8' to native encoding:
> svn: E22: fn{U+00B1}{U+00D7}{U+00F7}{U+00A7}{U+00B6}-utf8.tex
>
> Or, in ls terms:
> $ ls --quoting-style escape fn??*-utf8.tex
> fn\302\261\303\227\303\267\302\247\302\266-utf8.tex
>
> Clearly those UTF-8 code points cannot be "converted" by svn to the
> 7-bit ASCII locale that is "C". Fine; I don't expect it to. Is there a
> way to force svn to complete the checkout anyway?
Perhaps «export LC_ALL=C.UTF-8», if your platform has that encoding?
Good questions in the rest of the email but I'm ENOTIME to deal with them at
the moment.
Cheers,
Daniel
> That is, just check
> out the file and let the name be whatever the bytes are. I don't
> understand why any "conversion" by svn is necessary merely to operate on
> files.
>
> Sure, the name may show up as garbage when I do things in my terminal,
> but that's my problem, not svn's. I didn't ask (and don't want) svn to
> convert anything.
>
> Incidentally, this is not about UTF-8 specifically. The same commit
> included names in SJIS and EUC encodings (they are test files for a new
> feature in Japanese TeX). The question is, in general, why svn needs to
> "convert" filenames at all.
>
> I did some searching both in the mailing list archives and on the web,
> to no avail. People had related problems, but I didn't see this (more
> basic) question being asked.
>
> This is with a somewhat old svn that I compiled myself:
> svn, version 1.13.0 (r1867053)
>compiled Nov 10 2019, 18:06:58 on x86_64-unknown-linux-gnu
>
> I'm guessing svn behavior in this regard has not changed since 1.13.0,
> but if I'm wrong about that, sorry for the noise, and I'll happily
> recompile the latest.
>
> Thanks for any info,
> Karl
Re: Questions about setting up Subversion Server
Felix Natter wrote on Tue, Nov 15, 2022 at 09:03:49 +0100: > - In my svnserve wrapper I call this: > > exec /usr/bin/svnserve "$@" -r /repos > > -> Does the (Linux-) system automatically call this with "-t"? > The -t is added by the client when it invokes the tunnel command (ssh(1)). This is documented in the [tunnels] section of the default ~/.subversion/config file and in Chapter 6 of the book (just search for "-t"). > - If I use ssh auth (svn+ssh://), do I have to change the options in > /svnserve.conf, > like anon-access=none? (I just want access based on file system > permissions, and no > anon access) > If you don't run svnserve without -t, then the value of anon-access shouldn't matter. I'd probably still set it to «none» just in case. > - I am migrating from a 3rd party server which used a different directory > structure, > and system users (no LDAP), but also svn+ssh://. > Can I keep all the history, even if some old users in the 3rd party server > are not > available in the new LDAP auth? (but of course the user names of existing > users are kept) > In other words: does a SVN Server store the user information (history) as > strings or > something else that is linked to accounts? > As strings, in the svn:author revprop. > - I am using strict permissions for two (LDAP-)groups in the base > directories for the repos > (/repos/students, /repos/employees). The Howto I used [1] is using "umask > 002" in the > wrapper. I think for my case (groups) "007" is sufficient, right? > > [1] https://www.startupcto.com/server-tech/subversion/setting-up-svn > > Please CC: me, as I am not subscribed. > > Many Thanks and Best Regards! > > Felix > > -- > > *SIDACT GmbH > Simulation Data Analysis and > Compression Technologies > * > *Felix Natter* > /Software Developer / > > Auguststraße 29 > 53229 Bonn > Germany > > Phone : +49 228 5348 0430 > Direct : +49 228 4097 7118 > Email : felix.nat...@sidact.com > Web : http://www.sidact.com/ >
Re: Triage recovery of damaged Subversion repo
Michael K wrote on Fri, Oct 28, 2022 at 17:25:19 -0500: > I am working on an important Subversion repository that was hit by a > targeted ransomware attack. Apparently the backups were deleted securely as > well, though there is a backup from a few years back that was unaffected in > different storage. In brief, the ransomware encrypted and overwrote (up to) > the first 4 KB of data and also added some encrypted data and zero-padding > to the end of every file. Since Subversion has many small files, the data > has been slashed up badly and some is gone forever. But files larger than 4 > KB have original data remaining. > > My goal is to build a working repository with as much of the original data > that is remaining as I can, like a triage operation. I have a backup that > was not affected, but it does not contain the last few years of data. I > need to utilize the data that is affected by ransomware encryption. > > Eventually I plan to write a program that will work over all the affected > revs and revprops files required and output new files. I'm coming at this > without previous knowledge of the inner workings of Subversion, but I am > comfortable working in a hex editor and writing programs that process raw > data. So for now, I have been learning about Subversion from reading the > documentation and while working hands-on with the raw data of these files > in a hex editor. I've learned a bit about the "representations" within the > revs files. That will probably be helpful since those provide units that > each revs file can be broken down into. I can use that knowledge to try > keeping full "representations" and discard partial ones. > Yes, rev files have quite a bit of internal structure: reps, node-rev headers, changed-paths, P2L/L2P, final line. These are generally easy to parse out of surrounding contexts (revprop files use counted-length strings, reps have their header and "ENDREP" trailer, L2P-INDEX and P2L-INDEX know their own length and have ASCII before and after them, and everything else is ASCII in specific formats). Similarly, it should be easy to recognize where the appended cryptogram and padding start, since the part from L2P-INDEX to the last line is distinctive and self-checksummed. I don't know by heart what elements will be serialized into the first 4KB of a rev file in logical addressing mode. (By the way, it's worth looking up in the implementation what physical order it writes the items to the file in. Chances are this wasn't left to chance.) What you might find there is: - File reps. A rep is a compressed [see fsfs.conf, no relation to "self-compressed" in the sense of having no base] svndelta [see notes/svndiff], whose base, if there is one, might or might not be the preceding revision of the node [see notes/skip-deltas and fsfs.conf] [note: this means it's possible for rN+M of a file to be recoverable even if rN's rep is lost]. In principle, you can even dive down this rabbit hole of abstractions to recover data from the surviving tail ends of partially-overwritten reps. - Dir reps. These are like file reps but the content of the file is an svn_hash_write2() hash mapping basenames to node-rev id's. IIRC, the hashes are dumped in sorted order and the node-rev id's are also fairly predictable, and in any case they are repeated in the node-rev headers of the directory entries. It might even be possible to reconstruct an overwritten dir rep from the remainder of the rev file. - Node-rev headers. Parts of these are predictable (e.g., the "pred:" value), or can be regenerated (e.g., the checksums), or inferred from other parts of the rev file (e.g., "type: dir" can easily be guessed if you still have the rep itself). - Changed-paths. That's just an index/cache, IIRC, of information derivable from the remainder of the file. > Currently, I am trying to add a single new empty revision that Subversion > will accept after testing with the "svnadmin verify" and "svn info" > commands. I fabricated data for a revprops file on this new revision, I > adjusted the "current" file to the new revision number, and I'm working on > the revs file. If I can achieve that, I'll move on to adding a new revision > that contains some original data. > I assume you mean this: [[[ echo Hello world > foo svnadmin create r svnmucc -U file://$(pwd)/r -mm put foo iota # 'svn import' would do the trick too xxd < r/db/revs/1 ]]] Why would you need to /manually/ create a rev file with original data? You can use 'svn commit' to create rev files (on top of the old, good backup). I'd have thought you'd focus on trying to extract data from the partially-corrupted rev files (e.g., reconstruct the fulltexts of reps where it's possible to do so). Anyway, regarding creating rev files: The rev files you get by default have bells and whistles turned on. For instance, they use DELTA and self-DELTA reps even though it's a lot easier to fabricate a PLAIN rep, and
Re: Windows, svn propset svn:ignore *
Jon Daley via users wrote on Fri, 26 Aug 2022 13:13 +00:00: > I don't use Windows, so I can't help you on the escaping of the *, but I > often use propedit, rather than propset (partly because I can never > remember the order of the directory and the property value), Usually (not just in svn) the directory / local file target is last. This makes it easy to pass multiple targets, or to invoke the command via xargs(1), etc.. Also, it plays well with various shells' M-. keybinding to insert the last command's last word again.
Re: Invalid cross-link error when have a mounted subdirectory
Jon Daley via users wrote on Mon, Aug 08, 2022 at 05:12:37 -0400: > On Sun, 7 Aug 2022, Nico Kadel-Garcia wrote: > > On Sun, Aug 7, 2022 at 8:40 AM Jon Daley via users > > wrote: > > > > > > I'm not sure what version this change happened in, but I used to be able > > > to have my /home directory mounted and use subversion commands in my home > > > directory, even though the .svn directory exists at /.svn > > > > Which version are you using, and on which operating system? > Debian Bullseye 11.4. Subversion version 1.14.1 (r1886195). I had > assumed > this would be a known issue on a new version, so I hadn't looked into it > further, but I have another system with the exact same version, but it > works, so it must be a different of the repo or something? /.svn/format is > 12 for both. The term is "working copy". The format number is given by `sqlite3 .svn/wc.db 'pragma user_version;'`. > /etc/subversion/config and ~/.subversion/config are all empty. > > > Climbing up the directory tree past mountpoints is... well, it's a bit > > dangerous. > I understand, and I saw some people using removable drives, which sounds > dangerous to me. I don't see any issues with permanently mounted drives; I > understand there is probably a performance increase by doing some sort of > rename or hard-link or something that only works on one filesystem, but it'd > be nice to either detect that it is across filesystems (which would reduce > the performance increase) or have a config setting that puts it in slow > mode. I suspect it's done this way not for performance reasons but in order to take advantage of rename(2)'s atomicity guarantees. E.g., that's why an 'update' that's SIGKILLed partway, and leaves some items with 'L' in `svn status`, won't leave unversioned files or half-written versioned files behind. > I also realized after my first posting that I can probably checkout > /home > separately from the / repo, which will fix the problem, I think. +1 > I just noticed that I reported this same bug years ago, and it was > "resolved" by fixing the documentation, so I guess that is the end. > It is interesting that it works on some systems. > Presumably rename(2) doesn't return an error on those systems. > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=766285 > > https://lists.apache.org/thread/8qggy35hbft1q88jjqnyjlkb49pvr0zr > >
Re: What's the status of changing UUIDs of repos in working copies?
Ryan Schmidt wrote on Wed, 13 Jul 2022 21:45 +00:00: > [...] that way all your public users' working copies continue to work > as before, and you would just need to check out new working copies for > those hopefully fewer and more technically capable internal users who > were using the formerly private server with its old UUIDs. Or manually change the UUID in those working copies using sqlite3.
Re: What's the status of changing UUIDs of repos in working copies?
Thorsten Schöning wrote on Mon, Jul 11, 2022 at 16:06:11 +0200: > BUT: Each working copy stores a UUID of it associated repo and those > UUIDs are different between private and public repos. In the past, it > wans't supported to change that, one needed to either replace them in > config files or later in the SQLite database maintained in the working > copy. That's still the case. > Do you have any tips for the easiest possible change of the UUIDs? It'll be a lot easier to set the repository's UUID to the one expected by clients. See «svnadmin setuuid».
Re: svnadmin: E16004: Invalid r4422 footer. How to investigate deeper?
Dmitry Minsky wrote on Tue, 28 Jun 2022 13:18 +00:00:
>> What does the "folder with files" contain?
>
> Just a random files on my computer ;) It’s not from working copy or
> repository or anything else meaningful. Let’s assume that it’s just a
> bunch of random files which I want to put in the middle of repo and
> hope that it won’t blow up ;) Is that possible?
And to answer my own question about a simpler solution:
1. Create a copy of the repository up to r7448.
2. Create an empty r7449.
3. Copy over the remaining revisions, discarding (if dump/load: with
svndumpfilter; if svnsync: by using authz to make certain paths
unreadable) all changes to the files that r7449 touched.
This is lossy, but a lot easier.
Actually, you could just do this experiment and in step 2 manually
commit one of the candidate files to the path it was originally
committed in r7449 to, to see whether that candidate is the right one;
but then you'd have to load/sync all revisions between r7450 and the
next revision that touches the file-modified-in-r7449 in order to see
that the candidate isn't the right one.
The important thing is that once you've fabricated an r7449 you can dump
(--deltas --incremental) or svnsync r7450:HEAD on top of it… assuming
all _future_ revisions are fine.
You might want to check that no other revisions are truncated, and which
revisions after r7449 modified the file touched in r7449. ("file",
singular, per the rep-cache output.) If no other revision touched that
file, you can just do the "copy up to r7448, commit one of the
candidates, copy r7450:HEAD" thing with whichever candidate you prefer.
Daniel
Command-line tool for applying deltas? (was: Re: svnadmin: E16004: Invalid r4422 footer. How to investigate deeper?)
Good morning dev@, Anyone has a script that takes as input a file and an svndiff and emits to stdout the result of applying the latter to the former? This came up on users@ in the context of reconstructing a truncated rev file. I've checked tools/. Cheers, Daniel Daniel Shahaf wrote on Tue, 28 Jun 2022 17:58 +00:00: > Assuming I haven't missed any simpler solution, you'll want: ⋮ > 2. A script that takes as input a file and a delta, applies the latter > to the former, and outputs the result. We don't seem to have one of > those already. If you write one, do consider contributing it for our > tools/ directory.
Re: svnadmin: E16004: Invalid r4422 footer. How to investigate deeper?
d, meaning those five deltas comprise one file content delta (based on rep-cache.db) and four directory deltas — one for each directory level between the modified file and the repository root — which can be regenerated by hand. (This is delicate in case the svndiff — meaning the contents of the DELTA — has "copy" instructions that refer to the node-rev id inside the serialized directory node-rev, but possible.) Devs — anyone sees any simpler solution? If you've thought about this and _don't_ see a simpler solution, please say so. Assuming I haven't missed any simpler solution, you'll want: 1. To extract from the r7449 rev file what can be extracted from it. The code for that exists in libsvn_fs_fs, but you'll need to jump through hoops to arrange for it to be called even though r7449 is truncated. Basically, you need to either skip (in the debugger or with a custom patch) or fabricate (by editing rev files manually) everything that happens before libsvn_fs_fs seek()s to a particular offset in the revision file. 2. A script that takes as input a file and a delta, applies the latter to the former, and outputs the result. We don't seem to have one of those already. If you write one, do consider contributing it for our tools/ directory. 3. (possibly, depending on step #1) To regenerate the new dir reps of the truncated r7449 based on r7450 and following revisions. Daniel > % svnadmin dump /var/repo_serpico -r7451 > ~/sdb/test.dump > svnadmin: E160004: Corrupt representation '7449 21 25 159 > 8f3d18747d3388ff2b35096cafbd57ab - -' > svnadmin: E160004: Invalid r7449 footer > > > -- > Dmitry Minsky > >> On 28.06.2022, at 15:50, Daniel Shahaf wrote: >> >> Dmitry Minsky wrote on Tue, 28 Jun 2022 13:18 +00:00: >>>> What does the "folder with files" contain? >>> >>> Just a random files on my computer ;) It’s not from working copy or >>> repository or anything else meaningful. Let’s assume that it’s just a >>> bunch of random files which I want to put in the middle of repo and >>> hope that it won’t blow up ;) Is that possible? >> >> With enough effort, yes. >> >> Devs: In attempting to recreate db/revs/7/7449, what needs to be >> matched? Off the top of my head, it's rep-cache.db references, actual >> rep-sharing references in future rev files, and possibly node-rev id's. >> Anything else? >> >> What's the output of «sqlite3 rep-cache.db '.header on' 'SELECT * FROM >> rep_cache WHERE revision = 7449'»? >> >> Does any rev file after 7449 contain " 7449 " on a "text:" or >> "props:" line? >> >> Does any rev file after 7449 contain ".7449/"? >> >> Daniel >> >>>> On 28.06.2022, at 15:14, Daniel Shahaf wrote: >>>> >>>> Dmitry Minsky wrote on Tue, 28 Jun 2022 11:01 +00:00: >>>>> Ok. I’m pretty sure that db/revs/7/7449 is just truncated. Since there >>>>> aren’t any signs of any text readable data at the bottom of the file >>>>> and the top of file looks similar to 7448, 7450 and to any other >>>>> revision. >>>>> >>>>> So, let’s say I’m 85.23% sure about content of this particular >>>>> revision. How can I recreate revision from folder with files? This rev >>>>> contains only add-dir and add-file changes. >>>> >>>> What does the "folder with files" contain? >>>> >>>> Is it a working copy? A repository? An export? None of the above? >>>> >>>> Does it contain exactly the files and directories added in r7449 *as >>>> they were in that revision*, and nothing else?
Re: svnadmin: E16004: Invalid r4422 footer. How to investigate deeper?
Dmitry Minsky wrote on Tue, 28 Jun 2022 13:18 +00:00: >> What does the "folder with files" contain? > > Just a random files on my computer ;) It’s not from working copy or > repository or anything else meaningful. Let’s assume that it’s just a > bunch of random files which I want to put in the middle of repo and > hope that it won’t blow up ;) Is that possible? With enough effort, yes. Devs: In attempting to recreate db/revs/7/7449, what needs to be matched? Off the top of my head, it's rep-cache.db references, actual rep-sharing references in future rev files, and possibly node-rev id's. Anything else? What's the output of «sqlite3 rep-cache.db '.header on' 'SELECT * FROM rep_cache WHERE revision = 7449'»? Does any rev file after 7449 contain " 7449 " on a "text:" or "props:" line? Does any rev file after 7449 contain ".7449/"? Daniel >> On 28.06.2022, at 15:14, Daniel Shahaf wrote: >> >> Dmitry Minsky wrote on Tue, 28 Jun 2022 11:01 +00:00: >>> Ok. I’m pretty sure that db/revs/7/7449 is just truncated. Since there >>> aren’t any signs of any text readable data at the bottom of the file >>> and the top of file looks similar to 7448, 7450 and to any other >>> revision. >>> >>> So, let’s say I’m 85.23% sure about content of this particular >>> revision. How can I recreate revision from folder with files? This rev >>> contains only add-dir and add-file changes. >> >> What does the "folder with files" contain? >> >> Is it a working copy? A repository? An export? None of the above? >> >> Does it contain exactly the files and directories added in r7449 *as >> they were in that revision*, and nothing else?
Re: svnadmin: E16004: Invalid r4422 footer. How to investigate deeper?
Dmitry Minsky wrote on Tue, 28 Jun 2022 11:01 +00:00: > Ok. I’m pretty sure that db/revs/7/7449 is just truncated. Since there > aren’t any signs of any text readable data at the bottom of the file > and the top of file looks similar to 7448, 7450 and to any other > revision. > > So, let’s say I’m 85.23% sure about content of this particular > revision. How can I recreate revision from folder with files? This rev > contains only add-dir and add-file changes. What does the "folder with files" contain? Is it a working copy? A repository? An export? None of the above? Does it contain exactly the files and directories added in r7449 *as they were in that revision*, and nothing else?
Re: svnadmin: E16004: Invalid r4422 footer. How to investigate deeper?
Dmitry Minsky wrote on Sun, 19 Jun 2022 11:53 +00:00: > There is something in revisions before and after corrupted one: > > % strings repo/db/revs/7/7448 | tail -1 > 2244140591 fa0c1a8229575b0ce27ef0c5a8b898b4 2244140730 > 7d861109493094a15013c7ea105e33a1W > > % strings repo/db/revs/7/7450 | tail -1 > 51995736 59c66b6d95365e6bdb4be4ec3b2d3a34 51995799 > 72059006b7c456b03efb7f07e0557795S > > But on corrupted (actual number of corrupted revision is 7449) it doesn't > give anything meaningful > > % strings repo/db/revs/7/7449 | tail -1 > Y$Q8 > OK, so the last line isn't there at all. What you now need to do is figure out what happened to the revision file. For starters, check whether the change-list portion (a list of lines with "add-file" "modify-dir" and such on them, immediately before «L2P-INDEX») is still there. - If that's the case, the file was likely truncated, but the lost portion of the file can likely be reconstructed manually, though perhaps with some effort (what's the size of db/revs/7/7449 in bytes and the number of lines `svn log -qvr 7449`'s output should have?). - If it got truncated earlier, then it's likely that actual deltas (PLAIN..ENDREP or DELTA..ENDREP) got truncated, i.e., data loss has occurred, so you'd have to reconstruct the revision manually or proceed without the lost data. - And if it's not merely truncated but otherwise corrupted, well, it depends. Do you have any way to reconstruct the revision or its contents? Older backups, mirrors, commit mails, a working copy that committed this revision or updated to it and never got 'svn cleanup' run on it? (The thinking being to exploit issue #4071; cf. <https://subversion.apache.org/docs/release-notes/1.7#wc-pristines>.) Reminder that revision files contain binary data and should not be opened by editors that auto-fix whitespace and so on. And that repositories should be backed up before being manually operated on. > % < repo/db/revs/7/7449 xxd -s Y$Q8 -l 9 > : 4445 4c54 410a 5356 4e DELTA.SVN > That's likely because xxd(1) calls atoi(3) on its argv[2], which (assuming the shell parameter «Q8» is unset) has the value «Y», so the function call returns 0. The output appears sane, though. These bytes are exactly what the start of a revision file might look like. Cheers, Daniel P.S. For any future FSFS hackers out there, note that it's likely r7449 added a file or a directory. (Why? Because gung bhgchg fubjf n /frys-pbzcerffrq/ qrygn.) > On Sun, Jun 19, 2022 at 12:19 PM Daniel Shahaf > wrote: > >> Dmitry Minsky wrote on Sat, 18 Jun 2022 17:16 +00:00: >> > I have a pretty old repository and now going to move it to another >> machine. >> > When I start the dump process I stumbled upon this error in one of the >> old >> > revisions: >> > >> > svnadmin: E16004: Invalid r4422 footer >> > >> >> It's actually E160004. (Just saying this so search engines will find this >> thread.) >> >> > I now workaround with a skip over corrupted revision and continue >> > --incremental dump. But like every 10-50 revisions I still get this error >> > because I suppose there are files that depend on something from this old >> > corrupted revision. The question is, how and where to look in this >> revision >> > so I could manually fix the error by changing some files or checksum or >> > anything? >> >> At the very end of the file, like the following but on db/revs/4/4422: >> >> % strings db/revs/0/1 | tail -1 >> 417 b4657c89ff644471b6760fd6389d253c 445 ea755737e485eeb03c0012e5d6bc1b49I >> % < r/db/revs/0/1 xxd -s 417 -l 9 >> 01a1: 4c32 502d 494e 4445 58 L2P-INDEX >> % < r/db/revs/0/1 xxd -s 445 -l 9 >> 01bd: 5032 4c2d 494e 4445 58 P2L-INDEX >> % >> >> (The first command might pick up some of the P2L/L2P data too? I don't >> remember whether it's guaranteed that there'll be a non-printable >> character between that and the last line.) >> >>
Re: svnadmin: E16004: Invalid r4422 footer. How to investigate deeper?
Dmitry Minsky wrote on Sat, 18 Jun 2022 17:16 +00:00: > I have a pretty old repository and now going to move it to another machine. > When I start the dump process I stumbled upon this error in one of the old > revisions: > > svnadmin: E16004: Invalid r4422 footer > It's actually E160004. (Just saying this so search engines will find this thread.) > I now workaround with a skip over corrupted revision and continue > --incremental dump. But like every 10-50 revisions I still get this error > because I suppose there are files that depend on something from this old > corrupted revision. The question is, how and where to look in this revision > so I could manually fix the error by changing some files or checksum or > anything? At the very end of the file, like the following but on db/revs/4/4422: % strings db/revs/0/1 | tail -1 417 b4657c89ff644471b6760fd6389d253c 445 ea755737e485eeb03c0012e5d6bc1b49I % < r/db/revs/0/1 xxd -s 417 -l 9 01a1: 4c32 502d 494e 4445 58 L2P-INDEX % < r/db/revs/0/1 xxd -s 445 -l 9 01bd: 5032 4c2d 494e 4445 58 P2L-INDEX % (The first command might pick up some of the P2L/L2P data too? I don't remember whether it's guaranteed that there'll be a non-printable character between that and the last line.)
Re: Question about mergeinfo and 2-url merge
Christophe Royer wrote on Sat, 14 May 2022 23:28 +00:00:
> Thank you for looking into this, taking the time to clean up the script
> and share your comments. Lessons learnt.
>
> I will likely revert (maybe manually) those invalid mergeinfo.
>
Sure. I recommend that you use a record-only reverse merge, as opposed
to a propedit. The latter could cause rather a lot more fun if you
commit a syntax error to the repository.
> Still not quite sure of the consequences, and if this situation could be
> detected. Maybe a defect or enhancement should be logged - but I am not
> familiar how this done.
>
See <https://subversion.apache.org/reporting-issues.html>. You've
already posted a thread and gotten someone to validate/buddy your
complaint (me, hereby), so feel free to submit an issue.
If you wish, you can wait a day or two before filing anything, in case
any of the other devs respond to my previous post with the analysis and
the specific question.
> Thanks again.
>
You're welcome.
Daniel
> Christophe
>
> On 5/14/2022 5:39 AM, Daniel Shahaf wrote:
>> Christophe Royer wrote on Fri, May 13, 2022 at 10:45:14 -0700:
>>> Daniel, the first script I sent may confuse things a little, because of the
>>> tree conflict.
>>> Here is another script that avoid that issue. The problem is the same: it
>>> looks like the mergeinfo does not match the state of the branch.
>>
>> [ There's a longish preface here about the reproduction script and its
>> output. For Christophe's mergeinfo-related question, skip the
>> triple-braced part. ]
>>
>> {{{
>> Here's a Unix version of your script:
>>
>> [[[
>> #!/usr/bin/env zsh
>> alias '@ECHO'=: REM=: ECHO.=echo ECHO=echo md=mkdir SET='() {
>> repoRoot=file://$PWD/repo }' TYPE=cat
>> rd() { rm -rf -- ${argv:#/*} }
>> s="$(< script.bat tr '\\' '/' | sed -e 's/%repoRoot%/$repoRoot/g' -e
>> 's/FileA/fileA/g' -e 's/FileB/fileB/g' | perl -C -Mutf8 -pE $'y/()\'"/{}’”/
>> if /ECHO|REM/')"
>> setopt ignorebraces
>> set -ex
>> eval "$s"
>> ]]]
>>
>> That assumes ./script.bat contains the script you posted, without the
>> delimiter lines. It's written as a dense multilingual mix because it
>> doesn't need to be team-maintained indefinitely going forward, so I
>> optimized for whatever was fastest to implement. It'd be fairly easy to
>> port that to sh if need be.
>>
>> Also, it would also have been helpful if you hadn't used different
>> letter cases to refer to a single on-disk file, to make it easier to
>> port the script; if the script began by deleting ./repo ./WC
>> ./workingCopy, so it would be easier to re-run it; and if the script had
>> been attached rather than inlined, because long lines were hard-wrapped
>> along the way; and if you'd posted your script's _output_ as well as its
>> code, so those of us on other platforms would be able to analyze the
>> script without porting it first. Anyway, no harm done.
>>
>> Here's the script's output:
>>
>>> +(eval):1> : off
>>> +(eval):2> : -- Script to test handling of merge info during a 2-url merge
>>> +(eval):3> : -- this time, we setup to avoid tree conflict: we don’t add
>>> file after the branches are created,
>>> +(eval):4> : -- we just update them
>>> +(eval):7> : script.bat script.zsh
>>> +(eval):8> : 2 files in trunk: fileA and fileB
>>> +(eval):9> : 2 branches
>>> +(eval):10> : edit fileA in trunk, merge to foo
>>> +(eval):11> : edit fileB in foo
>>> +(eval):12> : diff between trunk and foo do NOT show fileA '{merged,' same
>>> im 'both}'
>>> +(eval):13> : 2-url merge trunk to foo into bar: should bring changes to
>>> fileB only
>>> +(eval):14> : script.bat script.zsh
>>> +(eval):17> echo
>>>
>>> +(eval):18> echo -- Create empty repo in current folder
>>> -- Create empty repo in current folder
>>> +(eval):19> mkdir repo
>>> +(eval):20> svnadmin create --fs-type fsfs repo/testRepo
>>> +(eval):22> '(anon)' 'repoRoot=file:///%CD:/=/%/repo'
>>> +(anon):0> repoRoot=file:///scratch/tmp.9SbJ9D2q2o/repo
>>> +(eval):24> echo
>>>
>>> +(eval):25> echo -- Create sructure to import in repo
>>> -- Create sructure to import in repo
>>> +(eval):26> mkdir WC
>>> +(eval):27> mkdir WC/trunk
>>> +(eval):28> mkdir WC/branches
>>> +(eval):29> svn import WC file:///scratch/tmp.9SbJ9D2q2o/repo/testRepo
>>> -mImporting
>>>
Re: Question about mergeinfo and 2-url merge
as merged in bar
> -- But the trunk revision where fileA was changed {r5} shows as merged in bar
> +(eval):93> echo -- rev5 of fileA in the trunk
> -- rev5 of fileA in the trunk
> +(eval):94> svn cat
> file:///scratch/tmp.9SbJ9D2q2o/repo/testRepo/trunk/fileA.txt@5
> Adding fileA in trunk
> Editing fileA in trunk
> +(eval):97> echo
>
> +(eval):98> echo -- mergeinfo porperty for bar
> -- mergeinfo porperty for bar
> +(eval):99> svn pg svn:mergeinfo workingCopy/branches/bar
> /branches/foo:3-7
> /trunk:4-5
> +(eval):100> echo
>
> +(eval):101> echo -- svn mergeinfo between trunk and bar
> -- svn mergeinfo between trunk and bar
> +(eval):102> svn mergeinfo workingCopy/trunk workingCopy/branches/bar
> youngest common ancestor
> | last full merge
> | |tip of branch
> | || repository path
>
> 3 58
> | ||
> ---| | trunk
> \ \
> \ \
>--| | branches/bar
>|
>WC
And 'log -v' of the resulting repository:
>
> r8 | daniel | 2022-05-14 11:55:46 + (Sat, 14 May 2022) | 1 line
> Changed paths:
>M /branches/bar
>M /branches/bar/fileB.txt
>
> Merging diff from trunk to foo into bar
>
> r7 | daniel | 2022-05-14 11:55:46 + (Sat, 14 May 2022) | 1 line
> Changed paths:
>M /branches/foo/fileB.txt
>
> Editing fileB in foo
>
> r6 | daniel | 2022-05-14 11:55:46 + (Sat, 14 May 2022) | 1 line
> Changed paths:
>M /branches/foo
>M /branches/foo/fileA.txt
>
> Merging latest from trunk to foo
>
> r5 | daniel | 2022-05-14 11:55:46 + (Sat, 14 May 2022) | 1 line
> Changed paths:
>M /trunk/fileA.txt
>
> Editing fileA in trunk
>
> r4 | daniel | 2022-05-14 11:55:46 + (Sat, 14 May 2022) | 1 line
> Changed paths:
>A /branches/bar (from /trunk:3)
>
> branching to bar
>
> r3 | daniel | 2022-05-14 11:55:46 + (Sat, 14 May 2022) | 1 line
> Changed paths:
>A /branches/foo (from /trunk:2)
>
> branching to foo
>
> r2 | daniel | 2022-05-14 11:55:46 + (Sat, 14 May 2022) | 1 line
> Changed paths:
>A /trunk/fileA.txt
>A /trunk/fileB.txt
>
> Adding 2 files to trunk
>
> r1 | daniel | 2022-05-14 11:55:46 + (Sat, 14 May 2022) | 1 line
> Changed paths:
>A /branches
>A /trunk
>
> Importing
>
}}}
So, here's r8:
[[[
% cd workingCopy
% svn di -c 8
Index: branches/bar/fileB.txt
===
--- branches/bar/fileB.txt (revision 7)
+++ branches/bar/fileB.txt (revision 8)
@@ -1 +1,2 @@
Adding fileB in trunk
+Editing fileB in foo
Index: branches/bar
===
--- branches/bar(revision 7)
+++ branches/bar(revision 8)
Property changes on: branches/bar
___
Added: svn:mergeinfo
## -0,0 +0,2 ##
Merged /trunk:r4-5
Merged /branches/foo:r3-7
]]]
The question is why r8 adds mergeinfo for trunk:5 even though it doesn't
bring in the changes to fileA.txt made in that revision.
I'm guessing the 2-URL merge added mergeinfo for trunk:4-5 because
trunk:4-5 appear in the diff between trunk@HEAD and foo@HEAD.
Similarly, the changes to fileA.txt in r5 weren't part of that diff and
therefore weren't added to bar.
The resulting state is definitely incorrect. It can be corrected by
an appropriate record-only reverse merge (e.g., «svn merge --record-only
-c-5 trunk branches/bar»).
Could Subversion get this right by default? Good question. I don't
remember whether this has been discussed before. If anyone else does,
do jump in ☺
Cheers,
Daniel
> -start of 2nd script
> ⋮
> -end of 2nd script
>
> Christophe
>
> On 5/12/2022 4:41 PM, Christophe Royer wrote:
> > Thank you Daniel, below is the script I used (line wrapping might mess
> > up a few lines, sorry)
> >
> &
Re: SVN Diff fails with error
Aravind Raj wrote on Mon, 09 May 2022 09:27 +00:00: > As the changes are huge we are getting the above error, this does not > happen when the change is short. Another important observation is, this > issue does not happen in Tortoise SVN version below 1.8. So, *assuming for > Subversion client with Neon library has no problem but with serf library > this error occurs.* > Not necessarily. 1.7 and earlier _could_ use serf. > Analysis done: > >- No connection drop was detected between Subversion server and >TortoiseSVN Client >- TCPDumps are reviewed no issues observed >- Several TCP/IP related settings were updated on Windows but still >issue persists. >- Tried changing the following token on svn client but no luck -- >'Timeout', 'http-max-connection', 'http-bulk-updates', 'http-compression'. > > The error is not from the server, its from the client. We don't see any > information about the error from the server logs. Double check that, please? The error message you quoted isn't the root cause. If there wasn't a TCP RST and there wasn't any second line to the error message, then there should be something httpd's error log (as opposed to the access log). At the very least, the six-digit error code should be different. Is there a proxy in front of httpd? > As we can also see the issue exists in client version above 1.8 but not > less than 1.8. > > We are also able to reproduce the issue in TortoiseSVN and Windows Command > line client. > Subversion server version: 1.10.2 > Subversion client versions tested: >1.8 and <1.14 > Try the latest 1.14.x. > Any assistance on this error is greatly appreciated. > > Thanks > Aravind
Re: Question about mergeinfo and 2-url merge
Could you post the script, please? It's hard to answer your question when it describes the details verbally rather than machine-readably. It sounds like a supported scenario. Cheers, Daniel Christophe Royer wrote on Fri, 06 May 2022 21:46 +00:00: > I recently saw some mergeinfo that I can explain but still look wrong to > me. Not sure if it’s a wrong usage of svn, or possibly a defect, and so > far I have not seen any post about this. Here is the scenario (I have a > script, batch file for windows, if needed): > > Seen first using svn 1.6.17, confirmed with with svn 1.14.2 > > Here is the setup: > >* Create repo with trunk and branches >* Add a file to trunk >* Create 2 branches from trunk, foo and bar (at this point, neither > branch has mergeinfo for trunk) >* In foo, edit the existing file >* Make some changes in trunk (I added a file) >* Merge trunk to foo (this bring the new file in and adds mergeinfo > for trunk, including the revision where the new file was added) > > Now the fun part: > >* Do a 2-url merge to merge the changes solely made in foo, and apply > them to bar > > The diff between trunk and foo shows only one file was edited (the > second file does not show, since merge made it the same in trunk and > foo). But the diff also shows mergeinfo changed between trunk and foo, > so those get merged to bar, which seem appropriate. > > However, it follows that bar now indicates that it has those changes > from trunk (esp. the one revision where the file was added in trunk) but > of course that file is not in bar…And this occurs whether or not I use > –ignore-ancestry. > > I could use cherrypicking instead, but I did not want to have to pick > specific changes (I really want all the changes made in foo, but only > those changes) > > So, > > 1)is that a misuse of svn, an unsupported scenario?. Or is there room > for improvement to the mergeinfo management? > > 2)should I manually delete those “unwanted” mergeinfo? If I don’t, I > have an idea of the issues I can run into. But I am afraid to miss > something here, and removing them would cause other issues. > > > Thanks > > Christophe
Re: configure script enhancement proposal
Ilari Jääskeläinen wrote on Sat, 16 Apr 2022 10:03 +00:00: > Should it probe for libexpat with pkg-config? Well, expat does register itself with pkg-config (on my system at least), so I imagine we'll accept patches for this. Sorry we didn't answer sooner. Cheers, Daniel
A New Feature[:] Film About Subversion
[ Follow-ups to dev@ only, please. ]
Hi, everybody!
As y'all may recall, issue #525 concerns implementing working copies
that need not store an unmodified copy ("pristine", formerly "text-base")
of every versioned file:
https://subversion.apache.org/issue/525
Our currently-envisioned design is documented in what will become
1.15.0's release notes:
https://subversion-staging.apache.org/docs/release-notes/1.15#bare-working-copies
Our development notes are on issue #525's feature branch:
https://svn.apache.org/repos/asf/subversion/branches/pristines-on-demand-on-mwf/BRANCH-README
One planned change is to add "hydrating" functions to the internal
interlibrary API:
https://github.com/apache/subversion/commit/dbfcd85cd12fe624d2fbb845da24036bb519aa28
(see the changes under subversion/include/private/)
We are also contemplating adding an «svn hydrate» command:
https://mail-archives.apache.org/mod_mbox/subversion-dev/202201.mbox/%3C877dapkri1.fsf%40red-bean.com%3E
(for further context, see )
The transition between the non-hydrated state and the hydrated state
will be done via "locks", so we'll be adding another paragraph to this
already-overlong sidebar:
https://svnbook.red-bean.com/nightly/en/svn.advanced.locking.html#svn.advanced.locking.meanings
We expect the ability to hydrate a working copy will be particularly
useful to users of the third-party client submerge(1):
https://manpages.debian.org/unstable/subcommander/submerge.1.en.html
We have prepared a short film explaining our vision. A preliminary
Internet Movie Database™ entry is here:
imdb://tt0xC27BF2/
(spoiler: gur yvax vf erny, ohg jr nerag nssvyvngrq jvgu vg;
jr whfg pbhyqag erfvfg gur cha)
It's not yet released, of course, but if you're interested in a preview,
let us know; a small number of beta copies are available.
Cheers,
Daniel
P.S. Sorry about that github URL. I would have linked to
(aka ), but both of these URLs just
redirect to that github URL. (With HTTP 301, too… ☹) Does anyone else
see this? Perhaps it's just the EU mirror? svn.a.o resolves to
13.90.137.153 here.
Re: merging into a read only file succeeds
Daniel Shahaf wrote on Fri, Feb 25, 2022 at 21:41:10 +:
> Daniel Shahaf wrote on Fri, Feb 25, 2022 at 15:18:44 +:
> > Nathan Hartman wrote on Thu, 24 Feb 2022 23:44 +00:00:
> > > On Thu, Feb 24, 2022 at 2:33 PM Mark Phippard wrote:
> > >> Maybe merge should just refuse to run at all if it detects any
> > >> svn:needs-lock properties in the WC?
> > >
> > > Then it would never be able to run; I think you meant: refuse to run
> > > if we don't hold the lock on a file that has the svn:needs-lock
> > > property. However, it would be rather irritating if the refusal to run
> > > were caused by files unaffected by the merge.
> >
> > Or if the user deliberately did the merge without intending to commit it.
> >
> > How about making this situation a conflict? "local file not locked,
> > incoming edit upon merge". Then the file is not silently changed, but
> > the user can use --accept or «svn resolve» to make the changes anyway if
> > they know that's what they want.
>
> Is it just this?
Had a "goto fail;" bug there. Corrected version:
[[[
Index: subversion/include/svn_wc.h
===
--- subversion/include/svn_wc.h.orig
+++ subversion/include/svn_wc.h
@@ -1657,7 +1657,10 @@ typedef enum svn_wc_conflict_reason_t
/** Object is moved away. @since New in 1.8. */
svn_wc_conflict_reason_moved_away,
/** Object is moved here. @since New in 1.8. */
- svn_wc_conflict_reason_moved_here
+ svn_wc_conflict_reason_moved_here,
+ /** #SVN_PROP_NEEDS_LOCK was set and the file wasn't locked.
+ * @since New in 1.15. */
+ svn_wc_conflict_reason_not_locked
} svn_wc_conflict_reason_t;
Index: subversion/libsvn_client/conflicts.c
===
--- subversion/libsvn_client/conflicts.c.orig
+++ subversion/libsvn_client/conflicts.c
@@ -244,6 +244,7 @@ static const svn_token_map_t map_conflic
{ "unversioned", svn_wc_conflict_reason_unversioned },
{ "moved-away", svn_wc_conflict_reason_moved_away },
{ "moved-here", svn_wc_conflict_reason_moved_here },
+ { "not-locked", svn_wc_conflict_reason_not_locked },
{ NULL, 0 }
};
@@ -1327,6 +1328,11 @@ describe_local_file_node_change(const ch
}
break;
}
+ case svn_wc_conflict_reason_not_locked:
+{
+ *description = _("hello world");
+ break;
+}
}
return SVN_NO_ERROR;
@@ -1477,8 +1483,8 @@ describe_local_dir_node_change(const cha
scratch_pool));
}
}
- }
- break;
+}
+break;
case svn_wc_conflict_reason_moved_here:
{
const char *moved_from_abspath;
@@ -1552,6 +1558,10 @@ describe_local_dir_node_change(const cha
}
}
}
+break;
+ case svn_wc_conflict_reason_not_locked:
+*description = _("hello world");
+break;
}
return SVN_NO_ERROR;
@@ -3169,6 +3179,9 @@ describe_local_none_node_change(const ch
*description = _("An item had been moved here in the working copy "
"at the time this conflict was recorded.");
break;
+case svn_wc_conflict_reason_not_locked:
+ *description = _("hello world");
+ break;
}
return SVN_NO_ERROR;
Index: subversion/libsvn_wc/conflicts.c
===
--- subversion/libsvn_wc/conflicts.c.orig
+++ subversion/libsvn_wc/conflicts.c
@@ -520,6 +520,7 @@ static const svn_token_map_t reason_map[
{ "replaced", svn_wc_conflict_reason_replaced },
{ "moved-away", svn_wc_conflict_reason_moved_away },
{ "moved-here", svn_wc_conflict_reason_moved_here },
+ { "not-locked", svn_wc_conflict_reason_not_locked },
{ NULL }
};
Index: subversion/libsvn_wc/merge.c
===
--- subversion/libsvn_wc/merge.c.orig
+++ subversion/libsvn_wc/merge.c
@@ -24,6 +24,7 @@
#include "svn_wc.h"
#include "svn_diff.h"
#include "svn_dirent_uri.h"
+#include "svn_hash.h"
#include "svn_path.h"
#include "svn_pools.h"
#include "svn_props.h"
@@ -1142,6 +1143,32 @@ svn_wc__internal_merge(svn_skel_t **work
cancel_func, cancel_baton,
scratch_pool, scratch_pool));
+{
+ svn_wc__db_lock_t *lock;
+ apr_hash_t *props;
+
+ /* TODO: short-circuit one of the two calls */
+ SVN_ERR(sv
Re: merging into a read only file succeeds
Daniel Shahaf wrote on Fri, Feb 25, 2022 at 15:18:44 +:
> Nathan Hartman wrote on Thu, 24 Feb 2022 23:44 +00:00:
> > On Thu, Feb 24, 2022 at 2:33 PM Mark Phippard wrote:
> >> Maybe merge should just refuse to run at all if it detects any
> >> svn:needs-lock properties in the WC?
> >
> > Then it would never be able to run; I think you meant: refuse to run
> > if we don't hold the lock on a file that has the svn:needs-lock
> > property. However, it would be rather irritating if the refusal to run
> > were caused by files unaffected by the merge.
>
> Or if the user deliberately did the merge without intending to commit it.
>
> How about making this situation a conflict? "local file not locked,
> incoming edit upon merge". Then the file is not silently changed, but
> the user can use --accept or «svn resolve» to make the changes anyway if
> they know that's what they want.
Is it just this?
[[[
Index: subversion/include/svn_wc.h
===
--- subversion/include/svn_wc.h (revision 1898427)
+++ subversion/include/svn_wc.h (working copy)
@@ -1657,7 +1657,10 @@ typedef enum svn_wc_conflict_reason_t
/** Object is moved away. @since New in 1.8. */
svn_wc_conflict_reason_moved_away,
/** Object is moved here. @since New in 1.8. */
- svn_wc_conflict_reason_moved_here
+ svn_wc_conflict_reason_moved_here,
+ /** #SVN_PROP_NEEDS_LOCK was set and the file wasn't locked.
+ * @since New in 1.15. */
+ svn_wc_conflict_reason_not_locked
} svn_wc_conflict_reason_t;
Index: subversion/libsvn_client/conflicts.c
===
--- subversion/libsvn_client/conflicts.c(revision 1898427)
+++ subversion/libsvn_client/conflicts.c(working copy)
@@ -244,6 +244,7 @@ static const svn_token_map_t map_conflict_reason[]
{ "unversioned", svn_wc_conflict_reason_unversioned },
{ "moved-away", svn_wc_conflict_reason_moved_away },
{ "moved-here", svn_wc_conflict_reason_moved_here },
+ { "not-locked", svn_wc_conflict_reason_not_locked },
{ NULL, 0 }
};
@@ -1327,6 +1328,11 @@ describe_local_file_node_change(const char **descr
}
break;
}
+ case svn_wc_conflict_reason_not_locked:
+{
+ *description = _("hello world");
+ break;
+}
}
return SVN_NO_ERROR;
@@ -1477,8 +1483,8 @@ describe_local_dir_node_change(const char **descri
scratch_pool));
}
}
- }
- break;
+}
+break;
case svn_wc_conflict_reason_moved_here:
{
const char *moved_from_abspath;
@@ -1552,6 +1558,10 @@ describe_local_dir_node_change(const char **descri
}
}
}
+break;
+ case svn_wc_conflict_reason_not_locked:
+*description = _("hello world");
+break;
}
return SVN_NO_ERROR;
@@ -3168,6 +3178,9 @@ describe_local_none_node_change(const char **descr
case svn_wc_conflict_reason_moved_here:
*description = _("An item had been moved here in the working copy "
"at the time this conflict was recorded.");
+ break;
+case svn_wc_conflict_reason_not_locked:
+ *description = _("hello world");
break;
}
Index: subversion/libsvn_wc/conflicts.c
===
--- subversion/libsvn_wc/conflicts.c(revision 1898427)
+++ subversion/libsvn_wc/conflicts.c(working copy)
@@ -520,6 +520,7 @@ static const svn_token_map_t reason_map[] =
{ "replaced", svn_wc_conflict_reason_replaced },
{ "moved-away", svn_wc_conflict_reason_moved_away },
{ "moved-here", svn_wc_conflict_reason_moved_here },
+ { "not-locked", svn_wc_conflict_reason_not_locked },
{ NULL }
};
Index: subversion/libsvn_wc/merge.c
===
--- subversion/libsvn_wc/merge.c(revision 1898427)
+++ subversion/libsvn_wc/merge.c(working copy)
@@ -24,6 +24,7 @@
#include "svn_wc.h"
#include "svn_diff.h"
#include "svn_dirent_uri.h"
+#include "svn_hash.h"
#include "svn_path.h"
#include "svn_pools.h"
#include "svn_props.h"
@@ -1141,6 +1142,31 @@ svn_wc__internal_merge(svn_skel_t **work_items,
SVN_ERR(maybe_update_target_eols(_abspath, prop_diff, left_abspath,
cancel_func, cancel_baton,
scratch_pool, scratch_pool));
+
+{
+ svn_wc__db_lock_t *lock;
+ apr_hash_t *props;
+
+
Re: merging into a read only file succeeds
Mark Phippard wrote on Fri, 25 Feb 2022 17:59 +00:00: > On Fri, Feb 25, 2022 at 12:29 PM Daniel Shahaf > wrote: >> >> Mark Phippard wrote on Fri, Feb 25, 2022 at 10:39:13 -0500: >> > On Fri, Feb 25, 2022 at 10:19 AM Daniel Shahaf >> > wrote: >> > > >> > > Nathan Hartman wrote on Thu, 24 Feb 2022 23:44 +00:00: >> > > > On Thu, Feb 24, 2022 at 2:33 PM Mark Phippard >> > > > wrote: >> > > >> >> > > >> On Thu, Feb 24, 2022 at 10:51 AM Lorenz wrote: >> > > >> >> > > >> > just discovered, that merging into a RO file succeeds. >> > > >> > I my case the file is RO because of a needs-lock property. >> > > >> > So when I try to commit I can't. >> > > >> > >> > > >> > This effect ist at least annoying. >> > > >> > There is a reason why the file is read only. >> > > >> >> > > >> It is interesting both that it was able to do this and also that this >> > > >> has never come up before. I am not sure I can wrap my head around what >> > > >> I think should even happen here. There is no way you could be expected >> > > >> to obtain locks on these files before merging. >> > > >> > > Why can't we expect that? >> > >> > I had not considered your proposal to just lock everything before the >> > merge. I agree that would be a solution. I cannot predict user >> > reaction because it is hard to know what kind of scale we are talking >> > about. >> >> To be clear, I didn't intend that as a "solution"; only as a "first >> approximation workaround", as I said there. >> >> > When I said we could "not expect" what I was getting at is that it >> > would require the user to figure out what files the merge was going to >> > modify BEFORE they do the merge and lock those files. That does not >> > seem reasonable ... again I did not consider the simpler lock >> > everything idea. >> >> Why wouldn't that be reasonable? > > Different people have different tolerance levels. For me, this would > be too much to ask and I would question what the tool is really even > doing for me and look for alternatives. Ah, I see. Yes, requiring the user to _manually_ lock all files changed in the branch doesn't sound right. It might work better as something libsvn_client/merge.c does by itself. (Although, as discussed in the remainder of the thread, the library should only lock files if the merge is intended to be committed, and shouldn't lock more files than it needs to, if possible.) Cheers, Daniel
Re: merging into a read only file succeeds
Mark Phippard wrote on Fri, Feb 25, 2022 at 10:39:13 -0500: > On Fri, Feb 25, 2022 at 10:19 AM Daniel Shahaf > wrote: > > > > Nathan Hartman wrote on Thu, 24 Feb 2022 23:44 +00:00: > > > On Thu, Feb 24, 2022 at 2:33 PM Mark Phippard wrote: > > >> > > >> On Thu, Feb 24, 2022 at 10:51 AM Lorenz wrote: > > >> > > >> > just discovered, that merging into a RO file succeeds. > > >> > I my case the file is RO because of a needs-lock property. > > >> > So when I try to commit I can't. > > >> > > > >> > This effect ist at least annoying. > > >> > There is a reason why the file is read only. > > >> > > >> It is interesting both that it was able to do this and also that this > > >> has never come up before. I am not sure I can wrap my head around what > > >> I think should even happen here. There is no way you could be expected > > >> to obtain locks on these files before merging. > > > > Why can't we expect that? > > I had not considered your proposal to just lock everything before the > merge. I agree that would be a solution. I cannot predict user > reaction because it is hard to know what kind of scale we are talking > about. To be clear, I didn't intend that as a "solution"; only as a "first approximation workaround", as I said there. > When I said we could "not expect" what I was getting at is that it > would require the user to figure out what files the merge was going to > modify BEFORE they do the merge and lock those files. That does not > seem reasonable ... again I did not consider the simpler lock > everything idea. Why wouldn't that be reasonable? When merging a feature branch, it's fairly easy to find "all files the branch touched" — in the UI, «svn diff --summarize» in the UI; in the FS backend, walking over the changed-paths information without having to parse dir reps — and, barring copies/renames on the merged-to branch, we wouldn't ever have to lock a file the branch didn't touch, would we? I don't know off the top of my head how we handle copies/renames in the merge target. > > >> I guess ideally merge would obtain locks but I just do not know what > > >> merge should do if it cannot obtain the lock. I guess it should > > >> behave the same way it would if the file was not present due to it > > >> being a sparse working copy. > > > > > > Hopefully there would be a visible warning if that happened! > > > > > >> Maybe merge should just refuse to run at all if it detects any > > >> svn:needs-lock properties in the WC? > > > > > > Then it would never be able to run; I think you meant: refuse to run > > > if we don't hold the lock on a file that has the svn:needs-lock > > > property. However, it would be rather irritating if the refusal to run > > > were caused by files unaffected by the merge. > > I am projecting a bit here based on memory of now 10+ year old discussions. > > At the beginning of merge, the process has no idea what it is going to > merge so it has no way to check for locks. > > I am assuming it is difficult for the merge process at the point of > merging a specific file to trigger a lock workflow before it does it. > The merge process kind of has to do all of this before it begins and > before it even knows what it is going to do. Those assumptions could > be wrong, or someone could rewrite the whole thing etc. I was just > being realistic that this is a problem that we are probably past the > point of solving. > > > Or if the user deliberately did the merge without intending to commit it. > > > > How about making this situation a conflict? "local file not locked, > > incoming edit upon merge". Then the file is not silently changed, but > > the user can use --accept or «svn resolve» to make the changes anyway if > > they know that's what they want. > > That seems like it could be a viable option given that we have > improved our process to include these sort of things with all of the > tree conflict work we have done and the additional workflows we added > for it. This definitely seems like the best idea for someone to > pursue. I wonder if it's possible to prototype this with svnraisetreeconflict(1) (from tools/). There's no existing enum value for "not locked", but "incoming edit" and "upon merge" both already exist in the API. Cheers, Daniel > Your lock everything idea before merging would be a good workaround if > we were to just make merge start to fail if it runs into a lock rather > than what it is doing today. > > Mark
Re: merging into a read only file succeeds
Nathan Hartman wrote on Thu, 24 Feb 2022 23:44 +00:00:
> On Thu, Feb 24, 2022 at 2:33 PM Mark Phippard wrote:
>>
>> On Thu, Feb 24, 2022 at 10:51 AM Lorenz wrote:
>>
>> > just discovered, that merging into a RO file succeeds.
>> > I my case the file is RO because of a needs-lock property.
>> > So when I try to commit I can't.
>> >
>> > This effect ist at least annoying.
>> > There is a reason why the file is read only.
>>
>> It is interesting both that it was able to do this and also that this
>> has never come up before. I am not sure I can wrap my head around what
>> I think should even happen here. There is no way you could be expected
>> to obtain locks on these files before merging.
Why can't we expect that?
>> I guess ideally merge would obtain locks but I just do not know what
>> merge should do if it cannot obtain the lock. I guess it should
>> behave the same way it would if the file was not present due to it
>> being a sparse working copy.
>
> Hopefully there would be a visible warning if that happened!
>
>> Maybe merge should just refuse to run at all if it detects any
>> svn:needs-lock properties in the WC?
>
> Then it would never be able to run; I think you meant: refuse to run
> if we don't hold the lock on a file that has the svn:needs-lock
> property. However, it would be rather irritating if the refusal to run
> were caused by files unaffected by the merge.
Or if the user deliberately did the merge without intending to commit it.
How about making this situation a conflict? "local file not locked,
incoming edit upon merge". Then the file is not silently changed, but
the user can use --accept or «svn resolve» to make the changes anyway if
they know that's what they want.
> Whether that is more or less annoying than merging to the read-only
> file is up to interpretation.
>
>> It is a hard problem to solve.
>
> I suggest to file an issue for now.
>
> Also, perhaps a workaround exists, or at least a way for users to
> check whether they should obtain locks before running a merge,
> preferably one that uses existing SVN commands. Some combination of
> 'svn merge --dry-run' and 'svn status -u', or something along those
> lines?
For a first approximation I'd go for:
svn lock ${all files with svn:needs-lock}
svn merge
svn unlock ${appropriate subset of the files just locked}
Cheers,
Daniel
Re: List modified on a branch
Stefan Sperling wrote on Fri, 11 Feb 2022 10:57 +00:00: > On Fri, Feb 11, 2022 at 08:29:35AM +, Mikael Stålhammar wrote: >> Is there a simple command that lets you list files that have been modified >> and committed on a feature branch but excluding any files that are committed >> on the branch that are only merged from trunk (to keep the branch up-to-date >> with trunk). I.e. I only want to see those files that actually contain >> branch specific changes. > > Find the most recent revision where a merge from trunk to the branch occurred. > If no such merge has occurred yet, find the revision in which the branch > was created. > tl;dr: A tools/ script to do this wouldn't go amiss. Here's pseudocode for this; it involves one easy but less-than-obvious command and one parsing task that's encapsulated in our APIs. [[[ svn log -r 0:HEAD --limit=1 --stop-on-copy -q -v ./ | parse_the_copyfrom_path_and_revision svn propget --strict svn:mergeinfo ./ | parse_and_extract_revisions_merged_from $COPYFROM_PATH if any revisions have been extracted: return max() of those else: return $COPYFROM_REVISION ]]] > Let's assume this was revision r100. > > Now you can run: svn diff --summarize ^/trunk@r100 ^/branches/mybranch
Re: Subversion environmental variables
Nathan Hartman wrote on Thu, 03 Feb 2022 19:12 +00:00: > On Thu, Feb 3, 2022 at 1:31 PM Jeffrey Walton wrote: >> I checked the man pages and could not find an answer... What >> environmental variables does a subversion client use? >> >> The problem I am trying to solve is, I am being prompted for a >> password in a script about 15 times. I want to know which variable to >> use for a password to avoid the prompts. > > Hi Jeffrey, > > Come to think of it, I don't think we have an exhaustive list of > environment variables that a Subversion client (or server, for that > matter) can use. That might be a useful thing to document. The file notes/knobs collects some of these. It covers some internal / easter eggs / unsupported stuff as well, and manages not to mention SVN_EDITOR at all.
zsh completion updated
If you use zsh, feel free to grab https://sourceforge.net/p/zsh/code/ci/04513dd5df31bc235691400b76a524d117cefede/tree/Completion/Unix/Command/_subversion?format=raw and drop it into a directory that's in your $fpath variable (`print -raC1 -- $fpath`). Earlier directories have precedence over later ones. The file will probably work with zsh as old as 5.3, but I haven't tested all the way back. Bug reports to zsh-us...@zsh.org. Thanks, Daniel
Re: SVN basic questions.
Daniel Sahlberg wrote on Fri, 20 Aug 2021 07:55 +00:00: > Den fre 20 aug. 2021 kl 05:02 skrev A Z : > > -Can multiple users add to a committed branch node, and que up > > adds, or is this in fact going to nothing, while granting a message? > > Is it the case that only the first add to the branch node will be added, > > queued, while others will be rejected until that first node is committed? > > You can commit as you would normally do in /trunk. Any conflicts will > be managed when updating the working copy (before commit, if needed). It might help to clarify the terminology here. Subversion's own documentation does not use the term "queueing" at all, and does not use the term "add" except in the narrow sense of an operation that results in a new versioned directory entry. «add» is a local operation (doesn't contact the server). It's not clear to me what sense you use those two terms in. In particular, when you write "until that first node is committed", you seem to be assuming some sort of interim state, a "node" that has been "added" but not been "committed", and that affects other users. There's no such thing. The repository is a versioned filesystem tree. Changes to that tree (commits) are made transactionally. A commit that has not completed does not affect other commits in any way (cf. ACID). > (You can configure a file to require locking, but that is a separate > question from branching. To edit a locked file you must aquire the lock > from the server and only one person at a time can hold the lock to a > certain file. There is no locking on folder level).
Re: Simple Trio of Questions about Apache SVN
Andreas Stieger wrote on Tue, 17 Aug 2021 08:24 +00:00: > Hi, > > > -In SVN, can you have multiple additions, queued up, marked with an A? > > No queuing feature, but changelists on file granularity. > See https://svnbook.red-bean.com/nightly/en/svn.advanced.changelists.html > > > If so, can the queued edition(s) be reviewed for marked files, > > Using svn diff which takes file names and changelists > > > and can those marked files be examined internally for change markers too? > > Yes by calling external tools or tools integrating svn. > > > If so, what are the command(s) involved? > > svn cl, svn diff > > > Can addition(s) be deleted before any comitting? > > Yes using diff tools, usually those implementing per-line reverts. Meld > is a generic tools, IDEs offer their own support. The question specified "marked with an A", which makes it sound like it's asking what's the way to cancel an uncommitted «svn add». That'd be «svn revert» (and wouldn't need to involve changelists).
Re: svn authz error: Access denied issue when running svn rm resource where the user should be able to have full access
Andrei Cusnir wrote on Sat, Jul 31, 2021 at 02:01:30 +0200: > Hi Daniel, > can you please let me know if this is a case for which I should open a bug > ticket, Yes, please, and please link to this thread from the ticket. > or this is something that has to do with the authz file contents. As I explained, I believe the existence of ** rules is what causes the recursive authz lookup to return a different result than the non-recursive one. Note that using svnserve is a workaround (given that your *.uasset is a file, according to the output you've posted). Cheers, Daniel > Kind Regards, > Andrei > > On Thu, Jul 22, 2021 at 4:57 PM Andrei Cusnir wrote: > > > Hi Daniel, > > thank for your reply, > > i was able to confirm this behaviour on both svn server versions 1.12.2 > > and 1.14.1 > > current apache server > > Apache/2.4.25 (Debian) SVN/1.14.1 OpenSSL/1.0.2u mod_wsgi/4.5.11 > > Python/2.7 configured -- resuming normal operations > > > > the user that reported it first is having most recent client > > SVN/1.14.1 (x64-microsoft-windows) serf/1.3.9 TortoiseSVN-1.14.1.29085 > > i have following client > > SVN/1.13.0 (x86_64-apple-darwin19.0.0) serf/1.3.9 > > > > > > On Thu, Jul 22, 2021 at 4:51 PM Daniel Sahlberg < > > daniel.l.sahlb...@gmail.com> wrote: > > > >> Den ons 21 juli 2021 kl 22:30 skrev Andrei Cusnir : > >> > >>> Hi, > >>> here is my authz config file > >>> > >>> [groups] > >>> r_users = Alexander_Lost > >>> > >>> [:glob:/] > >>> * = rw > >>> > >>> [:glob:/**/*.uasset] > >>> @r_users = rw > >>> > >>> [:glob:/**/Source/**] > >>> @r_users = > >>> > >>> > >>> user Alexander_Lost should be able to work with uasset type of files and > >>> should not see anything that has Source in the global path > >>> > >>> when i am trying to remove a path > >>> svn rm https://"Alexander_Lost"@ > >>> subversion.assembla.com/svn/repo_name/branches/project_name/Projects/ShooterGame/Content/Mods/project_name/Assets/Foliage/Trees/StepeeGrass/Grass01/skwnorp_2K_Normal.uasset > >>> -m "test uasset delete by andrei" > >>> svn: E175013: Access to > >>> '/svn/repo_name/!svn/txr/153436-3cur/branches/project_name/Projects/ShooterGame/Content/Mods/project_name/Assets/Foliage/Trees/StepeeGrass/Grass01/skwnorp_2K_Normal.uasset' > >>> forbidden > >>> > >>> in apache logs i can see the following > >>> > >>> [Wed Jul 21 19:47:42.187161 2021] [authz_svn:error] [pid 1510:tid > >>> 140015597156096] [client 188.26.223.19:42966] Access denied: > >>> 'Alexander_Lost' DELETE > >>> repo_name:/branches/project_name/Projects/ShooterGame/Content/Mods/project_name/Assets/Foliage/Trees/StepeeGrass/Grass01/skwnorp_2K_Normal.uasset > >>> > >>> > >>> Please let me know if this is a config error or it is an authz bug. > >>> > >> > >> Do you know what versions are you using for the server and for the client? > >> > >> There is an issue with the authorization rules in versions 1.10.0 until > >> 1.14.0, fixed in 1.10.7 and 1.14.1. I'm not sure if it would cause problems > >> for you but check the description in the release notes: > >> http://subversion.apache.org/docs/release-notes/1.14.html#compat-misc-authz > >> and see if it might apply. > >> > >> Kind regards, > >> Daniel Sahlberg > >> > >>
Re: svn authz error: Access denied issue when running svn rm resource where the user should be able to have full access
Daniel Shahaf wrote on Sat, Jul 31, 2021 at 00:41:48 +: > Now, the delete logic should request a *non* recursive access check if > the thing to be deleted is a plain file in HEAD. > subversion/libsvn_repos/commit.c:delete_entry() > does behave this way — when the .uasset path identifies a non-directory > in HEAD, which is why I asked about that above — but if I'm reading > mod_authz_svn's code correctly, it requires svn_authz_recursive access > on all DELETE calls, regardless of whether a file or a directory is > being requested to be deleted. > > Which is to say, this would seem to be a bug in mod_authz_svn. The following test results in SKIP/PASS/XFAIL on ra_local/ra_svn/ra_serf, respectively. It's a bit rough — no comments and line length style violation — but putting it here for anyone who may want to take this further. When testing this I commented out the AuthzSVNReposRelativeAccessFile line in davautocheck.sh for reasons discussed in the "1.14.x test failure under USE_HTTPV1=1: ra-test 13 commit_empty_last_change" thread (the secondary problem in that thread, not the one the subject line is about). Cheers, Daniel [[[ Index: subversion/tests/cmdline/authz_tests.py === --- subversion/tests/cmdline/authz_tests.py (revision 1891909) +++ subversion/tests/cmdline/authz_tests.py (working copy) @@ -1732,6 +1732,25 @@ def empty_group(sbox): sbox.repo_url) +@XFail(svntest.main.is_ra_type_dav) +@Skip(svntest.main.is_ra_type_file) +def delete_file_with_starstar_rules(sbox): + "delete file with ** rules" + + sbox.build(create_wc = False) + + write_restrictive_svnserve_conf(sbox.repo_dir) + + prefixed_rules = dict() + prefixed_rules[':glob:/**/lorem'] = '* = \n' + prefixed_rules['/'] = '%s = rw\n' % (svntest.main.wc_author,) + prefixed_rules['/A'] = '%s = \n' % (svntest.main.wc_author,) + prefixed_rules['/iota'] = '%s = rw\n' % (svntest.main.wc_author,) + write_authz_file(sbox, None, prefixed_rules = prefixed_rules) + + svntest.main.run_svn(None, 'rm', sbox.repo_url + '/iota', '-m', 'Delete by URL') + + # Run the tests @@ -1771,6 +1790,7 @@ test_list = [ None, inverted_group_membership, group_member_empty_string, empty_group, + delete_file_with_starstar_rules, ] serial_only = True ]]]
Re: svn authz error: Access denied issue when running svn rm resource where the user should be able to have full access
Daniel Sahlberg wrote on Thu, Jul 22, 2021 at 16:51:02 +0200: > Den ons 21 juli 2021 kl 22:30 skrev Andrei Cusnir : > > [groups] > > r_users = Alexander_Lost > > > > [:glob:/] > > * = rw > > > > [:glob:/**/*.uasset] > > @r_users = rw > > > > [:glob:/**/Source/**] > > @r_users = > > Do you know what versions are you using for the server and for the client? > > There is an issue with the authorization rules in versions 1.10.0 until > 1.14.0, fixed in 1.10.7 and 1.14.1. I'm not sure if it would cause problems > for you but check the description in the release notes: > http://subversion.apache.org/docs/release-notes/1.14.html#compat-misc-authz > and see if it might apply. That issue is about the interaction of [reposname:/path] and [/path] stanzas. It doesn't seem related. Cheers, Daniel
Re: svn authz error: Access denied issue when running svn rm resource where the user should be able to have full access
[dev@: tl;dr: mod_authz_svn shouldn't require svn_authz_recursive access on DELETE when the target is a file] Andrei Cusnir wrote on Wed, Jul 21, 2021 at 21:58:04 +0200: > Hi, > here is my authz config file > > [groups] > r_users = Alexander_Lost > > [:glob:/] > * = rw > > [:glob:/**/*.uasset] > @r_users = rw > > [:glob:/**/Source/**] > @r_users = > > > user Alexander_Lost should be able to work with uasset type of files and > should not see anything that has Source in the global path > > when i am trying to remove a path > svn rm https://"Alexander_Lost"@ > subversion.assembla.com/svn/repo_name/branches/project_name/Projects/ShooterGame/Content/Mods/project_name/Assets/Foliage/Trees/StepeeGrass/Grass01/skwnorp_2K_Normal.uasset > -m "test uasset delete by andrei" > svn: E175013: Access to > '/svn/repo_name/!svn/txr/153436-3cur/branches/project_name/Projects/ShooterGame/Content/Mods/project_name/Assets/Foliage/Trees/StepeeGrass/Grass01/skwnorp_2K_Normal.uasset' > forbidden > > in apache logs i can see the following > > [Wed Jul 21 19:47:42.187161 2021] [authz_svn:error] [pid 1510:tid > 140015597156096] [client 188.26.223.19:42966] Access denied: > 'Alexander_Lost' DELETE > repo_name:/branches/project_name/Projects/ShooterGame/Content/Mods/project_name/Assets/Foliage/Trees/StepeeGrass/Grass01/skwnorp_2K_Normal.uasset > > > Please let me know if this is a config error or it is an authz bug. What's the output of «svn info ^/branches/project_name/Projects/ShooterGame/Content/Mods/project_name/Assets/Foliage/Trees/StepeeGrass/Grass01/skwnorp_2K_Normal.uasset@HEAD»? I'm looking for the Node Kind line. With your authz file, I get: % svnauthz accessof /path/to/your/authz/file --username Alexander_Lost --path /branches/project_name/Projects/ShooterGame/Content/Mods/project_name/Assets/Foliage/Trees/StepeeGrass/Grass01/skwnorp_2K_Normal.uasset --repository repo_name rw % svnauthz accessof /path/to/your/authz/file --username Alexander_Lost --path /branches/project_name/Projects/ShooterGame/Content/Mods/project_name/Assets/Foliage/Trees/StepeeGrass/Grass01/skwnorp_2K_Normal.uasset --repository repo_name -R no The authz functionality doesn't deal in revision numbers. That implies it doesn't deal in node kinds either (since node kinds can't be determined without resolving a node's path at a particular revision (or transaction)). That, in turn, means authz must treat /foo/bar/baz.asset as something that, for all it knows, might be a directory (at least in some historical revisions). Therefore, as soon as there's a a glob rule that might apply to at least one descendant of /foo/bar/baz.asset, and that would grant Alexander_Lost "" (aka "no") access, the answer to the recursive access check ought to become "no". I think that's what explains the svnauthz(1) results above. Now, the delete logic should request a *non* recursive access check if the thing to be deleted is a plain file in HEAD. subversion/libsvn_repos/commit.c:delete_entry() does behave this way — when the .uasset path identifies a non-directory in HEAD, which is why I asked about that above — but if I'm reading mod_authz_svn's code correctly, it requires svn_authz_recursive access on all DELETE calls, regardless of whether a file or a directory is being requested to be deleted. Which is to say, this would seem to be a bug in mod_authz_svn. Cheers, Daniel
Re: svn: E160013: File not found: transaction '41-1j'
Nathan Hartman wrote on Thu, Jul 15, 2021 at 16:52:18 -0400: > Also, you might try using %20 in place of spaces and see if that makes > a difference. Perhaps by mistake one of those spaces is a different > codepoint, so looks like a regular space, but isn't. Good idea, but there's no reason to try blindly; running «svn ls» on the containing directory and «svn info» on the file, both of which you have suggested above, is sufficient to rule this out. Speaking of which, going back to the original error message, there's something I didn't notice at first: > > > > svn: E160013: File not found: transaction '41-1j', path > > > > '/Feasibility%20Studies/Use%20Case%20Analysis/TML/RIC_Initial_Use_Case_Analysis' This is a libsvn_fs_fs error message, and the path is percent-escaped. However, libsvn_fs_fs shouldn't be seeing percent-escaped paths at all. What's the version number on the client and server? > If it's a httpd-based server, do httpd's logs show anything that seems > significant? As mentioned upthread, svnserve has logs too, and successful commits would also show up in «svn log ^/». > Dumb question: Could the server be low on disk space? I don't see how that could cause an E160013? Cheers, Daniel
Re: svn: E160013: File not found: transaction '41-1j'
David Aldrich wrote on Thu, Jul 15, 2021 at 17:18:47 +0100: > Another team member had attempted the rename earlier today but it failed > with a permissions error. That's probably unrelated, but still, please quote the error message. > Could there be some incomplete transaction lurking in the database that is > causing a tree conflict? No. What you're seeing isn't a tree conflict. Conflicts are a client-side concept and are triggered by update/switch/merge operations. Besides, if an incomplete transaction _could_ somehow cause a conflict, then it could also be updated to, and we'd have noticed that. (Background reading pointers for interested devs: svn_fs_root_t; NODES.op_depth.) Cheers, Daniel
Re: svn: E160013: File not found: transaction '41-1j'
David Aldrich wrote on Thu, Jul 15, 2021 at 15:54:00 +0100: > I'm trying to rename a directory and I get this error on commit: > > Adding TML/RIC_Initial_Use_Case_Analysis > svn: E155011: Commit failed (details follow): > svn: E155011: Directory '//Feasibility Studies/Use Case > Analysis/TML/RIC_Initial_Use_Case_Analysis' is out of date > svn: E160013: File not found: transaction '41-1j', path > '/Feasibility%20Studies/Use%20Case%20Analysis/TML/RIC_Initial_Use_Case_Analysis' Hang on. "Feasibility Studies" is a child of the root in the E160013 error message, but child of something other than the root in the second E155011 line. Is that indeed so, or did you redact the two messages differently? > I guess the problem is associated with the fact that the path contains > spaces. > > Any advice on how to resolve this please? Review your start-commit and pre-commit hooks. Review your server's logs for any concurrent commit attempts. (This applies to everything except file://, not only to http/https. If you use file://, you can implement logging in pre-commit, or look at revisions that have in fact been committed right around the time of the failed commit attempt.) Cheers, Daniel
Re: svn log xml hangs and produces too many logentry closing tags
Attila wrote on Tue, Jul 13, 2021 at 23:48:48 +0200: > Hi > > I have a problem getting the svn log in a branch after sync-merging a commit > from trunk. > This commit in trunk is a merge of an old and complex branch with many > commits. > > The client accessing the repository over svn:// url. > (paths and text is redacted) > The head revision is: 10801 > > When I run the following command on the client (in the working copy), it > prints a long partial xml-log output, then hangs. > /usr/bin/svn log --xml -g -v -r 10701:HEAD /path/to/branch-wc > > When observing in "top", the command uses no visible CPU resources on hang. > (I waited ca. 2 minutes) > The hanging command does mot exits on CTRL-c, it does not exits on "kill > -TERM pid", I have to send "kill -KILL pid" to terminate it. > > When I run the command with strace it hangs at read(4, > ...SNIP... > read(4, " ( ) ( 4:file true false ) ) ( 1"..., 16384) = 16384 > mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = > 0x7f63f1d83000 > mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = > 0x7f63f1d81000 > mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = > 0x7f63f1d7f000 > mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = > 0x7f63f1d7d000 > mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = > 0x7f63f1d7b000 > mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = > 0x7f63f1d79000 > read(4, "***/-***_redacted_*_"..., 16384) = 14773 > mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = > 0x7f63f1d77000 > mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = > 0x7f63f1d75000 > mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = > 0x7f63f1d73000 > mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = > 0x7f63f1d71000 > mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = > 0x7f63f1d6f000 > mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = > 0x7f63f1d6d000 > read(4, > > When I observe the server, there is a CPU activity at the begin, but when the > client hangs, the server seems to be in idle. > Just a corresponding svnserve process is there with no visible cpu usage. > In svnserve.log is nothing relevant to see. > > The svnserve command is: > svnserve -d -r /path/to/repositories \ > --log-file=/var/log/svnserve.log \ > --memory-cache-size 1024 \ > --cache-txdeltas yes \ > --cache-fulltexts yes > > When I try to get the xml-log on the server with the corresponding file:// > repository URL: > /usr/bin/svn log --xml -g -v -r 10701:HEAD > file://path/to/local/repositories/project/branch > The command finishes in ca 5-10 seconds and I get the xml output, but the > output has a way too many lines. > > There are 1217 occurrences of the string “ the string "" in the output xml. > There are several thousand lines of in a row in many places in > repeated blocks. > > Details: > Client and Server OS: > Linux 4.19.0-8-amd64 #1 SMP Debian 4.19.98-1+deb10u1 (2020-04-27) x86_64 > GNU/Linux > > The repository is ca. 4 GB. > Running "svnadmin verify" on the server founds no errors. > I have no other problems with the server, checkout and commit works normal. > > svn --version > svn, version 1.10.4 (r1850624) >compiled Feb 10 2021, 20:15:45 on x86_64-pc-linux-gnu > > Copyright (C) 2019 The Apache Software Foundation. > This software consists of contributions made by many people; > see the NOTICE file for more information. > Subversion is open source software, see http://subversion.apache.org/ > > The following repository access (RA) modules are available: > > * ra_svn : Module for accessing a repository using the svn network protocol. > - with Cyrus SASL authentication > - handles 'svn' scheme > * ra_local : Module for accessing a repository on local disk. > - handles 'file' scheme > * ra_serf : Module for accessing a repository via WebDAV protocol using serf. > - using serf 1.3.9 (compiled with 1.3.9) > - handles 'http' scheme > - handles 'https' scheme > > The following authentication credential caches are available: > > * Plaintext cache in /home/username/.subversion > * Gnome Keyring > * GPG-Agent > * KWallet (KDE) > > > Googling around gives me these two somewhat related hits: > https://issues.apache.org/jira/browse/SVN-4856 > https://issues.apache.org/jira/browse/SVN-4711 > > But I do'nt use the --search parameter. > > Is this a bug or are there any suggestions how to solve this problem? It could be SVN-4856, which doesn't use --search either, or it could be something else that happens whenever the output is large enough. Could you try «svn log -v -r HEAD:0 '^/' >/dev/null» without -g? That might not actually be large enough (considering HEAD is ≈11k and your output had ≈33k cases of ), so you might have to create a
Re: Getting error E205009 upon doing SVN copy using URL
Ranajit Ghosh wrote on Wed, Jun 09, 2021 at 17:27:20 +0530: > Hi, Not much luck! If I put the text in the front, it is not being able to > process/recognize a multi word string and throwing error like below: > > If I put the below message in the command line > svn -m "jira:RTPS-9898 EPC 2021 install" copy What's the output of «which svn»? What's the output of: . printf \[%s\]\\n svn -m "jira:RTPS-9898 EPC 2021 install" copy SRC DST . ? What's the output of the last command in an interactive /bin/sh session?
Re: Getting error E205009 upon doing SVN copy using URL
Thorsten wrote on Wed, Jun 09, 2021 at 13:30:51 +0200: > My next guess would be that the -m paramter simply must be in the front. > For reference, options and positional arguments can be freely mixed (except when «--» is used to terminate options).
Re: Using IIS as a reverse proxy in front of Apache/SVN
Daniel Sahlberg wrote on Wed, Jun 09, 2021 at 08:18:04 +0200: > Hi, > > We are using VisualSVN server (basically Apache 2.4.48 and Subversion > 1.14.1 on Windows) on https://svn.companyname.tld, listening on port 443. > Currently this is on a separate server. I need to consolidate the servers > and would like to move Subversion to another server already running IIS > (serving multiple sites on both port 80 and 443). > > My thinking is that IIS should listen for the new hostname, do SSL > offloading and forward the traffic to 127.0.0.1:[some new port for Apache]. > I would like to avoid publishing the new port for Apache, since that would > mean to relocate all existing working copies. > > Does anyone have experience in using IIS as reverse proxy in front of > Apache? Not what you asked, but running the test suite under a reverse proxy configuration might be informative. Cheers, Daniel
Re: Is it possible to get the location of WC in a post-commit hook?
Some ideas: - Parse the operational log to determine the IP address of the commit - Pass the hostname and wc path in a revprop - Embed the hostname and wc path in the user-agent string and use persist-ephemeral-txnprops.py [admittedly a bit hacky; this isn't what the user-agent string is for] However, it sounds like you'll be better served by implementing some sort of dashboard that shows all known WC's and the current revision of each. That's the domain of monitoring and network management tools. Thorsten wrote on Wed, 09 Jun 2021 08:48 +00:00: > Hello, > > Short answer: no its not possible. > > I dont fully understand why you want to do this. Run "svn up" on any > device to get the latest commit. > > Run "svn st -u" or something else to check for incoming changes. > > Other than that you could also do something fancy like create an own > user for every working copy, that way you can differentiate them. > > Hope that helps, > > Thorsten Goetzke > > > Am 09/06/2021 um 09:38 schrieb Bo Berglund: > > I wrote a SVN mailer when we converted from CVSNT to SubVersion back in > > 2017. > > Its purpose is to send a nocely formatted commit message to subscribers and > > it > > has worked fine for years. > > > > But now I am missing one item in the mail, the location of the working copy > > from > > which the commit was done. > > > > I am working on a project where I have several WC in play, on my development > > device and the prototype and also a 3rd device. I have found it to be a bit > > difficult to keep track of which wc contains the latest commit so if it is > > possible to get this info on commit it would help a lot. These devices are > > all > > Linux. > > > > The mailer gets the data by calling svnlook. > > > > The svn server is on Windows Server 2016. > > >
Re: Unexpected behavior of svn log -r BASE:HEAD
Yasuhito FUTATSUKI wrote on Thu, 03 Jun 2021 04:25 +00:00:
> In article
>
> guillaume.brune...@gmail.com writes:
> > Den ons 2 juni 2021 12:47Yasuhito FUTATSUKI skrev:
> > Ah, thank you, I guess the help message should be fixed then?
>
> I have no idea about how accurate it should be in example section,
I think examples should be more accurate than this example is.
> and I can't also propose how it should be because of my poor English
> writing ability. However if you have some idea how it should be,
> please make a draft (or patch against subversion/svn/svn.c).
Seconded. :)
We could, say, have the help text point out that rBASE itself may or may
not be included in the output; or perhaps we could change the example to
one that can be explained without branches [in the CPU execution sense
of the word].
> > Is there another way to get what is described in the help message? To show
> > the log messages for any incoming changes to foo.c during the next 'svn
> > update'. This is exactly what I was trying to do.
>
> One of obvious way is something like this (although this can't take
> multiple target paths):
> [[[
> #!/bin/sh
>
> # if This is not in your command search paths, please change it.
> svn=svn
>
This would break «svn=/path/to/svn /path/to/this-script». Instead:
: ${svn:=svn}
> revision_base=`${svn} info -r BASE --show-item revision "$1"` || exit $?
> revision_head=`${svn} info -r HEAD --show-item revision "$1"` || exit $?
>
Recommend to add a «--» guard at the end of options.
Recommend to use peg rather than operative revisions. Doesn't make a
difference for the first line, but I think it does make a difference for
the second line (it would have to check that $1@HEAD and $1@BASE are the
same node-id).
> # This check is needed to avoid "No such revision error"
> if test "${revision_base}" = "${revision_head}" ; then exit 0; fi
>
> ${svn} log -r $((${revision_base}+1)):HEAD --incremental "$1"
Race condition: by the time this command is run, a commit might have
been made, in which case HEAD could be equal to revision_base+1, or even
greater than it. Recommend s/HEAD/${revision_head}/.
> ]]]
> or check the BASE revision number in advance and filter out the log
> on its revision number from the result of 'svn log -r BASE:HEAD'.
FWIW, one way to implement this is to use «svn log --xml», filter that,
and then synthesize to XML to plain «svn log» output in the same way
https://subversion.apache.org/docs/release-notes/#upcoming-patch-release
does (there's a link at the end of that to the Python implementation).
Aside: Looking at that section, it looks like it's time to release a 1.14.2.
> In addition, to get "exactly" what you want, you need forbidding
> everyone to commit change on the target path just after you examine
> the HEAD revision until you run 'svn update', and I think it is not
> good idea.
It's easier is to make note of the numeric value of HEAD and then run
«svn up -r N» to update precisely to that. Compare
tools/dist/release.py:bump_versions_on_branch()'s use of HEAD in
a similar situation involving commits rather than updates.
Cheers,
Daniel
Re: Need help in Subversion migration
Mark Phippard wrote on Mon, 17 May 2021 12:24 +00:00: > > > On May 17, 2021, at 8:21 AM, jackson wrote: > > > > > > Hello Everyone > > > > We are using subversion 1.7 running in LInux 6.9. > > we are working on subversion migration along with OS upgrade > > > > Currently we have built RHEL 8.3 OS with subversion 1.10 and did all the > > configuration , the 1.10 setup is up and working as expected . > > > > We need help in migrating the repo from current setup (v1.7 to the new setup > > > > The current setup (v1.7) has the repo size of 500GB. we are using svn load > > and svn dump method to migrate the repos from current setup (v1.7) to new > > setup(v1.10) , but it takes a lot of time due to the size of the repo > > (500GB) , Also the current setup (v1.7 is being currently used by the > > customer ,soo even after taking the dump of one particular repo and loading > > it in new setup , a newer revision is being made in the current setup > > (v1.7 by the customer , thus making us to do the svn dump and lsvn load > > again. > > > > now we need your help in identifying an efficient way in migrating the repo > > from the current setup (v1.7 to the ew setup(v1.10) , Need your expert > > advise on this please > > > > > > Also let us know we can do rsync for this? > > > > Thanks & Regards, > > Jackson J > > You do not have to do anything. Just point the new server at the > existing repositories or move them to new location. That's not necessarily correct for BDB repositories. > You can use rsync, tar or whatever method you prefer. That's not correct for live repositories. Bo and David covered the svnsync and «svnadmin hotcopy» approaches; all I have to add is that for «svnadmin dump», the --deltas, --incremental, and -r options should be used as necessary. Cheers, Daniel
Re: svnpubsub/commit-hook.py:65 - variable url assigned but not used
sebb wrote on Fri, 14 May 2021 11:44 +00:00: > On Fri, 14 May 2021 at 12:22, Daniel Shahaf wrote: > > sebb wrote on Thu, 13 May 2021 14:29 +00:00: > > > > Is it misspelt? > > > > ITYM "misspelled" ;-) > > No, I am using British English. I know. That's what the emoticon was there to acknowledge.
Re: svnpubsub/commit-hook.py:65 - variable url assigned but not used
sebb wrote on Thu, 13 May 2021 14:29 +00:00: > On Thu, 13 May 2021 at 15:16, sebb wrote: > > On Thu, 13 May 2021 at 15:11, Daniel Shahaf wrote: > > > > > > sebb wrote on Wed, May 12, 2021 at 11:49:41 +0100: > > > > As the subject says > > > > > > Assuming opener.open() actually returns a URL, I don't see the problem > > > here. The variable documents the return type for anyone who may want > > > to extend the function. > > > > The problem is that it is not clear whether the variable is supposed > > to be used or not. Then add a comment, or return the variable, etc.. > > Is it misspelt? ITYM "misspelled" ;-) > > AIUI the convention for intentionally unused variables is to prefix > > the name with an underscore, as this no longer triggers the warning in > > PyLint Last I checked, docs.python.org said (single) leading underscores designated class members that weren't part of the class's public API. > Furthermore, the method does not actually return a URL. Then the variable could be renamed. > The method behaves like the following: > https://python.readthedocs.io/en/v2.7.2/library/urllib2.html#urllib2.urlopen Thanks. Also, all the above discussion applies to revprop-change-hook.py, too. Daniel
Re: svnwcsub.py does not work across different SVN trees
sebb wrote on Thu, 13 May 2021 14:46 +00:00: > On Thu, 13 May 2021 at 15:03, Daniel Shahaf wrote: > > > > sebb wrote on Wed, May 12, 2021 at 10:47:15 +0100: > > > It looks like svnwcsub.py is not always able to update the workspace > > > > Don't use made-up terminology. The terms "repository" and "working > > copy" have specific meanings. Use those terms to refer to what they > > mean. Don't use those terms to mean other things and don't use other > > terms to mean those things. > > > > > if the source URL is updated in the configuration file. > > > > > > I am seeing an error of the form: > > > > > > svn: E195012: Path '/www/ace.apache.org' does not share common version > > > control ancestry with the requested switch location. Use > > > --ignore-ancestry to disable this check. > > > svn: E195012: 'https://svn-master.apache.org/repos/infra/sites/ace' > > > shares no common ancestry with '/www/ace.apache.org > > > > > > Is there any reason why the svn switch command does not include the > > > suggested option? > > > > You mean, why svnwcsub.py doesn't pass that option? For starters, > > because svnwcsub.py was written before that option existed. > > Any other reasons? > If you want to propose that that flag be added, the onus is on you to make the case for adding it, not on everyone else for making the case against adding it. You've been around long enough to know this. > > > Seems to me it ought to be possible to replace the checkout with a > > > different SVN repo without needing to manually remove the existing > > > checkout. > > > > -1. That would be data loss in case of a new svnwcsub deployment where > > the target dir already exists and is non-empty. This project is > > generally rather averse to the possibility of data loss. > > This is not about a new deployment, but about changing the URL for an > existing WC. How would the code tell the two cases apart? What are the proposed new semantics?
Re: svnwcsub.py - various pylint errors
sebb wrote on Wed, May 12, 2021 at 12:25:26 +0100: > pylint reports quite a few warnings and errors: > > svnwcsub.py:55:0: W0311: Bad indentation. Found 2 spaces, expected 4 > (bad-indentation) Don't hard-wrap program output in your emails. > svnwcsub.py:57:0: W0311: Bad indentation. Found 2 spaces, expected 4 > (bad-indentation) > svnwcsub.py:61:0: W0311: Bad indentation. Found 2 spaces, expected 4 > (bad-indentation) > svnwcsub.py:63:0: W0311: Bad indentation. Found 2 spaces, expected 4 > (bad-indentation) > svnwcsub.py:67:0: W0311: Bad indentation. Found 2 spaces, expected 4 > (bad-indentation) > svnwcsub.py:69:0: W0311: Bad indentation. Found 2 spaces, expected 4 > (bad-indentation) I know it's PEP8, but the churn isn't worthwhile, IMO, especially as the entire core tests directory uses non-PEP8 indentation. Thus, I'm voting WONTFIX. editorconfig(5) is already set correctly for these. > svnwcsub.py:87:4: W0612: Unused variable 'output' (unused-variable) I suppose you could change it to «_, errput = …», but it's rather a borderline case insofar as readability v. churn is concerned, IMO. > svnwcsub.py:115:12: W0612: Unused variable 'x' (unused-variable) How'd you rewrite it? > svnwcsub.py:439:38: W0613: Unused argument 'event_arg' (unused-argument) Looks like a false positive. Did you triage these warnings before you posted them to the list? > svnwcsub.py:46:0: W0611: Unused import errno (unused-import) > svnwcsub.py:58:0: W0611: Unused import time (unused-import) > svnwcsub.py:65:0: W0611: Unused import functools (unused-import) > svnwcsub.py:67:2: W0611: Unused import urlparse (unused-import) > svnwcsub.py:69:2: W0611: Unused urllib.parse imported as urlparse > (unused-import) Patch please. > svnwcsub.py:378:18: E0602: Undefined variable 'fname' (undefined-variable) > (should be self.fname) Did you reproduce the bug and confirm the proposed change fixes it? If so, post a complete patch, please.
Re: svnpubsub/commit-hook.py:65 - variable url assigned but not used
sebb wrote on Wed, May 12, 2021 at 11:49:41 +0100: > As the subject says Assuming opener.open() actually returns a URL, I don't see the problem here. The variable documents the return type for anyone who may want to extend the function.
Re: watcher.py no longer uses urlparse
sebb wrote on Wed, May 12, 2021 at 11:48:17 +0100: > As the subject says. +1 to commit, but the log message should follow the house style, for example: [[[ * tools/server-side/svnpubsub/watcher.py (urlparse): Remove unused import. Approved by: danielsh ]]]
Re: svnwcsub.py does not work across different SVN trees
sebb wrote on Wed, May 12, 2021 at 10:47:15 +0100: > It looks like svnwcsub.py is not always able to update the workspace Don't use made-up terminology. The terms "repository" and "working copy" have specific meanings. Use those terms to refer to what they mean. Don't use those terms to mean other things and don't use other terms to mean those things. > if the source URL is updated in the configuration file. > > I am seeing an error of the form: > > svn: E195012: Path '/www/ace.apache.org' does not share common version > control ancestry with the requested switch location. Use > --ignore-ancestry to disable this check. > svn: E195012: 'https://svn-master.apache.org/repos/infra/sites/ace' > shares no common ancestry with '/www/ace.apache.org > > Is there any reason why the svn switch command does not include the > suggested option? You mean, why svnwcsub.py doesn't pass that option? For starters, because svnwcsub.py was written before that option existed. > Seems to me it ought to be possible to replace the checkout with a > different SVN repo without needing to manually remove the existing > checkout. -1. That would be data loss in case of a new svnwcsub deployment where the target dir already exists and is non-empty. This project is generally rather averse to the possibility of data loss.
Re: svnpubsub/svnwcsub.py crashes if pidfile option is not supplied
sebb wrote on Tue, May 11, 2021 at 22:41:38 +0100:
> On Tue, 11 May 2021 at 19:33, Daniel Shahaf wrote:
> >
> > sebb wrote on Tue, 11 May 2021 16:25 +00:00:
> > > As the subject says, the code crashes as below unless --pidfile is
> > > provided
> > >
> > > File "./svnwcsub.py", line 559, in
> > > main(sys.argv[1:])
> > > File "./svnwcsub.py", line 548, in main
> > > d = Daemon('/dev/null', os.path.abspath(options.pidfile),
> > > File
> > > "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/posixpath.py",
> > > line 367, in abspath
> > > if not isabs(path):
> > > File
> > > "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/posixpath.py",
> > > line 54, in isabs
> > > return s.startswith('/')
> > > AttributeError: 'NoneType' object has no attribute 'startswith'
> >
> > Any chance of a patch? Feel free to just make --pidfile required, if
> > making it properly optional is a larger diff than you have tuits for.
Update the usage message, use «svn diff -x-p», root the diff at the
branch root, address it to dev@, include a proposed log message (that
follows our log message syntax and linguistic conventions and is free of
spelling errors); see https://subversion.apache.org/patches for details.
Re: svnpubsub/svnwcsub.py crashes if pidfile option is not supplied
sebb wrote on Tue, 11 May 2021 16:25 +00:00:
> As the subject says, the code crashes as below unless --pidfile is provided
>
> File "./svnwcsub.py", line 559, in
> main(sys.argv[1:])
> File "./svnwcsub.py", line 548, in main
> d = Daemon('/dev/null', os.path.abspath(options.pidfile),
> File
> "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/posixpath.py",
> line 367, in abspath
> if not isabs(path):
> File
> "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/posixpath.py",
> line 54, in isabs
> return s.startswith('/')
> AttributeError: 'NoneType' object has no attribute 'startswith'
Any chance of a patch? Feel free to just make --pidfile required, if
making it properly optional is a larger diff than you have tuits for.
Re: Syntax error at svnpubsub/daemonize.py:59
sebb wrote on Tue, 11 May 2021 16:22 +00:00: > The code reads: > > 58:except (ChildTerminatedAbnormally, ChildForkFailed, > 59:DaemonTerminatedAbnormally, DaemonForkFailed), e: > > The ',' is invalid syntax; it should be 'as'. +1 to commit. Feel free to also nominate it for backport to 1.14.x with my +1, if you need it backported. > There is also an invalid comment at line 20: > > 19: # This software lives at: > 20: #http://gstein.googlecode.com/svn/trunk/python/daemonize.py I don't know a forwarding address for this, sorry. Cheers, Daniel
Re: Enforce encryption client-side
Nils-Johan Andreasson wrote on Fri, May 07, 2021 at 15:41:47 +0200:
> On Thu, May 6, 2021 at 3:24 PM Daniel Shahaf wrote:
>
> > Nils-Johan Andreasson wrote on Thu, May 06, 2021 at 14:48:01 +0200:
> > > I have indeed considered svn+ssh but there are other details involved
> > which
> > > makes me prefer to stay with svnserve and svn:// if possible.
> >
> > Does your library consider svn+foo:// secure? If so, you can define an
> > svn+foo:// scheme that simply wraps plain old TCP sockets. You have two
> > options for that, even: either «svn+nc://hostname/path» that uses nc(1)
> > or socat(1) to wrap a plain TCP socket, or a scheme that hardcodes
> > a specific hostname, that you then use in URLs whose hostname component is
> > ignored (can actually be left empty).
>
> Clever!
> The library does indeed only consider svn:// insecure and it does not
> validate the scheme other than that. So, this would work (although it of
> course requires some configuration in the places I want to use it, but I
> can live with that).
>
> It took me some reading and fiddling though to get this to work completely.
> nc wants parameters as 'nc '.
> The command defined in subversion config under [tunnels] is called as:
> svnserve -t
⋮
> Which in the end makes a command such as this work like a charm:
> svn info svn+nc://hostname:3699
>
> Maybe you had a simpler way of achieving the same in mind?
No. You _could_ inline the wrapper script into the configuration, along
these lines:
.
[tunnels]
nc = $SUBVERSION_TUNNEL_NC sh -c 'nc ${1%:*} ${1##*:}' :
.
but other than that, what you did is exactly what I had in mind. Sorry
it wasn't clear the first time.
BTW, to support the default port:
.
nc = $SUBVERSION_TUNNEL_NC sh -c 'case $1 in *:*) ;; *) set -- ${1}:3690;;
esac && nc ${1%:*} ${1##*:}' :
I've committed a related clarification in https://svn.apache.org/r1889629.
Thanks for raising that.
The other approach I described was to define a tunnel such as
.
[tunnels]
bar = $BAR sh -c "nc lorem.bar.example 3690"
.
which can then be used as «svn+bar:///some/path» with an empty hostname
component. I used to do this, for two reasons: it made the URLs
shorter, and it let me change the SSH jumphost centrally across all my
working copies.
> In any case, thanks a lot all for your helpful inputs and suggestions -
> made my day!
You're welcome.
Daniel
Re: Enforce encryption client-side
Nils-Johan Andreasson wrote on Thu, May 06, 2021 at 13:03:28 +0200: > Thanks Stefan for your response and pointers! > > There is no doubt my connections are already encrypted. > The issue is that the external library I'm referring to supports specifying > paths to source code using multiple protocols, e.g. svn:// (but also > https://, http://, git://, ftp:// etc) and any svn:// source is generally > considered insecure, just like http:// (but not for example https://). It > is not verified in any way if that's actually the case, it's just based on > the scheme prefix svn:// > > Using something like that I can hopefully convince the library maintainer > to enable this flag by default and then consider an svn:// source secure by > default (as connection would otherwise not succeed). You could also point out to the library author that treating svn://foo/ as insecure would be wrong when foo is an IP address on a VPN, or a loopback address, or an address of a virtualized box running on the same hardware, and so on, even if no encryption is used at all. Cheers, Daniel > I understand your point with that as long as "use-sasl = true" is > configured on the server, the connection would abort if the client cannot > communicate securely. However, this means that if this external library is > pointed to a svn:// source with "use-sasl = false" the connection would > succeed, which is what I'm trying to prevent here to guarantee no > unencrypted connections are made. > > Hope my point came through better this time? > > Best regards, > > Nils-Johan > > > On Thu, May 6, 2021 at 11:40 AM Stefan Sperling wrote: > > > On Thu, May 06, 2021 at 11:21:57AM +0200, Nils-Johan Andreasson wrote: > > > Hi there! > > > > > > I'm serving a repository using svnserve with SASL to make sure > > > communication is always encrypted (config has use-sasl = true, > > > min-encryption = 128 and max-encryption = 256). > > > I know this enforces encryption server-side but is there any way to in > > > addition also "require" encryption client-side? E.g. let's say if I do > > 'svn > > > checkout svn://my-insecure-host/repository' I want the command to abort > > if > > > the connection is not encrypted. > > > > The min-encryption paramter maps directly to the Cyrus SASL > > secprops.min_ssf > > paramter which is described here: > > > > https://www.cyrusimap.org/sasl/sasl/developer/programming.html#security-layers > > > > Quote: > > "A connection supplying only integrity with no privacy would have an SSF > > of 1. > > A connection secured by 56-bit DES would have an SSF of 56. > > To require a security layer, set min_ssf to the minimum acceptable security > > layer strength." > > > > SVN servers and clients check the SASL_SSF property of their connection > > here > > and abort if SASL failed to negotiate encryption if encryption is > > configured: > > > > https://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_ra_svn/cyrus_auth.c?revision=1875971=markup#l726 > > > > So your connections should already be encrypted. Perhaps the encryption > > mechanism SASL is using is considered too weak by your external tool? > > Would setting min-encryption = 256 help? > > Or perhaps your external tool simply doesn't understand the SVN protocol? > > > > Cheers, > > Stefan > >
Re: Enforce encryption client-side
Nils-Johan Andreasson wrote on Thu, May 06, 2021 at 14:48:01 +0200:
> Is there a way (command) to remotely determine if the svn server behind
> 'hostname' has sasl enabled/uses encryption/enforces a min_ssf >= 128?
> Preferably without having to send any authentication credentials (to not
> risk sending anything in clear text).
Possibly. Look into the handshake in subversion/libsvn_ra_svn/protocol,
Section 2, specifically the «auth-request» line (the second thing the
server sends).
You can try it on a toy example like this:
% svnadmin create r
% ${EDITOR:-ed} r/conf/svnserve.conf
% printf '%s\n' '( 2 ( edit-pipeline ) 17:svn://localhost/r )' | svnserve
-i -r ./
Or on a real server with something like this:
% printf '( 2 ( edit-pipeline ) %d:%s )\n' "$(printf %s "$url" | wc -c)"
"$url" | nc $url 3690
When I do that I get "Could not obtain the list of SASL mechanisms"
(E170001 SVN_ERR_RA_NOT_AUTHORIZED), but you should get an actual
response. Does that response provide the information you seek?
> If so, an initialization step could hopefully be added which determines
> whether the source is deemed to be secure to access or not.
Re: Enforce encryption client-side
Nils-Johan Andreasson wrote on Thu, May 06, 2021 at 14:48:01 +0200: > I have indeed considered svn+ssh but there are other details involved which > makes me prefer to stay with svnserve and svn:// if possible. Does your library consider svn+foo:// secure? If so, you can define an svn+foo:// scheme that simply wraps plain old TCP sockets. You have two options for that, even: either «svn+nc://hostname/path» that uses nc(1) or socat(1) to wrap a plain TCP socket, or a scheme that hardcodes a specific hostname, that you then use in URLs whose hostname component is ignored (can actually be left empty).
Re: Does a WC contain all log messages?
Bo Berglund wrote on Sat, 24 Apr 2021 15:54 +00:00: > On Sat, 24 Apr 2021 11:05:33 -0400 (EDT), Robert Heller > wrote: > > >I believe svn log always connects to the "server". The difference is where > >the > >"url" comes from. In the first form, the user explicitly provides the url > >and > >in the second the url is formed from information in the .svn directories. > > This I could in fact test! :) > The svn server I use ison a local server at the office and is not exposed > externally. I am hooked up to the office network via VPN, so I could > disconnect > and re-test. > Sure enough, no log appeared. So it *is* a server side operation. Correct. > But there is no timeout on the client side, it just hangs there... > And reconnecting to the office while it is "waiting" does nothing at all. > Still > just hangs there. Depends on how long you waited, but the simplest explanation is that that's down to your TCP configuration. (SYN being dropped rather than responded to with an RST, and then the connection revived before the retransmission delay elapses, etc.) > Test done on Windows 10 with svn client version 1.12.0 > Server side svn version is 1.9.7, I believe.
Re: Does a WC contain all log messages?
Daniel Sahlberg wrote on Sat, 24 Apr 2021 15:18 +00:00: > Den lör 24 apr. 2021 16:52Bo Berglund skrev: > > 2) Run the command in the working copy dir: > > svn log -v > > > > I don't know if the second incarnation will only get whatever I have done > > myself > > when working on the project or if it somehow can show the complete log > > history. > > It would connect to the repository (at the server or in the local file > system, if you are using file://) and fetch the log. So you will see > the complete log. Not quite. In general, «svn $subcommand» with no positional arguments is equivalent to «svn $subcommand ./», so logs shown will be the logs of cwd, not of the whole repository. Second, accessing the logs via a client is subject to authz (that makes a difference if some of the revision numbers targeted touch paths the user has no read permission on).
Re: Should 'svn help' with APR lifetime debug enabled run without issues?
Rick van der Zwet wrote on Fri, Apr 23, 2021 at 02:57:02 +0200:
> I really liked the suggestions provided, they gave me the clearity I
> needed with regards the lifetime debug flagging on 'svn help' output,
> thanks!
You're welcome.
> FYI: I did some futher investigation on the specific problem within Trac
> I was trying to fix and I am getting more hints by experimenting with
> flags and combinations. If I compile APR (1.7.0) without debugging it
> produces a coredump with some endless(?) recursion, see attached
> gdb-output.txt for details. If I compile apr with
> '--enable-pool-debug=yes', the program executes without errors,
> to-be-continued. Since this is most likely not subversion related I
> think is best discussed on different mailinglist.
Ack, but the backtrace does go through Subversion's swig-py bindings and
libsvn_fs_fs, so we might be involved nevertheless. Does the problem
reproduce in «svnlook help»? In «svnlook youngest /path/to/repo»? In
«./fs-test» (in subversion/tests/*/)?
I'm guessing frame 87035 is subversion/libsvn_fs_fs/fs.c:fs_open().
That function uses a subpool, but I don't see why. A subpool would have
made sense if «subpool» had been passed to a function that was
documented to install pool cleanup handlers on its provided pool, for
example, but none of the uses of «subpool» are as anything other than
a called function's ordinary scratch_pool. So, could you try the
following patch?
[[[
Index: subversion/libsvn_fs_fs/fs.c
===
--- subversion/libsvn_fs_fs/fs.c(revision 1889073)
+++ subversion/libsvn_fs_fs/fs.c(working copy)
@@ -427,19 +427,15 @@ fs_open(svn_fs_t *fs,
apr_pool_t *scratch_pool,
apr_pool_t *common_pool)
{
- apr_pool_t *subpool = svn_pool_create(scratch_pool);
-
SVN_ERR(svn_fs__check_fs(fs, FALSE));
SVN_ERR(initialize_fs_struct(fs));
- SVN_ERR(svn_fs_fs__open(fs, path, subpool));
+ SVN_ERR(svn_fs_fs__open(fs, path, scratch_pool));
- SVN_ERR(svn_fs_fs__initialize_caches(fs, subpool));
+ SVN_ERR(svn_fs_fs__initialize_caches(fs, scratch_pool));
SVN_MUTEX__WITH_LOCK(common_pool_lock,
- fs_serialized_init(fs, common_pool, subpool));
-
- svn_pool_destroy(subpool);
+ fs_serialized_init(fs, common_pool, scratch_pool));
return SVN_NO_ERROR;
}
]]]
This passes fs-test and basic_tests.py (I didn't do a full «make
check» run).
To be clear, the scratch_pool usage in that function in HEAD is
non-idiomatic without an obvious reason, but I don't see why it would
result in an infinite loop.
About which, the pool address, 0x0804542028 (note I added a leading zero
to keep the number of nibbles even), looks suspiciously like ASCII: it's
"\t\004T (". That _could_ be a coincidence, especially since there's no
obvious mechanism by which the value of «subpool» could be corrupted
between the svn_pool_create() and the svn_pool_destroy(), but still,
a valgrind run on a pool-debug-enabled build might be a good idea.
> >
> > For future reference, your MUA hard-wrapped the various traces, which
> > made them difficult to read. Please disable hard wrapping for such
> > cases, or use attachments named *.txt.
>
> Thanks for the feedback, was unaware of hard-wrapping bt my MUA
> (roundcube), will try to prevent it in the future postings.
Thanks ☺
Cheers,
Daniel
Re: Should 'svn help' with APR lifetime debug enabled run without issues?
Rick van der Zwet wrote on Wed, Apr 21, 2021 at 16:10:57 +0200:
> Hi,
>
> I debugging crashes within the Trac (https://trac.edgewall.org/) when using
> the svn backend. The issue seems to be related to Apache Portable Runtime
> (APR) pool memory management at Trac together with subversion SWIG python
> bindings and libsvn.
>
> I am trying to run subverion with APR (lifetime) debugging enabled
> (./configure --enable-pool-debug=all --enable-debug) to give more insights,
> how-ever I got stuck with the basics.
I assume the use of «all» as the argument to --enable-pool-debug is
deliberate? I.e., that enabling fewer bitflags wouldn't suffice for
your use-case?
> First I tried 'svn' which completes fine:
>
> Next I tried 'svn help' which gives me apr_pool_integrity check [lifetime]
> exception:
> ===
> POOL DEBUG: [29126/34384113664] CREATEU ( 0/ 0/ 5029)
> 0x80174d640 "subversion/libsvn_subr/pool.c:160"
> (0/0/0)
> POOL DEBUG: [29126/34384113664]LIFE
> 0x80174d640
>
Looking at that file (in apr-1.6.5, since you didn't mention
a particular version):
1597 static void apr_pool_check_integrity(apr_pool_t *pool)
1598 {
1599 /* Rule of thumb: use of the global pool is always
1600 * ok, since the only user is apr_pools.c. Unless
1601 * people have searched for the top level parent and
1602 * started to use that...
1603 */
1604 if (pool == global_pool || global_pool == NULL)
1605 return;
1606
1607 /* Lifetime
1608 * This basically checks to see if the pool being used is still
1609 * a relative to the global pool. If not it was previously
1610 * destroyed, in which case we abort().
1611 */
1612 #if (APR_POOL_DEBUG & APR_POOL_DEBUG_LIFETIME)
1613 if (!apr_pool_is_child_of(pool, global_pool)) {
1614 #if (APR_POOL_DEBUG & APR_POOL_DEBUG_VERBOSE_ALL)
1615 apr_pool_log_event(pool, "LIFE",
1616 __FILE__ ":apr_pool_integrity check
[lifetime]", 0);
1617 #endif /* (APR_POOL_DEBUG & APR_POOL_DEBUG_VERBOSE_ALL) */
1618 abort();
1619 }
1620 #endif /* (APR_POOL_DEBUG & APR_POOL_DEBUG_LIFETIME) */
The Subversion caller is error.c (frame #6), and it has a comment saying
it deliberately uses a pool unrelated to the global pool:
69/* Strictly speaking, this is a memory leak, since we're creating an
70 unmanaged, top-level pool and never destroying it. We do this
71 because this pool controls the lifetime of the thread-local
72 storage for error locations, and that storage must always be
73 available. */
> Program received signal SIGABRT, Aborted.
> 0x000801088c2a in thr_kill () from /lib/libc.so.7
> (gdb) bt
> #0 0x000801088c2a in thr_kill () from /lib/libc.so.7
> #1 0x000801087084 in raise () from /lib/libc.so.7
> #2 0x000800ffd279 in abort () from /lib/libc.so.7
> #3 0x000800ea8a57 in apr_pool_check_integrity (pool=0x80174d640) at
> memory/unix/apr_pools.c:1618
> #4 0x000800ea8c3d in apr_pcalloc_debug (pool=0x80174d640, size=16,
> file_line=0x800e7c683 "threadproc/unix/threadpriv.c:28") at
> memory/unix/apr_pools.c:1795
> #5 0x000800ebbe49 in apr_threadkey_private_create (key=0x800873998
> ,
> dest=0x80081d770 , pool=0x80174d640) at
> threadproc/unix/threadpriv.c:28
> #6 0x00080081bd61 in locate_init_once (ignored_baton=0x0) at
> subversion/libsvn_subr/error.c:77
> #7 0x0008007f6d4e in str_init_func_wrapper (init_baton=0x7fffcfb0)
> at subversion/libsvn_subr/atomic.c:156
> #8 0x0008007f6bcd in init_once (global_status=0x800873990
> ,
> init_func=0x8007f6d30 , init_baton=0x7fffcfb0)
> at subversion/libsvn_subr/atomic.c:71
> #9 0x0008007f6ce0 in svn_atomic__init_once_no_error
> (global_status=0x800873990 ,
> str_init_func=0x80081bd30 , baton=0x0) at
> subversion/libsvn_subr/atomic.c:170
> #10 0x00080081bca7 in svn_error__locate (file=0x8007d85cd
> "subversion/libsvn_subr/io.c", line=3988)
> at subversion/libsvn_subr/error.c:128
> #11 0x00080082235d in svn_io_file_open (new_file=0x7fffd0e8,
> fname=0x8017523c0 "/etc/subversion/servers",
> flag=1, perm=4095, pool=0x80174d500) at subversion/libsvn_subr/io.c:3988
⋮
> #18 0x0025cc7f in main (argc=2, argv=0x7fffdaa0) at
> subversion/svn/svn.c:3325
> (gdb)
> ===
>
>
> I tried subversion 1.14.1 and subversion-trunk@1889032. Traces above are
> generated using the subversion-trunk codebase.
>
> I have also the unittests of the Apache Portable Runtime (APR) on which the
> relevant(?) ones completes successfully.
Re: Does anyone know of a statically linked SVN-client for IPfire?
Thorsten Schöning wrote on Thu, Mar 18, 2021 at 14:27:30 +0100:
> #!/bin/bash
>
> ##
> # Wrapper to make LD_LIBRARY_PATH easily available per invocation.
> #
> # This distribution of SVN is mostly self-contained, but it's own private
> shared libs need to be made
> # available somehow. The easiest approach is to use the environment variable
> LD_LIBRARY_PATH, so that
> # users don't need to care to much. Therefore this wrapper calculates the
> necessary path on its own
> # and sets the environment variable accordingly.
> #
> # While users are only interested in the SVN-client most likely, the wrapper
> is designed to calculate
> # the binary to forward all given arguments to based on the name of the
> wrapper itself. So it can
> # simply be stored using the naming scheme "svn.sh", "svnadmin.sh" etc. using
> symlinks and might be
> # used for other binaries as well this way.
> #
> # @param[in] ... All arguments are forwarded as-is to the binary of interest.
> #
>
> APP_NAME=$(basename "$0" ".sh")
> APP_DIR=$( dirname "$0")
> LIB_DIR=$( realpath "${APP_DIR}/../lib")
>
> export LD_LIBRARY_PATH=${LIB_DIR}
Note that this discards any preëxisting value of $LD_LIBRARY_PATH.
(Also, might want to use #!/bin/sh.)
Cheers,
Daniel
> "${APP_DIR}/${APP_NAME}" "$@"
Re: How to work around E000018: invalid cross-device link
Daniel Sahlberg wrote on Thu, Mar 11, 2021 at 08:56:27 +0100: > Den ons 10 mars 2021 kl 16:21 skrev Zhao Wu : > > 1. sensitive files must be kept on encrypted file system, but to put the > > entire repo is too slow; > > > > Since you most probably have multiple working copies and individual > developers can check out a new working copy wherever they want, I think it > will be difficult to implement such a policy across your organization. > Unless you have very specific hardware/software limitations I would suggest > to take the performance hit and encrypt everything. (This might also apply > to the disk on the server where the actual repository is stored). Don't forget about encrypting the relevant parts of .svn/, such as .svn/pristine/.
Re: Restoring svn database
Jean-Baptiste DUBOIS wrote on Tue, Mar 09, 2021 at 12:02:51 +:
> I have one last question regarding how svn tags are managed internally in
> fsfs.
> Is svn tag considered as a PLAIN data independant of previous revs or not ?
As far as FSFS is concerned, there's no such thing as a tag. What FSFS
deals with is a zero-indexed array of interrelated directory trees.
By "tag" people normally refer to a child of a well-known directory;
specifically, to its contents and copyfrom information. About these:
- The contents of any given file are stored in that file's `Last Changed
Revision` rev file or in an older rev file. The latter case is only
possible when rep-sharing is enabled, which in your case it's not,
because you use format 2 ("f2"). Watch:
.
1 % svn info --show-item=last-changed-revision file://$PWD/iota{0,1}@86
2 86 file://$PWD/iota0
3 85 file://$PWD/iota1
4 % svnlook tree --show-ids -r 86 ./
5 / <0.0.r86/428>
6 iota0 <2.0.r86/210>
7 iota1 <1.0.r85/210>
8 % xxd -s 210 -l 4096 -p db/revs/85 | xxd -r -p | sed -e '/^$/q'
9 id: 1.0.r85/210
10 type: file
11 pred: 1.0.r83/111
12 count: 42
13 text: 85 0 188 2340 b6cff4dd559998419b5210d8de7c7130
14 cpath: /iota1
15 copyroot: 0 /
16
iota1@86's node-rev header is in r85 (line 7). Its rep is also in r85
(line 13), but in a rep-sharing situation, it might've been in an
older rev file.
Line 8 was written for f2. For a variant supporting f3, f4, and f6, see
.
[For the curious, the bug manifesting in that post is #4129. It was
finally reproduced the following March:
.]
- The copyfrom information is stored in the node-rev header, so it'll be
stored in the rev file of the revision the tag was created in.
Under f2, which doesn't deltify directory reps (see
SVN_FS_FS__MIN_DELTIFICATION_FORMAT), the node-rev itself will be
accessible via the API, as discussed upthread.
Finally, note that "data independent of previous revs" and "PLAIN data"
aren't synonyms, since self-deltas are a thing.
Cheers,
Daniel
Re: Restoring svn database
Daniel Shahaf wrote on Fri, 05 Mar 2021 16:33 +00:00:
> Format 2 doesn't support rep-sharing and doesn't deltify directory reps,
> so simply running `svnlook changed -r 86` and then `svnlook cat -r 86`
> against each file printed thereby should extract everything extractable.
> Any given `svn cat` invocation might fail if a DELTA line refers to
s/svn cat/svnlook cat/
> a revision whose rev files has been lost. ("text:" and "props:" lines
> will always point into the rev file they themselves are in.)
>
> Use `svnlook propget` in addition to `svn cat` to extract versioned
s/svn cat/svnlook cat/
> properties. svn_hash_read2() will parse the format. (It's a public
> API and likely available via the bindings as well, but if needed,
svn_hash_read2() isn't needed in the format-2 case. It may, however, be
needed in the general case to parse directory reps or property reps.
Re: Connection was forcibly closed
Renzo Rosales wrote on Fri, Mar 05, 2021 at 14:53:02 +: > We have a few remote users who are unable to run "svn up" to an internal > server in specific paths in a repository but can access others. The error is > "svn: E730054: Error running context: An existing connection was forcibly > closed by the remote host." The server is in the US and some of the users are > in Russia, they communicate with the server over a VPN not using NAT. If they > use Putty to create a SSH port forward to the server, they can run svn up > without error. In the Putty case, what URL scheme do they use over the port forward? > The rule that allows traffic to traverse the VPN does not have any network > filtering in place. I know this server has an OS release and software dated > from 2011 and 2012 (details below), the httpd access logs don't show any > issues (HTTP code 200 and 207), the error log is bare and does not show a > relevant entry that shows why a sync was blocked or not permitted. > > What are some suggestions on where I can look to better troubleshoot the > issue? Run `svn up` over SSH port forwarding then `svn up -r BASE` over the VPN. That should be a no-op update. Does it fail the same way? What about, say, `svn ls -r BASE --depth=empty`? (That's a network operation) Check not just the httpd error log but also the system one, in case it's a packet filter or firewall that killed the connection (notwithstanding the rule you've reviewed). Also, naturally, what's the common thread to the paths that fail, and to the paths that don't. > Would a packet capture help me identify what is going on? If so, what should > I look for in the capture? Well, there might be a clue in there. Look for whether it was a FIN or a RST, and what happened immediately before it — e.g., silence (suggesting some sort of timeout), or client→server data, or server→client data. Cheers, Daniel
Re: Restoring svn database
Jean-Baptiste DUBOIS wrote on Fri, Mar 05, 2021 at 11:04:01 +:
> Inside the 'db' folder we have 'rev', 'revprops' 'transactions' folder
> contains files.
revs/ is where file content lives. See subversion/libsvn_fs_fs/structure.
(You can read it in trunk too: it covers all FSFS formats, not only the
newest one.)
> Some files are missing but the company told us that all files
> recovered are fully recovered (ie: file integrity is OK).
That's good. It's also a very different question than the one Philippe
asked.
>
> I know that we could not restore the entire database, but can we extract some
> 'SVN plain data' from file in 'revs' folder
>
> Hereunder a view of thoses files (ordered by Size) ...
>
> [revs]
For future reference, text is preferable to images. Copying the output
of `dir` would have been easier for us to consume.
> Is it possible to detect plain data with no dependency on previous revs
> inside these 'revs' files and extract them ?
Yes. In your case it'll actually be a four-liner loop in the scripting
language of your choice, but I'll give the full answer.
Let's take an example format-2 revision file:
[[[
% rm -rf r
% svnadmin create r --compatible-version=1.4
% cat r/db/format
2
% for i in {1..$(wc -l < ${fn::=~/src/svn/trunk/README})} ; do svnmucc -mm -U
file://$(pwd)/r put =(head -$i $fn) iota$((i % 2)) ; done > /dev/null
% cat_the_youngest_revision_file() { < r/db/revs/*(.om[1]) LC_ALL=C sed -e
's/[^ -~]/X/g' } # translate any octets other than printable ASCII to "X"
% cat_the_youngest_revision_file | nl -ba
1 DELTA 82 0 331
2 SVNX%XX(XXX& Finally, be sure to see Appendix B in the
Subversion Book. It
3contains a very quick overview of the major differences between
4CVS and Subversion.
5
6 ENDREP
7 id: 2.0.r86/210
8 type: file
9 pred: 2.0.r84/183
10 count: 42
11 text: 86 0 188 2341 c09759b8da81bf3e23647f4c517abbe0
12 cpath: /iota0
13 copyroot: 0 /
14
15 PLAIN
16 K 5
17 iota0
18 V 16
19 file 2.0.r86/210
20 K 5
21 iota1
22 V 16
23 file 1.0.r85/210
24 END
25 ENDREP
26 id: 0.0.r86/428
27 type: dir
28 pred: 0.0.r85/428
29 count: 86
30 text: 86 347 68 68 d3f8fb2002e019614ec0e47c79e2ac4c
31 cpath: /
32 copyroot: 0 /
33
34 2.0.t85-1 modify true false /iota0
35
36
37 428 558
%
]]]
(Aside: As an f8 repository, the svndiff delta would contain only the
last line of README, rather than the last three lines as in this
example.)
The parts you're interested in are:
- "DELTA %ld %ld %ld" lines (e.g., the «82» on line 1)
- "text:" and "props:" lines (e.g., the «86» on line 11)
- "DELTA\n" lines (without numbers)
- "PLAIN\n" lines
In the first two cases, the first number on the line identifies the
revision number in which depended-on data is found. See the
aforementioned «structure» file for details. The last two cases
identify data that's present inline.
Using this information, you could build a DAG of reachable reps (a "rep"
is the thing between the "DELTA" or "PLAIN" line and the "ENDREP" line)
and extract them. However, since you're on format 2, there's an easier
way.
Format 2 doesn't support rep-sharing and doesn't deltify directory reps,
so simply running `svnlook changed -r 86` and then `svnlook cat -r 86`
against each file printed thereby should extract everything extractable.
Any given `svn cat` invocation might fail if a DELTA line refers to
a revision whose rev files has been lost. ("text:" and "props:" lines
will always point into the rev file they themselves are in.)
Use `svnlook propget` in addition to `svn cat` to extract versioned
properties. svn_hash_read2() will parse the format. (It's a public
API and likely available via the bindings as well, but if needed,
I happen to have posted a pure Python implementation of that last week:
<https://mail-archives.apache.org/mod_mbox/subversion-dev/202102.mbox/%3C20210226173525.GA24828%40tarpaulin.shahaf.local2%3E>.)
Note: that's `svnlook cat -r`, not `svn cat -r`. The difference
matters: -r to svnlook denotes a peg revision, not an operational
revision. (Also, using svnlook(1) bypasses several layers of API.)
In newer formats, where directory reps may be deltified, it's possible
to get a case such as
.
r10: mkdir /A
r20: add /A/foo
r30: add /A/bar
.
with r20 lost. In this case, if the rep of /A in r30 happened to depend
on the rep of /A in r20, `svn ls ^/A@30` and `svn cat` of files
thereunder would both fail. However, if one figured out the location of
/A/bar's node-rev header or rep, one could still read those directly,
using the appropriate internal APIs.
Cheers,
Daniel
> I try to u
Re: Restoring svn database
Stefan Sperling wrote on Wed, Mar 03, 2021 at 11:15:26 +0100: > On Wed, Mar 03, 2021 at 08:03:08AM +, Philippe DEMOUSTIER wrote: > > Following an issue on our servers, we lost approximatively 30% of our svn > > database. > > Admin dump fails so we're trying to restore some data manually. _Which_ 30% did you lose? > > How can we restore data between SVN and ENDREP tags ? > > > > DELTA 30834 15564 155 > > SVN xx ENDREP > > > > Many thanks for your help, > > Philippe > > Hi Philippe, > > I am afraid the short answer is that you will need to restore your repository > from backups. If you cannot do that, then you have now learned the hard way > that an important SVN repository needs to be backed up. The data is gone. > > It is virtually impossible to restore the missing data manually. Well, it depends. If the data is part of a directory rep, it may be possible to reconstruct the directory rep from the node-rev headers in the same rev file. Moreover, even if the directory rep isn't reconstructed at all, so long as the file content reps are intact, extracting those into files might still be useful, even though that wouldn't restore filenames. For filenames, one could consult the changed-paths section of the rev file, commit notifications, TortoiseSVN log caches (if those cache changed-paths info?), log messages, git-svn(1) mirrors, and even nearby cpath node-rev headers. If the data is part of a property rep, it may be acceptable to accept the loss of those properties and restore the remaining data. Even if the data is part of a content rep, that isn't necessarily game over insofar as later revisions of the same file are concerned: it's likely that some later revisions will be able to be reconstructed despite the loss of one interim rep. (Because of skip-deltas and, IIUC, max-linear-deltification) > This data is sometimes full text, sometimes deltified content. Philippe specified that the data starts with "SVN", so it's unlikely to be a full text. It might be a self-delta, but it's most probably just a regular delta. > The only way to generate > equivalent data is to replay all the commits that have occurred throughout > repository history. The tool 'svnadmin load' can do this, provided you have > previously saved a dump file of the repository to backup storage when the > repository was still in a healthy condition. It's not that simple. Nothing guarantees that the lost rep and the rep resulting from the load would be the same length. One would have to check the lengths manually and deal with the differences. > Going forward, you should consider saving repository dump files to backup > storage, or saving backup copies of your repositories with tools such as > 'svnadmin hotcopy' or 'svnsync' For more information, see this page: > http://svnbook.red-bean.com/nightly/en/svn.reposadmin.maint.html#svn.reposadmin.maint.backup +1 Cheers, Daniel
Re: A more permanent home for the add-a-password-to-a-cached-username script? (was: Re: using svn cli with --non-interactive (in scripts) securely, without exposing password)
Daniel Sahlberg wrote on Wed, 03 Mar 2021 07:32 +00:00: > This brings up the question of translation. Futatuki translated the > text to Japanese last time - or can someone else volonteer? There is > also a Chinese translation, but it has not been updated for a long time > - is there someone who would be interested in updating the text? First, we shouldn't ask translators to translate moving targets. The text in staging/ may still change before it's published. As to translating the text once it's stable: we don't have a process for this. I suppose we could just email the relevant translators and advise them that faq.en.html has been updated and they may want to update their translations, just like we might ask translators to update subversion/po/ before a release — but we should keep in mind that some translators haven't been active in years, and may or may not be interested in contributing further. Both translations have text near the top identifying the English revision that was translated. There's some room for improvement there: 1. The Chinese translation needs to get the +840074 bump (per ^/subversion/README). 2. Both translations state the revision number that was translated, but leave the reader to do some footwork in order to determine whether the translation is up-to-date. They could, say, link to the ViewVC log view of the English FAQ. 3. Linking to the log view assumes some understanding of Subversion, and in context, that assumption may not be desirable. So, the English FAQ could state a "Last modified" date, and the translations could indicate the date of the English version that was translated. Cheers, Daniel
Re: cannot find libsvn_fs_x-1.so.0, 1.10.7, CentOS 7
Nathan Hartman wrote on Tue, 02 Mar 2021 14:32 +00:00:
> On Mon, Mar 1, 2021 at 6:36 PM Zube wrote:
> >
> > Trying to build 1.10.7 on CentOS 7 and failing. 1.10.6 builds fine.
> >
> > /usr/bin/ld: cannot find libsvn_fs_x-1.so.0: No such file or directory
> > collect2: error: ld returned 1 exit status
> > make: *** [build-outputs.mk:478: subversion/libsvn_fs_x/libsvn_fs_x-1.la]
> > Error 1
> >
> > One difference between .6 and .7 is from the created (from config.log)
> > subversion/libsvn_fs_x/libsvn_fs_x.pc:
> >
> > 8,12c8,12
> > < Version: 1.10.7
> > < Requires: apr-util-1, apr-1
> > < Requires.private: libsvn_delta, libsvn_subr, libsvn_fs_util
> > < Libs: -L${libdir} -lsvn_fs_x-1
> > < Cflags: -I${includedir}/subversion-1
> > ---
> > > Version: 1.10.6
> > > Requires: apr-util-1 apr-1
> > > Requires.private: libsvn_delta libsvn_subr libsvn_fs_util
> > > Libs: -L${libdir} -lsvn_fs_x
> > > Cflags: -I${includedir}
> >
> > In the .libs directory in .6, the libraries are built:
> >
> > libsvn_fs_x-1.a
> > libsvn_fs_x-1.la
> > libsvn_fs_x-1.lai
> > libsvn_fs_x-1.so
> > libsvn_fs_x-1.so.0
> > libsvn_fs_x-1.so.0.0.0
> > libsvn_fs_x-1.so.0.0.0T
> >
> > But they are missing from the .7 build. The .o files are in the
> > .libs directory, though.
> >
> > Any help is greatly appreciated.
>
> Hello,
>
> I've traced this to r1863987 and r1863990 on trunk, which were merged
> to the 1.10.x branch in r1866641.
>
> Part of the change was to respect pkg-config(1)'s requirement of a
> comma-separated list, so you can see the commas being introduced in
> your diff above.
>
> Another part of the change is in build/generator/gen_make.py, which is
> now adding the "-1" to "-lsvn_fs_x" making it "-lsvn_fs_x-1."
>
> I don't think there is an error in these changes; rather, I think that
> an additional change is needed elsewhere to make the built libsvn_fs_x
> compliant.
I think our build process doesn't consume the .pc files at all, which
would explain why 1.10.6 is buildable despite the lack of commas.
However, I don't see any other relevant change, so…
> I haven't yet determined what/where that change should be, nor what
> workaround to suggest, but I wanted to reply with what I know so far.
… try applying the reverse of r1866641 to the source tree. (Then, if you
correct .pc files, massage them by hand after configure runs.)
Alternatively, you could patch libsvn_fs_x out of the build entirely,
since it's an experimental component anyway.
I don't see a configure option for this.
One way is to patch fs_x mentions out of build.conf and Makefile.in and
re-run gen-make.py (normally done as part of autogen.sh, which in turn
is part of the build-from-checkout flow but not part of the build-from-
tarball flow). However, there may well be an easier way which I'm
overlooking.
Karl's advice in
https://subversion.apache.org/docs/community-guide/building.html#configury
stems from [a Debian libtool patch](https://bugs.debian.org/291641),
IIRC, so it wouldn't be my first suspicion for a CentOS build failure…
but it's not impossible.
Cheers,
Daniel
Re: A more permanent home for the add-a-password-to-a-cached-username script? (was: Re: using svn cli with --non-interactive (in scripts) securely, without exposing password)
Nathan Hartman wrote on Thu, Feb 25, 2021 at 13:29:48 -0500: > May I propose to have just one FAQ entry that simultaneously would answer: > * "what alternatives to plaintext caching are there?" > * "plaintext caching is supported but I want to *prevent* it" > * "plaintext caching is not supported but I want to use it anyway" Of course you may. _Prima facie_ I'm not sure that would be a good idea — one question per FAQ entry is similar to "Do one thing and do it well", and answering two questions per FAQ entry might make that entry less usable as a URL to be passed to someone who has either of these questions — but I'll reserve judgement until I've read the proposal. > > Password caching in plain-text on disk How about "Does Subversion cache passwords in plain text?" or "Configuring plain text password caching"? >title="Link to this section"> > > > To avoid having to type a password for each server operation, Subversion > can cache credentials. > > Whether and how Subversion caches credentials depends on several factors, > including the operating system, compile-time options, and settings in the > client's run-time config file. > Should this mention that ssh password prompts / keys aren't covered here? And what about client certs? > On some operating systems and configurations, Subversion can cache > passwords on disk in plain-text. Some users want this, while others want to > disallow it. This FAQ entry summarizes how credential caching works and > attempts to answer both of these questons: > > > How to prevent caching passwords on disk in plain-text (with > various alternatives provided), and > How to cache passwords on disk in plain-text > > See https://m.xkcd.com/1343/.¹ How about dropping the , deleting the above it, and in the latter's stead mention, in the Unix , that plaintext password caching is configurable because it's not everyone's cup of tea? > UNIX/Linux > > On UNIX/Linux, Subversion supports up to four credential caches: > > > GNOME Keyring > KWallet > GPG-Agent > Plaintext cache in ~/.subversion > > > To determine which credential caches your Subversion client supports, run > the svn --version command and look for "The following > authentication > credential caches are available" toward the end of its output. > > GNOME Keyring and KWallet both facilitate storing passwords on disk > encrypted. For Subversion to support these programs (since Subversion 1.6), > they need to be available at compile-time and at run-time. Otherwise, > Subversion may fallback to storing passwords in the Plaintext cache, > if support for that is built in; see below. Suggest s/if.*built in/if so configured/, since there are both run-time and compile-time knobs. The term "built in" in particular is ambiguous (could mean either "enabled at compile time" or "is available always rather than as a module"). > TODO: Discuss GPG-Agent. > > On UNIX/Linux, the Plaintext cache is always supported for > reading, > but support for writing new passwords to the cache depends on build > time configuration. Since Subversion 1.12, the default is not to > support writing new passwords to the Plaintext cache, unless specifically > enabled at build time, but Subversion will continue to use any previously > cached passwords that are "grandfathered in." This jumps back and forth between reading and writing. How about separating several concerns here: 1. State that writing new passwords to the cache is configurable because it's not everyone's cup of tea. 2. Describe the relevant compile- and run-time knobs, where they live, and their defaults on various versions: --enable-plaintext-password-storage, SVN_DISABLE_PLAINTEXT_PASSWORD_STORAGE, --no-auth-cache, store-passwords, store-auth-creds. 3. State that writing is only done if all those planets align (if all knobs have the right values). 4. State that reading is always done. 5. In the sections for "How to enable plaintext cachnig" and "How to disable plaintext caching", build on top of this description. 6. Consider adding discussion of when cache entries are deleted automatically (even if that's just "Never") and how to delete them manually. #5 would be a good reason to keep the answers to the two questions as h4's under a single top-level entry. > The directory which contains the cached passwords (usually > ~/.subversion/auth/) has permissions of 700, meaning only the user > (and root) can read them. > > "I want to prevent writing passwords to the Plaintext cache!" > > The following options are available in your run-time config file: > > > To allow encrypted stores like GNOME Keyring and KWallet, but not the > Plaintext cache, set store-plaintext-passwords = no. > To allow caching server certs but not passwords (encrypted or not), set > store-passwords = no. > To disable storing any kind of credentials (encrypted or not) set > store-auth-creds = no. It's noting here that the settings can be set in both
Re: A more permanent home for the add-a-password-to-a-cached-username script? (was: Re: using svn cli with --non-interactive (in scripts) securely, without exposing password)
Daniel Sahlberg wrote on Wed, Feb 24, 2021 at 23:27:18 +0100: > Learning Python has been on my todo list for ages, so I've cobbled > together something that seems to do the job. ☺ > There are basically two modes of operation: > > ./store-plaintext-password.py --listpassword|--list > > Which lists all cached realms/usernames (and passwords). I've intentionally > split it in two in case someone prefer not to put the password on-screen, > but I've also added the option to display to show that the password is > really in plain text. Isn't this what `svn auth` does? Also, "do one thing and do it well". Reasons: - So users have less code to audit before they run it. - Extensibility of interface - Clarity of interface (cf. «svn switch --relocate») > ./store-plaintext-password.py realm password [username] > > Which stores/updates the password for the given realm. If username is given > it is stored/updated as well. If there is no cached entry for the specified > realm a new file will be created (and username is a mandatory argument). Passwords shouldn't be passed in argv because the values in argv are visible to other non-root users. > TODO: > - Is the license ok? I stole it from svn.c I assume you're asking about the header format as opposed to the choice of license. Sure, svn.c's header ought to be fine. > - Improve Python code Reviewing. > - Improve documentation - especially on where to find the 'realm' string For one, by running `svn info` interactively and ^C'ing it at the prompt. > - Decide where to put it - I've gone with contrib for now as per Nathan's > and Daniel's suggestions This was a patch submission and my reply contains a review. We may want to move to dev@. > > > > Regarding the FAQ, currently we have [1] "Ahhh! I just discovered that > > > > my Subversion client is caching passwords in plain-text on disk! > > > > AHHH!" That is still applicable to 1.10, but now we need an entry to > > > > answer the opposite question: how to cache the password for svn use > > > > with cron jobs and non-X environments where Kwallet and GNOME-Keyring > > > > aren't applicable, and the particularly annoying case in which the > > > > machine itself has a GUI but the user is logged in via ssh; in this > > > > case the svn client will "freeze" while waiting for password entry in > > > > an inaccessible GUI window; I think this would occur with Kwallet, > > > > GNOME-Keyring, and macOS's Keychain.) > > > > > > > > But, as there doesn't seem to be one well-established way to handle > > > > this, other than just storing the password on disk, would the new FAQ > > > > entry say just that? Do we have any other concrete suggestions? > > > > If a cron job needs authentication, its credentials need to be stored > > somewhere, either in plaintext or in "as good as" plaintext. I think > > storing the passwords in unobfuscated plaintext was a deliberate > > decision, informed by CVS's design choices in this regard, but I wasn't > > around in the early days. > > > > On the other hand, for GUI-less environments and for headful > > environments ssh'd into, we should simply document the capabilities and > > workarounds of the libraries we use (possibly by pointing to those > > libraries' documentations). > > > > Suggestion for new FAQ entry: > [[[ > Ahhh! I just discovered that my Subversion client is NOT caching passwords > in plain-text on disk! AHHH! Having two entry titles that differ only by a "not" isn't a good idea. > Calm down, take a deep breath. > > This is the opposite of the previous question. After changing the compile > time default to not store passwords in plain-text there has been a number > of requests in the mailing lists to store the password. "A number of requests on the mailing lists" seems like too fine a level of abstraction. I think the context basically needs to be "The default is X but you may want Y; here's how to do that". > If you understand the security implications, you have ruled out other s/, you/, / > alternatives and still want to cache your password in plain-text on disk s/ and/, and/ > you can use the script > https://svn.apache.org/repos/asf/subversion/trunk/contrib/client-side/store-plaintext-password-py > to store the password in the directory which contains the cached passwords > (usually ~/.subversion/auth/). The script can also be used to list any > existing passwords stored in plain-text. This should just point to `svn auth`, surely? Should this explicitly say to run the script with --help to get its usage message? > ]]] > > I'm also suggesting to change the existing FAQ entry (Ahhh! I just > discovered that my Subversion client is caching passwords in plain-text on > disk! AHHH!) to mention the changed compile time default since 1.12 to not > store plain-text passwords: > > [[[ > s/Otherwise, the client will fall back/Otherwise, the client can fall back/ > > Since svn 1.12 the compile time default has been to disable storing new > passwords in
Re: A more permanent home for the add-a-password-to-a-cached-username script? (was: Re: using svn cli with --non-interactive (in scripts) securely, without exposing password)
Daniel Sahlberg wrote on Tue, Feb 23, 2021 at 16:50:07 +0100: > Den tis 23 feb. 2021 16:40Nathan Hartman skrev: > > I think it's a good candidate for contrib (though it might be better > > to port it to portable Bourne shell first). > > > > Would a Python version be useful? Porting isa rewrite and as such may introduce bugs. Porting will increase the number of developers able to maintain the script. Porting may let some users run the script without installing zsh first. However, it's perhaps worth noting (in any eventual FAQ entry as well) that running the script doesn't require chsh(1)ing to zsh: there's nothing preventing bash users from installing zsh just so as to run scripts written in it. Porting wouldn't be a matter of translating line-by-line, since the «select» builtin (http://zsh.sourceforge.net/Doc/Release/Shell-Grammar.html#index-select) doesn't have a direct equivalent in other languages. For translating to sh(1) specifically, the use of arrays would also be an issue. As to contrib/, even though it's deprecated, I don't have a better idea. (tools/ is probably not appropriate, unless more devs speak zsh than I know.) > > Regarding the FAQ, currently we have [1] "Ahhh! I just discovered that > > my Subversion client is caching passwords in plain-text on disk! > > AHHH!" That is still applicable to 1.10, but now we need an entry to > > answer the opposite question: how to cache the password for svn use > > with cron jobs and non-X environments where Kwallet and GNOME-Keyring > > aren't applicable, and the particularly annoying case in which the > > machine itself has a GUI but the user is logged in via ssh; in this > > case the svn client will "freeze" while waiting for password entry in > > an inaccessible GUI window; I think this would occur with Kwallet, > > GNOME-Keyring, and macOS's Keychain.) > > > > But, as there doesn't seem to be one well-established way to handle > > this, other than just storing the password on disk, would the new FAQ > > entry say just that? Do we have any other concrete suggestions? If a cron job needs authentication, its credentials need to be stored somewhere, either in plaintext or in "as good as" plaintext. I think storing the passwords in unobfuscated plaintext was a deliberate decision, informed by CVS's design choices in this regard, but I wasn't around in the early days. On the other hand, for GUI-less environments and for headful environments ssh'd into, we should simply document the capabilities and workarounds of the libraries we use (possibly by pointing to those libraries' documentations). > As for the script, IIRC there was a need for the username (?) to be cached > before running the script. Where should that be stored? In the md5(realm)-named file in ~/.subversion/auth/ which the script ed(1)s. (The --no-auth-cache flag and the store-auth-creds configuration knob may affect whether the username is saved to disk.) The easiest way to populate that is to run `svn info` on the URL and authenticate manually. A script that takes a URL/username/password and inserts that tuple into the cache would be nice, of course, but it'd need to compute the realm string of the URL. Cheers, Daniel
A more permanent home for the add-a-password-to-a-cached-username script? (was: Re: using svn cli with --non-interactive (in scripts) securely, without exposing password)
Nathan Hartman wrote on Mon, Feb 22, 2021 at 14:36:58 -0500: > In a recent discussion on our dev mailing list, there is an example > shell script (for zsh) that saves a password file. See [2] and note > that there were a few corrections to the script so be sure to use the > latest version in that mail list thread. > > [1] > https://subversion.apache.org/docs/release-notes/1.12.html#client-server-improvements > > [2] > https://lists.apache.org/thread.html/r0eef40236aeddd1db18bc7882454dd3b18bcd721d8fd8c9e21aca52a%40%3Cdev.subversion.apache.org%3E > > I hope the above is helpful; feel free to ask as many questions as you > need to, or propose improvements to the above-mentioned script or > Subversion itself. We have gotten quite a few questions about this and I'm starting to wonder if that script deserves a home more permanent than the end of a random dev@ thread whose subject line contains "WTF" and "?!"; e.g., perhaps that script should be linked from the FAQ or the release notes. Cheers, Daniel > it has been frustrating for anyone who uses svn as part of cron jobs > in non-X environments, where the available encrypted password stores, > Kwallet and Gnome-keyring, aren't much help, and GPG-Agent doesn't > persist the passwords indefinitely. We would be really grateful if > someone could propose a solution that works well in these scenarios > while alleviating people's concerns about storing passwords on disk in > plaintext. > > Cheers, > Nathan
Re: subversion 1.14.1: could not build python bindings (1.41.0 is Ok)
Nathan Hartman wrote on Tue, 16 Feb 2021 17:15 +00:00: > You can generate a diff to view the changes with: > > $ svn diff \ > https://svn.apache.org/repos/asf/subversion/trunk/subversion/bindings/swig/INSTALL\ > https://svn.apache.org/repos/asf/subversion/tags/1.14.1/subversion/bindings/swig/INSTALL For the archives, here it is again with a peg revision for future-proofing: $ svn diff \ https://svn.apache.org/repos/asf/subversion/trunk/subversion/bindings/swig/INSTALL@r1886601 \ https://svn.apache.org/repos/asf/subversion/tags/1.14.1/subversion/bindings/swig/INSTALL
Re: Migrate from StarTeam to Subversion
Mark Rutz wrote on Mon, 01 Feb 2021 19:21 +00:00: > Does anyone have procedures or tools for converting Micro Focus > StarTeam repositories to Subversion? We would like to convert the main > branch with history for each repository. For converting a single branch from any external VCS, the fallback strategy (if no preëxisting tool is available) is a plain old "checkout each commit in the history in sequence and re-commit it" loop. I hope you won't need to resort to it in your case, but I don't think we have a write-up of it, so I'll outline it here. The basic idea is (using svn syntax for both the old and new VCS's, since that's the only syntax everyone on this list is guaranteed to speak): % mkdir wc % $oldvcs checkout -r0 $oldURL wc % svnadmin create foo % printf '#!/bin/sh\ntrue\n' > foo/hooks/pre-revprop-change % chmod +x foo/hooks/pre-revprop-change % svn co file://$PWD/foo wc --force # --force suppresses a "Directory not empty" error % cd wc % # At this point, wc/ is a working copy of both $oldvcs and svn, both of them at r0 (an empty tree). % i=0 % while [ $i -le `$oldvcs info --show-item=revision ./` ]; do svn cleanup && # purge pristines $oldvcs up -r $((++i)) && svn-addremove && svn revert -R ./.$oldvcs && svn ci -m "Re-committing r$i" && for propname in svn:date svn:author ; do svn ps --revprop -r $i -- $propname "$($oldvcs pg --revprop -r $i -- $propname)" ; done done % where svn-addremove(1) should be a script that does «svn add» for all files that are '?' in `svn status` (= unversioned) and «svn rm» for all files that are '!' in `svn status` (= missing), like the eponymous hg(1) subcommand. There's an implementation in https://svn.apache.org/repos/asf/subversion/branches/addremove, but I don't know its status, and in any case it may be easier to find a script implementation or write one. There is no requirement to use file:// syntax; I only did that to make the example self-contained. Cheers, Daniel
Re: subversion on openbsd . is ssl-authority-files 1.14 broken ?
Sven F. wrote on Wed, 03 Feb 2021 00:41 +00:00: > so it probably 'tracked' down the problem into the lib ssl interraction > (serf?). libserf handles http/https. Subversion doesn't use SSL libraries directly for http. > I have no idea how to contact the serf developper, or how to test serv > alone https://serf.apache.org/ > i ignored : > buckets/bwtp_buckets.c:236:42: warning: format specifies type > 'unsigned long long' but the > argument has type 'apr_size_t' (aka 'unsigned long') [-Wformat] >ctx->channel, calc_header_size(ctx->headers), > > but it's probably very problematic .. long long is not long... Just add the cast to unblock investigating the other issues? There aren't any range issues with casting UL to ULL.
Re: Locked file - Windows - filesystem number instead of path is shown in error message
Martin Rupp wrote on Sun, 03 Jan 2021 03:27 +00:00: > What is the "filesystem '75510a91-dada-4f23-b575-03fe34008164'"? > > It seems it is the id for the svnserver (svnserve). > It's the repository UUID. The error message gives the repository UUID because there's no reason for the client to know the repository's disk path on the server. It appears in `svn auth` output because svnserve uses the UUID as the realm string when no realm is specified in svnserve.conf. Changing the realm string would invalidate client's cached credentials to that repository URL, since the cache is keyed on the "<$URL> $REALM" string, as shown by `svn auth`. > But how can I see in which path (working copy) on which computer the > file is locked? As mentioned, have clients supply this information in the lock message. You can enforce that with a pre-lock hook.
Re: Why is svn not saving my password in Pi-OS?
Nathan Hartman wrote on Tue, 29 Dec 2020 13:45 -0500: > I don't think it's Subversion that's broken in this case. It's an issue of > configuration. The problem of the GUI dialog while accessing a machine via > SSH is a known complication that affects other software too. It will happen > with any software that tries to use these credential caches. I don't have a > RPi that I could experiment with but maybe I can setup something reasonably > similar to try to reproduce this... Do the tools in this thread rely on environment variables to find their GUI? If they do, something like «tmux -L test start \; new \; detach» followed by «env -u DISPLAY tmux -L test attach» and therein «mtr 127.0.0.1», might work. ($DISPLAY would remain set in the environment of the $SHELL instance inside the tmux session; mtr(1) will run in a window if $DISPLAY is set, and in the terminal otherwise.) In the example the issue is that $DISPLAY is a per-process singleton but its desired value depends on which tmux client is currently active. The right answer is obviously to invent a way to query the $DISPLAY envvar via terminfo: that would let tmux(1) arbitrate between different clients' values of the envvar and supply, at any given time, the then-correct value… :P Cheers, Daniel
Re: Why is svn not saving my password in Pi-OS?
Bo Berglund wrote on Tue, 29 Dec 2020 19:00 +0100: > But for item 2 I drew a blank... > The Collabnet keyring_tool does not exist on my system: > $ which keyring_tool > (no output) > > So the instruction to "Create a keyring for your password." > by the operations below cannot be executed: > > keyring_tool --create=svn > or > keyring_tool --setdef=svn > > And apt is no help either: > > $ sudo apt install keyring_tool > Reading package lists... Done > Building dependency tree > Reading state information... Done > E: Unable to locate package keyring_tool Note that «sudo apt install svn» would fail too, even though svn(1) _is_ likely packaged in your distro. To find a package that contains a file of a given name, there's apt-file(1) from the eponymous package. (But there's no guarantee that the collabnet thing is packaged in the repositories your apt(8) uses…) > Seems such a pity that a tool like Subversion can be so broken by some > "upgrade" that it becomes unusable...
Re: mod_dav_svn and symbolic links
Ryan Schmidt wrote on Thu, 17 Dec 2020 09:59 +00:00: > > > On Dec 16, 2020, at 13:13, M.Eng. René Schwarz wrote: > > > I am using mod_dav_svn with Apache2 in combination with a SVN repository > > containing symbolic links. When a URL pointing to a symbolic link is > > accessed, the web server does not follow this link, but provides a text > > file with the content "link [path to file or directory]". > > > > I was wondering if there is a possibility to make mod_dav_svn actually not > > providing this text file but the contents of the linked file directly? > > > > Background: > > An installer is downloading several files for installation from the > > repository over the web server whereas the flexibility to specify a certain > > repository revision via the "!svn/bc/" special syntax is used, without the > > need for the installer to be aware of the SVN repo itself. For the > > installer, this address is just an ordinary URL. This worked well for a > > couple of years, but now symbolic links have been used in the repository > > which is causing the trouble for the installer now as it gets not the > > actual file, but the text representation of a symbolic link. > > As far as I know, mod_dav_svn does not have this capability. +1 > mod_dav_svn is a server process, and symlinks are a client-side-only > feature. The server has no knowledge of symlinks and just stores the > files and their properties. The Subversion client stores symlinks in > the repository exactly as you're experiencing: as a text file with the > contents "link [path to file or directory]" and a property > "svn:special" set to the value "*". > > In this case, your installer is acting as the Subversion client, so you > would have to modify your installer code to get it to notice when a > file it's accessing is a symlink and to make it send another request to > get the real file from its actual location. If you want to do it right, > you should consider that there could of course be multiple levels of > symlink indirection, and any of the directories leading up to the file > you want could be symlinks. The need for Ω(1) round trips is a good argument for adding a server-side symlink resolution thing (similar to the ?p= query argument already supported), though we'd have to be careful about dot-dot symlinks that point to above the repository root or into !svn (which is a perfectly valid dirent name to lower layers), symlink loops, etc.. It's possible some repository viewers (ViewVC-like tools) already provide this functionality. Note that !svn and everything under it are implementation details. Cheers, Daniel
Re: svn diff -c does not accept HEAD
Nathan Hartman wrote on Mon, 07 Dec 2020 20:50 +00:00: > On Mon, Dec 7, 2020 at 1:47 PM Yasuhito FUTATSUKI > wrote: > > > > As the error message says, '-c' option accepts only numeric revision. > > It seems it is a kind of bug that the help text is not kind enough. > > A possible rationale is that the HEAD revision could change without > your knowledge (e.g., another user commits something in the meantime) > and you wouldn't get the revision you were expecting. If this were the rationale, we wouldn't support «svn log -r HEAD» either. Use of «HEAD» to refer to any specific revision is inherently racy. Callsites that care about the race condition should handle it explicitly; see tools/dist/release.py:bump_versions_on_branch() for an example. There isn't any conceptual problem with defining «-c HEAD» to mean "Resolve HEAD to a revision number N and then behave as «-r N-1:N» would". That wouldn't be any more racy than any other use of HEAD. We just never did that. (Why? For one, because -c is a lot newer than -r: -c was added in 1.4.0 (sic), in 2006, while -r dates back to CVS. When we added -c, it was implemented directly in the cmdline client's option parser, without any API changes, so it was written to only support numeric arguments.) Note that «svn diff -c HEAD» isn't generally a useful call unless the repository root («^/») is given as the target. > When I want to see the diff of the most recent revision I use 'svn log > -l 1 --diff'. (Note, though, that will be from the BASE revision, not > HEAD.) There's also «svn log -r HEAD:0 -l 1 --diff». Cheers, Daniel
Re: svn log says text-mods="true" but there are no diffs
Vincent Lefevre wrote on Fri, 04 Dec 2020 01:08 +00:00: > I get the following: > > $ svn log --xml -v -r 1984 https://scm.gforge.inria.fr/anonscm/svn/mpfr > revision="1984"> > vlefevre > 2002-07-23T16:22:08.00Z > > prop-mods="false" >text-mods="true" >kind="file" >action="M">/trunk/mul.c > > Fixed permissions. > > > > > I'm wondering why text-mods="true" while > > svn diff -c 1984 https://scm.gforge.inria.fr/anonscm/svn/mpfr > > shows no diffs. You may be running into this (quoting from svn_fs_contents_changed()'s docstring): * @note svn_fs_contents_changed() was not designed to be used to detect * when two files have different content, but really to detect when the * contents of a given file have changed across two points in its history. * For the purposes of preserving accurate history, certain bits of code * (such as the repository dump code) need to care about this distinction. * For example, it's not an error from the FS API point of view to call * svn_fs_apply_textdelta() and explicitly set a file's contents to exactly * what they were before the edit was made. We have a pair of functions * that can answer both of these questions, svn_fs_contents_changed() and * svn_fs_contents_different(). See issue 4598 for more details. Or maybe that revision was created by a pre-1.0 version of Subversion that had some bug or another.
Re: svn.haxx.se is going away
Nathan Hartman wrote on Tue, 24 Nov 2020 21:27 +00:00: > On Tue, Nov 24, 2020 at 2:56 AM Daniel Sahlberg > wrote: > > Den tors 12 nov. 2020 kl 17:46 skrev Daniel Sahlberg > > : > >> Could ASF provide this server space (basically a VirtualHost)? The archive > >> is about 6.5 GB so it is not a huge amount. > > > > Any thoughts on this? > > I am looking into this; waiting for a reply... In the circumstances — it's Nov 25 and the site says it'll be taken down "in November 2020", not specifying a date — I'd say, better ask forgiveness than permission. Let's go ahead and grab all the data we need to stand up the site (we have the mboxes, but not the mapping of *.shtml files to message-id's, nor any of the HTML/CSS/images), and if possible, also set it up (on svn-qavm.a.o or wherever) to ensure we've got everything and to prepare for a DNS repointing, if Daniel agrees. We can figure out the "paperwork", Puppet PRs, etc., later. I'd say the highest priority is to save the mapping of .shtml URLs to message-id's (which are available as comments in the source HTML), whether via a recursive wget(1) invocation, or by asking Daniel to run an appropriate grep, or however else. Without that info, we won't be able to preserve old URLs. Maybe there's also a button we can press to sic the archive.org spider on svn.haxx.se. (We can't derive the message<->.shtml mapping from the mboxes we have. I only grabbed mboxes through the transition to ASF; for anything after that point, the order of .shtml files would be the order in which list mails reached haxx.se's MX, and we have no backups of that info.) Cheers, Daniel P.S. Yes, it's a bit https://m.xkcd.com/2337/ of me to refer to both Daniel and Daniel as "Daniel". :)
Re: SVN log-diff crashes if server not available
Ryan Schmidt wrote on Wed, 18 Nov 2020 05:59 +00:00: > > > On Nov 17, 2020, at 17:13, Dirk Helgemo wrote: > > > Right-click a file > > There is no "right-click" in Subversion, which is a command line > program. You seem to be using TortoiseSVN, which is a different product > made by different developers; [therefore,] Note that the entire log cache functionality is a TortoiseSVN thing that doesn't exist in the base Subversion product at all. > you should report the problem to them.
Re: svn.haxx.se is going away
Daniel Sahlberg wrote on Thu, 05 Nov 2020 11:16 +0100: > Den ons 4 nov. 2020 kl 22:32 skrev Nathan Hartman >: > > > On Wed, Nov 4, 2020 at 3:32 PM Mark Phippard wrote: > > > > > > Just a general fyi ... I went to https://svn.haxx.se/ today to search > > the lists and noticed there is a banner on the site saying it is going > > offline forever soon. > > > > > > I am not sure what the ramifications will be as I know there are a lot > > of historical links in the docs and site but I guess it is what it is. > > > > Daniel (danielsh) has been trying to get Infra to import the material > > from pre-2009 (pre-migration to ASF) into lists.apache.org to avoid > > losing the archives from the earliest period of development, which > > arguably contain some of the most important development information. > > > > See the discussion here: > > > > https://lists.apache.org/thread.html/r97c9c5208af706b067fd8e67a7cbe79b37255958bb087bf699b722f8%40%3Cdev.subversion.apache.org%3E > > And https://issues.apache.org/jira/browse/INFRA-20213 > > Possibly it's still mirrored at home.apache.org but I can't check at the > > moment. > > > > Nathan > > > > Would it be considered a good thing if we manage to keep svn.haxx.se > around? Even if Infra would get the old lists imported (I don't know what's > holding them back), there are a bunch of references to the archives in the > source (63 if I'm counting correctly), and in the website (87). > Those in the website should be covered by site/publish/.message-ids.tsv. (See site/tools/ for the generating scripts.) The logic for converting the message-ids into URLs is embedded in [1] (which I have tried to make discoverable, [2], but that seems to have regressed, and I'm ENOTIME to chase it). [1] https://svn.apache.org/repos/infra/infrastructure/trunk/projects/asf-generate-mail-archives-link [2] https://issues.apache.org/jira/browse/INFRA-19422 > I have reached out to Daniel Stenberg and he seems willing to discuss to > point the domain name to another server. I could probably volunteer to keep > the site alive, provided there is an agreement within @Dev this is a good > thing. Or is it better to just do the job and update the sources and > website? We should keep old links working, if possible. Ideally, not only links we happen to have lying around, but also other links (e.g., in people's non-public branches of https://github.com/apache/subversion). There's more than one way to preserve links (redirecting old URLs to new URLs for the same messages; keeping the site online but not updating; keeping the site online and updating, on ASF hardware, e.g., svn-qavm.a.o; etc.). Any and all assistance would be most welcome! > (Daniel S... seems to be a popular name!) It is, yes. And then there are people like danderson, who aren't named "Daniel" but still get in the way of tab-completing Daniels ☺ Cheers, Daniel
Re: svn.haxx.se is going away
Nathan Hartman wrote on Wed, 04 Nov 2020 16:32 -0500: > On Wed, Nov 4, 2020 at 3:32 PM Mark Phippard wrote: > > > > Just a general fyi ... I went to https://svn.haxx.se/ today to search the > > lists and noticed there is a banner on the site saying it is going offline > > forever soon. > > > > I am not sure what the ramifications will be as I know there are a lot of > > historical links in the docs and site but I guess it is what it is. > > Daniel (danielsh) has been trying to get Infra to import the material > from pre-2009 (pre-migration to ASF) into lists.apache.org to avoid > losing the archives from the earliest period of development, which > arguably contain some of the most important development information. > > See the discussion here: > https://lists.apache.org/thread.html/r97c9c5208af706b067fd8e67a7cbe79b37255958bb087bf699b722f8%40%3Cdev.subversion.apache.org%3E > > Possibly it's still mirrored at home.apache.org but I can't check at the > moment. It is — % ssh home.apache.org du -hs /home/danielsh/svn-haxx-se-mirror 245M/home/danielsh/svn-haxx-se-mirror % ssh svn-qavm.apache.org du -hs /x1/svn-haxx-se-mirror 245M/x1/svn-haxx-se-mirror — but I don't know that either of these is backed up, so please someone rsync either of those [they're identical] to their own hardware. Cheers, Daniel
Re: SVN and webdav
Marek Manukjan wrote on Mon, 05 Oct 2020 10:00 +00:00: > I'm not an expert on SVN's HTTP protocol so I don't know if there is > another variant of the Commit command like there are two modes of > Update, but from apache's access log it seems that Commit is done as > one initial POST request, then each file is uploaded as separate PUT > request, and the operation is finalized with MERGE request. Someone > else might want to confirm/disprove this, as it's based purely on > observation of the access log, not in-depth knowledge. This is how commits are performed over http when the client and server are both sufficiently new. There _is_ another way to perform commits over http, used when either the client or server is older. grep the source for for "create-txn". Cheers, Daniel > To clear up the difference between using SVNAllowBulkUpdates and > KeepAlive, as they control separate things: > * SVNAllowBulkUpdates controls how many HTTP *requests will be done in > total* for Checkout/Update/Switch operation. It doesn't affect Commit, > as it seems it always uses multiple requests > * KeepAlive controls how many HTTP *requests will be done per one TCP > connection* for any operation > From your initial question I can't quite make out which of the two you > want to actually control (not familiar with haproxy), but both should > be achievable using these configuration attributes. > > On Mon, Oct 5, 2020 at 10:49 AM Wokash Wolsku > wrote: > > This is helpful, thank you, however, reading the links especially > > SVNAllowBulkUpdates talks of getting the tree. What directives control > > updates to the tree. The situation I face is that some clients have to up > > load large graphics files, and video, these will be one object and > > presumably report request, on first reading that does not appear to be > > covered by the SVNAllowBulkUpdates. > > > > The KeepAlive seem also a corse way to achieve this... > > > > Thanks again. > > *From:* Marek Manukjan > > *Sent:* 05 October 2020 10:24 > > *To:* Wokash Wolsku > > *Cc:* users@subversion.apache.org > > *Subject:* Re: SVN and webdav > > > > It depends on server and client configuration. See following configuration > > attributes > > * SVNAllowBulkUpdates (On, Off, Prefer) > >* Bulk update means that all files will be received in a single REPORT > > request > >* Non-bulk (Skelta) mode, used by default in newer clients, means that > > each file will use its own GET request > >* see > > https://subversion.apache.org/docs/release-notes/1.8.html#serf-skelta-default > >* see http://svnbook.red-bean.com/en/1.7/svn.ref.mod_dav_svn.conf.html > > * standard Apache configuration attributes KeepAlive, KeepAliveTimeout, > > MaxKeepAliveRequests > >* they control how many HTTP requests can be done in a single connection > > to server > >* https://httpd.apache.org/docs/2.4/mod/core.html#keepalive > > > > On Mon, Oct 5, 2020 at 10:10 AM Wokash Wolsku > > wrote: > >> I am trying to use haproxy to rate control some svn clients which access > >> the SVN repro via svn+https. Some monitoring has thrown up some questions. > >> > >> 1. When for example doing a checkout or commit of a large number of > >> files is this implemented as > >>1. 1 https request or many > >>2. 1 web dav action or many? > >> 2. from the logs of haproxy (and I am by no means an expert) I see only > >> one connect and one https request. > >> I was hopping to rate limit the clients by IP address and thereafter http > >> requests hence to slow down large users so others get a share of the > >> processor. But without the volume of up load being related to http > >> requests I am struggling to see how to implement this. > >> > >> Can anyone offer any advise. > >> > >> Wocash
Re: Serialized hash missing terminator; svnadmin verify passes
[Re-adding the list to Cc. Please keep the list Cc'd by default.] Bruce Cowan wrote on Wed, 23 Sep 2020 11:38 -0500: > Thanks for the reply. I was unsure how to update with a solution that > worked for me. In such cases, a self-reply is fine and appreciated. For bonus points, put "[SOLVED]" in the subject line. > We were using dynamic memory allocation. We set the cache to 1gb and the > error went away. > > Currently : svnInMemoryCacheSize 1048576 > > WIth this and a restart of the apache service we are good. Hmm. Interesting. Did you try restarting the server _without_ bumping the cache size? If so, was the error reproducible at that point? > What does the DAV proxy have to do with this? We are using it on this > server. Our test server didn't have people testing the checkout(just the > application testing we goofed) so we missed this. > > Thanks again for the reply. I would appreciate any additional information > you can provide on why the DAV proxy could be the cause. Setups that use the DAV proxy have two copies of the repository, so it's possible that the copy the user checked out from and the copy you verified aren't the same one. Just wanted to rule that out. Thanks for sharing the workaround. Cheers, Daniel > Bruce > > On Wed, Sep 23, 2020 at 6:33 AM Daniel Shahaf > wrote: > > > Bruce Cowan wrote on Sun, 20 Sep 2020 13:52 -0500: > > > Just recently had a user unable to check out a folder. Going through the > > > logs found: > > > > > > Serialized hash missing terminator > > > Provider encountered an error while streaming a REPORT response > > > A failure occurred while driving the update report editor > > > Failed to parse revprops for r212893 > > > > > > Ran svnadmin verify on repo. No failures. What should I investigate? > > > > Are you using the DAV proxy/mirroring feature? > >
Re: Serialized hash missing terminator; svnadmin verify passes
Bruce Cowan wrote on Sun, 20 Sep 2020 13:52 -0500: > Just recently had a user unable to check out a folder. Going through the > logs found: > > Serialized hash missing terminator > Provider encountered an error while streaming a REPORT response > A failure occurred while driving the update report editor > Failed to parse revprops for r212893 > > Ran svnadmin verify on repo. No failures. What should I investigate? Are you using the DAV proxy/mirroring feature?
Re: svnsync and commit metadata
James Stocks wrote on Thu, 10 Sep 2020 15:01 +0100:
> [...], this wrongly asserts that the mirror is out of
> sync during the brief window when the mirror hasn't caught up.
>
> We would like to find a way of inspecting the time of a given commit
> being successfully synchronised to a mirror as opposed to the
> timestamp of the commit itself. We think that this might be stored as
> metadata in the subversion filesystem.
No, that timestamp isn't stored. However, you could write a simple
post-commit hook to record it.
> Alternatively, is it possible
> to inspect the SVN repo on the mirror and determine the time of the
> most recent successful svnsync?
Yes, it's possible. However, before I say how, I'll stress the
post-commit solution is preferable to the following one.
The following solution relies on implementation details: Check
$REPOS_DIR/{format,db/fs-type,db/format} in this order, then check the
timestamp of $REPOS_DIR/db/current. That file is updated whenever and
only when $REPOS_DIR is committed to (revprop changes don't count).
> I am aware that we could check the
> return code of svnsync running on the master, however, we need to be
> able to check this on the mirror end of the process.
>
What you can do is inspect the svn:sync-* revprops on r0 of the mirror —
svn:sync-lock, svn:sync-last-merged-rev, & svn:sync-currently-copying —
and cross-reference them with `svnlook youngest` on the mirror, and
with whether there's an active TCP connection to whichever IP svnsync
runs on. (The last part is needed in case svnsync exits abnormally,
leaving a stale svn:sync-currently-copying behind. The next «svnsync
sync» run will DTRT in such cases, but your monitoring will
nevertheless want to detect them.)
> We would appreciate advice anyone has with this situation, or tips
> from anyone who has dealt with anything similar.
Yet another solution is to push commit notifications (using svnpubsub,
post-commit emails, or anything else) to the the host that runs svnsync
(in your case it's the master, but svnsync can also be run from the
mirror or from a third host) and _then_ monitor the exit code of svnsync.
Cheers,
Daniel
Re: [ISSUE] Cancelling svn checkout leaves an open file handle
[ Moving to dev@ since earlier replies in the thread moved there. Please un-Cc users@ from replies. ] Uroš Jovanović wrote on Tue, 08 Sep 2020 11:31 +0200: > Hi Daniel, > > Not sure what are "relevant pools", I am not really too familiar with SVN :) > The C API manages lifetime of various resources — primarily memory, but also file handles — via the apr_pool_t abstraction. Nearly every API function takes at least one pool parameter. Given that you mentioned closing the TortoiseSVN window is a valid workaround for that version of the issue, I guess the file handle would be closed once the pools passed to the checkout API are cleared/destroyed, without requiring the process to be exited or low-level system calls to be resorted to. However, I don't know what the SharpSvn equivalent of clearing/destroying a pool or destroying a client context is. That'd be a question for the SharpSvn maintainers. (SharpSvn is a third party project, not part of the core Subversion project.) Cheers, Daniel > Best regards, > Uros > > > On Tue, Sep 8, 2020 at 12:51 AM Daniel Shahaf > wrote: > > > Uroš Jovanović wrote on Mon, 07 Sep 2020 19:25 +0200: > > > when using SharpSvn in context of a larger > > > application, this would mean the user needs to close the entire > > application > > > because one unmanaged file handle was left alive. > > > > Have you tried clearing the relevant pools? > > > > Could you help debugging this? (E.g., write a reproduction recipe or > > a test case or track down where in the code the file gets opened.) > > > > Cheers, > > > > Daniel > >
Re: [ISSUE] Cancelling svn checkout leaves an open file handle
Uroš Jovanović wrote on Mon, 07 Sep 2020 19:25 +0200: > when using SharpSvn in context of a larger > application, this would mean the user needs to close the entire application > because one unmanaged file handle was left alive. Have you tried clearing the relevant pools? Could you help debugging this? (E.g., write a reproduction recipe or a test case or track down where in the code the file gets opened.) Cheers, Daniel
Re: COTS : [urgent] : Information required on security-related patches under APACHE GROUP
Subscribe to the announce@ mailing list. https://subversion.apache.org/mailing-lists
Re: svnmucc: new commands: add and modify/update
Daniel Sahlberg wrote on Fri, 24 Jul 2020 05:53 +00:00: > Den fre 24 juli 2020 01:46sebb skrev: > > I am suggesting that 'add' functionality could be added to svnmucc itself. > > This would make it more versatile, especially for use in shell scripts. > > Unless I'm mistaken, add is a purely local action within a working > copy. And the point with svnmucc is that you work with a repository > without having a wc. How should add work with svnmucc? It would work exactly like «svnmucc put $dirent $URL», except that it would fail if $URL already exists, similar to open(O_EXCL) in C.
Re: Searching entire repository for a file (fastsvncrawler)
Stefan Sperling wrote on Thu, 23 Jul 2020 17:22 +0200: > Some servers will advertise the SVN version on pages which can be visited > with a web browser. But this depends on the server's configuration. You may > have to ask the administrator to be sure about the server's exact version. > > If the server supports the feature it will send an "svn/list" DAV header. > The full capability string to look for in a trace of an SVN HTTP session > would be: "http://subversion.tigris.org/xmlns/dav/svn/list; > I'm guessing the equivalent svnserve capability is called just "list". svnserve capabilities are printed as soon as a TCP channel is established, in the server's greeting. > I suppose the easiest way forward might be to simply try it out and see > how long it takes.
Re: svnmucc --revision 0 no longer works when creating a file
sebb wrote on Thu, 23 Jul 2020 11:18 +0100: > On Thu, 23 Jul 2020 at 01:17, Daniel Shahaf wrote: > > You can either parse stderr despite this complication, or use the > > API directly, in which case you'll sidestep this complication entirely > > (you'll get just one integer, rather than two). > > What API are you referring to here? The C API or the various language bindings, bypassing the cmdline client.
