Re: [External] : Re: Can't connect to github anymore

[Trent Fisher]

On 1/14/2024 2:52 PM, Nico Kadel-Garcia wrote:

On Sun, Jan 14, 2024 at 2:27 PM sean  wrote:

On 2024-01-13 16:11, Nico Kadel-Garcia wrote:


There are not many compelling reasons
to use Subversion anymore, except the ability to check out only
subdirectories from a branch and the insistence that a single central
repository is the only source of truth.

The ability to `svn lock` files is very useful if your repo has a lot of
non-mergeable files, like say MS Office documents.

I have never once found that feature to be useful since I first used
Subversion back around 2001.  Mind you, I'd treat Word documents as
binaries objects and not consider them suitable for incremental
changes in a source control system.


There are numerous documentation teams who I have worked with over the 
years who would differ with this. Version control is essential for their 
files (they may not be able to directly compare them, but they can tell 
when changes were made and who made them), and having locking is the 
only way to make this work (which is one of the reasons why Git will 
never work for them).


However, I would agree with you that binaries generated from some other 
source document is not appropriate for version control (e.g. checking in 
PDF versions of documents), but Word documents are the primary source 
documents, therefore needing version control.


Before I got these teams on to SVN, they were storing everything in a 
shared folder with numerous old copies of each document laying around 
(e.g. foo.doc, foo.doc.old, foo.doc.1jun2013, foo.doc.bak, etc), which 
is an accident waiting to happen.

Re: Disable svn service temporarily

[Trent Fisher]

On 3/14/2023 8:52 AM, Felix Natter wrote:


Dear subversion community,

I have setup an ALMA Linux 8 (~RH8) Server, to which I will migrate some
repositories from a Ubuntu 20.04 system. During the migration phase I 
would
like to disable the svn (svn+ssh://) service temporarily on the Ubuntu 
20.04

system.

Is there any easy way to do it? I would like to avoid modifying conf/* for
all of the ~100 repositories, making deep changes to the system and
shutting down the ssh service (because I need to pull the dumps to the new
system).

Later, I would also like to have this "maintenance mode" on the target 
ALMA8

system.

Do you have an idea?



You could move the "svnserve" command aside, that would block all 
svn+ssh access, but that could break a lot of other things, depending on 
your setup.


You could replace it with a shell wrapper which blocks access in certain 
situations, something like this:


if [ "$SSH_CONNECTION" -a -f /etc/BLACKOUT ]; then cat /etc/BLACKOUT; 
exit 1; fi


exec svnserve.real "$@"

So if they are coming in from SSH and you have a file called 
/etc/BLACKOUT that file would be sent to users (downtime notice). You 
might want to exempt the repository owner, so that any replication or 
migration efforts still work.


note: that script is off the top of my head, poorly written and 
untested; but it should give you the idea.


trent...

Re: [External] : Re: svnsync: E120106: ra_serf: The server sent a truncated HTTP response body.

[Trent Fisher]

I have seen this sort of thing, and I think the cause is that it is 
taking SVN a long time to put together the transaction and, during that 
time, no data is going through the HTTP pipe and the connection times 
out.  I am pretty sure the Apache setting to change here is "Timeout".


On 11/14/2022 8:58 AM, Daniel Sahlberg wrote:

Den mån 14 nov. 2022 kl 13:31 skrev JITHIN K :

Hello Team,

I use Subversion 1.13 in Ubuntu 20.04.5 LTS and sync a
repository size of 300GB to a mirror server ( same version of SVN
and OS ).

I get the following warning svnsync: E120106: ra_serf: The server
sent a truncated HTTP response body every time ( I had to take a
dump of specific revisions and load it in the mirror server ). 
Did anyone face this problem while in sync? Is there any solution?


Do you see any common pattern with the revisions causing trouble? 
Possibly if they are very large or contain a lot of files.


This was mentioned once before on the list [1]. I don't see if the 
user found a way around it, but he mentions that the troublesome 
revisions were large imports.


Kind regards,
Daniel

[1] https://lists.apache.org/thread/cmc872c221o0r5mvhk16lcjfnd4xwtl1 



I have seen this sort of thing, and I think the cause is that it is 
taking SVN a long time to put together the transaction and, during that 
time, no data is going through the HTTP pipe and Apache thinks the 
connection is idle and kills it.   I have seen this both with svn and 
user checkins, in the latter case the checkin contains thousands of 
changed files.  I was able to see this in action by looking at what 
files were changing within the (replica) repository itself and I could 
see files in "transactions" (or was it "txn-protorevs"?) and I could see 
them slowly growing as each change was written into them.


I am pretty sure the Apache setting to change here is "Timeout". There 
may be other settings I am forgetting about, it has been some time since 
I have seen this.


It is possible there may be some filesystem tuning which could speed up 
the file writes, but I have no direct experience with that.

Re: [External] : Re: What's the status of changing UUIDs of repos in working copies?

[Trent Fisher]

On 7/13/2022 9:25 AM, Daniel Shahaf wrote:

Thorsten Schöning wrote on Mon, Jul 11, 2022 at 16:06:11 +0200:


Do you have any tips for the easiest possible change of the UUIDs?

It'll be a lot easier to set the repository's UUID to the one expected
by clients.  See «svnadmin setuuid».


This is what I do, I have svnsync replicas at another site for disaster 
recovery, and I change the UUIDs to match so that, in a disaster, I can 
flip the DNS alias and they can keep working transparently (of course, 
making the replica read-write is also needed, but that's a different, 
more complicated, subject).


trent...

Re: svnadmin lslocks ambiguity

[Trent Fisher]

On 1/6/2020 1:42 PM, Daniel Shahaf wrote:

Trent Fisher wrote on Mon, 06 Jan 2020 18:22 +00:00:

This seems like a bug in lslocks, why would it not list *all* the locks?

If you run fsfs, it might be 
https://urldefense.proofpoint.com/v2/url?u=https-3A__subversion.apache.org_issue3750=DwIBAg=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE=ZbaTjKb7ivmk5rbWkuwu4Mqvrh_gmetmbEk2XMe0cVs=HvcvPbSptrAq78THvePuzylhdDLG4ABfI9Y5NXg15ac=c8ls12Aj8kxq6gQ4jXAltM3ipA4nFCLssJk0XndlTuU=
 .


So, from that, it sounds like copying a repository could result in the 
locks data structures in FSFS becoming inconsistent?  This would make 
sense as I am working with an rsync'ed copy of a repository (though the 
bug seems to be saying that even hotcopy can end up with this problem)


Is there a way to either fix the lock database to be consistent, or just 
to clear it?

svnadmin lslocks ambiguity

[Trent Fisher]

I just noticed something odd about "svnadmin lslocks"... perhaps a bug?

The (short version) backstory is that I am trying to clear all locks 
from a repository (they are all stale) with something like "svnadmin 
lslocks repos | xargs svnadmin rmlocks repos" (the exact script is more 
complicated, but you get the idea).  That seemed to work, but I still 
got errors due to the locks.


The svnadmin lslocks command listed no locks, but if I specified the 
pathname from the error message I get, the lock is, indeed, present:


$ svnadmin lslocks /scm/svn/repos/

$ svnadmin lslocks /scm/svn/repos/ '/some/file/omitted.foo'
Path: /some/file/omitted.foo
UUID Token: opaquelocktoken:52effd7a-fadc-11e9-9fa4-db671f4521f1
Owner: somebody
Created: 2019-10-30 06:13:00 + (Wed, 30 Oct 2019)
Expires:
Comment (1 line):
something

This seems like a bug in lslocks, why would it not list *all* the locks? 
  Or am I doing something wrong?


This is SVN 1.10.4 on Linux

++thanks,

trent...

Re: Perforce to Subversion

[trent . fisher]

On 06/14/2018 10:24 AM, Brady Cottam wrote:

Can anyone tell me if you can import data from Perforce into 
Subversion with any ease?


Thanks
Brady

Yes, the converter at http://p42svn.tigris.org/ does a good job. I'm not 
sure any conversion of any significance can ever be done "with ease", 
but I have done extensive work on this over the years to make it as easy 
as I could.


But keep in mind that it can take some time depending on the amount of 
history and the size of each rev;  e.g. my last conversion took a couple 
of months to do over 100k revs.


Let me know if you have any questions about it.

trent...

Bug with svnadmin dump --deltas??

[Trent Fisher]

I think I found a bug with svnadmin dump --deltas.  It all started with 
an svnsync replication which got "svnsync: E210008: Error while 
replaying commit".  I tried doing an svnrdump dump of the failing 
revision from the remote repository, and tried to load that and got this 
error.


svnadmin: E200014: Base checksum mismatch on 
'/blah/blah/blah/META-INF/MANIFEST.MF':

   expected:  91130ebbf7425816cc82ad08be4bb128
 actual:  80d4dede1743346affe4fe5e52cd280e

But then I tried doing an svnadmin dump via ssh, and that loaded 
cleanly.  Looking at the dump file from each, I noticed that the 
svnrdump had deltas and that the expected checksum, shown above, matched 
the 'Text-delta-base-md5' line in the dump file not the 
'Text-content-md5' line.  So then I tried doing an svnadmin dump 
--deltas, and it produced the exact same dump output as svnrdump. This 
makes me think that something about the --deltas output, or the 
interpretation of it at load time, is incorrect.  I'm not sure how to 
track it down to a reproducible case, any suggestions would be welcome.


My servers are both Linux OEL (Oracle's RHEL variant), both with 
Subversion 1.8.15.


++thanks,
trent...

Re: Unable to parse reversed revision range

[Trent Fisher]

On 02/11/2015 04:31 AM, Philip Martin wrote:

Trent Fishertrent.fis...@oracle.com  writes:

Is this a known problem (fixed since 1.8.9) or is there a workaround?

Issue 4414 applies:
http://subversion.tigris.org/issues/show_bug.cgi?id=4414

Your particular case is fixed by r1643074 and proposed for the next 1.8:
...


Many thanks!

However, in the mean time, I'll need to correct that svn:merginfo 
property so it is not invalid.  Am I correct that the only way to do 
this is via a dump/filter/load sequence?


++thanks,
trent...

Unable to parse reversed revision range

[Trent Fisher]

I am doing a dump/load of a repository, and ran into an odd (fatal) 
error on the load:


 Started new transaction, based on original revision 209
svnadmin: E200020: Unable to parse reversed revision range '29584-200'

I figured this would be easy enough to fix by adding 
--bypass-prop-validation to the load, but I got the same error.  I 
looked in the dump file, and sure enough, the rev numbers are backwards 
as stated.  However, I don't understand why load would fail.  I am just 
trying to do some repository migrations and I could care less whether 
those properties are correct or not, that's the users' problem (good 
attitude, eh? :)  Shouldn't --bypass-prop-validation cover this 
validation as well?


I thought about correcting the property, but since it is a versioned 
property, I'm not sure how it can be fixed (without doctoring the svn 
dump file).


I tried this with both SVN 1.6.11 and 1.8.9.  But both got the exact 
same error.


Is this a known problem (fixed since 1.8.9) or is there a workaround?

++thanks,
trent...

svnserve errors in the system logs

[Trent Fisher]

I just discovered tons of spam in /var/log/messages on one of my servers 
(Linux RHEL 5.8 with Subversion 1.8.9) :


Sep 17 06:30:41 adc4110305 svnserve: sql_select option missing
Sep 17 06:30:41 adc4110305 svnserve: auxpropfunc error no mechanism 
available
Sep 17 06:30:41 adc4110305 svnserve: auxpropfunc error invalid parameter 
supplied


This error pops up everytime someone accesses a repository via svn+ssh.  
It doesn't seem to hinder functionality.  I can reproduce this simply by 
running ssh adc4110305 svnserve -t (the error pops up before I can 
type anything).


From what I have gathered from my internet searches, it sounds like 
Cyrus SASL is getting involved.  Though I don't understand why, as I 
thought the tunnel mode was supposed to use the pre-authenticated 
username.  I tried messing with /etc/sasl2/svn.conf but only managed to 
make it worse.  One person suggested (on this very mailing list 5 years 
ago) removing the relevant cyrus-sasl packages, but that doesn't seem 
like a good idea.  What's the correct way of fixing this?


I should note that I built Subversion myself, so if I need to rebuild 
with some different config options to disable this behavior, I can do so.


++thanks,
trent...

Error while replaying commit

[Trent Fisher]

I have an svnsync replica which is running into this error when I sync:  
svnsync: E210008: Error while replaying commit.


I was able to work around the problem by doing an svnadmin dump of the 
troublesome revision and loading it on the other server, then tweaking 
the svnsync properties.  However, I decided to spend a bit of time 
trying to dig down to the root cause (since this error pops up 
periodically).


I am doing the svnsync via an svn+ssh url, doing the svnsync as a pull, 
and I am running as the owner of the repository (on both hosts), so 
there should be no permissions issues.  Both machines are running SVN 
1.8.9 on Linux.  To narrow it down to whether the problem is on the 
sending or the receiving side (the error message implies the latter), I 
ran svnrdump:


svnrdump dump --incremental -r511 svn+ssh://scmadm@somehost/some/svn/repos
svnrdump: E210008: Error while replaying commit

Ok, so I narrowed it down to being on the sending side. I ran svnrdump 
on the server itself using a file: url, and that worked. Then I brought 
up an svnserve process and tried using an svn: url, and that worked.  So 
it seems that SSH is getting in the way somehow.  I ran the first 
svnrdump with --config-option config:tunnels:ssh='$SVN_SSH ssh -vvv' 
but couldn't see anything wrong from what I could decipher.


I need to move on to other tasks for now, but I was hoping that someone 
may have some insights as to what might be going wrong here, or at least 
some pointers as to what to try next.  It would be awfully helpful if 
the error message actually said what was wrong and there were a way to 
get more verbose/debugging output from svnsync and svnrdump.


++thanks,
trent...

Bogus HOME causes segfault

[Trent Fisher]

I know this probably falls into the Doctor, it hurts when I do this 
category, but I found a probable bug in Subversion 1.8.5 where an 
invalid HOME directory results in a segfault.  For example:


$ HOME=/dev/null svn --version
svn: warning: W20: Can't open file '/dev/null/.subversion/servers': 
Not a directory

Segmentation fault (core dumped)

This does not happen with versions 1.7.8 or 1.6.16 (the other two I had 
handy).  On those, it gives me the same warning, but does not segfault.  
I have yet to try 1.8.8 as I don't have it installed anywhere yet.  This 
also happens with an unreadable directory (i.e. /root), but it does not 
happen if given a nonexistent directory (i.e. /foo/bar)


Why do I care?  The php script with repos-web does an svn log, and the 
HOME environment variable ends up containing /root which is not 
readable to the repository owner.  I have implemented a workaround, but 
it seems to me that a segfault is never a good answer, even if I am 
doing something stupid :)


++thanks,
trent...

Empty repository root via web...

[Trent Fisher]

I am working to upgrade my servers to SVN 1.7.8 and just ran into a 
weird issue.  If I look at the repository via the web, it shows up as 
empty.  At first I thought it was something with the repos-web 
stylesheet I'm using, but looking at the page source I see this:


svn version=1.7.8 (r1419691)
 href=http://subversion.apache.org/;
  index rev=119763 path=/ base=foobar
  /index
/svn

But if I use that exact same URL on the command line, it works as I expect:

$ svn ls https://adc4110316.us.oracle.com/svn/foobar/
README
branches/
tags/
trunk/

Furthermore my WebSVN pages show the repository correctly, so this isn't 
a show-stopper.  If I access a subdirectory it works correctly via all 
three methods.


FYI, here's my apache config (slightly redacted) for that repository 
(this is the exact same config I'm using for my 1.6 servers):


Location /svn/foobar
   DAV svn
   SVNPath /scm/svn/repos/foobar
   SVNIndexXSLT /repos-web/view/repos.xsl
   SVNPathAuthz on
   AuthzSVNAccessFile /scm/svn/repos/foobar/conf/authz
   AuthType Basic
   AuthLDAPURL  ldap://[...]
   AuthzLDAPAuthoritative On
   Require valid-user
   Satisfy Any
/Location

The authz file contains only [/] and $authenticated = r, so I 
certainly should be able to read it.


After a bit more testing, I think I figured it out.  When I access the 
top of the repository, it does not request authentication via the web 
(and, thus, gives me an empty page since anonymous access is not 
permitted), but it does request authentication via the svn client and 
websvn.  But if I access a subdirectory, it requests authentication in 
all three cases and works as expected.  However, my SVN 1.6 server will 
request authentication in all cases.  So this seems to be a change in 
1.7, is it a bug or something that changed in 1.7 and requires config 
file changes?


++thanks,
trent...

Re: Repairing or removing broken revisions

[Trent Fisher]

On 08/14/2012 06:13 PM, Daniel Shahaf wrote:

Trent Fisher wrote on Mon, Aug 13, 2012 at 22:10:57 -0400:

On 08/01/2012 10:49 AM, Johan Corveleyn wrote:

On Wed, Aug 1, 2012 at 2:24 PM, Joachim Sauer s...@gmx.net wrote:

Hello,

I'm currently reworking backups of multiple SVN repositories. In the
process I found out that one of those repositories has three broken
revisions. The problem is that the revision files in the revs
directory for those 3 revisions are of size 0 (those contain the
actual data of the revision, as far as I understand). This means that
I can't use svn dump (as it stops at that broken revision) and I can't
use svnsync.


...

But a broken repository is not desirable and I should attempt to fix
it as much as possible. Losing the information of those three
revisions (and a few related ones, probably) would not be a major
problem, but the repository as a whole should be in a consistent state
(to allow svnadmin dump to work, for example).

Since you know the affected paths, I think one possible way is to do
an svnsync, while excluding the corrupted paths by way of path-based
authz (i.e. make the affected paths unreadable by the svnsync user,
using an authz file on the source repository). After that, re-import
the corrupted files from one of your working copies.

I think everyone will have to re-checkout though, because you'll have
a new repository with slightly altered history. So it wouldn't be safe
to give this new repository the same repos-uuid, and act like it's the
same.

If you search the mailinglist archives, you might find some more posts
about this svnsync recovery trick (excluding broken files).


I had a similar situation, though I only had one damaged revision. I
did a dump in three segments, up to the bad rev, the bad rev and the
remaining revs, something like this (assuming rev 5 is the bad
one):

svnadmin dump -r0:4 /some/repos  p1
svnadmin dump --incremental -r5 /some/repos  p2

Did this actually work?  If the r5 rev file is 0 bytes long, it
should fail immediately.  (The same is true for the r5 revprops
file, but for a different reason.)



Yes, it did.  I guess it depends on the sort of corruption there is.  In 
my case, the dump generated a truncated, invalid dump file: It cut off 
right before the contents of a file.  So I removed that last header and 
modified the log entry.


But, you're right, the original poster said his files were zero length.  
In that case you'd have to cobble together a fake dump file for that 
broken revision.


The rest of the process would stand, however.

trent...

Re: Repairing or removing broken revisions

[Trent Fisher]

On 08/01/2012 10:49 AM, Johan Corveleyn wrote:

On Wed, Aug 1, 2012 at 2:24 PM, Joachim Sauer s...@gmx.net wrote:

Hello,

I'm currently reworking backups of multiple SVN repositories. In the
process I found out that one of those repositories has three broken
revisions. The problem is that the revision files in the revs
directory for those 3 revisions are of size 0 (those contain the
actual data of the revision, as far as I understand). This means that
I can't use svn dump (as it stops at that broken revision) and I can't
use svnsync.

For the backup itself I can still use svn hotcopy (the previous way of
doing a backup), so this will be my workaround.

I was able to find out the affected paths in the repository and
luckily the latest revision of those paths can be checked out without
a problem (so we only lost history, not current data).

But a broken repository is not desirable and I should attempt to fix
it as much as possible. Losing the information of those three
revisions (and a few related ones, probably) would not be a major
problem, but the repository as a whole should be in a consistent state
(to allow svnadmin dump to work, for example).

Is there a way to remove the broken revisions and still keep the rest
of the repository intact? Ideally without requiring everyone to make
new clean checkouts because the repository became incompatible.

regards
Joachim Sauer

P.S.: unfortunately there are no working backups of the broken
revisions, as the corruption seems to be pretty old (older than our
last full backup).

Since you know the affected paths, I think one possible way is to do
an svnsync, while excluding the corrupted paths by way of path-based
authz (i.e. make the affected paths unreadable by the svnsync user,
using an authz file on the source repository). After that, re-import
the corrupted files from one of your working copies.

I think everyone will have to re-checkout though, because you'll have
a new repository with slightly altered history. So it wouldn't be safe
to give this new repository the same repos-uuid, and act like it's the
same.

If you search the mailinglist archives, you might find some more posts
about this svnsync recovery trick (excluding broken files).

I had a similar situation, though I only had one damaged revision. I did 
a dump in three segments, up to the bad rev, the bad rev and the 
remaining revs, something like this (assuming rev 5 is the bad one):


svnadmin dump -r0:4 /some/repos  p1
svnadmin dump --incremental -r5 /some/repos  p2
svnadmin dump --incremental -r6:HEAD /some/repos  p3

Then I manually doctored p2 to indicate that the rev is broken (but 
leave the original comment intact), then load all three into a fresh 
repository (cat p1 p2 p3 | svnadmin load /some/new/repos)


The advantage of this is that the uuid and rev numbers remain the same, 
so existing workspaces will work.  And this would preserve all other 
history of the given file.


I just did this on a production repository two weeks ago, and no 
complaints so far.


trent...

Re: Possible svnsync data loss bug?

[Trent Fisher]

On 06/26/2012 05:02 PM, Stefan Sperling wrote:

On Tue, Jun 26, 2012 at 10:13:50PM +0200, Johan Corveleyn wrote:

On Tue, Jun 26, 2012 at 8:07 PM,kmra...@rockwellcollins.com  wrote:

Any reason svnsync couldn't grow a --fail-on-access-error option?

Sure, I think that could be useful (as long as it's not the default
(backwards compat)).

Would printing a warning not be sufficient?
That would be an improvement, but since it seems like sometimes this 
behavior may be desired or expected, it may be better to always print a 
warning, and bomb out given an extra option like that suggested by 
Kevin, above.

Re: Possible svnsync data loss bug?

[Trent Fisher]

On 06/22/2012 08:17 PM, Nico Kadel-Garcia wrote:

On Fri, Jun 22, 2012 at 5:21 PM, Trent Fishertrent.fis...@oracle.com  wrote:

On 06/22/2012 01:59 PM, Mark Phippard wrote:

On Fri, Jun 22, 2012 at 1:52 PM, Trent Fishertrent.fis...@oracle.com
  wrote:

I just ran into a rather frightening problem.  We do replications via
svnsync for backup purposes.  I just found two of the replicated
repositories which had these entries for *every* version:

  r20927 | (no author) | (no date) | 1 line
  Changed paths:

And svnlook tree shows there is nothing in the repositories.

I believe this matches the behavior you would get if you were using
path-based permissions and the svnsync user did not have read
permission to the paths in the revision.  Could that be the reason?
You could also get this if you did not specify the URL to the root of
the repository.  Any revision that does not touch the path you
specified would just sync an empty revision.


Duh!  Of course.  I just tested both of the possible reasons you gave and,
sure enough, both will produce revs like the one in my first message.  But I
think the only way to get a replica, like mine, with *all* empty revs would
be via path-based permissions.  If we were syncing only a subdirectory of a
repository which we, otherwise, had read access to, the commits outside of
the sync'ed url would have proper usernames and dates, but no files.  (I
tested that too)

I was about to ask if svnsync would get access denied errors... but I did
another test which, I think, answered my own question:  If I run svn log
against a repository to which I have only access to part of, revs in the
areas I don't have access to will show up as (no author) | (no date).
  Presumably svnsync is doing some variant of a log call and then getting
the contents of each rev and faithfully reproduces what seem to be empty
revs.

So, I guess svnsync is arguably doing the right thing.  But even if it were
declared to be incorrect, and a fix were checked in right now, it wouldn't
address any of my existing replicas.   So, I'll have to develop some
auditing for these sorts of things.  Which brings up another question:  is
there any other reason you would ever see (no date) in a log entry?

Ouch. svnsync failing to transfer due to permissions, but leaving
empty logs, is.. That's a silent failure that's going to be really
problematic in the field.

What happens after the permissions are opened? Do the logs get
recovere successfully?


I agree this is problematic.   I might have a total loss on two 
repositories due to this.  I have no idea if or how this could be fixed, 
I've never really delved into the guts of SVN, though maybe I will after 
I take care of my current crisis.


As for later opening up permissions, I don't think that's going to help 
at all with old revisions.  Even though they are empty, svnsync will not 
go back to reconsider them, only newly created revisions would be 
affected.  But if you created a new replica with svnsync and start over, 
it should pick up the revisions correctly.

Possible svnsync data loss bug?

[Trent Fisher]

I just ran into a rather frightening problem.  We do replications via 
svnsync for backup purposes.  I just found two of the replicated 
repositories which had these entries for *every* version:


  r20927 | (no author) | (no date) | 1 line
  Changed paths:

And svnlook tree shows there is nothing in the repositories.

We have never seen any errors while running svnsync, however, these 
replicas have been around so long that most of the logs are long since 
gone, though I did find a recent log entry for the very version shown 
above  (note: the first and last lines are output by my scripts):


  Running command: svnsync sync file:///svn/repos/blah-blah 
--non-interactive

  Committed revision 20927.
  Copied properties for revision 20927.
  No errors, not sending email

It appears that svnsync is failing somehow but never indicating that in 
the output or exit status and, instead, creating empty revisions.


These two repositories are the only two we are replicating from this 
host, and I have yet to find any other repositories in the same state 
(though I am still looking).  So, I guess it could be a bug on the 
remote end, which svnsync is unable to detect.  My machine is running 
SVN 1.6.16 (on a Linux machine running our variant of RHEL 5.8).  I 
can't tell you much about the remote end as it is currently down ;(


I searched through the SVN bugdb but didn't see anything reminiscent of 
this.  Has anyone seen this behavior?


++thanks,
trent...

Re: Possible svnsync data loss bug?

[Trent Fisher]

On 06/22/2012 01:59 PM, Mark Phippard wrote:

On Fri, Jun 22, 2012 at 1:52 PM, Trent Fishertrent.fis...@oracle.com  wrote:

I just ran into a rather frightening problem.  We do replications via
svnsync for backup purposes.  I just found two of the replicated
repositories which had these entries for *every* version:

  r20927 | (no author) | (no date) | 1 line
  Changed paths:

And svnlook tree shows there is nothing in the repositories.

I believe this matches the behavior you would get if you were using
path-based permissions and the svnsync user did not have read
permission to the paths in the revision.  Could that be the reason?
You could also get this if you did not specify the URL to the root of
the repository.  Any revision that does not touch the path you
specified would just sync an empty revision.


Duh!  Of course.  I just tested both of the possible reasons you gave 
and, sure enough, both will produce revs like the one in my first 
message.  But I think the only way to get a replica, like mine, with 
*all* empty revs would be via path-based permissions.  If we were 
syncing only a subdirectory of a repository which we, otherwise, had 
read access to, the commits outside of the sync'ed url would have proper 
usernames and dates, but no files.  (I tested that too)


I was about to ask if svnsync would get access denied errors... but I 
did another test which, I think, answered my own question:  If I run 
svn log against a repository to which I have only access to part of, 
revs in the areas I don't have access to will show up as (no author) | 
(no date).  Presumably svnsync is doing some variant of a log call 
and then getting the contents of each rev and faithfully reproduces what 
seem to be empty revs.


So, I guess svnsync is arguably doing the right thing.  But even if it 
were declared to be incorrect, and a fix were checked in right now, it 
wouldn't address any of my existing replicas.   So, I'll have to develop 
some auditing for these sorts of things.  Which brings up another 
question:  is there any other reason you would ever see (no date) in a 
log entry?


Thanks!
trent...

Non unicode characters in log messages

[Trent Fisher]

I think I found a bug relating to handling of non-unicode characters in 
log messages.  When I do an svn log via the file:// url it works fine, 
but when I do it via http:// the history is truncated and ends with this 
message:


svn: REPORT of 
'/svn/tugbu/!svn/bc/236146/QA/CCB/main%20V2.4.X/Documentation/Test%20Plans/Rate%20Engine/Version%202': 
200 OK (https://adc4110279.us.oracle.com)


I figured out that the next revision which would have been displayed 
contained non-unicode characters.  From what I have gathered via some 
Google searches, Subversion shouldn't accept such things, though old 
versions did, and svndump load will let such things in (which means I 
have a bug in my import script).


One side effect of this error is that TortoiseSVN thinks the sever has 
gone down and offers to go into offline mode.


If I got rid of the non-unicode characters in the log message, that 
error went away.  FYI the following command seemed to do the trick for 
getting rid of the bad characters, since propget displayed the bad 
characters with backslash escaped octal sequences (i.e. ascii).


svn propget --revprop -r236146 svn:log file:///scm/svn/repos/tugbu | svn 
propset -F - --revprop -r236146 file:///scm/svn/repos/tugbu


So, it seems to me that this is a bug.  Subversion should degrade in a 
more graceful way when this happens, no?  I looked a bit through the 
Issues list, but didn't see anything like this.


++thanks,
trent...

Re: Non unicode characters in log messages

[Trent Fisher]

The server has Subversion 1.6.16 and Apache 2.2.15.  It happens on most 
clients I have tried including Tortoise SVN 1.6.7 and I think the person 
who discovered it was running 1.7, and from the command line I tried 
versions 1.6.6 and 1.6.16.


Perhaps this is something fixed in 1.7?

On 03/26/2012 12:59 PM, Daniel Shahaf wrote:

What version of svn on the client and on the server?  (Go to http://../
in a browser and check hte footer)

Trent Fisher wrote on Mon, Mar 26, 2012 at 12:35:27 -0400:

I think I found a bug relating to handling of non-unicode characters
in log messages.  When I do an svn log via the file:// url it
works fine, but when I do it via http:// the history is truncated
and ends with this message:

svn: REPORT of 
'/svn/tugbu/!svn/bc/236146/QA/CCB/main%20V2.4.X/Documentation/Test%20Plans/Rate%20Engine/Version%202':
200 OK (https://adc4110279.us.oracle.com)

I figured out that the next revision which would have been displayed
contained non-unicode characters.  From what I have gathered via
some Google searches, Subversion shouldn't accept such things,
though old versions did, and svndump load will let such things in
(which means I have a bug in my import script).

One side effect of this error is that TortoiseSVN thinks the sever
has gone down and offers to go into offline mode.

If I got rid of the non-unicode characters in the log message, that
error went away.  FYI the following command seemed to do the trick
for getting rid of the bad characters, since propget displayed the
bad characters with backslash escaped octal sequences (i.e. ascii).

svn propget --revprop -r236146 svn:log file:///scm/svn/repos/tugbu |
svn propset -F - --revprop -r236146 file:///scm/svn/repos/tugbu

So, it seems to me that this is a bug.  Subversion should degrade in
a more graceful way when this happens, no?  I looked a bit through
the Issues list, but didn't see anything like this.

++thanks,
trent...