Re: List modified on a branch

[Stefan Sperling]

On Fri, Feb 11, 2022 at 08:29:35AM +, Mikael Stålhammar wrote:
> Is there a simple command that lets you list files that have been modified 
> and committed on a feature branch but excluding any files that are committed 
> on the branch that are only merged from trunk (to keep the branch up-to-date 
> with trunk). I.e. I only want to see those files that actually contain branch 
> specific changes.

Find the most recent revision where a merge from trunk to the branch occurred.
If no such merge has occurred yet, find the revision in which the branch
was created.

Let's assume this was revision r100.

Now you can run: svn diff --summarize ^/trunk@r100 ^/branches/mybranch

Re: Does reintegration merge the logs?

[Stefan Sperling]

On Fri, Dec 17, 2021 at 08:49:03PM +, Mun Johl wrote:
> Hi all,
> 
> We're using SVN version 1.8.19 on Linux.

You should consider upgrading to 1.14.1, it has many important
bug fixes and nice new features.

> I was wondering if I execute an "svn merge -reintegrate" to reintegrate a 
> branch back to trunk, will SVN merge the logs from the various check-ins done 
> on the branch back into the trunk as well?  Empirical data shows that it does 
> not; but I'm not sure if we didn't do something correctly during the 
> reintegration or if that is how SVN works.  Although, I realize I can still 
> access the branch's logs for check-in information.

Simply reading branch commit logs is how one would usually learn about
the individual commits that were made on a branch, yes.
A merge commit will contain all the changes from the branch squashed
into a single commit, and it is up to the person who runs the merge
command to write a new log message for this merge commit. Conceivably,
one could include the log messages of all commits from the branch in a
merge commit's log message. But that is redundant and not common practice.

Re: ASF Subversion version

[Stefan Sperling]

On Mon, Dec 13, 2021 at 07:14:46AM -0600, Luke Mauldin wrote:
> It looks like the stashing functionality is still experimental. Do you think 
> there will be more funding in the future to complete it or do you think it 
> may remain unfinished for a long period of time?
> 

I do not know any specifics about the funding arragements that were
made for the staging feature. What I do know is what anyone can tell
from public records such as our mailing list archives and in our commit
history. Julian, like many others, is working in other jobs nowadays and
has not actively contributed much to SVN during the last year or so.

I suspect there are still some developers who would be open to the idea
of returning to continue working on in-progress features if they could
put food on the table in return. A simple fact is that any altruism
these people have spent over years in order to provide this tool to
communities and companies has largely run dry by now. While SVN grew
up because many of its developers were on paid time since 2000, many
were also willing to invest some of their free time to help out because
this was a fun and challenging project to work on.

But not many can afford to do such work in their free time forever, given
the complexity of the subject matter, the expectations of quality that
need to be met, and the fact that most users are now businesses which
simply use SVN because it does not cost them a cent compared to similar
proprietary offerings which are usually very expensive. I know this
because over the course of about a decade I have helped many such
companies move over to SVN from such platforms, and SVN works so well
for many of them that most do not even see a need for a paid support
contract to get help in case things go wrong. I have seen people in
companies who wanted to pitch such things to their bosses but found
themselves in a difficult spot when trying to justify a budget for
SVN tooling ("wait, did we not just got rid of ClearCase in order
to save such costs...?")

And naturally, the longer people have moved on with their lives,
the less interest they will have in returning to past projects.

Re: ASF Subversion version

[Stefan Sperling]

On Sun, Dec 12, 2021 at 07:21:44PM -0600, Luke Mauldin wrote:
> That really is unfortunate to hear about the lack of funding. I see in the 
> past few years there has been a lot of work done on the conflict resolver in 
> version 10 and multiple stash implementations in versions 11+. Are those 
> efforts being driven by people just working on it in their “spare time”?
> 

Those efforts were funded while development on was on-going.

Re: ASF Subversion version

[Stefan Sperling]

On Sat, Dec 11, 2021 at 06:59:31AM -0600, Luke Mauldin wrote:
> Does the subversion project receive any funding from the ASF to hire
> professional developers to complete more complex tasks or is development 100%
> community driven and supported?

The ASF does not pay anyone for development. I think this is an unfortunate
situation because many ASF projects slowly die off as funding dries up.
I believe the ASF is unlikely to change this long-standing practice, even
though there are other open source foundations which fund developers.
The FreeBSD and OpenBSD foundations pay some development (see their
financial reports), and apparently a new PHP foundataion is starting up
with the sole purpose of funding PHP developers.

In the past many SVN developers were employed by companies who ran with
business models related to Subversion. This is the funding model the ASF
is promoting. However, as of a few years ago most such companies changed
direction and are no longer employing any SVN developers. Many people have
moved on as a result and are no longer active.

(Disclaimer: I still receive a small amount of indirect SVN-related funding
via elego's SVN customer support. I occasionally use some of this time
to work on various things in Subversion, even though this budget is not
intended to fund development beyond customer-specific issues which can
only be fixed in the code base. And it is not enough to cover complex tasks.)

Re: Compare revisions on different devices, why different?

[Stefan Sperling]

On Fri, Dec 10, 2021 at 04:57:03PM +0100, Bo Berglund wrote:
> So the update did nothing except fix the revision number and time stamp.
> 
> Do you need to do an svn up after each svn ci in order to fix the state?

Yes. See here for an explanation:
https://subversion.apache.org/faq.html#hidden-log

Re: svndumpfilter: finding source commit for error?

[Stefan Sperling]

On Thu, Dec 09, 2021 at 01:04:24PM -0600, Luke Mauldin wrote:
> Did this problematic commit occur due to a bug at the time in Subversion or a 
> user error?
> 
> Luke

The log message suggests it was a deliberate choice, and there was
some discussion about it after the fact.
See https://svn.apache.org/r1295006 and 
https://marc.info/?t=13986609788=1=2

Anyway, since this code lives in the ASF repository there was nothing
that could be done to undo the copy without impacting every ASF project.

Re: svndumpfilter: finding source commit for error?

[Stefan Sperling]

On Thu, Dec 09, 2021 at 06:29:14PM +, Kristofer wrote:
> Hi Stefan and thanks for the hints. Then I need to line up a lot of
> arguments, right? There's no "read from file" option that I can see. I'll try
> that on the next failure, if I get one *fingers crossed*

Indeed, the list of path prefixes must be passed on the command line.
Support for reading them from a file is not implemented, unfortunately.

> Btw, I also have this really silly commit sequence where someone managed to
> delete the entire branches/ directory followed by a commit that brings it
> back (not sure if the commiter used a proper reverse-merge or a copy). I
> haven't understood if that can be "fixed" with svndumpfilter or if there's
> some other way to do it. Those two are basically a null operation, but it
> messes with things like "log --stop-on-copy"

I don't think there is an easy way to fix that via dump/load once other
commits have been stacked on top. Any newer commit might refer to data
stored in the problematic commit, due to deltification and other meta-data
relationships between revisions. Copies in particular refer to node-rev-IDs
which are generally hidden from the user, but which can be seen in the dump
file, and which are not supposed to be changed.

There is a commit in Subversion's own trunk history which unfortunately
did exactly the same thing. But it is water under the bridge at this point.
People rarely have a need to go far enough back in history to cross it.

Re: svndumpfilter: finding source commit for error?

[Stefan Sperling]

On Thu, Dec 09, 2021 at 05:07:09PM +, Kristofer wrote:
> And of course, soon after mailing, I found a workaround. I am not sure why it 
> worked but anyways:
> The problem was caused by the branch being merged to trunk, but the contents 
> of the "bad directory" was never merged. Then I thought it would be fine to 
> filter out the bad directory itself, especially since there were no subtree 
> mergeinfo on it.
> But it turned out that I could filter "baddir/badcontent1" and 
> "baddir/badcontent2" and then the dumpfilter operation worked fine. I still 
> don't know which revision the actual problem showed up, but it doesn't matter 
> at this point :)
> I suppose there is something with the mergeinfo that I do not understand, but 
> since I managed to get around it, no need to bother this list further. Sorry 
> about the noise.
> 

Nowadays we also support --include and --exclude options in "svnadmin dump"
itself which avoids having to pass the dump stream through svndumpfilter.

I don't know if this would have prevented your mergeinfo-related problem.
But I wanted to mention this in case you would like to try it out.
Or just keep it in mind in case you run into such a problem again.

In general, the built-in --include and --exclude options can be smarter
because they have access to context information while the dump stream
is being created. Whereas svndumpfilter can only obtain information from
the generated dump stream.

$ svnadmin help dump
[...]
Using --exclude or --include gives results equivalent to authz-based
path exclusions. In particular, when the source of a copy is
excluded, the copy is transformed into an add (unlike in 'svndumpfilter').

Valid options:
[...]
  --exclude ARG: filter out nodes with given prefix(es) from dump
  --include ARG: filter out nodes without given prefix(es) from dump
[...]

Re: Install SVN 1.6.11 on RHEL 7.7

[Stefan Sperling]

On Tue, Dec 07, 2021 at 03:19:45PM +0530, Abhishek Kumar wrote:
> Dear Team,
> There is a Planned Linux OS Upgradation of Server from existing version RHEL
> 6 to RHEL 7.7 where SVN Server(SVN V1.6.11)  setup is done.
> 
> Kindly provide the installation steps and compatibility for the same.
> 
> *Existing*
> 
> SVN Version Linux Version
> V1.6.11 RHEL 6.10
> *Planned*
> SVN Version Linux Version
> V1.6.11 RHEL 7.7

Subversion 1.6.11 was released in April 2010 (almost 12 years ago!)
and is affected by numerous security issues listed here:
https://subversion.apache.org/security/

You should consider upgrading SVN to a supported version, instead
of upgrading only the RHEL system itself.

For latest SVN source releases, see:
https://subversion.apache.org/download.cgi#source-releases

For SVN binary packages specific to RHEL, see:
https://subversion.apache.org/packages.html#redhat
 
> Looking forward to a quick reply and prompt support.

This mailing list is a volunteer forum. As such, no specific guarantees
can be made regarding support requests.

Regards,
Stefan

Re: Changed file permissions after committing user-crontabs

[Stefan Sperling]

On Wed, Nov 24, 2021 at 04:20:12PM +0100, Thorsten Schöning wrote:
> Guten Tag Stefan Sperling,
> am Mittwoch, 24. November 2021 um 13:33 schrieben Sie:
> 
> > This will be due to apr_file_mktemp() using default permissions
> > on a freshly created temporary file. The committed file is run
> > through translation (for keywords, newlines, and such) which
> > involves a temporary file that gets reinstalled into the working copy.
> 
> Is there any way to avoid that temporary file at all? I don't use
> keywords and the newlines are as expected. Can e.g. the eol-property
> or some other be set in a way to avoid all translations?

I don't know. I have told you all I could figure out by skimming the code,
and I don't have time to dive into more details.

If a workaround is not enough for your needs, then someone else (not me)
can study the libsvn_wc code in detail to figure out a proper solution.

There is a function in libsvn_subr/io.c which copies unix perms from
one file to another. This function is already being used in places.
Perhaps a call to this function is missing from the commit code path
such that permissions of files installed after committing them will
be preserved.

Cheers,
Stefan

Re: Changed file permissions after committing user-crontabs

[Stefan Sperling]

On Wed, Nov 24, 2021 at 12:32:40PM +0100, Thorsten Schöning wrote:
> Guten Tag Stefan Sperling,
> am Mittwoch, 24. November 2021 um 11:17 schrieben Sie:
> 
> > Subversion does not store file permission mode bits, except that the
> > "executable" bit is preserved via the svn:mime-type property.
> 
> I know and don't want SVN to store permissions. The file has correct
> permissions before committing and doesn't anymore afterwards, that's
> the problem I'm facing. Something is changing permissions of the file
> during/after the commit and that need to be found and disabled.

This will be due to apr_file_mktemp() using default permissions
on a freshly created temporary file. The committed file is run
through translation (for keywords, newlines, and such) which
involves a temporary file that gets reinstalled into the working copy.

As a workaround you could try running 'umask 077' before running
svn commit on affected files, such that newly created files get
the permission flags expected by cron. The default umask is 022.

Re: Changed file permissions after committing user-crontabs

[Stefan Sperling]

On Wed, Nov 24, 2021 at 11:09:08AM +0100, Thorsten Schöning wrote:
> Hi everyone,
> 
> for various reasons I put /var/spool/cron/crontabs under version
> control. I have one Ubuntu 12.04 legacy system running SVN 1.7.22 and
> recognized that after committing changes to the file for ROOT, it's
> permissions changed.
> 
> > 655748 4 -rw--- 1 root crontab 1597 Nov 24 10:09 root
> 
> vs.
> 
> > 655748 4 -rw-r--r-- 1 root crontab 1597 Nov 24 10:09 root
> 
> This made CRON ignoring the file with the following message:
> 
> > Nov 23 12:56:01 [...] cron[1346]: (root) INSECURE MODE (mode 0600 expected) 
> > (crontabs/root)

Subversion does not store file permission mode bits, except that the
"executable" bit is preserved via the svn:mime-type property.

You will want to use a wrapper tool such as 'asvn' for this use case:
https://svn.apache.org/repos/asf/subversion/trunk/contrib/client-side/asvn

Cheers,
Stefan

Re: SVN API: Identify repositories on server filesystem

[Stefan Sperling]

On Tue, Nov 16, 2021 at 03:38:11PM +, Jens Restemeier wrote:
> Hi Stefan,
> No, I hadn't seen that page.
> 
> Is the repository automatically closed when the pool gets released, or is 
> there a function I need to call to release any locks/resources allocated when 
> opening it?
> 

Yes, this is handled via pools. The pool that you pass in should not be
cleared or destroyed while the repository data structure is in use.

Re: SVN API: Identify repositories on server filesystem

[Stefan Sperling]

On Mon, Nov 15, 2021 at 07:05:35PM +, Jens Restemeier wrote:
> Hi,
> Not sure if this is a better dev or user question...
> Is there an official way to tell if a file path points to a svn repository in 
> the C API? I looked at the documentation, but it seem to require quite some 
> knowledge about the architecture, and it is sometimes hard to tell if 
> something is referring to a repository path or a host filesystem path.
> 
> I am on the server, and have access to the directory the repositories are 
> served from. I could obviously check if the directory contains "format", 
> "conf" or "db". I am linking against the svn libraries anyway, and I'd like a 
> future-proof method. Is there anything like svn_is_repository(path)? I could 
> try to open a path and see if I get an error?
> 
> Cheers,
> Jens
> 

Hi Jens,

In case you have not seen it yet, this page lists some things you may
need to know in order to make sense of Subversion's code base:
https://subversion.apache.org/docs/community-guide/conventions.html

I guess the simplest thing you could do is call svn_repos_open()
on a given path.

If your input path is not UTF-8 encoded then run it through
svn_utf_cstring_to_utf8() beforehand. Then you should run the
resulting UTF-encoded path through svn_dirent_internal_style()
in order to ensure that the path will be considered valid.
Code in subversion/svnadmin/svnadmin.c contains examples of this.

svn_open_repos() is declared in svn_repos.h and expects just a path
and an APR pool as input. It should raise some sort of error if
the path does not contain a repository. It is marked deprecated
but it can be called regardless. Newer versions of this function
only add more parameters you do not need, and this deprecated
version will fill in suitable defaults for you.

/** Similar to svn_repos_open2() with @a fs_config set to NULL.
 *
 * @deprecated Provided for backward compatibility with 1.6 API.
 */
SVN_DEPRECATED
svn_error_t *
svn_repos_open(svn_repos_t **repos_p,
   const char *path,
   apr_pool_t *pool);

Re: Use-after-free of object-pools in subversion/libsvn_repos/authz.c

[Stefan Sperling]

On Wed, Nov 03, 2021 at 05:21:19PM +0100, Thomas Weißschuh wrote:
> On 2021-11-03 17:14+0100, Stefan Sperling wrote:
> > On Wed, Nov 03, 2021 at 04:47:39PM +0100, Thomas Weißschuh wrote:
> > > The svn_atomic__init_once() inside svn_repos_authz_initialize() seems to 
> > > work
> > > correctly. synchronized_authz_initialize() is only executed once.
> > > 
> > > I am arguing that it should *not* be executed only once because the pools 
> > > that
> > > it gets called with the first time will be invalid when it is called the 
> > > second
> > > time. So it should rebuild authz_pool with the new parent pool.
> > > 
> > > Replacing the call through svn_atomic__init_once() with an unconditional 
> > > call
> > > makes the issue go away.
> > 
> > I see. So for some reason, under httpd's control, the SVN module's
> > post_config hook is invoked several times, each time with a different
> > pool? Do we know why this is happening?
> 
> I'll investigate that a bit more.
> 
> > If this is valid httpd behaviour then the code in authz.c is making
> > an invalid assumption. And there could similar problems elsewhere,
> > in svn_fs_initialize() for example.
> 
> Should we keep this discussion on the ML or should I open a ticket for it?

Both :)

I agree that this warrants a new ticket. Once the problem is fully
understood we can still decide to close this ticket in case the real
issue turns out to be somewhere else. Please feel free to create a
new ticket for this in our tracker. Thanks!

Re: Use-after-free of object-pools in subversion/libsvn_repos/authz.c

[Stefan Sperling]

On Wed, Nov 03, 2021 at 04:47:39PM +0100, Thomas Weißschuh wrote:
> The svn_atomic__init_once() inside svn_repos_authz_initialize() seems to work
> correctly. synchronized_authz_initialize() is only executed once.
> 
> I am arguing that it should *not* be executed only once because the pools that
> it gets called with the first time will be invalid when it is called the 
> second
> time. So it should rebuild authz_pool with the new parent pool.
> 
> Replacing the call through svn_atomic__init_once() with an unconditional call
> makes the issue go away.

I see. So for some reason, under httpd's control, the SVN module's
post_config hook is invoked several times, each time with a different
pool? Do we know why this is happening?

If this is valid httpd behaviour then the code in authz.c is making
an invalid assumption. And there could similar problems elsewhere,
in svn_fs_initialize() for example.

Re: Use-after-free of object-pools in subversion/libsvn_repos/authz.c

[Stefan Sperling]

On Wed, Nov 03, 2021 at 04:21:34PM +0100, Thomas Weißschuh wrote:
> Hi everybody,
> 
> While investigating persistent segmentation faults in mod_dav_svn I found
> invalid uses of objectpools in subversion/libsnv_repos/authz.c.
> 
> In svn_repos_authz_initialize() the objectpools passed in during the
> configuration phase are stored in static variables.
> For some reason the configuration phase runs multiple times and the previously
> used objectpools are freed.
> 
> Because cached references to these freed objectpools are still used inside
> authz.c accesses to that memory will read invalid data from other parts of
> Apache, leading to segmentation faults.
> 
> Maybe the issue happens especially on Alpine Linux because they are using musl
> libc with its own memory allocator which may behave differently than glibc and
> more directly reuse freed memory.
> 
> I was able to work around the issue by removing the caching logic in
> svn_repos_authz_initialize() and always call synchronized_authz_initialize().
> 
> Thanks,
> Thomas

I guess this means the implemention of apr_atomic_cas32() is not working
correctly on your system? svn_atomic__init_once() relies on this.

There are regression tests for atomics in APR.
You could try to run these tests to see if there are any failures.
If broken atomics turn out to be the root cause of this problem
then please file an issue with the APR project instead.

Hope this helps,
Stefan

Re: Current project status

[Stefan Sperling]

On Thu, Oct 28, 2021 at 11:19:43AM -0400, Mark Phippard wrote:
> On Thu, Oct 28, 2021 at 11:15 AM Stefan Sperling  wrote:
> >
> > On Thu, Oct 28, 2021 at 10:24:55AM -0400, Mark Phippard wrote:
> > > SVN's design handles renames well from a version control history point
> > > of view, but it does not handle them well when doing things like merge
> > > or even update if you have local mods to a file that was renamed. Git
> > > actually handles this much better. SVN has tried to solve it with the
> > > tree conflict feature and incremental progress has been made to
> > > resolve those tree conflicts better but it still leaves a lot to be
> > > desired.
> >
> > What is left to be desired when updating or merging a renamed file
> > with local mods? As far as I know, in current versions of SVN, this
> > is working quite well.
> 
> That is good to hear. I have tried to follow all of the progress you
> have made over the last few releases but I mainly use Git in day to
> day work now and have not gone to trouble of setting up examples to
> try it all out.
> 
> Does it all happen automatically now? I thought at one point we still
> created the tree conflict and then you had to run svn resolve or
> something like that to trigger the resolution?

The tree conflict conflict will still be created. The client will
invoke the resolver automatically when this happens. If the resolver
detects just one possible conflict resolution option then the client
uses this option automatically. The client will prompt only when the
resolver offers multiple options. In some cases we might prompt anyway
even if just one option is found, but non-ambiguous moves should be
followed without further interaction required by the user.

The logic for presenting and picking resolution options is fully
under the client's control. So some clients might show different
behaviour if they have not been updated to make use of new APIs.
Clients do not need to care in any way which options are available
and how their resolution strategies are implemented. That is all
handled by the library.
Clients only need to take care of presentation, and perhaps make a
decision on behalf of the user if there is only one option.

> Does any of the resolution just happen automatically in the libraries
> or for something like Subclipse would a bunch of new stuff need to be
> exposed to JavaHL and then we would have to fill in an implementation?

JavaHL might still be missing some bits, yes. I am not quite sure.
Your best starting point would be to check which svn_client_conflict APIs
the command line client using and see whether any of them still need to
be mapped to Java.

Re: Current project status

[Stefan Sperling]

On Thu, Oct 28, 2021 at 10:24:55AM -0400, Mark Phippard wrote:
> SVN's design handles renames well from a version control history point
> of view, but it does not handle them well when doing things like merge
> or even update if you have local mods to a file that was renamed. Git
> actually handles this much better. SVN has tried to solve it with the
> tree conflict feature and incremental progress has been made to
> resolve those tree conflicts better but it still leaves a lot to be
> desired.

What is left to be desired when updating or merging a renamed file
with local mods? As far as I know, in current versions of SVN, this
is working quite well.

Like with Git's heuristics-based approach, some cases are handled much
better than others. But simple file renames should be fine.
I don't know what the various SVN GUI clients have implemented on top
of the functionality we have built. But our command line client should
not even require additional user input in most such cases and just do
the "right thing" automatically.

Re: Current project status

[Stefan Sperling]

On Thu, Oct 28, 2021 at 09:01:02AM -0500, Luke Mauldin wrote:
> You mention renames as an example of a problem that are unlikely to ever be
> resolved.  Can you please elaborate on the issue?  My understanding is that
> especially the newer versions of SVN potentially do a better job of tracking
> renames than Git because SVN actually tracks more file level information
> whereas Git only tracks the contents of files.

I guess what Mark was referring to is a solution that would involve
server-side changes. Many potential solutions were discussed over time,
and some of them were even partially implemented but never went beyond
experimental status. Several proposals included changes to the repository
storage format as well as network procotol changes.

Our current solution for resolving tree conflicts is client-side only,
and as such it may not fit everyone's definition of "rename tracking".

Essentially, the client runs the equivalent of 'svn log -v' and traces
copyfrom information and deletions of relevant paths to figure out whether
a rename might have occurred. In other words, it does exactly what a
competent human would do to understand a rename conflict situation.
The client has to fall back to human guidance in case of ambiguity, where
multiple copies map to the same deleted path.
In a perfect world the client should not have to fall back on human guidance
because the user did already inform Subversion about the rename when they
ran 'svn move'. However, a move operation is translated to copy+delete when
a commit is made, so any move information the client had recorded is lost.

On the upside, our current approach works against any server, which is
important since it is generally easier to install up-to-date clients
than upgrading existing server deployments.

And the conflict resolution framework we have in the client has been
designed with extensibility in mind. It is possible to add support for
additional conflict cases by implementing a couple of funtions of C code.
The current coverage is not complete, most importantly it doesn't handle
cases where files have become directories or vice versa. But the only
reason it does not handle such cases is lack of time and motivation to
work on the missing bits. There is no technical reason it couldn't be done.
What we have now seems to be good enough to cover what most people need.
We do not hear complaints about this topic anymore so either people who used
to complain have abandoned SVN or the solution we have is good enough.
We cannot really tell the difference :)

Re: SVN hook

[Stefan Sperling]

On Wed, Aug 25, 2021 at 09:32:04AM +0200, Justin MASSIOT | Zentek wrote:
> * As for the "can do / can't do", the standard way would be to enforce
> "path-based
> authorization
> ",
> but it isn't flexible at all because it doesn't support wildcards.

Good news! Path-based authz does support wildcards as of 1.10.0:
http://subversion.apache.org/docs/release-notes/1.10.html#authzperf

Re: Crash in token.c after incomplete cherry pick merges in 1.15

[Stefan Sperling]

On Tue, Aug 10, 2021 at 02:46:50PM +0200, Stefan Sperling wrote:
> On Mon, Aug 09, 2021 at 06:19:38PM -0400, Joshua Kordani wrote:
> > Attached is a script to reproduce the error.
> > 
> > I also have a packed rr debugger session that I can provide (I highly
> > recommend the rr reversible debugger).  Its ~30meg
> 
> Thank you Joshua! The patch below makes your test script succeed for me.
> 
> Ideally I should commit a regression test together with this patch.
> I will need to find time to convert your script into a regression test.

I have finally gotten around this now.

Test added in https://svn.apache.org/r1892470
Fix committed in https://svn.apache.org/r1892471
Nominated for backport to 1.14.x in https://svn.apache.org/r1892472

Thanks for your report, Joshua!

Regards,
Stefan

Re: Crash in token.c after incomplete cherry pick merges in 1.15

[Stefan Sperling]

On Mon, Aug 09, 2021 at 06:19:38PM -0400, Joshua Kordani wrote:
> Attached is a script to reproduce the error.
> 
> I also have a packed rr debugger session that I can provide (I highly
> recommend the rr reversible debugger).  Its ~30meg

Thank you Joshua! The patch below makes your test script succeed for me.

Ideally I should commit a regression test together with this patch.
I will need to find time to convert your script into a regression test.

In the meantime can you confirm that the patch helps?

Regards,
Stefan

Index: subversion/libsvn_client/conflicts.c
===
--- subversion/libsvn_client/conflicts.c(revision 1892118)
+++ subversion/libsvn_client/conflicts.c(working copy)
@@ -3028,12 +3028,12 @@ conflict_tree_get_details_local_missing(svn_client
   deleted_basename,
   conflict->pool);
   details->moves = moves;
+  details->wc_move_targets = apr_hash_make(conflict->pool);
   if (details->moves != NULL)
 {
   apr_pool_t *iterpool;
   int i;
 
-  details->wc_move_targets = apr_hash_make(conflict->pool);
   iterpool = svn_pool_create(scratch_pool);
   for (i = 0; i < details->moves->nelts; i++)
 {
Index: subversion/libsvn_wc/wc_db.c
===
--- subversion/libsvn_wc/wc_db.c(revision 1892057)
+++ subversion/libsvn_wc/wc_db.c(working copy)
@@ -16732,7 +16732,7 @@ svn_wc__db_find_working_nodes_with_basename(apr_ar
   SVN_ERR(svn_sqlite__get_statement(, wcroot->sdb,
 STMT_SELECT_PRESENT_HIGHEST_WORKING_NODES_BY_BASENAME_AND_KIND));
   SVN_ERR(svn_sqlite__bindf(stmt, "ist", wcroot->wc_id, basename,
-kind_map, kind));
+kind_map_none, kind));
   SVN_ERR(svn_sqlite__step(_row, stmt));
 
   *local_abspaths = apr_array_make(result_pool, 1, sizeof(const char *));
@@ -16776,7 +16776,7 @@ svn_wc__db_find_copies_of_repos_path(apr_array_hea
   SVN_ERR(svn_sqlite__get_statement(, wcroot->sdb,
 STMT_SELECT_COPIES_OF_REPOS_RELPATH));
   SVN_ERR(svn_sqlite__bindf(stmt, "ist", wcroot->wc_id, repos_relpath,
-kind_map, kind));
+kind_map_none, kind));
   SVN_ERR(svn_sqlite__step(_row, stmt));
 
   *local_abspaths = apr_array_make(result_pool, 1, sizeof(const char *));

Re: Segfault in libsvn_client/conflicts.c

[Stefan Sperling]

On Mon, Aug 09, 2021 at 07:56:27PM -0400, Joshua Kordani wrote:
> On my work repo, this causes the crash.  In a small repo where I have
> repeated these steps, it does not.

Most likely in your test repository the history is arranged such
that the deletions can be properly correllated by logging history
of the various paths involved. Your test script probably creates
history which "makes sense", and your work repository probably
contains something unexpected which isn't aligned with "best practices"
and might be difficult to spot.

You will need to dig deeper into why the resolver is being mislead
in your work repository. If all else fails, you could try printing URLs
and revisions which are being logged by the resolver during its search.
That might help with understanding why the resolver fails to correlate
the deletions with one another.

Re: Segfault in libsvn_client/conflicts.c

[Stefan Sperling]

On Mon, Aug 09, 2021 at 02:09:40PM -0400, Joshua Kordani wrote:
> The shell script is easy.  I run resolve on my codebase :-).  The trick will
> be recreating the repo history that reproduces this problem, and that has
> always been gnarly to me.  I could use some advice for this.
> 
> Basically, what I will try to do is create a folder with files. create a
> branch at this point, move a file in the branch, cherry pick this commit
> with the move into trunk, and then try a top level merge of the branch into
> trunk.  Those are the things I did to the repo, I'm not sure if others
> mucked with it.  If that doesn't work, I'll have to sleuth my repo and I'll
> need help to do that

svn log -v can help you figure out what copies and deletions have
occurred. I suspect you have a case where some parent directory of the
affected file has also been deleted or replaced somewhere along the line.
Otherwise we should not get a NULL result when looking for the deleted
node in the parent directory's history.

Re: Segfault in libsvn_client/conflicts.c

[Stefan Sperling]

On Sun, Aug 08, 2021 at 10:26:43AM +0200, Daniel Sahlberg wrote:
> On 2021-08-07 17:14, Joshua Kordani wrote:
> > Yes it does appear to fix the issue.  Thank you!
> 
> Hi Joshua (and list),
> 
> I've been trying to reproduce your issue both to have test case for
> backporting the fix and possibly also adding to the test suite. However I
> can't reproduce it, possibly I misunderstand some of your steps.
> 
> Do you know the exact steps to reproduce (when running without the patch,
> obviously)?
> 
> Kind regards,
> 
> Daniel Sahlberg

I have committed my patch in https://svn.apache.org/r1892118

I agree that having a test case would be nice to prevent this issue from
resurfacing accidentally. It could also help us to identify other similar
scenarios that don't work right. If we had a shell script which reproduced
the crash I could write a corresponding C test for the resolver. What we
lack is the sequence of changes and merges which trigger the issue.

Joshua, do you think you would be able to provide this sequence of steps,
ideally as a self-contained shell script?
We offer a suitable shell script template here:
https://subversion.apache.org/docs/community-guide/repro-template.sh

Regards,
Stefan

Re: Segfault in libsvn_client/conflicts.c

[Stefan Sperling]

On Sat, Aug 07, 2021 at 06:32:34AM -0400, Joshua Kordani wrote:
> Thank you for explaining the convention, it makes sense now that you
> describe it like that.
> 
> It was delightfully short work to hunt down with the rr debugger ;-)
> 
> I'm having trouble testing it right now but I suspect it will work, I'll get
> back to you on this.
> 
> There is another caller of find_related_node, and again In the few minutes
> of looking it wasn't obvious to me if that callsite also needed to correctly
> track  the first argument remaining NULL after the call?  Line 5148.

The other caller should be fine because find_revision_for_suspected_deletion()
will deal with related_repos_relpath being NULL.

The problem in the case at hand is that we're passing related_repos_relpath
to svn_path_url_add_component2() which cannot handle a NULL argument.

Re: Segfault in libsvn_client/conflicts.c

[Stefan Sperling]

On Fri, Aug 06, 2021 at 07:41:42PM -0400, Joshua Kordani wrote:
> I am merging two branches together, A into B.  In A, I deleted a file and
> cherrypicked that onto B.  Later, I want to merge at the toplevel from A
> into B, and I get a segfault when svn tries to figure out how to resolve the
> resulting tree conflict (file missing, incoming file deleted).
> 
> I have narrowed it down to this line.  At first, the string pointed to by
> related_repos_relpath is set to null, and related_peg_rev is set to
> SVN_INVALID_REVNUM.  No matter the exit paths, related_peg_rev will either
> be a valid rev, or SVN_INVALID_REVNUM.  But, if we can't find a related node
> in line 2473, we bail, but related_repos_relpath is set to null.  It seems
> like there are only two ways that the function will exit.  In successful
> cases, the related_repos_relpath is updated appropriately.  Otherwise I
> guess that when the search fails, the conflict resolution stops and the
> original value of related_repos_relpath is unimportant.
> 
> I release all rights to this patch, or assign the rights to the project
> license as appropriate.

Hi Joshua,

Thank you very much for tracking this down!

> Index: subversion/libsvn_client/conflicts.c
> ===
> --- subversion/libsvn_client/conflicts.c  (revision 1875623)
> +++ subversion/libsvn_client/conflicts.c  (working copy)
> @@ -2422,7 +2422,7 @@
>svn_node_kind_t related_node_kind;
>svn_ra_session_t *ra_session;
>  
> -  *related_repos_relpath = NULL;
> +  /*  *related_repos_relpath = NULL; */
>*related_peg_rev = SVN_INVALID_REVNUM;
>  
>SVN_ERR(svn_client_conflict_get_repos_info(_root_url, NULL,

By convention, functions are supposed to initialize their output arguments.
Otherwise we run a risk of callers using uninitialized variables which
could be even worse than a NULL pointer since it might clobber unrelated
memory rather than crashing immediately.

I would introduce a separate variable in the caller such that the
original pointer is left untouched if we hit the case you have found.
Also, the caller must deal with the case where the search for the
related node fails.

Does this alternative patch fix the crash for you, too?

Index: subversion/libsvn_client/conflicts.c
===
--- subversion/libsvn_client/conflicts.c(revision 1892057)
+++ subversion/libsvn_client/conflicts.c(working copy)
@@ -2847,13 +2847,27 @@ conflict_tree_get_details_local_missing(svn_client
   /* Make sure we're going to search the related node in a revision where
* it exists. The younger incoming node might have been deleted in HEAD. */
   if (related_repos_relpath != NULL && related_peg_rev != SVN_INVALID_REVNUM)
-SVN_ERR(find_related_node(
-  _repos_relpath, _peg_rev,
-  related_repos_relpath, related_peg_rev,
-  (old_rev < new_rev ? old_repos_relpath : new_repos_relpath),
-  (old_rev < new_rev ? old_rev : new_rev),
-  conflict, ctx, scratch_pool, scratch_pool));
+{
+  const char *older_related_repos_relpath;
+  svn_revnum_t older_related_peg_rev;
+  SVN_ERR(find_related_node(
+_related_repos_relpath, _related_peg_rev,
+related_repos_relpath, related_peg_rev,
+(old_rev < new_rev ? old_repos_relpath : new_repos_relpath),
+(old_rev < new_rev ? old_rev : new_rev),
+conflict, ctx, scratch_pool, scratch_pool));
+  if (older_related_repos_relpath != NULL &&
+  older_related_peg_rev != SVN_INVALID_REVNUM)
+{
+  related_repos_relpath = older_related_repos_relpath;
+  related_peg_rev = older_related_peg_rev;
+}
+}
 
+  /* Bail if we are unable to find the related node. */
+  if (related_repos_relpath == NULL || related_peg_rev == SVN_INVALID_REVNUM)
+return SVN_NO_ERROR;
+
   /* Set END_REV to our best guess of the nearest YCA revision. */
   url = svn_path_url_add_component2(repos_root_url, related_repos_relpath,
 scratch_pool);

Re: Enforce encryption client-side

[Stefan Sperling]

On Thu, May 06, 2021 at 09:15:44AM -0400, Mark Phippard wrote:
> I am not familiar with the SASL options in the client config

Neither was I, and after some investigation of the code I concluded
that there are none. They're only available in svnserve/libsvn_repos.
Client-side support for these options would need to be added in order
to satisfy the use case under discussion.

Re: Enforce encryption client-side

[Stefan Sperling]

On Thu, May 06, 2021 at 01:03:28PM +0200, Nils-Johan Andreasson wrote:
> Thanks Stefan for your response and pointers!
> 
> There is no doubt my connections are already encrypted.
> The issue is that the external library I'm referring to supports specifying
> paths to source code using multiple protocols, e.g. svn:// (but also
> https://, http://, git://, ftp:// etc) and any svn:// source is generally
> considered insecure, just like http:// (but not for example https://). It
> is not verified in any way if that's actually the case, it's just based on
> the scheme prefix svn://
> 
> So, what I'm looking for is a way to ensure an svn client command is only
> allowed to execute if using an encrypted transport. E.g. something like:
> svn --enforce-encryption checkout svn://hostname/repository
> svn --min-encryption 128 checkout svn://hostname/repository
> (or --disallow-unencrypted-transport, --require-sasl etc)
> 
> Using something like that I can hopefully convince the library maintainer
> to enable this flag by default and then consider an svn:// source secure by
> default (as connection would otherwise not succeed).
> 
> I understand your point with that as long as "use-sasl = true" is
> configured on the server, the connection would abort if the client cannot
> communicate securely. However, this means that if this external library is
> pointed to a svn:// source with "use-sasl = false" the connection would
> succeed, which is what I'm trying to prevent here to guarantee no
> unencrypted connections are made.
> 
> Hope my point came through better this time?

Yes. It seems indeed impossible to enforce a min_sff parameter other than
zero on the client side at present, which is an unfortunate omission.
I am sure there is no technical reason which would prevent this from
becoming configurable.

If you or the third-party library maintainer are compiling your own
client binaries, the easy way to fix this would be to set min_ssf here:
https://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_ra_svn/cyrus_auth.c?view=markup#l276

When adding this an official Subversion feature, I would suggest to
expose the SASL configuration parameters that svnserve is already using
as part of the client-side configuration, and then ensuring that these
parameters are applied after svn_ra_svn__default_secprops() is called.
Note that any configuration file parameters can also be specified on
the command line via the --config-option option so you won't necessarily
need to add a new separate option to the command line interface if this
feature gets implemented via config file options.

I agree that this would be useful. However, someone would need to write
a patch, and then you would have to wait for a Subversion 1.15 release
to ship since this feature cannot be added in a 1.14 patch release due
to our compatibility promises (a clean roll-back from 1.14.Y to 1.14.X
without any configuration/parameter changes must remain possible). And
your third party library would need to upgrade to that version once it
has been released. All of this could take many months.

Have you considered using svn+ssh instead?

Cheers,
Stefan

Re: Enforce encryption client-side

[Stefan Sperling]

On Thu, May 06, 2021 at 11:21:57AM +0200, Nils-Johan Andreasson wrote:
> Hi there!
> 
> I'm serving a repository using svnserve with SASL to make sure
> communication is always encrypted (config has use-sasl = true,
> min-encryption = 128 and max-encryption = 256).
> I know this enforces encryption server-side but is there any way to in
> addition also "require" encryption client-side? E.g. let's say if I do 'svn
> checkout svn://my-insecure-host/repository' I want the command to abort if
> the connection is not encrypted.

The min-encryption paramter maps directly to the Cyrus SASL secprops.min_ssf
paramter which is described here:
https://www.cyrusimap.org/sasl/sasl/developer/programming.html#security-layers

Quote:
"A connection supplying only integrity with no privacy would have an SSF of 1.
A connection secured by 56-bit DES would have an SSF of 56.
To require a security layer, set min_ssf to the minimum acceptable security
layer strength."

SVN servers and clients check the SASL_SSF property of their connection here
and abort if SASL failed to negotiate encryption if encryption is configured:
https://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_ra_svn/cyrus_auth.c?revision=1875971=markup#l726

So your connections should already be encrypted. Perhaps the encryption
mechanism SASL is using is considered too weak by your external tool?
Would setting min-encryption = 256 help?
Or perhaps your external tool simply doesn't understand the SVN protocol?

Cheers,
Stefan

Re: Restoring svn database

[Stefan Sperling]

On Wed, Mar 03, 2021 at 08:03:08AM +, Philippe DEMOUSTIER wrote:
> Hi all,
> 
> Following an issue on our servers, we lost approximatively 30% of our svn 
> database.
> Admin dump fails so we're trying to restore some data manually.
> 
> How can we restore data between SVN and ENDREP tags ?
> 
> DELTA 30834 15564 155
> SVN xx ENDREP
> 
> Many thanks for your help,
> Philippe

Hi Philippe,

I am afraid the short answer is that you will need to restore your repository
from backups. If you cannot do that, then you have now learned the hard way
that an important SVN repository needs to be backed up. The data is gone.

It is virtually impossible to restore the missing data manually. This data
is sometimes full text, sometimes deltified content. The only way to generate
equivalent data is to replay all the commits that have occurred throughout
repository history. The tool 'svnadmin load' can do this, provided you have
previously saved a dump file of the repository to backup storage when the
repository was still in a healthy condition.

Going forward, you should consider saving repository dump files to backup
storage, or saving backup copies of your repositories with tools such as
'svnadmin hotcopy' or 'svnsync'  For more information, see this page:
http://svnbook.red-bean.com/nightly/en/svn.reposadmin.maint.html#svn.reposadmin.maint.backup

Regards,
Stefan

[SECURITY][ANNOUNCE] Apache Subversion 1.10.7 released

[Stefan Sperling]

I'm happy to announce the release of Apache Subversion 1.10.7.
Please choose the mirror closest to you by visiting:

https://subversion.apache.org/download.cgi#supported-releases

This is a stable bugfix and security release of the Apache Subversion
open source version control system.

THIS RELEASE CONTAINS AN IMPORTANT SECURITY FIX:

  CVE-2020-17525
  "Remote unauthenticated denial-of-service in Subversion mod_authz_svn"

The full security advisory for CVE-2020-17525 is available at:
  https://subversion.apache.org/security/CVE-2020-17525-advisory.txt

A brief summary of this advisory follows:

  Subversion's mod_authz_svn module will crash if the server is using
  in-repository authz rules with the AuthzSVNReposRelativeAccessFile
  option and a client sends a request for a non-existing repository URL.

  This can lead to disruption for users of the service.

  We recommend all users to upgrade to the 1.10.7 or 1.14.1 release
  of the Subversion mod_dav_svn server.

  As a workaround, the use of in-repository authz rules files with
  the AuthzSVNReposRelativeAccessFile can be avoided by switching
  to an alternative configuration which fetches an authz rules file
  from the server's filesystem, rather than from an SVN repository.

  This issue was reported by Thomas Åkesson.

SHA-512 checksums are available at:

https://www.apache.org/dist/subversion/subversion-1.10.7.tar.bz2.sha512
https://www.apache.org/dist/subversion/subversion-1.10.7.tar.gz.sha512
https://www.apache.org/dist/subversion/subversion-1.10.7.zip.sha512

PGP Signatures are available at:

https://www.apache.org/dist/subversion/subversion-1.10.7.tar.bz2.asc
https://www.apache.org/dist/subversion/subversion-1.10.7.tar.gz.asc
https://www.apache.org/dist/subversion/subversion-1.10.7.zip.asc

For this release, the following people have provided PGP signatures:

   Stefan Sperling [2048R/4F7DBAA99A59B973] with fingerprint:
8BC4 DAE0 C5A4 D65F 4044  0107 4F7D BAA9 9A59 B973
   Branko Čibej [4096R/1BCA6586A347943F] with fingerprint:
BA3C 15B1 337C F0FB 222B  D41A 1BCA 6586 A347 943F
   Johan Corveleyn [4096R/B59CE6D6010C8AAD] with fingerprint:
8AA2 C10E EAAD 44F9 6972  7AEA B59C E6D6 010C 8AAD

These public keys are available at:

https://www.apache.org/dist/subversion/subversion-1.10.7.KEYS

Release notes for the 1.10.x release series may be found at:

https://subversion.apache.org/docs/release-notes/1.10.html

You can find the list of changes between 1.10.7 and earlier versions at:

https://svn.apache.org/repos/asf/subversion/tags/1.10.7/CHANGES

Questions, comments, and bug reports to users@subversion.apache.org.

Thanks,
- The Subversion Team

--
To unsubscribe, please see:

https://subversion.apache.org/mailing-lists.html#unsubscribing

[SECURITY][ANNOUNCE] Apache Subversion 1.14.1 released

[Stefan Sperling]

I'm happy to announce the release of Apache Subversion 1.14.1.
Please choose the mirror closest to you by visiting:

https://subversion.apache.org/download.cgi#recommended-release

This is a stable bugfix and security release of the Apache Subversion
open source version control system.

THIS RELEASE CONTAINS AN IMPORTANT SECURITY FIX:

  CVE-2020-17525
  "Remote unauthenticated denial-of-service in Subversion mod_authz_svn"

The full security advisory for CVE-2020-17525 is available at:
  https://subversion.apache.org/security/CVE-2020-17525-advisory.txt

A brief summary of this advisory follows:

  Subversion's mod_authz_svn module will crash if the server is using
  in-repository authz rules with the AuthzSVNReposRelativeAccessFile
  option and a client sends a request for a non-existing repository URL.

  This can lead to disruption for users of the service.

  We recommend all users to upgrade to the 1.10.7 or 1.14.1 release
  of the Subversion mod_dav_svn server.

  As a workaround, the use of in-repository authz rules files with
  the AuthzSVNReposRelativeAccessFile can be avoided by switching
  to an alternative configuration which fetches an authz rules file
  from the server's filesystem, rather than from an SVN repository.

  This issue was reported by Thomas Åkesson.

SHA-512 checksums are available at:

https://www.apache.org/dist/subversion/subversion-1.14.1.tar.bz2.sha512
https://www.apache.org/dist/subversion/subversion-1.14.1.tar.gz.sha512
https://www.apache.org/dist/subversion/subversion-1.14.1.zip.sha512

PGP Signatures are available at:

https://www.apache.org/dist/subversion/subversion-1.14.1.tar.bz2.asc
https://www.apache.org/dist/subversion/subversion-1.14.1.tar.gz.asc
https://www.apache.org/dist/subversion/subversion-1.14.1.zip.asc

For this release, the following people have provided PGP signatures:

   Stefan Sperling [2048R/4F7DBAA99A59B973] with fingerprint:
8BC4 DAE0 C5A4 D65F 4044  0107 4F7D BAA9 9A59 B973
   Branko Čibej [4096R/1BCA6586A347943F] with fingerprint:
BA3C 15B1 337C F0FB 222B  D41A 1BCA 6586 A347 943F
   Johan Corveleyn [4096R/B59CE6D6010C8AAD] with fingerprint:
8AA2 C10E EAAD 44F9 6972  7AEA B59C E6D6 010C 8AAD

These public keys are available at:

https://www.apache.org/dist/subversion/subversion-1.14.1.KEYS

Release notes for the 1.14.x release series may be found at:

https://subversion.apache.org/docs/release-notes/1.14.html

You can find the list of changes between 1.14.1 and earlier versions at:

https://svn.apache.org/repos/asf/subversion/tags/1.14.1/CHANGES

Questions, comments, and bug reports to users@subversion.apache.org.

Thanks,
- The Subversion Team

--
To unsubscribe, please see:

https://subversion.apache.org/mailing-lists.html#unsubscribing

Re: svn diff -c does not accept HEAD

[Stefan Sperling]

On Tue, Dec 08, 2020 at 03:44:21PM +0300, Anton Shepelev wrote:
> Nathan Hartman:
> 
> > A possible rationale is that the HEAD revision could
> > change without your knowledge (e.g., another user commits
> > something in the meantime) and you wouldn't get the
> > revision you were expecting.
> >
> > When I want to see the diff of the most recent revision I
> > use 'svn log -l 1 --diff'. (Note, though, that will be
> > from the BASE revision, not HEAD.)
> 
> Thanks for the explanation, Nathan and Daniel, but svn diff
> -c does not support `BASE' either. Futhermore, from the
> viewpoint of consistency, it would be natural for -c and -r
> to accept exactly the same set of values, whose parsing
> would be implemented in the same piece of code.

I agree that it would seem natural. However, it was not done that way.
Here is the change which introduced the -c option 15 years ago:
https://svn.apache.org/r857128
The log message explicitly states "change number". So, unfortunately,
for whatever reason the author who contributed this code didn't write
the code to do what you expect it to do, and this inconsistency has
never been fixed.

If there is someone with enough motivation to fix this, please do.
Improvements to the consistency of the command line interface will
always be welcome. But someone would need to get the work done,
and these days we're no longer blessed with a huge amount of active
developers who have a lot of time on their hands :-/

Cheers,
Stefan

Re: Updated documentation

[Stefan Sperling]

On Fri, Sep 11, 2020 at 02:13:26PM -0500, Blake McBride wrote:
> Greetings,
> 
> I am a long time user and lover of svn.  It seems like the main
> documentation for SVN is for version 1.7 yet svn is now at 1.14.  Since the
> version hasn't gotten to 2.x, I presume that the changes are not overly
> significant.  So, is there better documentation, or are there plans to
> update the red-bean version?
> 
> Thanks!
> 
> Blake McBride

Hi Blake,

The best source of information about what changed since 1.7 are
release notes: https://subversion.apache.org/docs/release-notes/

It would be great if the book were expanded to reflect changes that
appeared after 1.7. Contributions to the book are welcome. See the
section "Feeback/Contributing" on http://svnbook.red-bean.com/

Regards,
Stefan

Re: Possible bug: "Searching tree conflict details" takes forever

[Stefan Sperling]

On Tue, Aug 04, 2020 at 06:37:07PM +, Jacob Weber wrote:
> Hi there. I'm doing a merge which seems to be doing a very long-running 
> operation (over an hour so far) when it gets to the "Searching tree conflict 
> details" step. I'm wondering if there's any way to avoid this.
> 
> I'm merging from a branch X where a directory was removed, into a branch Y 
> where that same directory was removed. In each case, the removal was actually 
> done as part of a reintegrate merge from a different branch.
> 
> I can reproduce this every time I revert and restart the merge. Unfortunately 
> I can't share the content of my repository.
> 
> The output is something like this:
> 
> $ cd branchY
> $ svn update
> $ svn merge ^/branchX
> --- Recording mergeinfo for merge of r1 through r2 into '.':
>  U   .
> --- Merging r20001 through r20100 into '.':
> Afoo
> Abar
>  G   .
> --- Recording mergeinfo for merge of r20001 through r20100 into '.':
>  G   .
> Summary of conflicts:
>   Text conflicts: 2
>   Tree conflicts: 7

The above per-path output shows no new conflicts. I suppose you were actually
seeing lines such as "Cfoo" somewhere, and they are just missing from
your example?

> Searching tree conflict details for 'path/to/conflict/dir' in repository:
> Checking r9000...
> 
> At this point it slowly starts decreasing the revision after "Checking". But 
> it seems to be going through my entire repository history, which will take 
> forever.
> 
> Is there any way to avoid this?

Yes. The option: --accept=postpone
for svn merge will bypass the conflict resolver and flag a tree conflict
in the working copy.

This conflict still needs to be resolved before the merge result can be
committed. Note that 'svn resolve' will try to scan history again, so you
will need to use some non-interactive variant of this command.

If one of the --accept option paremeters achieve the result you want to
resolve to, then you could use --accept with a suitable parameter (see
the output of 'svn help resolve' for a list).

Otherwise, use this command to simply clear the conflict marker:
svn resolve --accept=working path/to/conflict/dir
And then resolve the actual conflict manually as required.

> Mac OS 10.14.6
> SVN client 1.14.0, compiled Jul  4 2020, 20:57:11 on x86_64-apple-darwin18.7.0
> SVN server 1.8.13

Some conflicts may be avoided if the server was upgraded to SVN 1.10 or later.
Newer servers can help with avoiding some tree conflicts.
See for example, see https://svn.apache.org/r1760570 -- this bug should
affect you since it was first fixed in 1.9.5. Though I doubt it is related
to your problem at hand since it looks like your conflict is on a directory
rather than a file.

Unfortunately, deep history crawls cannot always be avoided. Since information
shown by the resolver can save people a lot of time trying to figure out what
happened, it is considered acceptable that the resolver may take a while to
obtain this information.

Of course, an hour is outside of this acceptable scope. There could also be
a bug to fix in this case. We've found and fixed similar problems before,
for example here: https://svn.apache.org/r1839662 (this particular problem
does *not* affect you since are using a 1.14 client).
But in order to figure this out we'd need a lot more information from you.
Ideally, a script which starts off with an empty repository, populates it
(with deep history if necessary), and then runs a series of SVN commands
which ends in a merge that triggers the problem.

Cheers,
Stefan

Re: Searching entire repository for a file (fastsvncrawler)

[Stefan Sperling]

On Thu, Jul 23, 2020 at 08:05:12AM -0700, Kenneth Porter wrote:
> --On Thursday, July 23, 2020 12:54 PM +0200 Stefan Sperling 
> wrote:
> 
> > Are you aware of the built-in svn list --search feature, which has
> > been available since SVN 1.10.0?
> > 
> > For example:
> > 
> > $ svn list --depth=infinity --search svn.c ^/subversion/trunk
> > subversion/svn/svn.c
> > $
> > 
> > Also as of SVN 1.10 the server supports a special-purpose 'list' request
> > to speed this up.
> > 
> > In any case, the fastest way to search will likely be with a file:// URL,
> > assuming you can get direct access to the repository for this purpose.
> 
> I was not. Very nice, particularly the pattern matching. Can I determine the
> server version remotely to see if this is supported?

Some servers will advertise the SVN version on pages which can be visited
with a web browser. But this depends on the server's configuration. You may
have to ask the administrator to be sure about the server's exact version.

If the server supports the feature it will send an "svn/list" DAV header.
The full capability string to look for in a trace of an SVN HTTP session
would be: "http://subversion.tigris.org/xmlns/dav/svn/list;

I suppose the easiest way forward might be to simply try it out and see
how long it takes.

> (Alas, I don't have
> direct access.) Is the globbing done on the server with the 1.10 support?

Yes, globbing would be done on the server side.

Re: Searching entire repository for a file (fastsvncrawler)

[Stefan Sperling]

On Thu, Jul 23, 2020 at 11:54:52AM +0200, Stefan Sperling wrote:
> On Wed, Jul 22, 2020 at 03:36:36PM -0700, Kenneth Porter wrote:
> > I need to locate a file in a client's large repository. I found
> > fastsvncrawler which uses svn_ra_do_status2 to rapidly dump the entire
> > repository as a directory listing. Has anyone built a Windows binary? Or
> > perhaps it's made it into the distribution? (I access the repo over a Cisco
> > VPN from Windows, or I'd just build it on Linux. I fear I'll have to learn
> > how to build Subversion on Windows, which looks daunting.)
> > 
> > <https://github.com/mithro/fastsvncrawler>
> > 
> > How it works:
> > 
> > <http://vcs.atspace.co.uk/2012/07/15/subversion-remote-api-listing-repository-with-status-request/>
> > 
> 
> Are you aware of the built-in svn list --search feature, which has
> been available since SVN 1.10.0?
> 
> For example:
> 
> $ svn list --depth=infinity --search svn.c ^/subversion/trunk
> subversion/svn/svn.c
> $

I forgot to mention that that this feature supports pattern matching,
and that the pattern argument may need quoting. From 'svn help list':

  --search ARG : use ARG as search pattern (glob syntax, case-
 and accent-insensitive, may require quotation marks
 to prevent shell expansion)

> Also as of SVN 1.10 the server supports a special-purpose 'list' request
> to speed this up.
> 
> In any case, the fastest way to search will likely be with a file:// URL,
> assuming you can get direct access to the repository for this purpose.
> 
> Regards,
> Stefan

Re: Searching entire repository for a file (fastsvncrawler)

[Stefan Sperling]

On Wed, Jul 22, 2020 at 03:36:36PM -0700, Kenneth Porter wrote:
> I need to locate a file in a client's large repository. I found
> fastsvncrawler which uses svn_ra_do_status2 to rapidly dump the entire
> repository as a directory listing. Has anyone built a Windows binary? Or
> perhaps it's made it into the distribution? (I access the repo over a Cisco
> VPN from Windows, or I'd just build it on Linux. I fear I'll have to learn
> how to build Subversion on Windows, which looks daunting.)
> 
> 
> 
> How it works:
> 
> 
> 

Are you aware of the built-in svn list --search feature, which has
been available since SVN 1.10.0?

For example:

$ svn list --depth=infinity --search svn.c ^/subversion/trunk
subversion/svn/svn.c
$

Also as of SVN 1.10 the server supports a special-purpose 'list' request
to speed this up.

In any case, the fastest way to search will likely be with a file:// URL,
assuming you can get direct access to the repository for this purpose.

Regards,
Stefan

Re: invalid status for updating properties

[Stefan Sperling]

On Tue, Jul 07, 2020 at 09:01:50AM +, Mikael Stålhammar wrote:
> Hi
> I'm having an issue with merging/releasing a feature branch to trunk, getting:
> 
> 
> --- Recording mergeinfo for merge between repository URLs into '.':
> 
> svn: E155023: Can't set properties on 
> 'E:\SVN\WorkingCopies\repo\trunk\folder\file.txt': invalid status for 
> updating properties.

This looks like the problem tracked in this issue:
https://issues.apache.org/jira/browse/SVN-4859

A fix is being worked on and has already been committed to trunk.
See https://svn.apache.org/r1879192

[ANNOUNCE] Apache Subversion 1.14.0 released

[Stefan Sperling]

I'm happy to announce the release of Apache Subversion 1.14.0.
Please choose the mirror closest to you by visiting:

https://subversion.apache.org/download.cgi#recommended-release

This is a stable feature release of the Apache Subversion open source
version control system.

SHA-512 checksums are available at:

https://www.apache.org/dist/subversion/subversion-1.14.0.tar.bz2.sha512
https://www.apache.org/dist/subversion/subversion-1.14.0.tar.gz.sha512
https://www.apache.org/dist/subversion/subversion-1.14.0.zip.sha512

PGP Signatures are available at:

https://www.apache.org/dist/subversion/subversion-1.14.0.tar.bz2.asc
https://www.apache.org/dist/subversion/subversion-1.14.0.tar.gz.asc
https://www.apache.org/dist/subversion/subversion-1.14.0.zip.asc

For this release, the following people have provided PGP signatures:

   Stefan Sperling [2048R/4F7DBAA99A59B973] with fingerprint:
8BC4 DAE0 C5A4 D65F 4044  0107 4F7D BAA9 9A59 B973
   Julian Foad [4096R/1FB064B84EECC493] with fingerprint:
6011 63CF 9D49 9FD7 18CF  582D 1FB0 64B8 4EEC C493
   Nathan Hartman (CODE SIGNING KEY) [4096R/583F00ADF981C39F] with fingerprint:
3F8E 467C B336 6E30 13E1  120D 583F 00AD F981 C39F
   Branko Čibej [4096R/1BCA6586A347943F] with fingerprint:
BA3C 15B1 337C F0FB 222B  D41A 1BCA 6586 A347 943F
   James McCoy (Debian) [4096R/DFE691AE331BA3DB] with fingerprint:
91BF BF4D 6956 BD5D F7B7  2D23 DFE6 91AE 331B A3DB
   Johan Corveleyn [4096R/B59CE6D6010C8AAD] with fingerprint:
8AA2 C10E EAAD 44F9 6972  7AEA B59C E6D6 010C 8AAD

These public keys are available at:

https://www.apache.org/dist/subversion/subversion-1.14.0.KEYS

Release notes for the 1.14.x release series may be found at:

https://subversion.apache.org/docs/release-notes/1.14.html

You can find the list of changes between 1.14.0 and earlier versions at:

https://svn.apache.org/repos/asf/subversion/tags/1.14.0/CHANGES

Questions, comments, and bug reports to users@subversion.apache.org.

Thanks,
- The Subversion Team

--
To unsubscribe, please see:

https://subversion.apache.org/mailing-lists.html#unsubscribing

[ANNOUNCE] Apache Subversion 1.14.0-rc2 released

[Stefan Sperling]

I'm happy to announce the release of Apache Subversion 1.14.0-rc2.
Please choose the mirror closest to you by visiting:

https://subversion.apache.org/download.cgi#pre-releases

SHA-512 checksums are available at:

https://www.apache.org/dist/subversion/subversion-1.14.0-rc2.tar.bz2.sha512
https://www.apache.org/dist/subversion/subversion-1.14.0-rc2.tar.gz.sha512
https://www.apache.org/dist/subversion/subversion-1.14.0-rc2.zip.sha512

PGP Signatures are available at:

https://www.apache.org/dist/subversion/subversion-1.14.0-rc2.tar.bz2.asc
https://www.apache.org/dist/subversion/subversion-1.14.0-rc2.tar.gz.asc
https://www.apache.org/dist/subversion/subversion-1.14.0-rc2.zip.asc

For this release, the following people have provided PGP signatures:

   Stefan Sperling [2048R/4F7DBAA99A59B973] with fingerprint:
8BC4 DAE0 C5A4 D65F 4044  0107 4F7D BAA9 9A59 B973
   James McCoy (Debian) [4096R/DFE691AE331BA3DB] with fingerprint:
91BF BF4D 6956 BD5D F7B7  2D23 DFE6 91AE 331B A3DB
   Nathan Hartman (CODE SIGNING KEY) [4096R/583F00ADF981C39F] with fingerprint:
3F8E 467C B336 6E30 13E1  120D 583F 00AD F981 C39F
   Julian Foad [4096R/1FB064B84EECC493] with fingerprint:
6011 63CF 9D49 9FD7 18CF  582D 1FB0 64B8 4EEC C493
   Branko Čibej [4096R/1BCA6586A347943F] with fingerprint:
BA3C 15B1 337C F0FB 222B  D41A 1BCA 6586 A347 943F
   Johan Corveleyn [4096R/B59CE6D6010C8AAD] with fingerprint:
8AA2 C10E EAAD 44F9 6972  7AEA B59C E6D6 010C 8AAD

These public keys are available at:

https://www.apache.org/dist/subversion/subversion-1.14.0-rc2.KEYS

This is a pre-release for what will eventually become version 1.14.0 of the
Apache Subversion open source version control system.  It may contain known
issues, a complete list of 1.14.0-blocking issues can be found
here:


https://issues.apache.org/jira/issues/?jql=project%20%3D%20SVN%20AND%20resolution%20%3D%20Unresolved%20AND%20fixVersion%20%3D%201.14.0%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC

A pre-release means the Subversion developers feel that this release
is ready for widespread testing by the community.  There are known issues
(and unknown ones!), so please use it at your own risk, though we do
encourage people to test this release thoroughly.  Of particular note, please
remember that persistent data, such as the working copy or repository
formats may change before the final release, and there may not be an
upgrade path from the pre-releases to the final.

As a note to operating system distro packagers: while we wish to have this
release candidate widely tested, we do not feel that it is ready for packaging
and providing to end-users through a distro package system.  Packaging a
release candidate poses many problems, the biggest being that our policy lets
us break compatibility between the release candidate and the final release, if
we find something serious enough.  Having many users depending on a release
candidate through their distro would cause no end of pain and frustration that
we do not want to have to deal with.  However, if your distro has a branch that
is clearly labeled as containing experimental and often broken software, and
explicitly destined to consenting developers and integrators only, then we're
okay with packaging the release candidate there.  Just don't let it near the
end users please.


Release notes for the 1.14.x release series may be found at:

https://subversion.apache.org/docs/release-notes/1.14.html

You can find the list of changes between 1.14.0-rc2 and earlier versions at:

https://svn.apache.org/repos/asf/subversion/tags/1.14.0-rc2/CHANGES

Questions, comments, and bug reports to users@subversion.apache.org.

Thanks,
- The Subversion Team

--
To unsubscribe, please see:

https://subversion.apache.org/mailing-lists.html#unsubscribing

Re: Preferred Subversion 1.13 MPM

[Stefan Sperling]

On Tue, Apr 07, 2020 at 09:43:40AM +0200, Stefan Sperling wrote:
> On Tue, Apr 07, 2020 at 12:05:49AM +0200, Carmen Alzzer wrote:
> > hi guys,
> > 
> > so im starting to tune a Subversion installation, that runs slow on merges.
> > So im starting looking into apache MPM - current its setup to event MPM in
> > Apache2.4, but i figure it perhaps should be preforked. Whats your
> > recommmandations ?
> 
> What makes you think changing the MPM would solve the problem?

To clarify, I was asking this to understand your reasoning, not trying
to question your competence (I"m sorry if it came across that way).

And if none of the suggestion I made apply to your situation, please
provide more details so we can try to look for more suggestions :)

Re: Preferred Subversion 1.13 MPM

[Stefan Sperling]

On Tue, Apr 07, 2020 at 12:05:49AM +0200, Carmen Alzzer wrote:
> hi guys,
> 
> so im starting to tune a Subversion installation, that runs slow on merges.
> So im starting looking into apache MPM - current its setup to event MPM in
> Apache2.4, but i figure it perhaps should be preforked. Whats your
> recommmandations ?

What makes you think changing the MPM would solve the problem?
 
> the installation is 80GBcode in 150+ repos - apache 2.4.43 and im upgrading
> subversion from 1.12.2 to 1.13 tomorrow - it runs on FreeBSD 11.3 and theres
> on LAN traffic going through http, and traffic is on an average day 700.000
> requests to the webserver all inclusive :)
> 
> The developers are complaining that merging takes way too long and theyd
> rather shift for git enterprise bling bling. any help is much appreciated :)

You're not providing much details but I'll write a list of common problems
below.

Keep in mind most of the work performed during merges occurs client-side.
If people are running old clients, ask them to upgrade.

There are some server-configuration factors which can increase latency
between client and servers, or which affect server-side performance in
general, and those could be relevant:

- Make sure that authentication is fast. You're not providing any details,
  but if you're using a centralized service like LDAP or AD make sure that
  the SVN server will get a response from this service as fast as possible.
  Avoid configurations which result in per-request round-trips between SVN
  server and another server, and if possible even avoid such round-trips
  altogether. Cache authorized credentials! (e.g. use LDAP cache in mod_ldap)

- If the SVN server is talking SSL/TLS to any other service for any purpose,
  ensure that certificates are valid and avoid certificate revocation
  mechanisms that require sending additional requests like OCSP.
  (Especially if OCSP info in certificates somewhere in the chain points at
  a non-existent hostname it will add a lot of overhead; I have seen this
  in the wild with certs generated from AD)

- Configure high TCP Keepalive limits to avoid too many authentication
  handshakes (i.e. bump MaxKeepAliveRequests, see:
  https://subversion.apache.org/docs/release-notes/1.8.html#neon-deleted)

- Make sure that path-based-authorization rules with mod_authz_svn are used
  only to the minimum extent necessary. Lookups can be very expensive so
  ideally you'd only put path-based authorization rules in place if you
  need to *hide* something from the view of some users and otherwise let
  anyone who manages to authenticate read anywhere in the repository.

- If your performance problems are due to disk reads (i.e. you see a very
  heavy I/O load on the SVN server) tune your server's in-memory caching:
  
https://subversion.apache.org/docs/release-notes/1.7.html#server-performance-tuning
  And consider a dump/load of large repositories that aren't FSFS v7/v8 yet:
  https://subversion.apache.org/docs/release-notes/1.9.html#fsfs-improvements
  And make sure to enable lz4 with FSFS v8 if you do a full dump/load cycle:
  https://subversion.apache.org/docs/release-notes/1.10.html#lz4

Re: Automatically supply the origin URL in svn merge

[Stefan Sperling]

On Thu, Mar 26, 2020 at 08:46:35PM +, Daniel Shahaf wrote:
> Stefan Sperling wrote on Thu, 26 Mar 2020 10:45 +0100:
> > Renames are modelled as copy+delete in SVN.
> 
> They are, but we can behave differently depending on whether or not the
> adding-with-history was accompanied by a deletion in the same revision.
> That is, if cwd is a working copy of foo, we'd behave differently in the
> case that foo had been created by «svnmucc cp HEAD bar foo» and in the
> case that foo had been created by «svnmucc cp HEAD bar foo rm bar».

It's not that simple, as the conflict resolver demonstates.
You're not taking cases into account where multiple copies within a
signle revision correspond to a single deletion.

So if you want to handle that then you must either error out when the
heuristic fails, or morph 'svn merge' into an interactive question and
answer game with the user, much like the conflict resolver can behave
when it finds ambiguity trying to match up copies and deletions.

All instead of requiring the user to simply type one simple argument on
the command line to run a merge!

The ultimate goal here is to allow users to type less characters when
starting merges. Scripting tab-completion for SVN URL arguments into
your favourite shell would also be a usable and effective solution.

Re: Automatically supply the origin URL in svn merge

[Stefan Sperling]

On Thu, Mar 26, 2020 at 01:10:25AM +0300, Anton Shepelev wrote:
> Daniel Shahaf:
> > however, I don't think the lack of these distinctions is
> > necessarily a blocker.  It just means we need to be more careful
> > about not writing automation that will help some cases and
> > backfire in others.
> 
> Certainly not.  I still hope that my proposal can be made safe.

There is a way to find out, but it requires some work:

Write a patch that implements your proposal.

And then ensure that all tests in Subversion's regression test suite
keep passing with that patch applied and with the URL argument removed
from every merge command that runs a sync-style merge throughout the
entire regression test suite.

I'm confident that the regression test suite is comprehensive enough to
catch any problems and if needed inspire further discussion about those 
problems in detail. It's hard to thoroughly evaluate your idea without
knowing which of the test cases will break and why.

I've used the test suite many times to try out random ideas I've had.
This approach works really well and is often quite enlightening!

In any case, what you're asking for implies that at a minimum either you
or someone else would have to invest time into actually doing the above
work in order to verify your idea and make it happen, regardless of which
potential problems are being discussed now.

Cheers,
Stefan

Re: Automatically supply the origin URL in svn merge

[Stefan Sperling]

On Thu, Mar 26, 2020 at 01:10:25AM +0300, Anton Shepelev wrote:
> Daniel Shahaf:
> > However, I can also see circumstances in which this smartness
> > could be counter-productive:
> > 
> > 1. In repository restructurings, such as running «svn merge» in a
> > working copy of ^/thebarproject/trunk after that project had been
> > renamed from ^/thefooproject/trunk.  Under the proposal, that
> > would attempt to merge from ^/thefooproject/trunk, and it's not
> > clear to me that that's a more useful behaviour than just
> > throwing a usage error.
> 
> You say the project was renamed, which means that `svn cp' was not
> involved.

Renames are modelled as copy+delete in SVN.

Cheers,
Stefan

Re: Automatically supply the origin URL in svn merge

[Stefan Sperling]

On Wed, Mar 25, 2020 at 04:45:29PM +0300, Anton Shepelev wrote:
> Hello, all
> 
> Why does even the basic sync-merging require that the user
> specify the source URL, as in:
> 
>   svn merge ^/project/trunk
> 
> I think this requirement is redundant because SVN know
> exactly from which original original directory the branch
> was created by svn cp.  Will you consider simplifying the
> basic merge syntax to:
> 
>   svn merge
>   svn merge 
> 
> so that SVN shall supply the origin URL automatically?  Or
> am I wrong, and my proposal makes no sense?

Recall that SVN cannot know whether a copy operation is supposed to
create a new branch, or to copy a folder within the scope of a branch.
This distinction exists only in the human mind, not in the tool itself.

Also recall that users have the ability to check out a working copy
from any level of the tree.

When we combine your idea with the above two aspects of SVN's design,
we run into a problem.

Example:

svn mkdir ^/trunk
svn mkdir ^/trunk/dir1
svn copy ^/trunk ^/branches/mybranch
svn copy ^/mybranch/dir1 ^/mybranch/dir2
svn checkout ^/mybranch/dir2 my-dir2-working-copy
cd my-dir2-working-copy
svn merge

What will happen now?

Re: Help needed with a unique type of branch creation

[Stefan Sperling]

On Wed, Mar 04, 2020 at 04:45:45AM +, Mun Johl wrote:
> Hi all,
> 
> Off of trunk in our repo are various directories.  I am trying to branch a 
> specific revision of two of the directories and am not being successful ... 
> the branch is always off of the HEAD revision.  Here's one attempt:
> 
> $ svn mkdir ^/branches/temp
> $ svn copy -r 1 svn://domain.com/trunk/dir-x 
> svn://domain.com/branches/temp
> $ svn copy -r 1 svn://domain.com/trunk/dir-y 
> svn://domain.com/branches/temp

> I'd rather not branch the entire trunk due to it's size and mostly unneeded 
> files.

What you are trying to do may look like a benefit in the short term. But once
you run sync merges from trunk, or merge your changes back to trunk, it will
work a lot better if SVN sees your branch as a full copy of trunk.

So I would recommend to always branch the entire trunk.

On the server this does not use any additional space.

On the client you can make your working copy "shallow" (also called "sparse")
to avoid the space issues and unneeded files.
See http://svnbook.red-bean.com/nightly/en/svn.advanced.sparsedirs.html

Regards,
Stefan

Re: How to improve search performance for moved directories and files?

[Stefan Sperling]

On Tue, Feb 25, 2020 at 09:09:14AM +0100, Thorsten Schöning wrote:
> Guten Tag Daniel Shahaf,
> am Montag, 24. Februar 2020 um 18:27 schrieben Sie:
> 
> > If the remote repository uses https://, you could set up mod_dav_svn on
> > localhost in a proxy configuration.  For svn:// the equivalent would be
> > to set up an svnsync mirror and do «svn relocate»'s to and from it by
> > hand.[...]
> 
> Thanks for the suggestions, but I can't expect my coworkers to do
> that. Some of them would simply prefer discussing if to keep using
> SVN at all in favour of ... we all know what. ;-)
> 
> I'm regularly getting the SVN-repos from the remote host using RSYNC
> locally anyway. So while not as correct as using svnsync in theory, I
> can simply do a 2 URL-merge using unrelated file-URIs with my local
> backups of the repos. That at least saves me from the relocate. The
> only thing I'm missing this way is merge tracking and merge recording.
> At least the latter can can be done after merging using the remote
> target again by telling the SVN-client to record the merge only. That
> is fast enough as no conflicts are triggered at all.
> 
> Two additional questions:
> 
> 1. Why does the number of revisions seem to matter that much?
> 
> This kind of merge conflict seems to become slower and slower as the
> number of revisions increases, even if all of those commits belong to
> totally unrelated branches. Additionally, the commits moving the
> directories and triggering the conflicts are not that far in the
> past, only very few hundreds of commits.
> 
> Something like the following: 100 auto-commits in branchA, very few
> commits moving directories in branchB, 100 auto-commits in branchA
> again. I would have expected the SVN-client focussing on branchB and
> finding the possible move targets in that branch pretty early.
> 
> 2. Really no other handbrake somewhere?
> 
> When doing the merge locally, I have a very high CPU-usage, but very
> little I/O, like constantly something around 40 Kbit/s. That doesn't
> matter locally especially in case using a SSD of course, but does
> remotely because of the additional latencies I guess.
> 
> So, is that simply how things work? Lots of small reads in those
> cases introducing lots of latency slowing things down heavily? And
> that can't be easily optimized further by e.g. any setting of the
> SVN-client?

The primary goal of the conflict resolver is not to be fast.

Consider the situation we had before the conflict resolver existed:
Each and every conflict had to be analyzed and resolved by a human, and
it was very easy to make mistakes. This cost literally hours and hours
of human time everywhere SVN was deployed.

The human conflict resolving timeframe is what the design of the conflict
resolver was up against.
The goal was to reduce these hours spent on resolving tree conflicts over
and over to a couple of minutes. The resolver tries to be accurate in its
detection of conflicts, provide sufficient flexibility when resolving
conflicts, and is also designed to be extensible (if there is a conflict
case that is not covered yet but should be, all that needs to be done is
adding about 3 functions, written in C code).

Another constraint is that the resolver should be able to work against old
SVN servers, since clients are more regularly updated to new releases than
already deployed servers. This means the resolver needs to do round-trips.
As it discovers information it keeps going back to the server until it has a
complete picture of the conflict situation. The server has no idea what the
client is really asking it for.

If you're unhappy with the result, I would suggest you become involved in
improving the implementation yourself. There should be room for improvement,
especially if the server was made smarter.

A situation with high latency tunnelling is naturally very hard to improve
with a client<->server roundtrip-heavy design.
For best performance you really want your SVN server on the LAN.

Re: How to help SVN finding moved directories during a sync-merge?

[Stefan Sperling]

On Tue, Feb 25, 2020 at 08:50:00AM +0100, Thorsten Schöning wrote:
> Guten Tag Stefan Sperling,
> am Montag, 24. Februar 2020 um 21:04 schrieben Sie:
> 
> > If several possible move targets exist, the 'svn' client will offer
> > 'd' and 'w' conflict options which may help you to set the right
> > destination paths.
> 
> It did, thanks. That's what I had expected from TSVN as well. But do
> you really mean "w" or "m"? Because I have been offered "d" and "m"
> only:
> 
> > Select: (p) Postpone, (r) Mark as resolved,
> > (m) Apply to move destination,
> > (d) Set repository move destination path, (h) Help,
> > (q) Quit resolution: d
> 
> Just wondering if "w" is something I need to keep in mind
> additionally.

It depends on the particular situation. "w" only happens in case there
is more than one possible move target inside the working copy, which
also depends on the particular repository-side path selected with "d".
If the path selected with "d" maps to a single path in the working copy
then "w" is not needed.

> > This feature only works if the SVN client implements support for it.
> > If TSVN did not do that, then they are missing this feature.
> > I am not sure how far TortoiseSVN went with their resolver implementation.
> 
> Am going to ask them, mabye I simply did something wrong or missed
> some setting or else.

Thanks! Perhaps this is indeed just missing in their UI.

Re: How to help SVN finding moved directories during a sync-merge?

[Stefan Sperling]

On Mon, Feb 24, 2020 at 06:03:05PM +0100, Thorsten Schöning wrote:
> Hi all,
> 
> I have a repo with 178'000 revisions, and need to sync-merge 2
> features branches: "feature1" and "feature2" created from "trunk" at
> different times and "feature1" merged into "feature2" successfully
> some days ago already. That merge needs to be repeated now, but I'm
> runnig into trouble.
> 
> The problem is that "feature2" is about refactoring the directory
> layout of "feature1", especially some directories got moved. I used
> "svn mv" for that so that it could keep track of that in theory. The
> following is a simplified example:
> 
> * feature1
>   * mod1/src/de/File1.java
>   * mod2/src/de/File2.java
> * feature2
>   * mod1/src/main/java/de/File1.java
>   * mod2/src/main/java/de/File2.java
> 
> During merging SVN runs into conflicts because the merge-target dirs
> of "mod1" and "mod2" don't exist anymore. I'm using TortoiseSVN and
> that tries to find the new merge targets and does so successfully for
> "mod1". The problem is that for the changes of "mod2", SVN suggests
> the target dir of "mod1" as well again instead of "mod2" and this way
> the conflict can't be resolved ever.
> 
> Is there some way to tell SVN the correct target dir? Or how should
> that be resolved? The only thing I can currently think of is manually
> applying patches.

Can you try this with the command line instead of TortoiseSVN please?

If several possible move targets exist, the 'svn' client will offer
'd' and 'w' conflict options which may help you to set the right
destination paths.

This feature only works if the SVN client implements support for it.
If TSVN did not do that, then they are missing this feature.
I am not sure how far TortoiseSVN went with their resolver implementation.

Re: How to resolve between working copies?

[Stefan Sperling]

On Mon, Feb 17, 2020 at 10:33:59AM +0100, Bo Berglund wrote:
> On Mon, 17 Feb 2020 10:10:53 +0100, Stefan Sperling 
> wrote:
> 
> >> So what can I do to get rid of this problem?
> >
> >First, you should put the file back on disk :)
> >Moving files away behind the back of SVN is never a good idea.
> 
> Yes, but the file was a non-versioned duplicate so it should not
> matter that I renamed it, right.

If a conflict is already recorded for a file, then it is best to leave
it on disk because the resolver may try to do someting with it based
on the options chosen by the users.

If you had moved the file aside before 'svn update' the conflict would
not have appeared in the first place.
'svn status' will show such unversioned files with '?'.

> >Then run 'svn resolve' again. When you are unsure what to choose, the option
> >to "merge" the files should be the best one. It will boil down 'add vs add'
> >to a text-conflict or perhaps even a clean merge, depending on file content.
> 
> I have never even seen the resolve command before...
> 
> >There are more options (try the 'h' option to see them), but generally
> >any "merge" options in the conflict resolver will always produce a useful
> >result when possible.
> >
> >Hope this helps!
> 
> Yes it does, thank you!
> Running svn resolve got me a chance to fix the problem.

Great, glad it helped :)

Re: How to resolve between working copies?

[Stefan Sperling]

On Mon, Feb 17, 2020 at 10:04:46AM +0100, Bo Berglund wrote:
> I have a project I am working on in both Windows and Linux.
> So the project is checked out on both places.
> Now I have a php file that was created in the Windows WC and copied to the 
> Linux WC 
> where its functions were tested on Apache after copying over to the website.
> Then I svn added it in the Linux WC and committed it along with other files.
> 
> Now when I want to update the Windows WC I get a message at the end of the 
> file listing:
> 
> Updated to revision 2867.
> Summary of conflicts:
>   Tree conflicts: 1
> Searching tree conflict details for 'ServerConfig\putcmdfile.php' in 
> repository:
> Checking r2861... done
> Tree conflict on 'ServerConfig\putcmdfile.php':
> A new file appeared during update to r2867; it was added by bosse in r2861.
> An unversioned file was found in the working copy.
> Select: (p) Postpone, (r) Mark as resolved, (m) Merge the files, (h) Help,
> (q) Quit resolution: q
> Summary of conflicts:
>   Tree conflicts: 1
> 
> Since I was unsure about how to deal with this I selected q to quit, but this 
> was apparently
> no good because now I cannot get rid of the error message whatever I do.
> I have tried renaming the file in the file system and then do a svn up to get 
> a fresh file from
> the server but it also fails:
> 
> ServerConfig>svn up
> Updating '.':
> At revision 2867.
> ServerConfig>svn st
> D C putcmdfile.php
>   >   local file unversioned, incoming file add upon update
> Summary of conflicts:
>   Tree conflicts: 1
> 
> So what can I do to get rid of this problem?

First, you should put the file back on disk :)
Moving files away behind the back of SVN is never a good idea.

Then run 'svn resolve' again. When you are unsure what to choose, the option
to "merge" the files should be the best one. It will boil down 'add vs add'
to a text-conflict or perhaps even a clean merge, depending on file content.

There are more options (try the 'h' option to see them), but generally
any "merge" options in the conflict resolver will always produce a useful
result when possible.

Hope this helps!

Re: False conflict with interleaved merge commits

[Stefan Sperling]

On Sat, Feb 08, 2020 at 09:57:47AM +0100, Stefan Sperling wrote:
> So, indeed, if there was a way to extend the heuristic to detect this case
> the problem could be avoided. I suspect the heuristic only ever looks at
> server-side mergeinfo and does not take mergeinfo in the local working copy
> into account? See client_find_automatic_merge() in libsvn_client/merge.c.

I have filed https://issues.apache.org/jira/projects/SVN/issues/SVN-4847

Re: False conflict with interleaved merge commits

[Stefan Sperling]

On Sat, Feb 08, 2020 at 04:01:06AM +, Daniel Shahaf wrote:
> That's the desired result, isn't it?  If so, is there an underlying rule
> here that we could teach Subversion to follow automatically?

The underlying issue is a decision made by the "automatic merge" heuristic.

Note the output of the "problematic" merge performed by the script:

--- Merging differences between repository URLs into 'wcb':
Cwcb/A/mu

"Merging differences between repository URls" means that the automatic
merge heuristic has decided to run a "reintegrate merge", rather than a
"sync merge". This decision is consistent with the state of mergeinfo in
the repository at the time the merge command is executed.

Changing the last line in the script as follows to force a sync merge
has the same result as your manual cherry-picking merges:

--- Merging r5 through r7 into 'wcb':
Uwcb/A/mu

So, indeed, if there was a way to extend the heuristic to detect this case
the problem could be avoided. I suspect the heuristic only ever looks at
server-side mergeinfo and does not take mergeinfo in the local working copy
into account? See client_find_automatic_merge() in libsvn_client/merge.c.

--- merge-interleave-3.sh.orig  Sat Feb  8 09:47:55 2020
+++ merge-interleave-3.sh   Sat Feb  8 09:46:37 2020
@@ -118,7 +118,7 @@
 # Step 9, the bug.
 # The following trunk->branch1 merge should complete successfully, as A/mu has 
only been edited on trunk and the first edit has already been merged to 
branch1, but instead it shows a conflict on A/mu as if the first merge never 
took place and as if A/mu was edited directly in branch1.
 ${SVN} up wc wcb
-${SVN} merge ${URL}/trunk wcb
+${SVN} merge -r0:HEAD ${URL}/trunk wcb
 
 # Put kill command in a file, in case need to run it manually.
 # echo "kill -9 `cat svnserve-pid`" > k

Re: False conflict with interleaved merge commits

[Stefan Sperling]

On Thu, Feb 06, 2020 at 03:37:10PM -0500, Daniel Dickison wrote:
> Strangely, the conflict goes away if you flip the order of steps 2 and 3,
> or commit the merge from step 4 first.

I would say it is working as designed, but the design has its flaws.

As you have found out, the basic problem is that unlike when developers
work on the same branch, SVN does not enforce an 'svn update' style
operation when merges and commits interleave.

This is indeed a problem but it is not easy to fix in SVN itself.
If this happens to your team a lot you might want to let your developers
know about the issue and show them how to work around it.

An easy way to work around it is the following, which essentially
simulates the missing 'svn update' style step. After a merge, and
before committing this merge, if the other branch has new changes
which were not yet merged, merge those changes as follows to "catch up":

svn merge -r0:HEAD ^/the/other/branch

Patch for your script:

--- merge-interleave-3.sh   Fri Feb  7 08:19:28 2020
+++ merge-interleave-4.sh   Fri Feb  7 08:29:59 2020
@@ -111,6 +111,7 @@
 ${SVN} merge ${URL}/branches/branch1 wc # Step 5
 ${SVN} pl -v -R wc
 ${SVN} commit wc -m 'Merge branch1 into trunk'  # Step 6
+${SVN} merge -r0:HEAD ${URL}/trunk wcb
 ${SVN} commit wcb -m 'Merge trunk into branch1' # Step 7
 
 # Step 8. edit A/mu one more time in trunk.

Note that I'm using -r0:HEAD to force a "sync merge" at that point.
Once the merge to the other branch has been committed, Subversion will
try to run the 'reintegrate' strategy which will refuse to operate on
a working copy which already contains local modifications.

To understand why your problem is happening, look at the generated mergeinfo.

With your script, the mergeinfo generated by the merge into 'wc' lacks r5:

Properties on 'wcb':
  svn:mergeinfo
/trunk:2-4

Properties on 'wc':
  svn:mergeinfo
/branches/branch1:2-4

As you point out, when merges and commits are done sequentually it works
as expected. Mergeinfo contains this instead:

Properties on 'wcb':
  svn:mergeinfo
/trunk:2-4

Properties on 'wc':
  svn:mergeinfo
/branches/branch1:2-5

r5 is the revision which records the merge from trunk to the branch.
Without r5 in trunk's mergeinfo, 'svn merge' will pick a merge strategy
that includes r5, which includes 'Edit 1'.

Which is why I see no way to fix this in SVN itself.
If we made SVN record r5 during the original merge (before r5 existed),
that would be wrong.
And if we made SVN skip A/mu edits from r5 during the conflicting merge,
that would be wrong, too.

So to me it looks like this needs to be handled by synchronizing
your developers. Sorry :-/


$ svn log -r5 --diff ^/
[...]
Index: branches/branch1/A/mu
===
--- branches/branch1/A/mu   (revision 4)
+++ branches/branch1/A/mu   (revision 5)
@@ -1 +1 @@
-This is the file 'A/mu'.
+Edit 1 of A/mu in trunk
Index: branches/branch1
===
--- branches/branch1(revision 4)
+++ branches/branch1(revision 5)

Property changes on: branches/branch1
___
Added: svn:mergeinfo
## -0,0 +0,1 ##
   Merged /trunk:r2-4

Re: question about file commit

[Stefan Sperling]

On Sat, Jan 11, 2020 at 10:20:01AM +0800, guan.j...@byd.com wrote:
> When I commit file to svn， a warning as follow， and failed to commit
> modified file

TortoiseSVN 1.8.7 is very old.
Please upgrade, and this problem should go away.
See https://tortoisesvn.net/downloads.html

Regards,
Stefan

Re: SVN Patch Vs GNU Patch

[Stefan Sperling]

On Fri, Jan 03, 2020 at 11:27:01PM +, Dipu Haridasan wrote:
> Hi Team,
> 
> I observed that for `svn patch` to apply properly, the spaces between 
> filename and revision in the patch must be a tab. However, the GNU patch 
> works irrespective of the number of spaces between filename and revision. For 
> example, the below patch can be applied using the GNU patch, however with 
> `svn patch` it fails to apply the patch as there are more spaces between 
> filename and revision argument (brcm_cos_utils.c (revision 1070911)).
> 
> Index: brcm_cos_utils.c
> ===
> --- brcm_cos_utils.c (revision 1070911)<<< Extra spaces 
> between filename and revision.
> +++ brcm_cos_utils.c   (working copy)
> @@ -1216,8 +1216,8 @@
> 
> Wondering if this can be considered as a feature request for svn, as GNU 
> (Unix) patch handles it irrespective of the spaces between filenames and 
> revision.
> 
> Thanks,
> Dipu H
> 

This was discussed just a few days ago on the dev@ list:
https://svn.haxx.se/dev/archive-2019-12/0108.shtml

'svn patch' works with the output of regular diff tools, 'svn diff', and the
diff output produced by other version control systems such as git and hg.
If a patch does not apply after transfer that means it was broken in transit.

Patch files can be mangled during transfer in any number of ways.
Adding workarounds for mangled patches is not a sustainable approach.

Re: Perforce checksum mismatch error while importing in svn

[Stefan Sperling]

On Mon, Sep 23, 2019 at 01:53:58PM +0200, Branko Čibej wrote:
> On 23.09.2019 13:20, Kumar, Raushan wrote:
> >
> > Hi Team,
> >
> >
> > I created a perforce dump using p42svn perl script, it created
> > successfully as showing in below screen.
> >
> > it is created successfully but while importing it on svn server
> > showing checksum mismatch error, as showing in below screen.
> >
> 
> This is obviously a bug in the p42svn script. It's calculating the
> checksums incorrectly, and there's not much Subversion can do after the
> fact.
> 
> Sure, we could add an option to 'svnadmin load' to ignore checksums in
> the dump file, but IMO that would be a really silly thing to do, because
> the purpose of those checksums is verifying data integrity, and without
> that verification one might as well move the repository to /dev/null.
> 
> You really should report this to the authors of the p42svn script.

If using SVN 1.8, try using 'svnadmin load -M 0'.

1.8 has a caching bug in 'svnadmin load' which results in this kind of error.
Setting the memory cache size to zero with the -M option works around it.

Re: Remote renames create subtree mergeinfo

[Stefan Sperling]

On Tue, Aug 27, 2019 at 09:41:37AM -0700, bryce.scho...@gmail.com wrote:
> FWIW, I found the explanations in these two emails from the same thread to
> be easier to understand as a user:
> https://svn.haxx.se/users/archive-2010-11/0408.shtml
> https://svn.haxx.se/users/archive-2010-11/0466.shtml
> 
> This sounds like yet another UX flaw caused by the constraints of
> subversion's characteristic "flexibility" afforded by its nearly-complete
> agnosticism regarding repository branching and tagging structure. As I use
> git more and more for all of my daily development, I continue to run into
> UX problems like this that are made so much less helpful and more
> surprising, all in the name of that ultimate "everything is just a
> sub-tree" flexibility. I am coming to strongly believe that this design
> paradigm is SVN's fatal flaw keeping it from being the best long-term
> centralized VCS competitor to git & other DVCSes.
> 
> If subversion were to have a configurable "strict structure" mode that
> could both enforce and simplify its behavior in use-cases like this, it
> could be a big win. It could even provide simplified "tag" and "branch"
> aliases to "svn copy", as well as "--tag" and "--branch" options to switch,
> merge, etc. I suppose directions like that may have been taken before with
> svn wrapper systems, but those never catch on or survive unless they are
> upstream.

Many such ideas have been proposed and discussed over the past years.

What has always been getting in the way of making such changes now are SVN"s
strong backwards-compatibility promises. These have been of great advantage
to both the project and its users, but the flip side is that making invasive
changes is getting increasingly hard as more features get added over time.

If the people involved in designing merge-tracking for SVN 1.5 had known
what we know today, things would likely look very different. Mergeinfo
would ideally be stored as pure meta-data and remain mostly invisible to
the user. Tree conflict detection should have been built in from day one.
But all that is water under the bridge at this point.

Furthermore, today, the project lacks sufficient developer resources to
make very large changes in the first place. We will keep maintaining the
code for years to come, but large new features are unlikely to be developed
unless the development community grows again.

Cheers,
Stefan

Re: Upgraded to SVN 12 and now receiving very odd results via mod dav svn.

[Stefan Sperling]

On Tue, Jul 23, 2019 at 02:33:59PM -0400, ken edward wrote:
> Hello,
> 
> I am configured as below:
> 
> Apache/2.4.39 (Unix) OpenSSL/1.0.2r SVN/1.12.0 mod_auth_gssapi/1.6.1
> mod_dav_svn configured
> 
> Apache+SVN comes up clean, no errors.
> 
> I pointed the httpd.conf repo locations and access control files to
> the legacy 1.8 repo's and get varying results. Some of the repo's
> display normally, but many just show the header and revision number,
> but everything else is blank (shows nothing in the project, but there
> is).
> 
> I tried reloading the 1.8 repos into clean 1.12 repo's, but same
> issue. Zero errors in the log file. If I try to view the repo via the
> browser or tortise SVN I just see:
> 
> camd_repo1 - Revision 451: /
> [should see content here, but is empty]
> 
> 
> ANY IDEAS?

Check release notes of 1.9, 1.10, 1.11, and 1.12 for compatibility notes:
http://subversion.apache.org/docs/release-notes/

My best guess is your access control files are causing problems:
http://subversion.apache.org/docs/release-notes/1.10.html#authz-compatibility

Re: Is Permanently Accept SSL Certificate gone in 1.10.4 ?

[Stefan Sperling]

On Fri, Jul 19, 2019 at 09:52:32PM +, Daniel Shahaf wrote:
> Stefan Sperling wrote on Fri, 19 Jul 2019 18:45 +00:00:
> > It looks like the interactive prompt omits an option to save the cert
> > if it sees a certificate failure of class 'other' from the above list.
> > I am not sure why this decision was made but that's what the current
> > code seems to do.
> 
> The rationale is that if we don't know what the failure reason _is_, we
> don't know whether it's safe to ignore it permanently.  In other words,
> it only offers "permanently" if the failure bits are all whitelisted.
> 
> The downside is that there's no easy way for a user to say "I know what
> I'm doing, and I _do_ want to ignore this permanently; make it so", such
> as a utility that takes a PEM form certificate (on, say, stdin) and
> marks it as permanently trusted.

At the point where we're already asking the user, we might as well
let the user decide what to do, in any case.

Yes, some people might then save a bad cert without knowing any better.

But as a user I find it infuriating when software I use contains
artificial restrictions like this. We should assume our users know
what they are doing. Subversion is not a web browser.

Re: Is Permanently Accept SSL Certificate gone in 1.10.4 ?

[Stefan Sperling]

On Fri, Jul 19, 2019 at 08:38:57PM +0200, Stefan Sperling wrote:
> On Fri, Jul 19, 2019 at 01:40:52PM +0200, Pierre Fourès wrote:
> > Hi all,
> > 
> > I have a script accessing an old svn server whom SSL certificate have
> > expired a long time ago. Up to now, I was permanently accepting the
> > certificate on the first run of the script and then everything was
> > sailling smooth. I reinstalled a couple of months ago a new box where
> > this script was intented to run and the (p)ermanently option seems not
> > provided anymore.
> 
> If you're scripting 'svn' you should be using the --non-interactive option.
> 
> In which case your script can use the --trust-server-cert-failures
> option to accept a cert in pre-determined failure cases.
> 
> 'svn help update', for example, displays the following information
> section about the --trust-server-cert-failures option:
> 
>   --trust-server-cert-failures ARG : with --non-interactive, accept SSL server
>  certificates with failures; ARG is 
> comma-separated
>  list of 'unknown-ca' (Unknown Authority),
>  'cn-mismatch' (Hostname mismatch), 'expired'
>  (Expired certificate), 'not-yet-valid' (Not yet
>  valid certificate) and 'other' (all other not
>  separately classified certificate errors).
> 
> Once your script uses this option it should work out of the box against
> your problematic server and there should be no need to save the cert.

Follow-up regarding your actual question:

It looks like the interactive prompt omits an option to save the cert
if it sees a certificate failure of class 'other' from the above list.
I am not sure why this decision was made but that's what the current
code seems to do. So I suspect your SSL cert is failing for some reason
other than unknown-ca, cn-mismatch, expired, not-yet-valid.

Additionally, the ability to save a cert is also disabled if the
--no-auth-cache option is used.

Re: Is Permanently Accept SSL Certificate gone in 1.10.4 ?

[Stefan Sperling]

On Fri, Jul 19, 2019 at 01:40:52PM +0200, Pierre Fourès wrote:
> Hi all,
> 
> I have a script accessing an old svn server whom SSL certificate have
> expired a long time ago. Up to now, I was permanently accepting the
> certificate on the first run of the script and then everything was
> sailling smooth. I reinstalled a couple of months ago a new box where
> this script was intented to run and the (p)ermanently option seems not
> provided anymore.

If you're scripting 'svn' you should be using the --non-interactive option.

In which case your script can use the --trust-server-cert-failures
option to accept a cert in pre-determined failure cases.

'svn help update', for example, displays the following information
section about the --trust-server-cert-failures option:

  --trust-server-cert-failures ARG : with --non-interactive, accept SSL server
 certificates with failures; ARG is comma-separated
 list of 'unknown-ca' (Unknown Authority),
 'cn-mismatch' (Hostname mismatch), 'expired'
 (Expired certificate), 'not-yet-valid' (Not yet
 valid certificate) and 'other' (all other not
 separately classified certificate errors).

Once your script uses this option it should work out of the box against
your problematic server and there should be no need to save the cert.

Regards,
Stefan

Re: E235000 assertion failed in commit_util.c

[Stefan Sperling]

On Thu, May 23, 2019 at 09:46:56AM +1200, Jason Pickens wrote:
> Hi,
> 
> I am getting an error when trying to use the SlickSVN client from within
> IntelliJ. I initially contacted SlickSVN but they said that the issue is in
> the SVN library. I am using version 1.8.5.

SVN 1.8 is no longer supported by this community.
Please try to reproduce the problem with a supported version of SVN.
See http://subversion.apache.org/docs/release-notes/#supported-versions
for a list of supported versions (I would recommend 1.12.0 on the
client side, and 1.10 on the server side).

> The error is:
> svn: E235000: In file '..\..\..\subversion\libsvn_client\commit_util.c'
> line 211: assertion failed (repos_root_url && repos_relpath)
> 
> I was unable to commit at the time. I have discarded all my changes and now
> it seems to be working although at the time I was able to reproduce it. It
> seems like a bug.

Yes, but it is unlikely to get fixed without more information since
it is unclear how this problem could be reproduced by someone else.
Could you try to find a reproduction recipe which produces the problem
in isolation on a fresh repository?

Thanks,
Stefan

Re: svn version 1.10 lack of robustness in presence of flaky network

[Stefan Sperling]

On Wed, Apr 24, 2019 at 12:55:47AM +0200, Johan Corveleyn wrote:
> On Mon, Apr 22, 2019 at 9:22 AM Marlow, Andrew
>  wrote:
> > Hello everyone,
> >
> > I got this error below during an svn co command. It left my workspace in a 
> > bad state from which I had to do svn cleanup before trying again (the retry 
> > worked):
> >
> > svn: E200033: Another process is blocking the working copy database, or the 
> > underlying filesystem does not support file locking; if the working copy is 
> > on a network filesystem, make sure file locking has been enabled on the 
> > file server
> > svn: E200033: sqlite[S5]: database is locked
> > svn: E200042: Additional errors:
> > svn: E200033: sqlite[S5]: database is locked
> > svn: E200030: sqlite[S1]: cannot start a transaction within a transaction
> > svn: E200030: sqlite[S1]: cannot start a transaction within a transaction
> >
> > I think this happens when the network is flaky. This error happened on 
> > windows but I have also seen it happen on solaris 10.  Has anyone else seen 
> > this? If it is due to network flakiness then perhaps svn should retry to 
> > work around this transparently, and thus be more robust? Perhaps it could 
> > retry up to 3 times with a sleep a 1 second between retries?
> >
> 
> Is your working copy on a network filesystem (CIFS, NFS, ...)? And are
> you talking about a flaky network between your machine and its
> networked filesystem? If so, I think there is not much we can do about
> it ... the filesystem you're checking out to should reliable. There is
> already a retry loop in some places for putting checked out files into
> place, to work around locks from antivirus software etc, ... (but the
> sqlite database itself should be reliably available).

While working copies on networks filesystems should generally work,
such use is strongly discouraged.

So far, all reasons I've heard for putting working copies on network
drives turned out to be backed by bad or misinformed decisions about
the development process or allocation of hardware resources.
Moving to local disks improved not just the SVN user experience but
also repaired a broken process in such cases.

So put working copies on a local disk, preferrably SSDs. Working copies
should be considered temporary and disposable copies of your data.
The repository on the server is the important and permanent database
which must be protected and backed up, not the working copy.

If your working copy is too large for a modern SSD (really?),
consider sparse working copies
(http://svnbook.red-bean.com/nightly/en/svn.advanced.sparsedirs.html)
and/or reorganize your project such that parts of it can be checked
out and built in isolation.

That said, if you really must use a network filesystem, you should
look into tweaking the following options in Subversion's 'config' file:

### Section for configuring working copies.
[working-copy]
### Set to a list of the names of specific clients that should use
### exclusive SQLite locking of working copies.  This increases the
### performance of the client but prevents concurrent access by
### other clients.  Third-party clients may also support this
### option.
### Possible values:
###   svn(the command line client)
# exclusive-locking-clients =
### Set to true to enable exclusive SQLite locking of working
### copies by all clients using the 1.8 APIs.  Enabling this may
### cause some clients to fail to work properly. This does not have
### to be set for exclusive-locking-clients to work.
# exclusive-locking = false
### Set the SQLite busy timeout in milliseconds: the maximum time
### the client waits to get access to the SQLite database before
### returning an error.  The default is 1, i.e. 10 seconds.
### Longer values may be useful when exclusive locking is enabled.
# busy-timeout = 1

Re: question on merging revisions/svn patch

[Stefan Sperling]

On Mon, Apr 22, 2019 at 01:29:15PM -0400, Kathleen Warren wrote:
> Hi all,
> 
> I've gotten myself into a bit of a svn merging funk and I'd like to know if
> my proposed solution will cause further errors.
> 
> Due to a misunderstanding of how svn handles mergeinfo, I merged several
> commits from branch A into trunk like so: svn merge -c 1,2,3 ^/branches/A,
> with the idea of merging all three of those commits into branch B and only
> one into branch C.
> I realized when attempting to merge from trunk to branches B, C that I
> couldn't merge from trunk to get the content of those merges unless I
> specified the merge commit (let's call it revision 4) rather than any one
> or all of merges 1,2,3 as before.
> So I merged the merge commit into branch B, so now branch B has the content
> changes I want.
> But what about branch C? I only want one of those commits. My idea was to
> record a mergeinfo change and manually apply the patch and commit THAT -
> will that cause bad merge conflicts down the line?
> Thanks,
> Kathleen

Hi Kathleen,

Have you considered merging the change in question directly from branch A
to branch C, instead of merging the merged change from trunk to branch C?
Merging into C like this should work: svn merge -c 2 ^/branches/A

Whether or not you'll see conflicts down the line is hard to say
without knowing more about the nature of these changes.

Regards,
Stefan

Re: Segfault in svnserve on UB 16.04 LTS sometimes

[Stefan Sperling]

On Fri, Apr 12, 2019 at 11:42:54AM +0200, Thorsten Schöning wrote:
> Does the error tell anyone of you anything already?

No.

> What should I do
> to get some hints about which repo is affected with which operation?

Install a package with debug symbols, enable core dumps, and get
a backtrace with gdb from a core dump.

Re: apr-1.7.0 breaks subversion build

[Stefan Sperling]

On Tue, Apr 09, 2019 at 05:52:30PM +0200, Stefan Sperling wrote:
> You are right, this looks like a problem in Subversion's configure script.
> It assumes that APR uses "d", "ld" or "lld" for this format string which
> is now a wrong assumption.

Pierre, could you please try this patch against Subversion and let
me know if it helps?

Index: build/ac-macros/swig.m4
===
--- build/ac-macros/swig.m4 (revision 1857217)
+++ build/ac-macros/swig.m4 (working copy)
@@ -132,6 +132,10 @@ AC_DEFUN(SVN_FIND_SWIG,
 dnl Sun Forte adds an extra space before substituting APR_INT64_T_FMT
 dnl gcc-2.95 adds an extra space after substituting APR_INT64_T_FMT
 dnl thus the egrep patterns have a + in them.
+dnl
+dnl APR >= 1.7.0 uses PRId64 from inttypes.h on many platforms, so if
+dnl we fail to match APR_INT64_T_FMT to a known format string, try to
+dnl deduce the format string from inttypes.h instead of apr.h.
 SVN_PYCFMT_SAVE_CPPFLAGS="$CPPFLAGS"
 CPPFLAGS="$CPPFLAGS $SVN_APR_INCLUDES"
 AC_CACHE_CHECK([for apr_int64_t Python/C API format string],
@@ -154,6 +158,24 @@ AC_DEFUN(SVN_FIND_SWIG,
   MaTcHtHiS APR_INT64_T_FMT EnDeNd],
  [svn_cv_pycfmt_apr_int64_t="i"])
   fi
+  if test "x$svn_cv_pycfmt_apr_int64_t" = "x"; then
+AC_EGREP_CPP([MaTcHtHiS +\"lld\" +EnDeNd],
+ [#include 
+  MaTcHtHiS PRId64 EnDeNd],
+ [svn_cv_pycfmt_apr_int64_t="L"])
+  fi
+  if test "x$svn_cv_pycfmt_apr_int64_t" = "x"; then
+AC_EGREP_CPP([MaTcHtHiS +\"ld\" +EnDeNd],r
+ [#include 
+  MaTcHtHiS PRId64 EnDeNd],
+ [svn_cv_pycfmt_apr_int64_t="l"])
+  fi
+  if test "x$svn_cv_pycfmt_apr_int64_t" = "x"; then
+AC_EGREP_CPP([MaTcHtHiS +\"d\" +EnDeNd],
+ [#include 
+  MaTcHtHiS PRId64 EnDeNd],
+ [svn_cv_pycfmt_apr_int64_t="i"])
+  fi
 ])
 CPPFLAGS="$SVN_PYCFMT_SAVE_CPPFLAGS"
 if test "x$svn_cv_pycfmt_apr_int64_t" = "x"; then

Re: apr-1.7.0 breaks subversion build

[Stefan Sperling]

On Tue, Apr 09, 2019 at 05:44:31PM +0200, Stefan Sperling wrote:
> On Tue, Apr 09, 2019 at 04:57:56PM +0200, Pierre Labastie wrote:
> > Hi,
> > 
> > Following the instructions for reporting a bug, I write to this list.
> > I'm not subscribed to the list, so thanks for CCing me when replying.
> > I'm Pierre Labastie, one of the maintainers of BLFS [1]. I use the build
> > method described in the BLFS development book.
> > Please ask if you need further details.
> > 
> > Here are the last lines of the output of configure for subversion-1.11.1,
> > with apr-1.7.0 installed:
> 
> > configure: error: failed to recognize APR_INT64_T_FMT on this platform
> > -
> > I think it is because apr.h now defines APR_INT64_T_FMT as PRId64,
> > instead of formerly %ld. I've not found anything related in the subversion
> > repository, nor in the bug tracker. Please forgive me if this has already
> > been reported.
> 
> This was an intentional change in APR: https://svn.apache.org/r1856042
> 
> If your platform has problems with this change, there's nothing the
> Subversion project can do about this. Please talk to the APR project
> instead: https://apr.apache.org

I was a bit too fast with my assessment of your situation, sorry.

You are right, this looks like a problem in Subversion's configure script.
It assumes that APR uses "d", "ld" or "lld" for this format string which
is now a wrong assumption.

As a workaround, try setting the svn_cv_pycfmt_apr_int64_t environment
variable manually before running the configure script:
export svn_cv_pycfmt_apr_int64_t="L"

Re: apr-1.7.0 breaks subversion build

[Stefan Sperling]

On Tue, Apr 09, 2019 at 04:57:56PM +0200, Pierre Labastie wrote:
> Hi,
> 
> Following the instructions for reporting a bug, I write to this list.
> I'm not subscribed to the list, so thanks for CCing me when replying.
> I'm Pierre Labastie, one of the maintainers of BLFS [1]. I use the build
> method described in the BLFS development book.
> Please ask if you need further details.
> 
> Here are the last lines of the output of configure for subversion-1.11.1,
> with apr-1.7.0 installed:

> configure: error: failed to recognize APR_INT64_T_FMT on this platform
> -
> I think it is because apr.h now defines APR_INT64_T_FMT as PRId64,
> instead of formerly %ld. I've not found anything related in the subversion
> repository, nor in the bug tracker. Please forgive me if this has already
> been reported.

This was an intentional change in APR: https://svn.apache.org/r1856042

If your platform has problems with this change, there's nothing the
Subversion project can do about this. Please talk to the APR project
instead: https://apr.apache.org

Re: svn and svnserve hanging

[Stefan Sperling]

On Mon, Apr 08, 2019 at 05:05:47PM +0200, Vincent Lefevre wrote:
> On 2019-04-08 16:38:37 +0200, Stefan Sperling wrote:
> > Since you have a way to reproduce the problem, even if unreliably,
> > you're in a position to help.
> 
> But it could take weeks...

Perhaps you can find a way to trigger it on purpose?

> > If you don't try, we'll have to wait until someone else figures out where
> > the hang occurs or provides a clear and working reproduction recipe.
> > 
> > Sorry, but bugs don't get fixed by magic ;)
> 
> Well, I forgot, there's at least an issue with svnserve. I got that
> in the past, and could reproduce it: once I kill the svn client
> with Ctrl-C and kill the SSH master too, the sshd terminates, but
> svnserve is still running, and now has 1 as its PPID:
> 
> UIDPID  PPID  C STIME TTY  TIME CMD
> vlefevre 48724 1  0 12:40 ?00:00:00 svnserve -t

Hmm... that looks like the svnserve process was in zombie state
and is not being terminated. It is the init system's job to call
waitpid() for such processes.
I cannot tell how this is supposed to work in your server environment.
 
> If svnserve was waiting for something from sshd, it should at least
> have noticed that the connection is no longer there.

There is no connection as far as svnserve is concered.
In tunnel mode, svnserve reads from stdin and write to stdout, and
it is done when these get closed or the process is terminated for
some reason.

I suspect the problem is caused either in SSH or somewhere else on
your server.

> lsof still shows:
> 
> svnserve  48724  vlefevre  cwd   DIR   0,32 4096 
> 110673531 /home/users/vlefevre (...:/forge_home)
> svnserve  48724  vlefevre  rtd   DIR  254,0 4096 
> 2 /
> svnserve  48724  vlefevre  txt   REG  254,086256  
> 9413 /usr/bin/svnserve
> svnserve  48724  vlefevre0r FIFO0,8  0t0 
> 957950313 pipe
> svnserve  48724  vlefevre1w FIFO0,8  0t0 
> 957950314 pipe
> svnserve  48724  vlefevre2w FIFO0,8  0t0 
> 957950315 pipe
> svnserve  48724  vlefevre3r  CHR1,9  0t0  
> 7109 /dev/urandom
> 
> BTW, is the permanent open of /dev/urandom normal?

Those should all be open file handles that will get closed when the
process exists. Subversion relies on APR pool cleanup handlers to close
files in many cases, and perhaps those handlers haven't been run yet.

Re: svn and svnserve hanging

[Stefan Sperling]

On Mon, Apr 08, 2019 at 04:21:37PM +0200, Vincent Lefevre wrote:
> On 2019-04-08 15:26:15 +0200, Stefan Sperling wrote:
> > On Mon, Apr 08, 2019 at 02:38:15PM +0200, Vincent Lefevre wrote:
> > > On 2019-04-08 13:57:32 +0200, Stefan Sperling wrote:
> > > > There is insufficient information in your report for anyone to act upon.
> > > > Most importantly, it is unclear which component in the chain is causing
> > > > the problem. Is it SVN? Is it SSH? Is it TCP?
> > > > Please try to find answers to these questions.
> > > 
> > > I'm using SSH connection sharing and the master (started at 12:39,
> > > i.e. the same master as with the hanging svn) is still working.
> > 
> > Interesting. Does the problem also occur if you disable SSH
> > connection sharing completely? Perhaps it is related?
> 
> This problem occurs not often enough to know (except by doing
> background tests in a loop).

Since you have a way to reproduce the problem, even if unreliably,
you're in a position to help.

If you don't try, we'll have to wait until someone else figures out where
the hang occurs or provides a clear and working reproduction recipe.

Sorry, but bugs don't get fixed by magic ;)

Re: svn and svnserve hanging

[Stefan Sperling]

On Mon, Apr 08, 2019 at 02:38:15PM +0200, Vincent Lefevre wrote:
> On 2019-04-08 13:57:32 +0200, Stefan Sperling wrote:
> > There is insufficient information in your report for anyone to act upon.
> > Most importantly, it is unclear which component in the chain is causing
> > the problem. Is it SVN? Is it SSH? Is it TCP?
> > Please try to find answers to these questions.
> 
> I'm using SSH connection sharing and the master (started at 12:39,
> i.e. the same master as with the hanging svn) is still working.

Interesting. Does the problem also occur if you disable SSH
connection sharing completely? Perhaps it is related?
 
> > You could use strace to figure out which system call programs are
> > sleeping in, e.g.: strace -p 48724
> 
> Unfortunately strace is not installed on the server.
> 
> Concerning the svn client, that's
> 
> strace: Process 19549 attached
> read(6,
> 
> and for the SSH client:
> 
> strace: Process 19565 attached
> restart_syscall(<... resuming interrupted restart_syscall ...>

These look as expected.

> > You could try to get some information out of SSH? Perhaps the sshd
> > server logs contain clues?
> 
> I've asked the sysadmin, but since the SSH master still works,
> I doubt that this would be related.

How can you know that? My and your guesses are only as good as any
other guesses. Until more information is surfaced which pins things
down further we simply don't know anything at all about this problem.

> I don't know how to reproduce the problem. The master is already
> started with -vvv, but there are no dates. However, I don't see
> anything suspicious.
> 
> After doing a SSH connection via the master and quitting, I get
> concerning the status:
> 
> debug3: channel 3: status: The following connections are open:
>   #1 mux-control (t16 r2 i0/0 o0/0 e[closed]/0 fd 6/6/-1 sock 6 cc -1)
>   #2 client-session (t4 r0 i0/0 o0/0 e[write]/0 fd 7/8/9 sock -1 cc 1)
>   #3 mux-control (t16 nr0 i3/0 o3/0 e[closed]/0 fd 10/10/-1 sock 10 cc -1)
> 
> #2 can only be the session of the hanging svn.

That looks alright to me.

Re: svn and svnserve hanging

[Stefan Sperling]

On Mon, Apr 08, 2019 at 01:44:37PM +0200, Vincent Lefevre wrote:
> I've run a "svn diff" with the -c option, and it is hanging.
> The corresponding "svnserve -t" on the server is hanging too.
> 
> After one hour, on the client side, the svn command is still running,
> together with
> 
> vinc17   19550 19549  0 12:40 pts/11   00:00:00 zsh /home/vinc17/scripts/ssh 
> gforge svnserve -t
> vinc17   19565 19550  0 12:40 pts/11   00:00:00 ssh -F 
> /home/vinc17/.ssh/config -C gforge svnserve -t
> 
> and on the server side:
> 
> root 48651   878  0 12:39 ?00:00:00 sshd: vlefevre [priv]
> vlefevre 48655 48651  0 12:39 ?00:00:00 sshd: vlefevre@notty
> vlefevre 48724 48655  0 12:40 ?00:00:00 svnserve -t
> 
> Such an issue occurs from time to time.
> 
> Version on the client side:
> 
> svn, version 1.10.4 (r1850624)
>compiled Jan 23 2019, 03:41:34 on x86_64-pc-linux-gnu
> 
> Version on the server side:
> 
> svnserve, version 1.8.10 (r1615264)
>compiled May 21 2018, 00:10:07 on x86_64-pc-linux-gnu
> 
> A search on the web shows many reports about hanging svn.

Hi Vincent!

There is insufficient information in your report for anyone to act upon.
Most importantly, it is unclear which component in the chain is causing
the problem. Is it SVN? Is it SSH? Is it TCP?
Please try to find answers to these questions.

You could use strace to figure out which system call programs are
sleeping in, e.g.: strace -p 48724

You could try to get some information out of SSH? Perhaps the sshd
server logs contain clues?

On the client side you can do this: export SVN_SSH='ssh -q -vvv --'
and try to reproduce the problem. Maybe the SSH client will print
something useful when the problem occurs?

Re: Replacing directory by circular symlink produces malformed XML

[Stefan Sperling]

On Thu, Feb 28, 2019 at 10:44:25PM +0100, Denis Excoffier wrote:
> Hello,
> 
> The situation of the replacement of a directory by a circular symlink
> generates an XML fragment which is invalid, hence triggers a failure
> within the calling system. See hereafter the recipe.
> 
> Starting from a clean folder, do the following:
> 
> % svn mkdir 1
> A 1
> % rmdir 1
> % ln -s 1 1
> % touch 2
> % svn status --xml
> 
> 
> path=".">
> path="1">
> item="obstructed"
>revision="-1"
>props="none">
> 
> 
> svn: E62: Can't open directory '/Users/dexco/svntest/svn/1': Too many 
> levels of symbolic links
> %
> 
> There is a missing end tag for  and .
> Another issue is missing output: path '2' is not listed.
> 
> The correction of these issues will be much appreciated (i use 1.11.1).
> 
> Regards,
> 
> Denis Excoffier.

Hi Denis,

This problem is not specific to symbolic links.
There are quite a few cases where --xml produces invalid XML
when it runs into some kind of error. Perhaps the command
line client should be fixed to close open XML tags when an
error occurs, though this also risks people or scripts not
noticing such errors.

I suppose that, ideally, our XML output would embed errors
inside the XML stream in a well-defined manner, as well as
printing errors on stderr.

So fixing this would require some non-trivial amount of effort.
Would you have time and skills to work on this issue?

Thanks,
Stefan

Re: Weird behavior of `svn --non-interactive`

[Stefan Sperling]

On Wed, Feb 27, 2019 at 01:27:15PM -0800, Alexey Neyman wrote:
> On 2/26/19 11:22 PM, Stefan Sperling wrote:
> > Since you have several password stores running, the best bet to get reliable
> > behaviour is to pick the one you actually want to use and configure it in
> > the ~/.subversion/config file.
> > 
> > By the way, 'svn auth' will list cached credentials and also show which
> > password cache they are stored in.
> 
> Ok, I left only gpg-agent configured since its the one giving the trouble.
> 
> Issue still reproducible after a period of SVN inactivity.
 
The way gpg-agent works is that it tries to run a 'pinentry' program
to get the password. Once cached, after some time, the password expires.
What is supposed to happen after the password expires is that gpg-agent runs
the pinentry tool again to refresh its password store when SVN wants to
retrieve the password. However, with --non-interactive, SVN passes the
'--no-ask' option to gpg-agent which disables the pinentry program.

So what you could do in this situation is to manually invoke a pinentry
tool (there are several with various graphical and text frontends), type
the password, and SVN will keep working again for some time.

My conclusion is that the tools you are trying to use are at odds with
each other. You will want to use a more persistent SVN password store than
gpg-agent in your use case: plaintext storage, gnome-keyring, or kwallet
(the latter two should last throughout an entire desktop session).

The idea behind SVN's gpg-agent support is to avoid having the plaintext
password be written to disk. The password is kept in memory in plaintext
instead. But by design, this in-memory copy of the password will always
time out after a while and needs to be refreshed.

See also http://subversion.apache.org/docs/release-notes/1.8.html#gpg-agent

Re: Weird behavior of `svn --non-interactive`

[Stefan Sperling]

On Tue, Feb 26, 2019 at 10:15:32PM -0800, Alexey Neyman wrote:
> How does SVN decide when to use gpg-agent and when to use gnome-keyring? By
> the way, I am running KDE so I'd assume the kwallet would be the default -
> but it isn't...

Subversion does not know that you're running it in KDE.

By default it will prefer gpg-agent, detected by checking for the existence
of a shared socket (the location of which unfortunately changes between
versions of gpg-agent and between Linux distros). If that socket doesn't
exist or has no gpg-agent listening on it, SVN will keep trying other
auth stores in the default order given in the config file you've found.

> If you need me to build Subversion with some kind of debugging patch, let me
> know.

Since you have several password stores running, the best bet to get reliable
behaviour is to pick the one you actually want to use and configure it in
the ~/.subversion/config file.

By the way, 'svn auth' will list cached credentials and also show which
password cache they are stored in.

Re: Weird behavior of `svn --non-interactive`

[Stefan Sperling]

On Mon, Feb 25, 2019 at 05:41:13PM -0800, Alexey Neyman wrote:
> Hi all,
> 
> I am encountering some weird behavior after upgrading my workstation to
> Ubuntu 18.10 - which also upgraded the SVN to version 1.10.0 (r1827917).
> 
> An attempt to query anything from the server using the `--non-interactive`
> flag fails, unless there has been a recent "interactive session" with this
> server.
> 
> aneyman@yehat:~/work/lsk-ranges$ svn --non-interactive pl ^/
> svn: E170013: Unable to connect to a repository at URL
> 'svn://svn.lynx.com/lynxsecure'
> svn: E170001: Can't get username or password
> aneyman@yehat:~/work/lsk-ranges$ svn pl ^/
> Properties on 'svn://svn.lynx.com/lynxsecure':
>   reviewboard:url
> aneyman@yehat:~/work/lsk-ranges$ svn --non-interactive pl ^/
> Properties on 'svn://svn.lynx.com/lynxsecure':
>   reviewboard:url
> 
> This happens during various actions by `rbt` (RBTools) which runs svn with
> --non-interactive flag.
> 
> Note that the "interactive" run of svn does not even query the password - it
> happily uses the stored password and proceeds. Why isn't the non-interactive
> invocation doing the same?
> 
> I also tried the development version of Subversion a couple of weeks ago; it
> has the same behavior.
> 
> Regards,
> Alexey.

I agree this looks like a bug.

To find the bug we'll likely need to know which password store is
actually being used by your installation of Subversion.
plaintext? gpg-agent? gnome-keyring? kwallet?

Re: working copy hook

[Stefan Sperling]

On Mon, Feb 25, 2019 at 05:11:25PM +0800, wuzhouhui wrote:
> Could Subversion support working copy hook? Because I want to ensure
> new commit won't introduce any building error, e.g. always running
> "make html" before "svn commit".
> 
> Thanks.

There are only server-side hooks.

But you can use branches to solve this problem.
You could commit to a "staging" branch, and have a post-commit hook on the
server which runs a build test on commits to this branch, and then performs
an automatic merge to a "production" branch if the build was successful.
If the build fails, such a server-side hook could avoid merging changes to
the "production" branch until the "staging" branch has been fixed in a
future commit.

Re: svn cleanup options

[Stefan Sperling]

On Thu, Feb 14, 2019 at 03:29:20PM +0100, Stefan Sperling wrote:
> On Thu, Feb 14, 2019 at 01:55:10PM +, Cooke, Mark wrote:
> > Is there any way to say "ignore errors" or "ignore read-only" or even 
> > "remove read-only"?
> > 
> 
> Well, it should already work without errors.
> I am not sure why it does not work for you :-/

Oh, if I remember correctly, Windows has this odd quirk where it is
unable to delete files which are being held open by an application.

Is this is happening in your case? If so, you will need to close these
files first. This is not Subversion-specific; every program on Windows
is affected by this issue.

Re: svn cleanup options

[Stefan Sperling]

On Thu, Feb 14, 2019 at 01:55:10PM +, Cooke, Mark wrote:
> Hello list,
> 
> Windows 7, using the 1.10.3 command line tools (from TortoiseSVN).
> 
> I am trying to use `svn cleanup` in a build script (batch file) to remove 
> some ignored / unversioned files that may have been generated by a previous 
> build.  However, some of them are marked as "read-only" and cleanup stops 
> with an error.
> 
> 

Which error message and code are you seeing exactly?

The cleanup code calls a functions which is supposed to remove
files even if they don't have write-permissions, and even on Windows.
It looks like in your case this is going wrong for some reason.

On Unix, it is working as expected for me:

$ touch foo
$ chmod 444 foo
$ svn st
?   foo
$ svn cleanup --remove-unversioned
D foo
$ svn st
$

I don't have a Windows machine.

> I have a couple of questions:
> 
> Is it valid to use both --remove-ignored and --remove-unversioned in the same 
> command?  This is not clear in the help text although it seems to work (until 
> it hits the read-only files).

Yes, passing both flags at the same time is valid.

> Is there any way to say "ignore errors" or "ignore read-only" or even "remove 
> read-only"?
> 

Well, it should already work without errors.
I am not sure why it does not work for you :-/

Regards,
Stefan

Re: E155011 + E160013: how to clear this mess?

[Stefan Sperling]

On Thu, Feb 14, 2019 at 12:55:02PM +, Niemann, Hartmut wrote:
> Hello!
> 
> After some copying of trees, and renaming I seem to have brought my WC into a 
> messy state.
> 
> when I try to commit changes of a folder, I get
> 
> Deleting   #REQ#B4#1#.xlsx
> svn: E155011: Commit failed (details follow):
> svn: E155011: File 'D:\localpath...\#REQ#B4#1#.xlsx' is out of date
> svn: E160013: 
> '/repos/locoproc/!svn/txr/15246-c5h/remotepath.../%23REQ%23B4%231%23.xlsx' 
> path not found
> 
> If I look into the 'repo browser', the file that is marked as "delete" isn't 
> there.
> So SVN wants to commit a delete for a file that is not committed.
> 
> cleanup doesn't help.

Cleanup is not supposed to help with this type of problem.

> revert doesn't help.

But reverting the local deletion should indeed be possible and prevent
'svn commit' from trying to delete this path.

So why does revert not help? How did you attempt to revert the deleted
path? What output did you see when you tried to revert the path? 

Regards,
Stefan

Re: svn problem

[Stefan Sperling]

On Sun, Feb 10, 2019 at 09:18:49AM +, Chagai Nota wrote:
> Our company now working with 1.8 its will be problematic to move to 1.11 
> (also for other users it’s not happen)

Subversion 1.8 is no longer officially supported, which means that
problems you find while using Subversion 1.8 won't be fixed anymore.

Of course, if Subversion 1.8 works for you, you are free to keep using it.
But we can support you only if you upgrade to a supported version first.
I understand upgrading can be difficult in some environments, but please
consider the fact that we are a volunteer community which does not have
the resources to keep supporting old relaeses forever.

Note also that clients and server can be upgraded separately because
clients and servers in all released versions of Subversion are compatible
with each other. You can start using Subversion 1.11 clients with your 1.8
server today -- there won't be a problem.

I recommend upgrading your SVN server to Subversion 1.10 ASAP.
Subversion 1.10 is our current long-term support (LTS) release which
will be supported until April 2022.
See https://subversion.apache.org/roadmap.html#release-planning

Re: Tree conflict when an update attempts to remove a directory with unversioned files

[Stefan Sperling]

On Tue, Jan 29, 2019 at 01:06:09PM +0100, Stefan Sperling wrote:
> On Tue, Jan 29, 2019 at 10:48:21AM +0100, Stefan Sperling wrote:
> > I will look at making 'svn update' detect this case if possible
> 
> See https://svn.apache.org/r1852436 

I have nominated this change for the 1.11.x, 1.10.x, and 1.9.x branches.
If this proposed changes passes our review and voting process it will
become part of future patch releases made from those branches.

Re: Tree conflict when an update attempts to remove a directory with unversioned files

[Stefan Sperling]

On Tue, Jan 29, 2019 at 10:48:21AM +0100, Stefan Sperling wrote:
> I will look at making 'svn update' detect this case if possible

See https://svn.apache.org/r1852436 

Re: Tree conflict when an update attempts to remove a directory with unversioned files

[Stefan Sperling]

On Tue, Jan 29, 2019 at 10:48:21AM +0100, Stefan Sperling wrote:
> Please note that releases older than SVN 1.10 are no longer supported by
> this community. The behaviour of SVN 1.9 won't ever be changed.

Sorry, I got mixed up here. Our release planning was recently changed
and SVN 1.9.x is indeed still supported.
See http://subversion.apache.org/docs/release-notes/#supported-versions

Re: Tree conflict when an update attempts to remove a directory with unversioned files

[Stefan Sperling]

On Mon, Jan 28, 2019 at 09:24:41PM +0100, bonsma wrote:
> Hi all,

Hi Erwin,
 
> When we upgraded from Subversion 1.8.17 to Subversion 1.9.7 we
> noticed a change in behaviour of svn update. This happens when the
> update pulls in a change that deletes a directory while this directory
> contains one or more unversioned files in your local working copy.
> 
> What used to happen with svn 1.8 was the following:
> - The versioned files inside the removed directory are removed
> - Any unversioned files remain present
> 
> What now happens with svn 1.9 is:
> - A tree conflict is signalled with the following details:
>   > local dir edit, incoming dir delete or move upon update
> 
> This subsequently requires action by the developer to resolve the
> conflict before he can use the working copy again. Furthermore, when
> resolved incorrectly, the deleted directory may (unintentially) be
> added again.
> 
> In our archive we have the build generate files throughout the
> working copy (obj folders, files generated from template, output
> folders per sub-codebase, etc). We also regularly delete directories
> as part of refactoring, so the tree conflicts occur quite frequently
> and are considered an inconvenience by our developers.
> 
> So we would like to know, is the new behaviour intentional or is it
> an accidental side-effect of another change? We did not see this
> change documented in the release notes.
> 
> Kind regards,
> Erwin

Yes, this behaviour is expected in SVN 1.9. Tree conflict detection was made
more aggressive in anticipation of the conflict resolver released in SVN 1.10
and later. This conflict resolver depends on conflict markers to operate.
See http://subversion.apache.org/docs/release-notes/1.10.html#conflict-resolver

The basic idea is that developers should be able to resolve tree conflicts
in a semi-automated way by picking an item from a conflict resolution menu.
There are many possible tree conflicts and corresponding resolutions, and
implementing many resolution strategies takes a lot of time and effort. The
situation is improving with each release of SVN but it is not yet perfect.

As for your specific case, it unfortunately looks like there is currently no
way to make the resolver preserve the unversioned files you want to preserve.
As you point out, SVN 1.9 and onwards will re-add the directory by default.
The intention here is to preserve any local changes in this directory by
default when the user hits the 'mark resolved' button without really knowing
what this button will do. We don't want people to complain about SVN deleting
their uncommitted changes. Of course, if "local changes" are composed only of
unversioned items, keeping the directory under version control makes no sense.
I will look at making 'svn update' detect this case if possible, or will add
a resolver menu option which handles this conflict in a way that preserves
unversioned items on disk.

Please note that releases older than SVN 1.10 are no longer supported by
this community. The behaviour of SVN 1.9 won't ever be changed.

Going forward, please use SVN 1.11.x releases for resolving tree conflicts.
That line of releases contains the most up-to-date resolver implementation and
is being fed with improvements and new conflict resolution options over time.
Some improvements are also being ported back to 1.10.x releases.
Please refer to our CHANGES file for notes about conflict resolver
improvements in current and future releases:
https://svn.apache.org/repos/asf/subversion/trunk/CHANGES

Regards,
Stefan

Re: Subversion Version Control

[Stefan Sperling]

On Fri, Jan 25, 2019 at 08:07:11PM +, Gillead, Gennar wrote:
> I haven't used SVN but my client currently uses it and
> looking for a certain functionality before considering replacing it.  To be
> more specific can you define your own keyword replacement variables?

Hi Gennar,

It is not entirely clear to me if you mean "keyword variables" as
in Subversion's keyword substitution feature, or something else?

Defining custom keywords in SVN is possible to a certain degree. See here:
http://subversion.apache.org/docs/release-notes/1.8.html#custom-keywords
Perhaps that already does what you need?

> Based on documentation I have found it doesn't allow that functionality.

Which documentation have you consulted so far?

Features of newer SVN releases tend to be documented in the release notes,
rather than the SVN Book: http://subversion.apache.org/docs/release-notes/
This is mostly due to a lack of volunteers working on the SVN Book.

Regards,
Stefan

Re: Reintegration problem

[Stefan Sperling]

On Fri, Jan 18, 2019 at 01:09:19PM +, David Aldrich wrote:
> Hi Stefan
> 
> Thanks for your reply. I am using svn 1.11.
> 
> I tried the command you suggested but, although it succeeded, there was
> nothing to commit afterwards.

Does the file not have an svn:mergeinfo property?

Otherwise, what does svn:mergeinfo on this file contain?
Does it already mention the missing path/revision?

Re: Reintegration problem

[Stefan Sperling]

On Fri, Jan 18, 2019 at 12:31:12PM +, David Aldrich wrote:
> Hi
> 
> I have a branch to which I have carefully merged all changes from the trunk
> over the past year.  Today I want to reintegrate the branch into the trunk.
> When I attempt to do so I get error:
> 
> Reintegrate can only be used if revisions 8908 through 10479 were
> previously merged from /trunk
> to the reintegrate source, but this is not the case:
> branches/TRY_NECA_01Feb2018/Simulations/Leda5G/RegressionTests/LEDA_DLBF_32Tx_8User_EPA5_MCS17_Rank2_RosellaBF.xml
> Missing ranges:
> /trunk/Simulations/Leda5G/RegressionTests/LEDA_DLBF_32Tx_8User_EPA5_MCS17_Rank2_RosellaBF.xml:9491
> 
> The merge info for the branch is:
> 
> /branches/TML_TRY_MassiveMIMO_25April2016:7280-8557
> 
> /trunk:8908-10478

Mergeinfo on /branches/TML_TRY_MassiveMIMO_25April201 could be overridden
by svn:mergeinfo properties on deeper paths. The above mergeinfo only
applies to sub-paths which do not have their own svn:mergeinfo property.
 
> How should I fix this?

Try running a direct merge to the problematic path on your branch.
Something like this should do it:

svn merge -c9491   
^/trunk/Simulations/Leda5G/RegressionTests/LEDA_DLBF_32Tx_8User_EPA5_MCS17_Rank2_RosellaBF.xml
   
Simulations/Leda5G/RegressionTests/LEDA_DLBF_32Tx_8User_EPA5_MCS17_Rank2_RosellaBF.xml

This might only modify svn:mergeinfo which is fine. Commit that change
and then try your reintegration merge again.

> Why does the merge info contain information about branches when I have
> always merged from the trunk?

Most likely because other branches got reintegrated into trunk while you
were working on your own branch. If your branch didn't receive such mergeinfo,
and you merged changes from those other branches to your branch, 'svn merge'
would try to redundantly apply changes which arrived on your branch via trunk.


By the way, you didn't mention which version of Subversion you are using.
Please make sure to run SVN 1.10 or 1.11 clients if you are not already
doing so. Many mergeinfo problems present in earlier releases have been
fixed over time, and you could be running into one of them. In some cases,
such issues are rooted in history committed with older clients. The sooner
every user of the repository upgrades, the better.

Regards,
Stefan

Re: Homebrew SVN 1.11 not working

[Stefan Sperling]

On Wed, Jan 16, 2019 at 09:39:12AM -0500, Mark Phippard wrote:
> I am trying to update my Homebrew and it looks like it wants to move me
> from SVN 1.10 to 1.11 but it is failing.  I do not recall even asking it to
> install the Ruby bindings but perhaps I did once. I thought I just had
> --with-java since I do need JavaHL.  Any ideas?
> 
> ==> make swig-rb EXTRA_SWIG_LDFLAGS=-L/usr/lib
> Last 15 lines from /Users//Library/Logs/Homebrew/subversion/14.make:
> 2019-01-16 09:36:12 -0500

You will need to somehow modify the homebrew build script to avoid
calling 'make swig-rb' if ruby is not present. There's no way to
disable this build step via the configure script, and it is not
part of the default build target invoked with just 'make'.

> line 1760: none: command not found

Our configure script sets the variables which point to python, ruby,
or perl to "none" by default. There is faulty logic somewhere in the
swig bindings build which doesn't detect the non-presence of those binaries
and still allows an attempt to compile bindings regardless. This should
be fixed so that the make swig-* targets exit early in this situation.

Re: Old repo backup, checkout current, lost repo, create new repo?

[Stefan Sperling]

On Thu, Dec 13, 2018 at 07:18:23AM -0600, Tom Browder wrote:
> Sorry, Nico, I probably didn’t use the correct terms in my problem
> description. Basically the subversion repos on my remote server were
> current as of about six months ago when they were established there
> and a hotcopy was made.

You should reconsider your backup strategy.

The biggest concern is that backups should be created immediately after
a change has been committed, and that your backup should preferrably go to
a different physical machine. Use the post-commit hook to trigger a backup
(and the post-revprop-change hook, too, if your repository is configured
to allow revprop changes).

As for getting a copy of repository data, there are a number of options.

The fastest and smallest backup is provided by:
  svnadmin hotcopy --incremental /var/svn/repo /data/backup/repo
It is safe to run this command in a post-commit hook.
But a local mount point for the destination is required. Incremental hotcopy
relies on file sizes and last-modified timestamps recorded in the filesystems
of source and destination.
If the destination is a network drive, make sure clocks between machine stay
in sync. No harm will be done when the clocks are out of sync, but it can
take a lot more time if timestamps don't match since data might be copied
redundantly.

If you need to backup a repository across a network connection, then
take a look at 'svnsync' (requires a second SVN server) and 'svnrdump'
(backup requires read-access over network, backup can be restored
locally with 'svnadmin load').

There is also 'svnadmin freeze' which is intended for use with an
underlying filesystem which supports snapshots (e.g. ZFS). You can
freeze the repository while a snapshot is created, and then copy
data from that snapshot however you want.

> There have been few updates since, so is the hotcopy of value or not,
> history wise?

Only you can decide whether the history of your data is valuable to you.
 
> I started to update on one and am getting these messages (which you
> warned me about):
> 
>   Clean Up: Failed to run the WC DB work queue associated with
> 'C:\Users\Tom\Documents\0-mydocs-svn', work item 636 (file-install
> Personal/TomB/sto/Misc/llftpar2.exe 1 0 1 1) Can't open file
> 'C:\Users\Tom\Documents\0-mydocs-svn\.svn\pristine\7e\7eddb1479c338c0a0fb4a08e21e2b81a8d6c1b61.
> svn-base': The system cannot find the file specified.

This error means the working copy is corrupt. This error has nothing to do
with the repository. The working copy's work queue contains instructions to
copy a temporary file into the area where you see the files you work with.
One possible reason is that an operation (probably an update) was interrupted
in a bad way, i.e. not by a 'Cancel' button but because SVN crashed.
Another possibility is that you moved this working copy between machines
and somehow lost the temporary files in that process.

Generally, SVN's design considers working copies as disposable
because they can always be checked out again, Your resistence to
transferring your data again is working against that design assumption.

> Is there anything I can do to fix something like that?  Or do I have
> to go through creating new repos and populating them from he original
> repo files and dirs?

I'll note that you still haven't fixed your problem which we first
heard about last Sunday. Would it take you more than a week to transfer
the data? Maybe you should just do something that allows you to move on,
instead of waiting until you find a potentially perfect solution?

Re: Old repo backup, checkout current, lost repo, create new repo?

[Stefan Sperling]

On Mon, Dec 10, 2018 at 03:37:46PM -0600, Tom Browder wrote:
> On Mon, Dec 10, 2018 at 4:55 AM Tom Browder  wrote:
> >
> > On Mon, Dec 10, 2018 at 12:10 AM Nico Kadel-Garcia  wrote:
> > > On Sun, Dec 9, 2018 at 6:31 PM Tom Browder  wrote:
> > ...
> > > > Given that history will be lost, does anyone see any problems with my 
> > > > recovery plan?
> > ...
> > > If you have working copies and you don't care about history, why are
> > > you spending any cycles on doing anything with hotcopy? You've lost
> > > history anyway, why keep any of it?
> ...
> > So, again, would my plan work to re-establish my repo and work flow?
> 
> Ping?  I really need some expert opinion here so I can proceed: is my
> plan to replace the lost repos with their hotcopies okay?
> 
> Thanks.
> 
> -Tom

Your current working copy will be invalidated if repositories are
back-dated. The SVN client will complain that it cannot find revisions
which are not present in the old hotcopy.

So if you use the old hotcopy, you need to check out a fresh working
copy and then move your current files from your current working copy
over to the fresh working copy, record any tree structure changes with
'svn add', 'svn remove', or 'svn move' as appropriate, and then commit
the result. This giant commit will then contain a combined version of all
the commits you have lost. Then you can continue working with SVN as usual.

Re: Problems accessing GitHub's SVN-bridge with SVN 1.11

[Stefan Sperling]

On Mon, Dec 10, 2018 at 10:08:10AM +0100, Thorsten Schöning wrote:
> If you come to the conclusion that you don't do this kind of hacking
> anymore or even at all including this issue, users of the SVN-bridge
> would simply need to change their workflow to something else. I'm
> only using the bridge because it was the easiest way to stay with my
> former workflow and how I manage some versioned libs.
> 
> Or do you think it's not worth discussing that fundamentally (yet)?

Perhaps Github's customers will just need to press a little bit
harder for a fix to be applied at Github's end? As Branko explained
earlier, the fix amounts to sending another HTTP header in the response.
Github's web developers should be able to manage that...

Re: subversion 1.11.0 on FreeBSD 11.2: ... does not support the HTTP/DAV protocol

[Stefan Sperling]

On Tue, Nov 06, 2018 at 11:09:53AM +0100, Branko Čibej wrote:
> On 06.11.2018 10:20, Dr. Rolf Jansen wrote:
> > Said all this, I would have simply amended this error message by a hint. 
> > „... check the URL is pointing to a SVN repository!“.
> 
> ... and this hint implies that only the URL could be wrong, so it's
> misleading.

I agree with Rolf that users will appreciate a hint about the URL
potentially being wrong. If it is phrased in a way that does not suggest
the URL as the only possible root cause for the user's problem, then I
don't see a downside.
This is similar to the message we print when the working copy has a newer
revision than the repository. In that case we add a hint to the error
message that the repository might be hosted on a write-through proxy
that could be out of date, an assumption which might not be true at all.

I don't agree with Rolf about creating drama around this issue.
The likely outcome is that github will adapt their implementation soon.
Until that happens we can stay friends and go about our lives, rather
than accusing each other of violating sacred principles of software
engineering.

Re: subversion 1.11.0 on FreeBSD 11.2: ... does not support the HTTP/DAV protocol

[Stefan Sperling]

On Mon, Nov 05, 2018 at 08:21:20PM -0200, Dr. Rolf Jansen wrote:
> and of course the GitHub server does support HTTP/DAV.

> # svn co https://github.com/cyclaero/ContentCGI.git/trunk ContentCGI
> svn: E170013: Unable to connect to a repository at URL 
> 'https://github.com/cyclaero/ContentCGI.git/trunk'
> svn: E175003: The server at 
> 'https://github.com/cyclaero/ContentCGI.git/trunk' does not support the 
> HTTP/DAV protocol

There is a problem at Github's end with SVN 1.11.

See the thread titled "Problems accessing GitHub's SVN-bridge with SVN 1.11"
on this very mailing list: https://svn.haxx.se/users/archive-2018-11/0008.shtml