unsubscribe
unsubscribe
Re: Issue in svn E195020
On Tue, Mar 31, 2015 at 2:56 AM, Scott Aron Bloom sc...@towel42.com wrote: ~~Scott Original message From: Johan Corveleyn jcor...@gmail.com Date:03/30/2015 12:48 (GMT-08:00) To: Scott Aron Bloom sc...@towel42.com Cc: users@subversion.apache.org Subject: Re: Issue in svn E195020 Its has been common, that after the merge, if another merge is attempted we get the E195020 error, (cannot merge into mixed-revision), a simple svn up Fixes the issue, however lately, the svn up fails, and the E195020 continues. Why does svn up fail? Error message? It would be interesting to address this first, because merging into a uniform revision working copy is still the recommended workflow. = I get error E195020 every time. You said svn up fails. Does 'svn up' give an error message then? I find it hard to believe that 'svn up' would say: svn: E195020: Cannot merge into mixed-revision working copy [X:Y]; try updating first So does 'svn up' really fail (i.e. gives an error)? If so, can you please copy-paste its error message? Or do you mean that 'svn up' succeeds, but it doesn't fix the problem of the mixed-rev working copy? If the latter, maybe your problem is similar to this report on Stackoverflow (related to a file external, which apparently makes svn see the working copy as mixed rev -- and the file external doesn't get updated to the uniform revision): http://stackoverflow.com/questions/5046075/unfixable-mixed-revision-working-copy-in-svn -- Johan
RE: unsubscribe
Please read http://subversion.apache.org/mailing-lists.html before sending another useless request. Thanks.
Apache Subversion 1.8.13 released
I'm happy to announce the release of Apache Subversion 1.8.13. This release addresses 3 security issues. CVE-2015-0202: Subversion HTTP servers with FSFS repositories are vulnerable to a remotely triggerable excessive memory use with certain REPORT requests. CVE-2015-0248: Subversion mod_dav_svn and svnserve are vulnerable to a remotely triggerable assertion DoS vulnerability for certain requests with dynamically evaluated revision numbers CVE-2015-0251: Subversion HTTP servers allow spoofing svn:author property values for new revisions For details see the advisories at: http://subversion.apache.org/security/CVE-2015-0202-advisory.txt http://subversion.apache.org/security/CVE-2015-0248-advisory.txt http://subversion.apache.org/security/CVE-2015-0251-advisory.txt Please choose the mirror closest to you by visiting: http://subversion.apache.org/download/#recommended-release The SHA1 checksums are: aa0bd14ac6a8f0fb178cc9ff325387de01cd7452 subversion-1.8.13.tar.bz2 a8ac829dd0d575461424fbd2335820f9d094c379 subversion-1.8.13.zip 437cf662b7ed27d2254aa7ca334fdd74b49262ef subversion-1.8.13.tar.gz PGP Signatures are available at: http://www.apache.org/dist/subversion/subversion-1.8.13.tar.bz2.asc http://www.apache.org/dist/subversion/subversion-1.8.13.tar.gz.asc http://www.apache.org/dist/subversion/subversion-1.8.13.zip.asc For this release, the following people have provided PGP signatures: Bert Huijben [4096R/CCC8E1DF] with fingerprint: 3D1D C66D 6D2E 0B90 3952 8138 C4A6 C625 CCC8 E1DF Branko Čibej [4096R/A347943F] with fingerprint: BA3C 15B1 337C F0FB 222B D41A 1BCA 6586 A347 943F Ivan Zhakov [4096R/F6AD8147] with fingerprint: 4829 8F0F E47F 4B8A 43FD 6525 919F 6F61 F6AD 8147 Johan Corveleyn [4096R/010C8AAD] with fingerprint: 8AA2 C10E EAAD 44F9 6972 7AEA B59C E6D6 010C 8AAD Julian Foad [4096R/4EECC493] with fingerprint: 6011 63CF 9D49 9FD7 18CF 582D 1FB0 64B8 4EEC C493 Philip Martin [2048R/ED1A599C] with fingerprint: A844 790F B574 3606 EE95 9207 76D7 88E1 ED1A 599C Stefan Fuhrmann [4096R/57921ACC] with fingerprint: 056F 8016 D9B8 7B1B DE41 7467 99EC 741B 5792 1ACC Stefan Sperling [2048R/9A59B973] with fingerprint: 8BC4 DAE0 C5A4 D65F 4044 0107 4F7D BAA9 9A59 B973 Release notes for the 1.8.x release series may be found at: http://subversion.apache.org/docs/release-notes/1.8.html You can find the list of changes between 1.8.13 and earlier versions at: http://svn.apache.org/repos/asf/subversion/tags/1.8.13/CHANGES Questions, comments, and bug reports to users@subversion.apache.org. Thanks, - The Subversion Team
Re: Branching slow 1.8.11 https
On Tue, Mar 31, 2015 at 2:19 AM, Johan Corveleyn jcor...@gmail.com wrote:
On Sun, Mar 29, 2015 at 7:57 PM, Johan Corveleyn jcor...@gmail.com wrote:
On Sat, Mar 28, 2015 at 5:09 PM, Bert Huijben b...@qqmail.nl wrote:
-Original Message-
From: Johan Corveleyn [mailto:jcor...@gmail.com]
Sent: vrijdag 27 maart 2015 22:03
To: users@subversion.apache.org
Subject: Branching slow 1.8.11 https
Does the following ring a bell for someone?
Recently upgraded our server (on Solaris 10 SPARC) from 1.5.4 to
1.8.11 (CollabNet package). Some time after that, we discovered that
branching was very slow. I'm talking about pure server-side branching
('svn copy $URL/trunk $URL/branches/br1'). I'm testing with a 1.8.11
client (tried both from same machine as the server, and from another
machine on the LAN (100 Mbit)).
- Branching trunk (containing many directories and files): 6-8 minutes
- Branching a subfolder of trunk: 20-30 seconds (still very slow)
- Branching a single file is fast ( 0.5s or so).
So it seems the performance degrades depending on the depth or size of the
tree.
Now, it gets more interesting:
- The resulting rev file on the server is always very small (as it
should be, it contains only a lightweight 'copy' of the trunk node).
- Our repos is currently served via https (Apache 2.2.29).
- Branching with file:/// urls is fast (branching trunk takes 0.6s).
- When starting an svnserve instance serving the same repository, and
branching with svn:// urls, it's fast as well (also 0.6s).
- We reproduced it on a copy of the production repo.
- Experimenting with the test copy, we found that
$repos/dav/activities.d contains ~2000 files. When we clear that
directory, the branching times go down by more than half (~2 minutes
for trunk, ~10s for subdir of trunk --- i.e. still slow, but it
definitely has an impact).
- With a 1.7 client connecting with neon, the problem is the same.
- During the 'svn copy', an httpd child consumes a lot of cpu (around
half a core).
- There is no authz configured for this repo (SVNPathAuthz off).
- Backend is still in 1.5 format (we have not run svnadmin upgrade
yet, a dump+load is planned in a couple of weeks).
So it seems clearly mod_dav_svn related (and not for instance related
to the FSFS backend).
I don't think we have anything special in our httpd config:
[[[
Location /test_svn
SVNInMemoryCacheSize 131072
SVNCacheFullTexts on
SVNCacheTextDeltas on
SSLRequireSSL
AuthName TEST Subversion Repository
AuthType Basic
AuthBasicProvider ldap
AuthBasicAuthoritative off
AuthLDAPURL ldap://redacted:389;
AuthLDAPBindDN redacted
AuthLDAPBindPassword redacted
Require ldap-group redacted
DAV svn
SVNPath /path/to/test_repos
SVNPathAuthz off
/Location
]]]
Any ideas?
Why the cpu usage by the server, what's it doing?
What is the dav/activities.d directory for? How come it contains so
many files? Is it ok to purge the old files from that directory?
Httpd's mod_dav was updated in some recent version to do a full lock
traversal on copies and moves. I think we already applied some
optimizations, but the real fix would be that mod_dav shouldn't do this
work (which our repos layer already does).
I'm not sure which release we applied the first set of optimizations.
Thanks for refreshing my memory.
So the problem is known as issue #4531 (server-side copy (over dav)
uses too much memory) [1]. The memory usage issue has been fixed in
SVN 1.8.11 and 1.7.19 (see CHANGES), but a performance problem remains
(copy is no longer O(1), but depends on the size of the tree being
copied). That's a direct violation of one of Subversion's old selling
points vs. CVS: that branching / tagging is O(1). Branching / tagging
taking several minutes brings back fond memories from CVS' days.
As Philip pointed out in his last comment on #4531 [2]: This issue is
related to a change in mod_dav in 2.2.25 to fix PR54610 which
added a walk over the copy source looking for lock tokens. (also
released in 2.4.5; so both httpd 2.2.25+ and 2.4.5+ are affected --
older httpd's won't have this problem I guess).
Again quoting Philip: Apache knows in advance that the walk is
redundant in cases such as Subversion's URL-to-URL copy but Subversion
cannot avoid the read access. We should attempt to fix mod_dav to
avoid the walk where possible.
So my hope rests with Philip and others who might have the necessary
knowledge to fix this in mod_dav. It's really not acceptable that
branching / tagging (or I'm guessing also: moving a large tree with a
server-side move) takes several minutes.
[1] http://subversion.tigris.org/issues/show_bug.cgi?id=4531
[2] http://subversion.tigris.org/issues/show_bug.cgi?id=4531#desc12
I think I've found a workaround: it seems the tree walk by mod_dav is
avoided when the request has a header Depth with value 0. I've
Apache Subversion 1.7.20 released
I'm happy to announce the release of Apache Subversion 1.7.20. This release addresses two security issues: CVE-2015-0248: Subversion mod_dav_svn and svnserve are vulnerable to a remotely triggerable assertion DoS vulnerability for certain requests with dynamically evaluated revision numbers CVE-2015-0251: Subversion HTTP servers allow spoofing svn:author property values for new revisions For details see the advisories at: http://subversion.apache.org/security/CVE-2015-0248-advisory.txt http://subversion.apache.org/security/CVE-2015-0251-advisory.txt Please choose the mirror closest to you by visiting: http://subversion.apache.org/download/#supported-releases The SHA1 checksums are: f600c68010d2fd9a23fc8c6b659099aedac12900 subversion-1.7.20.tar.bz2 675ac5a843e01dbb4a30d6333a809fd048c5ce0c subversion-1.7.20.zip e861f85e9df1b5aca903aa6eda15919c454cbda5 subversion-1.7.20.tar.gz PGP Signatures are available at: http://www.apache.org/dist/subversion/subversion-1.7.20.tar.bz2.asc http://www.apache.org/dist/subversion/subversion-1.7.20.tar.gz.asc http://www.apache.org/dist/subversion/subversion-1.7.20.zip.asc For this release, the following people have provided PGP signatures: Bert Huijben [4096R/CCC8E1DF] with fingerprint: 3D1D C66D 6D2E 0B90 3952 8138 C4A6 C625 CCC8 E1DF Branko Čibej [4096R/A347943F] with fingerprint: BA3C 15B1 337C F0FB 222B D41A 1BCA 6586 A347 943F Ivan Zhakov [4096R/F6AD8147] with fingerprint: 4829 8F0F E47F 4B8A 43FD 6525 919F 6F61 F6AD 8147 Johan Corveleyn [4096R/010C8AAD] with fingerprint: 8AA2 C10E EAAD 44F9 6972 7AEA B59C E6D6 010C 8AAD Julian Foad [4096R/4EECC493] with fingerprint: 6011 63CF 9D49 9FD7 18CF 582D 1FB0 64B8 4EEC C493 Philip Martin [2048R/ED1A599C] with fingerprint: A844 790F B574 3606 EE95 9207 76D7 88E1 ED1A 599C Stefan Fuhrmann [4096R/57921ACC] with fingerprint: 056F 8016 D9B8 7B1B DE41 7467 99EC 741B 5792 1ACC Stefan Sperling [2048R/9A59B973] with fingerprint: 8BC4 DAE0 C5A4 D65F 4044 0107 4F7D BAA9 9A59 B973 Release notes for the 1.7.x release series may be found at: http://subversion.apache.org/docs/release-notes/1.7.html You can find the list of changes between 1.7.20 and earlier versions at: http://svn.apache.org/repos/asf/subversion/tags/1.7.20/CHANGES Questions, comments, and bug reports to users@subversion.apache.org. Thanks, - The Subversion Team
Re: Branching slow 1.8.11 https
On Mar 31, 2015, at 8:13 AM, Johan Corveleyn jcor...@gmail.com wrote:
On Tue, Mar 31, 2015 at 2:19 AM, Johan Corveleyn jcor...@gmail.com wrote:
On Sun, Mar 29, 2015 at 7:57 PM, Johan Corveleyn jcor...@gmail.com wrote:
On Sat, Mar 28, 2015 at 5:09 PM, Bert Huijben b...@qqmail.nl wrote:
-Original Message-
From: Johan Corveleyn [mailto:jcor...@gmail.com]
Sent: vrijdag 27 maart 2015 22:03
To: users@subversion.apache.org
Subject: Branching slow 1.8.11 https
Does the following ring a bell for someone?
Recently upgraded our server (on Solaris 10 SPARC) from 1.5.4 to
1.8.11 (CollabNet package). Some time after that, we discovered that
branching was very slow. I'm talking about pure server-side branching
('svn copy $URL/trunk $URL/branches/br1'). I'm testing with a 1.8.11
client (tried both from same machine as the server, and from another
machine on the LAN (100 Mbit)).
- Branching trunk (containing many directories and files): 6-8 minutes
- Branching a subfolder of trunk: 20-30 seconds (still very slow)
- Branching a single file is fast ( 0.5s or so).
So it seems the performance degrades depending on the depth or size of the
tree.
Now, it gets more interesting:
- The resulting rev file on the server is always very small (as it
should be, it contains only a lightweight 'copy' of the trunk node).
- Our repos is currently served via https (Apache 2.2.29).
- Branching with file:/// urls is fast (branching trunk takes 0.6s).
- When starting an svnserve instance serving the same repository, and
branching with svn:// urls, it's fast as well (also 0.6s).
- We reproduced it on a copy of the production repo.
- Experimenting with the test copy, we found that
$repos/dav/activities.d contains ~2000 files. When we clear that
directory, the branching times go down by more than half (~2 minutes
for trunk, ~10s for subdir of trunk --- i.e. still slow, but it
definitely has an impact).
- With a 1.7 client connecting with neon, the problem is the same.
- During the 'svn copy', an httpd child consumes a lot of cpu (around
half a core).
- There is no authz configured for this repo (SVNPathAuthz off).
- Backend is still in 1.5 format (we have not run svnadmin upgrade
yet, a dump+load is planned in a couple of weeks).
So it seems clearly mod_dav_svn related (and not for instance related
to the FSFS backend).
I don't think we have anything special in our httpd config:
[[[
Location /test_svn
SVNInMemoryCacheSize 131072
SVNCacheFullTexts on
SVNCacheTextDeltas on
SSLRequireSSL
AuthName TEST Subversion Repository
AuthType Basic
AuthBasicProvider ldap
AuthBasicAuthoritative off
AuthLDAPURL ldap://redacted:389;
AuthLDAPBindDN redacted
AuthLDAPBindPassword redacted
Require ldap-group redacted
DAV svn
SVNPath /path/to/test_repos
SVNPathAuthz off
/Location
]]]
Any ideas?
Why the cpu usage by the server, what's it doing?
What is the dav/activities.d directory for? How come it contains so
many files? Is it ok to purge the old files from that directory?
Httpd's mod_dav was updated in some recent version to do a full lock
traversal on copies and moves. I think we already applied some
optimizations, but the real fix would be that mod_dav shouldn't do this
work (which our repos layer already does).
I'm not sure which release we applied the first set of optimizations.
Thanks for refreshing my memory.
So the problem is known as issue #4531 (server-side copy (over dav)
uses too much memory) [1]. The memory usage issue has been fixed in
SVN 1.8.11 and 1.7.19 (see CHANGES), but a performance problem remains
(copy is no longer O(1), but depends on the size of the tree being
copied). That's a direct violation of one of Subversion's old selling
points vs. CVS: that branching / tagging is O(1). Branching / tagging
taking several minutes brings back fond memories from CVS' days.
As Philip pointed out in his last comment on #4531 [2]: This issue is
related to a change in mod_dav in 2.2.25 to fix PR54610 which
added a walk over the copy source looking for lock tokens. (also
released in 2.4.5; so both httpd 2.2.25+ and 2.4.5+ are affected --
older httpd's won't have this problem I guess).
Again quoting Philip: Apache knows in advance that the walk is
redundant in cases such as Subversion's URL-to-URL copy but Subversion
cannot avoid the read access. We should attempt to fix mod_dav to
avoid the walk where possible.
So my hope rests with Philip and others who might have the necessary
knowledge to fix this in mod_dav. It's really not acceptable that
branching / tagging (or I'm guessing also: moving a large tree with a
server-side move) takes several minutes.
[1] http://subversion.tigris.org/issues/show_bug.cgi?id=4531
[2] http://subversion.tigris.org/issues/show_bug.cgi?id=4531#desc12
I think I've found a workaround: it seems the tree walk by
RE: Issue in svn E195020
Here is some more info: First I show the results of the svnversion, showing the mixed revision. Then I do a svn merge (with -allow-mixed-revisions) and get the E195020 error Then I do an svn up Then another svnversion still showing the mixed revision Then finally the merge showing the problem still exists.. Scott $ svnversion 28594:29395S scott@Vader ~/sb/bps/9.0.29202 $ svn merge --allow-mixed-revisions ^/branches/vijay/vhdlBitWidthFix_1 . svn: E195020: Cannot merge into mixed-revision working copy [28594:29395]; try updating first $ svn up Updating '.': Fetching external item into 'UI\nlview': External at revision 26116. Fetching external item into 'verific': External at revision 29173. Fetching external item into 'tcl': Fetching external item into 'tcl\Install': External at revision 23404. At revision 28171. Fetching external item into 'gmock': External at revision 11153. Fetching external item into 'qtiocompressor': External at revision 29045. Fetching external item into 'Licensing': External at revision 28829. Fetching external item into 'tcllib': External at revision 23595. Fetching external item into 'Prerequisites': External at revision 28594. Fetching external item into 'libicu': External at revision 28899. Fetching external item into 'libxcb': External at revision 28929. Fetching external item into 'Installation\FPGALibraries': External at revision 29142. Fetching external item into 'UI\QtGraphVizLib\libcgraphviz': External at revision 28492. At revision 29395. $ svnversion 28594:29395S $ svn merge --allow-mixed-revisions ^/branches/vijay/vhdlBitWidthFix_1 . svn: E195020: Cannot merge into mixed-revision working copy [28594:29395]; try updating first From: Scott Aron Bloom [mailto:sc...@towel42.com] Sent: Tuesday, March 31, 2015 6:46 AM To: Johan Corveleyn Cc: users@subversion.apache.org Subject: RE: Issue in svn E195020 Sorry, I misunderstood your question. Svn up succeeds but does not fix th3 svn merge issue. I'll look at the stack overflow issue since we do have a bunch of externals. ~~Scott Original message From: Johan Corveleyn jcor...@gmail.commailto:jcor...@gmail.com Date:03/31/2015 02:29 (GMT-08:00) To: Scott Aron Bloom sc...@towel42.commailto:sc...@towel42.com Cc: users@subversion.apache.orgmailto:users@subversion.apache.org Subject: Re: Issue in svn E195020 On Tue, Mar 31, 2015 at 2:56 AM, Scott Aron Bloom sc...@towel42.commailto:sc...@towel42.com wrote: ~~Scott Original message From: Johan Corveleyn jcor...@gmail.commailto:jcor...@gmail.com Date:03/30/2015 12:48 (GMT-08:00) To: Scott Aron Bloom sc...@towel42.commailto:sc...@towel42.com Cc: users@subversion.apache.orgmailto:users@subversion.apache.org Subject: Re: Issue in svn E195020 Its has been common, that after the merge, if another merge is attempted we get the E195020 error, (cannot merge into mixed-revision), a simple svn up Fixes the issue, however lately, the svn up fails, and the E195020 continues. Why does svn up fail? Error message? It would be interesting to address this first, because merging into a uniform revision working copy is still the recommended workflow. = I get error E195020 every time. You said svn up fails. Does 'svn up' give an error message then? I find it hard to believe that 'svn up' would say: svn: E195020: Cannot merge into mixed-revision working copy [X:Y]; try updating first So does 'svn up' really fail (i.e. gives an error)? If so, can you please copy-paste its error message? Or do you mean that 'svn up' succeeds, but it doesn't fix the problem of the mixed-rev working copy? If the latter, maybe your problem is similar to this report on Stackoverflow (related to a file external, which apparently makes svn see the working copy as mixed rev -- and the file external doesn't get updated to the uniform revision): http://stackoverflow.com/questions/5046075/unfixable-mixed-revision-working-copy-in-svn -- Johan
Aw: start-commit log message.
Hello, validate the log message [...] start-commit [...] [...] if the client is using an older version (like 1.7) the commit message obtained using svnlook is always empty durng the start-commit. In this case the commit message is available only during pre-commit. Yes, and this is expected, documented [1] and will not change. Quoting: [[[ Note: Subversion does not require that commit transaction properties (such as the revision log message) be attached to the transaction as part of its initialization. As such, some clients will still not provide that information to the server until after the start-commit hook has been invoked. Here is a list of such clients we are aware of: Pre-1.8 clients communicating via HTTP Clients communicating via HTTP when mod_dav_svn's SVNAdvertiseV2Protocol option has been set to off Administrators should consider modularizing the tests that their hooks perform on transaction properties, invoke those tests from both the start-commit and pre-commit hook scripts. ]]] You will need to run the same hook again as pre-commit. [1] https://subversion.apache.org/docs/release-notes/1.8.html#hooks-start-commit Andreas
start-commit log message.
Hi, I'm trying to validate the log message using the new 1.8 start-commit
capability of inspect the transaction beeing created.
My problem is that if the client is using an older version (like 1.7) the
commit message obtained using svnlook is always empty durng the
start-commit. In this case the commit message is available only during
pre-commit.
All of this is is working well if the client is using 1.8 version.
The text bellow is an copy of my start-commit hook:
#!/usr/bin/env python
# The start-commit hook is invoked before a Subversion txn is created
# in the process of doing a commit. Subversion runs this hook
# by invoking a program (script, executable, binary, etc.) named
# 'start-commit' (for which this file is a template)
# with the following ordered arguments:
#
# [1] REPOS-PATH (the path to this repository)
# [2] USER (the authenticated user attempting to commit)
# [3] CAPABILITIES (a colon-separated list of capabilities reported
# by the client; see note below)
# [4] TXN-NAME (the name of the commit txn just created (1.8 or
newer))
#
import sys
import subprocess
def get_svn_log(txn, repos):
child =
subprocess.Popen(['/usr/bin/svnlook','log','-t',txn,repos],stdout=subprocess.PIPE,stderr=subprocess.PIPE)
out,err = child.communicate()
if child.returncode:
return ('',err)
return (out,'')
repo = sys.argv[1]
capabilities = sys.argv[3].split(':')
txn = sys.argv[4]
if 'mergeinfo' not in capabilities:
sys.stderr.write('Commits from merge-tracking-unaware clients are not
permitted.\n'
'Please upgrade to Subversion 1.5 or newer.\n')
sys.exit(1)
log, err = get_svn_log(txn,repo)
if err:
sys.stderr.write('Error verifying log message: '+err)
sys.exit(1)
elif not log.strip():
sys.stderr.write('Commits without log message are not permitted.\n'
'Please enter the log message.\n')
sys.exit(1)
sys.exit(0)
RE: Issue in svn E195020
Sorry, I misunderstood your question. Svn up succeeds but does not fix th3 svn merge issue. I'll look at the stack overflow issue since we do have a bunch of externals. ~~Scott Original message From: Johan Corveleyn jcor...@gmail.com Date:03/31/2015 02:29 (GMT-08:00) To: Scott Aron Bloom sc...@towel42.com Cc: users@subversion.apache.org Subject: Re: Issue in svn E195020 On Tue, Mar 31, 2015 at 2:56 AM, Scott Aron Bloom sc...@towel42.com wrote: ~~Scott Original message From: Johan Corveleyn jcor...@gmail.com Date:03/30/2015 12:48 (GMT-08:00) To: Scott Aron Bloom sc...@towel42.com Cc: users@subversion.apache.org Subject: Re: Issue in svn E195020 Its has been common, that after the merge, if another merge is attempted we get the E195020 error, (cannot merge into mixed-revision), a simple svn up Fixes the issue, however lately, the svn up fails, and the E195020 continues. Why does svn up fail? Error message? It would be interesting to address this first, because merging into a uniform revision working copy is still the recommended workflow. = I get error E195020 every time. You said svn up fails. Does 'svn up' give an error message then? I find it hard to believe that 'svn up' would say: svn: E195020: Cannot merge into mixed-revision working copy [X:Y]; try updating first So does 'svn up' really fail (i.e. gives an error)? If so, can you please copy-paste its error message? Or do you mean that 'svn up' succeeds, but it doesn't fix the problem of the mixed-rev working copy? If the latter, maybe your problem is similar to this report on Stackoverflow (related to a file external, which apparently makes svn see the working copy as mixed rev -- and the file external doesn't get updated to the uniform revision): http://stackoverflow.com/questions/5046075/unfixable-mixed-revision-working-copy-in-svn -- Johan
Re: start-commit log message.
Sorry for asking without proprer documentation read and thanks. There is a way to detect that the log message is not sent because of an older client version? My intent is to block the commit early if I can. On Tuesday, March 31, 2015 at 12:28:37 PM UTC-3, Andreas Stieger wrote: Hello, validate the log message [...] start-commit [...] [...] if the client is using an older version (like 1.7) the commit message obtained using svnlook is always empty durng the start-commit. In this case the commit message is available only during pre-commit. Yes, and this is expected, documented [1] and will not change. Quoting: [[[ Note: Subversion does not require that commit transaction properties (such as the revision log message) be attached to the transaction as part of its initialization. As such, some clients will still not provide that information to the server until after the start-commit hook has been invoked. Here is a list of such clients we are aware of: Pre-1.8 clients communicating via HTTP Clients communicating via HTTP when mod_dav_svn's SVNAdvertiseV2Protocol option has been set to off Administrators should consider modularizing the tests that their hooks perform on transaction properties, invoke those tests from both the start-commit and pre-commit hook scripts. ]]] You will need to run the same hook again as pre-commit. [1] https://subversion.apache.org/docs/release-notes/1.8.html#hooks-start-commit https://www.google.com/url?q=https%3A%2F%2Fsubversion.apache.org%2Fdocs%2Frelease-notes%2F1.8.html%23hooks-start-commitsa=Dsntz=1usg=AFQjCNG5AarLHZr4kAfA6cuRwPNj6Qeohg Andreas
RE: Issue in svn E195020
The stackoverflow issue seems to be similar, I cant confirm it's the same issue however, since there is no tell tail sign of failure However, I am using 1.8.11, and the article implies this bug has been fixed since 1.7.0 Scott From: Scott Aron Bloom [mailto:sc...@towel42.com] Sent: Tuesday, March 31, 2015 6:46 AM To: Johan Corveleyn Cc: users@subversion.apache.org Subject: RE: Issue in svn E195020 Sorry, I misunderstood your question. Svn up succeeds but does not fix th3 svn merge issue. I'll look at the stack overflow issue since we do have a bunch of externals. ~~Scott Original message From: Johan Corveleyn jcor...@gmail.commailto:jcor...@gmail.com Date:03/31/2015 02:29 (GMT-08:00) To: Scott Aron Bloom sc...@towel42.commailto:sc...@towel42.com Cc: users@subversion.apache.orgmailto:users@subversion.apache.org Subject: Re: Issue in svn E195020 On Tue, Mar 31, 2015 at 2:56 AM, Scott Aron Bloom sc...@towel42.commailto:sc...@towel42.com wrote: ~~Scott Original message From: Johan Corveleyn jcor...@gmail.commailto:jcor...@gmail.com Date:03/30/2015 12:48 (GMT-08:00) To: Scott Aron Bloom sc...@towel42.commailto:sc...@towel42.com Cc: users@subversion.apache.orgmailto:users@subversion.apache.org Subject: Re: Issue in svn E195020 Its has been common, that after the merge, if another merge is attempted we get the E195020 error, (cannot merge into mixed-revision), a simple svn up Fixes the issue, however lately, the svn up fails, and the E195020 continues. Why does svn up fail? Error message? It would be interesting to address this first, because merging into a uniform revision working copy is still the recommended workflow. = I get error E195020 every time. You said svn up fails. Does 'svn up' give an error message then? I find it hard to believe that 'svn up' would say: svn: E195020: Cannot merge into mixed-revision working copy [X:Y]; try updating first So does 'svn up' really fail (i.e. gives an error)? If so, can you please copy-paste its error message? Or do you mean that 'svn up' succeeds, but it doesn't fix the problem of the mixed-rev working copy? If the latter, maybe your problem is similar to this report on Stackoverflow (related to a file external, which apparently makes svn see the working copy as mixed rev -- and the file external doesn't get updated to the uniform revision): http://stackoverflow.com/questions/5046075/unfixable-mixed-revision-working-copy-in-svn -- Johan
Re: Branching slow 1.8.11 https
On Tue, Mar 31, 2015 at 2:19 AM, Johan Corveleyn jcor...@gmail.com wrote:
...
I think I've found a workaround: it seems the tree walk by mod_dav is
avoided when the request has a header Depth with value 0. I've tried
adding
If %{REQUEST_METHOD} == 'COPY'
RequestHeader set Depth 0
/If
Apparently this workaround is specific to httpd 2.4 or higher
(If/If is only available as of 2.4). Since the problem also exists
in httpd 2.2.25 or higher, this might be a better way to do this:
SetEnvIf Request_Method COPY method_is_copy
RequestHeader set Depth 0 env=method_is_copy
This should work both in 2.4 and 2.2.
--
Johan
Re: start-commit log message.
Only to record.
I solved the problem looking for ephemeral properties on commit. They are
only sent for 1.8 clients.
To do this I changed my start-commit hook was to this:
#!/usr/bin/env python
# The start-commit hook is invoked before a Subversion txn is created
# in the process of doing a commit. Subversion runs this hook
# by invoking a program (script, executable, binary, etc.) named
# 'start-commit' (for which this file is a template)
# with the following ordered arguments:
#
# [1] REPOS-PATH (the path to this repository)
# [2] USER (the authenticated user attempting to commit)
# [3] CAPABILITIES (a colon-separated list of capabilities reported
# by the client; see note below)
# [4] TXN-NAME (the name of the commit txn just created (1.8 or
newer))
#
import sys
import subprocess
def get_svn_txn_proplist(repos,txn):
child =
subprocess.Popen(['/usr/bin/svnlook','proplist','--revprop',repos,'-t',txn],
stdout=subprocess.PIPE,stderr=subprocess.PIPE)
out,err = child.communicate()
if child.returncode:
return ([],err)
return ([p.strip() for p in out.strip().split('\n')],'')
def get_svn_txn_log(repos,txn):
child = subprocess.Popen(['/usr/bin/svnlook','log','-t',txn,repos],
stdout=subprocess.PIPE,stderr=subprocess.PIPE)
out,err = child.communicate()
if child.returncode:
return ('',err)
return (out,'')
capabilities = sys.argv[3].split(':')
if 'mergeinfo' not in capabilities:
sys.stderr.write('Commits from merge-tracking-unaware clients are not
permitted.\n'
'Please upgrade to Subversion 1.5 or newer.\n')
sys.exit(1)
repos = sys.argv[1]
txn = sys.argv[4]
log,err = get_svn_txn_log(repos,txn)
if err:
sys.stderr.write('Error inspecting commit log: '+err)
sys.exit(1)
elif not log.strip():
proplist,err = get_svn_txn_proplist(repos,txn)
if err:
sys.stderr.write('Error inspecting commit properties: '+err)
sys.exit(1)
elif 'svn:txn-user-agent' in proplist:
sys.stderr.write('Commits without log message are not permitted.\n'
'Please enter the log message.\n')
sys.exit(1)
sys.exit(0)
On Tuesday, March 31, 2015 at 12:34:36 PM UTC-3, Carlos Alberto Costa
Beppler wrote:
Sorry for asking without proprer documentation read and thanks.
There is a way to detect that the log message is not sent because of an
older client version?
My intent is to block the commit early if I can.
On Tuesday, March 31, 2015 at 12:28:37 PM UTC-3, Andreas Stieger wrote:
Hello,
validate the log message [...] start-commit [...]
[...]
if the client is using an older version (like 1.7) the commit message
obtained
using svnlook is always empty durng the start-commit. In this case the
commit
message is available only during pre-commit.
Yes, and this is expected, documented [1] and will not change. Quoting:
[[[
Note: Subversion does not require that commit transaction properties
(such as the revision log message) be attached to the transaction as part
of its initialization. As such, some clients will still not provide that
information to the server until after the start-commit hook has been
invoked. Here is a list of such clients we are aware of:
Pre-1.8 clients communicating via HTTP
Clients communicating via HTTP when mod_dav_svn's
SVNAdvertiseV2Protocol option has been set to off
Administrators should consider modularizing the tests that their hooks
perform on transaction properties, invoke those tests from both the
start-commit and pre-commit hook scripts.
]]]
You will need to run the same hook again as pre-commit.
[1]
https://subversion.apache.org/docs/release-notes/1.8.html#hooks-start-commit
https://www.google.com/url?q=https%3A%2F%2Fsubversion.apache.org%2Fdocs%2Frelease-notes%2F1.8.html%23hooks-start-commitsa=Dsntz=1usg=AFQjCNG5AarLHZr4kAfA6cuRwPNj6Qeohg
Andreas
Re: 1.8 bug(?): svn:mergeinfo set for tree-conflicted files in subdirs
Hello. Firstly I'd like to say thank you, Pete, for bringing up this issue and discussing it so clearly. I had a think about this the other day and chatted with the others on IRC [1]. I picked this old email from this thread to reply to, as it's got me in the CC and is as good a one as any to reply to. Much of this has been said already, but here's my 'take' on the problem. On 2015-03-16, Pete Harlan wrote: On 2015-03-14, Pete Harlan wrote: My expectation as a naive user is: I initiated a merge from the root of a branch (or trunk), I told svn to merge the root of another branch (or trunk), and I resolved all reported conflicts (however complex). Unless I've explicitly told svn otherwise, I expect svn to consider this a full merge, and not create separate mergeinfo for any interior nodes. Yes, I completely agree. The user has requested a full merge and sorted out all the physical and semantic conflicts. The mergeinfo is meant to record the intent of the merge, as you correctly argued, rather than to record exactly where the physical content changes were made in the target branch in order to achieve the desired result [2]. Therefore the merge should be recorded, in the end, as a complete merge, with no subtree mergeinfo needed. There are two or three things going on. (I'm going to be critical of Subversion here.) 1. The user should always have the choice to record a subtree as 'merged' or as 'not merged'. Subversion currently doesn't make this choice easily available, and doesn't clearly report or document what it's doing, in all the cases that can occur during a merge such as - a conflict occurs - skipped because subtree is missing, obstructed, or still conflicted from an earlier merge - subtree is switched and across different operations on the subtree, such as - revert changes to subtree - copy/move/rename the subtree In some of these cases it records mergeinfo indicating 'merged', and in other cases indicating 'not merged'. 2. Subtree mergeinfo was never designed to be an absolute indicator that a subtree merge had been done. In other words, there was no specific rule that a complete tree merge must result in no subtree mergeinfo. After 1.5 was released and some work flows resulted in lots of subtree mergeinfo (especially, I remember, copies added subtree mergeinfo), there was a succession of refinements aimed at reducing the number of nodes having subtree mergeinfo. When Subversion creates (or fails to elide) unnecessary subtree mergeinfo, we classify that as suboptimal performance but not an outright bug. In fact, the plan for eliding mergeinfo is not simply always elide any unnecessary explicit mergeinfo. In some cases it's more like elide unnecessary mergeinfo if we're touching that node anyway, but don't touch other nodes just to elide their mergeinfo. In other words, trying to minimize the number of changes to be committed. I don't know the details of when we do what. 3. The Working Copy and its user interface doesn't keep track of what merge(s) you have done, it just tracks the resulting mergeinfo properties. As a consequence, for example, after doing a merge to the root of the WC if you try to commit a subtree, it won't notice that actually to commit the subtree along with the subtree's mergeinfo it needs to put a mergeinfo property on the subtree. What you will find is that the merge implementation has not carefully followed a consistent set of rules or a 'mental model'. There have been rules and mental models but they have not been formally identified and kept consistent. I'd love to be able to point you to the rules as documented, but unfortunately can't. And there are still cases where it does something that's clearly wrong or inconsistent. What to do? In the case you identified, I certainly agree it's bad behaviour, not what we want. But it's not easy to know exactly which cases need fixing -- or if fixing this case would make it inconsistent with other cases -- without first cataloguing the existing behaviour over all or most cases. If somebody wants to work on fixing this case, that's fine, what I'm saying is I'm more interested in figuring out the bigger solution. Pete, you seem to have a good grasp of this stuff. Would you be willing to help? Either in a small way, anything from saying please just fix this case upwards, or in a bigger way by helping to catalogue the existing behaviour and decide what we should do about it overall? No worries if not, but we do rely on those who have an interest. - Julian So I think it would be worth updating svn resolve to, by default, take action to trust the user and mark this as a full resolution. If the user needs to take an extra step or add a new parameter to get that effect, would that not feel like a regression compared with 1.7? A colleague argued that creating the mergeinfo for a subtree in this case (root-root merge) is a simple bug because mergeinfo says what inputs were considered to come up
RE: Branching slow 1.8.11 https
-Original Message-
From: Johan Corveleyn [mailto:jcor...@gmail.com]
Sent: dinsdag 31 maart 2015 14:13
To: users@subversion.apache.org
Cc: Bert Huijben; Philip Martin; Ben Reser
Subject: Re: Branching slow 1.8.11 https
On Tue, Mar 31, 2015 at 2:19 AM, Johan Corveleyn jcor...@gmail.com
wrote:
On Sun, Mar 29, 2015 at 7:57 PM, Johan Corveleyn jcor...@gmail.com
wrote:
On Sat, Mar 28, 2015 at 5:09 PM, Bert Huijben b...@qqmail.nl wrote:
-Original Message-
From: Johan Corveleyn [mailto:jcor...@gmail.com]
Sent: vrijdag 27 maart 2015 22:03
To: users@subversion.apache.org
Subject: Branching slow 1.8.11 https
Does the following ring a bell for someone?
Recently upgraded our server (on Solaris 10 SPARC) from 1.5.4 to
1.8.11 (CollabNet package). Some time after that, we discovered that
branching was very slow. I'm talking about pure server-side branching
('svn copy $URL/trunk $URL/branches/br1'). I'm testing with a 1.8.11
client (tried both from same machine as the server, and from another
machine on the LAN (100 Mbit)).
- Branching trunk (containing many directories and files): 6-8 minutes
- Branching a subfolder of trunk: 20-30 seconds (still very slow)
- Branching a single file is fast ( 0.5s or so).
So it seems the performance degrades depending on the depth or size of
the
tree.
Now, it gets more interesting:
- The resulting rev file on the server is always very small (as it
should be, it contains only a lightweight 'copy' of the trunk node).
- Our repos is currently served via https (Apache 2.2.29).
- Branching with file:/// urls is fast (branching trunk takes 0.6s).
- When starting an svnserve instance serving the same repository, and
branching with svn:// urls, it's fast as well (also 0.6s).
- We reproduced it on a copy of the production repo.
- Experimenting with the test copy, we found that
$repos/dav/activities.d contains ~2000 files. When we clear that
directory, the branching times go down by more than half (~2 minutes
for trunk, ~10s for subdir of trunk --- i.e. still slow, but it
definitely has an impact).
- With a 1.7 client connecting with neon, the problem is the same.
- During the 'svn copy', an httpd child consumes a lot of cpu (around
half a core).
- There is no authz configured for this repo (SVNPathAuthz off).
- Backend is still in 1.5 format (we have not run svnadmin upgrade
yet, a dump+load is planned in a couple of weeks).
So it seems clearly mod_dav_svn related (and not for instance related
to the FSFS backend).
I don't think we have anything special in our httpd config:
[[[
Location /test_svn
SVNInMemoryCacheSize 131072
SVNCacheFullTexts on
SVNCacheTextDeltas on
SSLRequireSSL
AuthName TEST Subversion Repository
AuthType Basic
AuthBasicProvider ldap
AuthBasicAuthoritative off
AuthLDAPURL ldap://redacted:389;
AuthLDAPBindDN redacted
AuthLDAPBindPassword redacted
Require ldap-group redacted
DAV svn
SVNPath /path/to/test_repos
SVNPathAuthz off
/Location
]]]
Any ideas?
Why the cpu usage by the server, what's it doing?
What is the dav/activities.d directory for? How come it contains so
many files? Is it ok to purge the old files from that directory?
Httpd's mod_dav was updated in some recent version to do a full lock
traversal on copies and moves. I think we already applied some optimizations,
but the real fix would be that mod_dav shouldn't do this work (which our repos
layer already does).
I'm not sure which release we applied the first set of optimizations.
Thanks for refreshing my memory.
So the problem is known as issue #4531 (server-side copy (over dav)
uses too much memory) [1]. The memory usage issue has been fixed in
SVN 1.8.11 and 1.7.19 (see CHANGES), but a performance problem remains
(copy is no longer O(1), but depends on the size of the tree being
copied). That's a direct violation of one of Subversion's old selling
points vs. CVS: that branching / tagging is O(1). Branching / tagging
taking several minutes brings back fond memories from CVS' days.
As Philip pointed out in his last comment on #4531 [2]: This issue is
related to a change in mod_dav in 2.2.25 to fix PR54610 which
added a walk over the copy source looking for lock tokens. (also
released in 2.4.5; so both httpd 2.2.25+ and 2.4.5+ are affected --
older httpd's won't have this problem I guess).
Again quoting Philip: Apache knows in advance that the walk is
redundant in cases such as Subversion's URL-to-URL copy but Subversion
cannot avoid the read access. We should attempt to fix mod_dav to
avoid the walk where possible.
So my hope rests with Philip and others who might have the necessary
knowledge to fix this in mod_dav. It's really not acceptable that
branching / tagging (or I'm guessing
Re: Branching slow 1.8.11 https
On 31.03.2015 14:43, Mark Phippard wrote:
On Mar 31, 2015, at 8:13 AM, Johan Corveleyn jcor...@gmail.com wrote:
On Tue, Mar 31, 2015 at 2:19 AM, Johan Corveleyn jcor...@gmail.com wrote:
On Sun, Mar 29, 2015 at 7:57 PM, Johan Corveleyn jcor...@gmail.com wrote:
On Sat, Mar 28, 2015 at 5:09 PM, Bert Huijben b...@qqmail.nl wrote:
-Original Message-
From: Johan Corveleyn [mailto:jcor...@gmail.com]
Sent: vrijdag 27 maart 2015 22:03
To: users@subversion.apache.org
Subject: Branching slow 1.8.11 https
Does the following ring a bell for someone?
Recently upgraded our server (on Solaris 10 SPARC) from 1.5.4 to
1.8.11 (CollabNet package). Some time after that, we discovered that
branching was very slow. I'm talking about pure server-side branching
('svn copy $URL/trunk $URL/branches/br1'). I'm testing with a 1.8.11
client (tried both from same machine as the server, and from another
machine on the LAN (100 Mbit)).
- Branching trunk (containing many directories and files): 6-8 minutes
- Branching a subfolder of trunk: 20-30 seconds (still very slow)
- Branching a single file is fast ( 0.5s or so).
So it seems the performance degrades depending on the depth or size of
the
tree.
Now, it gets more interesting:
- The resulting rev file on the server is always very small (as it
should be, it contains only a lightweight 'copy' of the trunk node).
- Our repos is currently served via https (Apache 2.2.29).
- Branching with file:/// urls is fast (branching trunk takes 0.6s).
- When starting an svnserve instance serving the same repository, and
branching with svn:// urls, it's fast as well (also 0.6s).
- We reproduced it on a copy of the production repo.
- Experimenting with the test copy, we found that
$repos/dav/activities.d contains ~2000 files. When we clear that
directory, the branching times go down by more than half (~2 minutes
for trunk, ~10s for subdir of trunk --- i.e. still slow, but it
definitely has an impact).
- With a 1.7 client connecting with neon, the problem is the same.
- During the 'svn copy', an httpd child consumes a lot of cpu (around
half a core).
- There is no authz configured for this repo (SVNPathAuthz off).
- Backend is still in 1.5 format (we have not run svnadmin upgrade
yet, a dump+load is planned in a couple of weeks).
So it seems clearly mod_dav_svn related (and not for instance related
to the FSFS backend).
I don't think we have anything special in our httpd config:
[[[
Location /test_svn
SVNInMemoryCacheSize 131072
SVNCacheFullTexts on
SVNCacheTextDeltas on
SSLRequireSSL
AuthName TEST Subversion Repository
AuthType Basic
AuthBasicProvider ldap
AuthBasicAuthoritative off
AuthLDAPURL ldap://redacted:389;
AuthLDAPBindDN redacted
AuthLDAPBindPassword redacted
Require ldap-group redacted
DAV svn
SVNPath /path/to/test_repos
SVNPathAuthz off
/Location
]]]
Any ideas?
Why the cpu usage by the server, what's it doing?
What is the dav/activities.d directory for? How come it contains so
many files? Is it ok to purge the old files from that directory?
Httpd's mod_dav was updated in some recent version to do a full lock
traversal on copies and moves. I think we already applied some
optimizations, but the real fix would be that mod_dav shouldn't do this
work (which our repos layer already does).
I'm not sure which release we applied the first set of optimizations.
Thanks for refreshing my memory.
So the problem is known as issue #4531 (server-side copy (over dav)
uses too much memory) [1]. The memory usage issue has been fixed in
SVN 1.8.11 and 1.7.19 (see CHANGES), but a performance problem remains
(copy is no longer O(1), but depends on the size of the tree being
copied). That's a direct violation of one of Subversion's old selling
points vs. CVS: that branching / tagging is O(1). Branching / tagging
taking several minutes brings back fond memories from CVS' days.
As Philip pointed out in his last comment on #4531 [2]: This issue is
related to a change in mod_dav in 2.2.25 to fix PR54610 which
added a walk over the copy source looking for lock tokens. (also
released in 2.4.5; so both httpd 2.2.25+ and 2.4.5+ are affected --
older httpd's won't have this problem I guess).
Again quoting Philip: Apache knows in advance that the walk is
redundant in cases such as Subversion's URL-to-URL copy but Subversion
cannot avoid the read access. We should attempt to fix mod_dav to
avoid the walk where possible.
So my hope rests with Philip and others who might have the necessary
knowledge to fix this in mod_dav. It's really not acceptable that
branching / tagging (or I'm guessing also: moving a large tree with a
server-side move) takes several minutes.
[1] http://subversion.tigris.org/issues/show_bug.cgi?id=4531
[2] http://subversion.tigris.org/issues/show_bug.cgi?id=4531#desc12
I think I've found a workaround: it
