svn commit: r1297402 - /subversion/trunk/notes/subversion-design.html
Author: julianfoad Date: Tue Mar 6 10:02:47 2012 New Revision: 1297402 URL: http://svn.apache.org/viewvc?rev=1297402view=rev Log: * notes/subversion-design.html (server.fs.api): Mention the FSFS structure document alongside the reference to the original FS structure document. Modified: subversion/trunk/notes/subversion-design.html Modified: subversion/trunk/notes/subversion-design.html URL: http://svn.apache.org/viewvc/subversion/trunk/notes/subversion-design.html?rev=1297402r1=1297401r2=1297402view=diff == --- subversion/trunk/notes/subversion-design.html (original) +++ subversion/trunk/notes/subversion-design.html Tue Mar 6 10:02:47 2012 @@ -2340,7 +2340,10 @@ write/Makefile:6 pOnce you've done this, read Jim Blandy's own structural overview, which explains how nodes and revisions are organized (among other things) in the filesystem implementation: -tt class=filenamesubversion/libsvn_fs_base/notes/structure/tt./p +tt class=filenamesubversion/libsvn_fs_base/notes/structure/tt. +(Some details in that document are specific to the BDB-based +filesystem implementation. Details specific to FSFS are recorded in +tt class=filenamesubversion/libsvn_fs_fs/structure/tt.)/p pFinally, read the well-documented API in tt class=filenamesubversion/include/svn_fs.h/tt./p
svn propchange: r1297239 - svn:log
Author: julianfoad Revision: 1297239 Modified property: svn:log Modified: svn:log at Tue Mar 6 11:10:24 2012 -- --- svn:log (original) +++ svn:log Tue Mar 6 11:10:24 2012 @@ -1 +1,2 @@ -Revert r1297223 and r1297231. +Revert r1297223 and r1297231. See the IRC thread starting at +http://colabti.org/irclogger/irclogger_log/svn-dev?date=2012-03-05#l334.
[Subversion Wiki] Update of MultiLayerMoves by PhilipMartin
Dear Wiki user, You have subscribed to a wiki page or wiki category on Subversion Wiki for change notification. The MultiLayerMoves page has been changed by PhilipMartin: http://wiki.apache.org/subversion/MultiLayerMoves?action=diffrev1=15rev2=16 Now scan_deletion(A/B/C/D) needs to tell us moved_to is Q at op-depth=4, Z/D at op-depth=3, X at op-depth=2. Quite how it does this is an open question. Does the caller pass the op-depth? Where would the caller get it? Does the function return some sort of array of (op-depth, moved-to) pairs? - = Copy/Move Differences = + = Nested Moves = || op-depth || local-relpath || presence || moved-to || ||0 ||A || normal || || ||0 ||A/F|| normal || || + Move A to B and then B/F to B/G - Move A to B and B/F to B/G (or A/F to A/G and A to B): - - '''gstein''': does the B/F to B/G have an op-depth of 2? That would change the table below. The current depiction seems very difficult to construct a commit process: if you move A to B, then how do you move A/F (no longer existing) to B/G?? And you can't do the A/F to B/G first since B does not exist. I think this seeming difficulty is based on the wrong op-depth for the B/F to B/G move. - - '''philip''': that's essentially the inconsistency mentioned below: it's not clear what the op-depth should be for nested moves. The node starts at A/F and ends at B/G. The current non-multi-layer move stuff records that move in base. Perhaps multi-layer move doesn't record that move at all, but records the A-B and B/F-B/G. In the short term move information is not used to contruct a commit, the commit still consists of copies and deletes. || op-depth || local-relpath || presence || moved-to || ||0 ||A || normal || || ||0 ||A/F|| normal || || - ||1 ||A || base-deleted || B || + ||1 ||A || base-deleted ||B || - ||1 ||A/F|| base-deleted || B/G || + ||1 ||A/F|| base-deleted || || ||1 ||B || normal || || ||1 ||B/F|| normal || || + + || op-depth || local-relpath || presence || moved-to || + ||0 ||A || normal || || + ||0 ||A/F|| normal || || + ||1 ||A || base-deleted ||B || - ||2 ||B/F|| base-deleted || || + ||1 ||A/F|| base-deleted || || + ||1 ||B || normal || || + ||1 ||B/F|| normal || || + ||2 ||B/F|| base-deleted ||B/G || ||2 ||B/G|| normal || || - We end up with two moves recorded A-B and A/F-B/G. We don't record B/F-B/G. + Alternatively, move A/F to A/G and then A to B + + || op-depth || local-relpath || presence || moved-to || + ||0 ||A || normal || || + ||0 ||A/F|| normal || || + ||2 ||A/F|| base-deleted || A/G|| + ||2 ||A/G|| normal || || + + || op-depth || local-relpath || presence || moved-to || + ||0 ||A || normal || || + ||0 ||A/F|| normal || || + ||1 ||A || base-deleted ||B || + ||1 ||A/F|| base-deleted || || + ||1 ||B || normal || || + ||1 ||B/F|| normal || || + ||2 ||B/F|| base-deleted ||B/G || + ||2 ||B/G|| normal || || + + The final database state is the same independent of the order of the moves. The second move in the second case causes the explicit moved-to associated with A/F to be removed. + + = Copy compared to Move = Now consider copying A instead of moving it, so copy A to C then move C/F to C/G. @@ -120, +144 @@ ||2 ||C/F|| base-deleted || C/G|| ||2 ||C/G|| normal || || - There is only one move recorded C/F-C/G. + There is only one move recorded C/F-C/G. The move inside the copy is recorded in a similar way to the move inside a move. - It would be possible to both of these in the same working copy, either A to C then A to B, or A to B then B to C and we end up with three moves recorded A-B, A/F-B/G and C/F-C/G. There is an inconsistency, we record C/F to C/G but not B/F to B/G. Does this matter? Perhaps it is wrong to record A/F-B/G, perhaps we should be recording B/F-B/G instead? - - If we recorded moves-in-moves within the move, the above move A to B and B/F to B/G would
[Subversion Wiki] Update of MultiLayerMoves by PhilipMartin
Dear Wiki user, You have subscribed to a wiki page or wiki category on Subversion Wiki for change notification. The MultiLayerMoves page has been changed by PhilipMartin: http://wiki.apache.org/subversion/MultiLayerMoves?action=diffrev1=16rev2=17 There is only one move recorded C/F-C/G. The move inside the copy is recorded in a similar way to the move inside a move. + = Move rewrites moved-to = + + Move has to rewrite moved-to both inside an outside the moved tree. The first case is moved-to entirely within the moved tree: move A to B after A/F to A/G. This causes moved-to on A/F to be rewritted (to null) and moved-to on B/F to be added: + + || op-depth || local-relpath || presence || moved-to || + ||0 ||A || normal || || + ||0 ||A/F|| normal || || + ||2 ||A/F|| base-deleted || A/G|| + ||2 ||A/G|| normal || || + + || op-depth || local-relpath || presence || moved-to || + ||0 ||A || normal || || + ||0 ||A/F|| normal || || + ||1 ||A || base-deleted ||B || + ||1 ||A/F|| base-deleted || || + ||1 ||B || normal || || + ||1 ||B/F|| normal || || + ||2 ||B/F|| base-deleted ||B/G || + ||2 ||B/G|| normal || || + + The second case is moved-to from inside the moved-tree to outside the moved tree: move A to B after A/F to G. Like the first case this requires the moved-to on A/F to be rewritten (to null) and moved-to on B/F to be added. Compared to the first case the value of the rewritten moved-to doesn't change: + + || op-depth || local-relpath || presence || moved-to || + ||0 ||A || normal || || + ||0 ||A/F|| normal || || + ||2 ||A/F|| base-deleted || G || + ||1 ||G || normal || || + + || op-depth || local-relpath || presence || moved-to || + ||0 ||A || normal || || + ||0 ||A/F|| normal || || + ||1 ||A || base-deleted || B || + ||1 ||A/F|| base-deleted || || + ||1 ||B || normal || || + ||1 ||B/F|| normal || || + ||2 ||B/F|| base-deleted || G || + ||1 ||G || normal || || + + The third case is move-to from outside the moved tree into the moved tree: move A to B after F to A/F. This causes moved-to on F, outside the moved tree, to be rewritten: + + || op-depth || local-relpath || presence || moved-to || + ||0 ||A || normal || || + ||0 ||F || normal || || + ||1 ||F || base-deleted || A/G|| + ||2 ||A/G|| normal || || + + || op-depth || local-relpath || presence || moved-to || + ||0 ||A || normal || || + ||0 ||F || normal || || + ||1 ||A || base-deleted || B || + ||1 ||F || base-deleted || B/G|| + ||1 ||B || normal || || + ||2 ||B/G|| normal || || +
[Subversion Wiki] Update of MultiLayerMoves by PhilipMartin
Dear Wiki user, You have subscribed to a wiki page or wiki category on Subversion Wiki for change notification. The MultiLayerMoves page has been changed by PhilipMartin: http://wiki.apache.org/subversion/MultiLayerMoves?action=diffrev1=17rev2=18 ||2 ||C/F|| base-deleted || C/G|| ||2 ||C/G|| normal || || - There is only one move recorded C/F-C/G. The move inside the copy is recorded in a similar way to the move inside a move. + There is only one move recorded C/F-C/G. The move inside the copy is recorded in a similar way to the move inside a move. The move can happen before the copy: + || op-depth || local-relpath || presence || moved-to || + ||0 ||A || normal || || + ||0 ||A/F|| normal || || + ||2 ||A/F|| base-deleted || A/G|| + ||2 ||A/G|| normal || || + + || op-depth || local-relpath || presence || moved-to || + ||0 ||A || normal || || + ||0 ||A/F|| normal || || + ||2 ||A/F|| base-deleted || A/G|| + ||2 ||A/G|| normal || || + ||1 ||C || normal || || + ||1 ||C/F|| normal || || + ||2 ||C/F|| base-deleted || C/G|| + ||2 ||C/G|| normal || || + + Irrespective of the order of the copy and move operations we end up with moved-to for C/F to C/G. = Move rewrites moved-to = Move has to rewrite moved-to both inside an outside the moved tree. The first case is moved-to entirely within the moved tree: move A to B after A/F to A/G. This causes moved-to on A/F to be rewritted (to null) and moved-to on B/F to be added:
Re: svn commit: r1241050 - /subversion/trunk/subversion/mod_dav_svn/mod_dav_svn.c
Daniel Shahaf wrote: Daniel Shahaf wrote on Mon, Feb 06, 2012 at 18:20:28 +0200: Stefan Sperling wrote on Mon, Feb 06, 2012 at 17:12:32 +0100: On Mon, Feb 06, 2012 at 05:59:04PM +0200, Daniel Shahaf wrote: This still strips whitespace around ='s in the value: SVNHooksEnv name = x = y will result in setenv(name, x=y, 1) whereas I believe it should result in setenv(name, x = y, 1) (and, to be honest, I'd be happy with setenv(name , x = y, 1) as well). WDYT? How should it behave? I agree. would telling svn_cstring_split() to no strip whitespace suffice? I assume that should result in the third setenv() case above, so +1. Ping? trunk@HEAD still strips whitespace around equal signs in the value. My tuppence-worth? I agree that the current behaviour as stated above is wrong. Unless there is precedent to the contrary, I think it should do no stripping at all. If you can find precedent for some stripping in such a setting, then follow the precedent. Note that it's not only possible to strip spaces before and/or after the first '=' character, but also before name and/or after x = y. - Julian
svn commit: r1297523 - /subversion/branches/1.7.x/STATUS
Author: rhuijben Date: Tue Mar 6 15:29:11 2012 New Revision: 1297523 URL: http://svn.apache.org/viewvc?rev=1297523view=rev Log: * STATUS: Nominate r1297522. Modified: subversion/branches/1.7.x/STATUS Modified: subversion/branches/1.7.x/STATUS URL: http://svn.apache.org/viewvc/subversion/branches/1.7.x/STATUS?rev=1297523r1=1297522r2=1297523view=diff == --- subversion/branches/1.7.x/STATUS (original) +++ subversion/branches/1.7.x/STATUS Tue Mar 6 15:29:11 2012 @@ -111,6 +111,14 @@ Candidate changes: Votes: +1: danielsh + * r1297522 + Resolve issue 4136, Deep commit followed by --depth immediates update + triggers checksum failure + Justification: + User reported regression since 1.7.0 + Votes: + +1: rhuijben + Veto-blocked changes: =
[Subversion Wiki] Update of InheritedProperties by pburba
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Subversion Wiki for
change notification.
The InheritedProperties page has been changed by pburba:
http://wiki.apache.org/subversion/InheritedProperties?action=diffrev1=31rev2=32
Comment:
Start tweaking design so *any* versioned property can be inherited (if a user
can at least read the parent path).
- = Inherited Properties =
+ = Property Inheritance =
== Background ==
=== What's This? ===
- Some ideas on how we might implement simple inherited properties.
+ Some ideas on how we might make versioned properties inheritable.
- === What Are Inherited Properties? ===
- Inherited properties are versioned properties that apply not only to the path
the property is explicitly set on, but also to all of that path's path-wise
descendants (at the same revision) which do not explicitly have the same
property set.
+ === What Is Property Inheritance? ===
+ Property inheritance is a mechanism by which a versioned property set on a
given path applies also to that path's path-wise descendants.
=== What's Driving This? ===
Desire for some form of inherited properties has existed almost from the dawn
of Subversion. This latest effort is in response to a recent proposal
regarding [[ServerDictatedConfiguration|Server Dictated Configuration]] (see
http://svn.haxx.se/dev/archive-2012-01/0032.shtml). That proposal made use of
a new mechanism by which server dictated configurations (notably auto-props and
global-ignores) would be communicated from the server to the client. In the
feedback on the proposal several people pointed out that versioned properties
provide a possible alternative solution (and that the TortoiseSVN project was
already using pseudo-inheritable properties to address some of the same
problems -- see
http://tortoisesvn.net/docs/nightly/TortoiseSVN_en/tsvn-dug-propertypage.html).
Despite its origins in the server defined configuration work, this wiki
describes only a generic inheritable properties design (''though my the
ultimate goal is to use these inheritable properties to implement the server
dictated configuration feature -- pburba'').
@@ -14, +14 @@
=== What This Design Is and What It Isn't ===
This design provides the bare minimum to support the basics of inherited
properties:
- * A way to identify properties that are inheritable vs. those that are not.
- * A way to set inheritable properties (not really anything to do here, we
support this today).
- * A way to get a path's inherited properties.
+ * A way to get a path's inherited properties from the repository.
+ * A way to cache a working copy's inherited properties locally so that
disconnected WC operations can remain disconnected.
- That's it, it's really just a small extension of our existing versioned
property capabilities. If your own personal vision of inherited properties
includes something outside of these three bullets (e.g. merging inherited
properties), then that is outside of this design's scope (at least initially).
+ That's it, it's really just a small extension of our existing versioned
property capabilities. Since any versioned property can be inherited, there is
no concept of setting inheritable properties. Our existing APIs are
sufficient. If your own personal vision of inherited properties includes
something outside of these three bullets (e.g. merging inherited properties),
then that is outside of this design's scope (at least initially).
=== How Is This Design Different? ===
It's not really. A lot of the ideas here come from the dev list, #svn-dev,
and conversations with other devs. The difference, if there is any, is that
this design aims to be as simple as possible and steer clear of complications
which have thwarted inheritable properties in the past. If there is more than
one reasonable behavior to choose from, it usually goes with the one that is
simpler to explain/understand/implement/maintain. ''This means that not
everyone will be happy, it may not cover every use-case, but I hope it covers
most of them -- pburba.''
@@ -46, +45 @@
}}}
== Design ==
- === Differentiating Inheritable Vs. 'Normal' Properties ===
+ === Differentiating 'Inheritable' Vs. 'Normal' Properties ===
- Inheritable properties will be identified by a prefix on the property name.
+ This is easy, there is no difference. Versioned properties, both
Subversion's reserved properties and custom user properties, can be interpreted
as inheritable, but otherwise will function as they always have in terms of
valid property names and values.
- 1. Custom user properties will use the svn:inheritable: prefix.
- 1. Existing Subversion reserved properties will 'not' become
inheritable, they will function as they always have. New reserved Subversion
properties may be introduced that are inheritable by definition, but such
properties are not required to use any special namespace, beyond
Re: svn commit: r1241050 - /subversion/trunk/subversion/mod_dav_svn/mod_dav_svn.c
On Tue, Mar 06, 2012 at 12:39:27PM +, Julian Foad wrote: Daniel Shahaf wrote: Daniel Shahaf wrote on Mon, Feb 06, 2012 at 18:20:28 +0200: Stefan Sperling wrote on Mon, Feb 06, 2012 at 17:12:32 +0100: On Mon, Feb 06, 2012 at 05:59:04PM +0200, Daniel Shahaf wrote: This still strips whitespace around ='s in the value: SVNHooksEnv name = x = y will result in setenv(name, x=y, 1) whereas I believe it should result in setenv(name, x = y, 1) (and, to be honest, I'd be happy with setenv(name , x = y, 1) as well). WDYT? How should it behave? I agree. would telling svn_cstring_split() to no strip whitespace suffice? I assume that should result in the third setenv() case above, so +1. Ping? trunk@HEAD still strips whitespace around equal signs in the value. My tuppence-worth? I agree that the current behaviour as stated above is wrong. Unless there is precedent to the contrary, I think it should do no stripping at all. If you can find precedent for some stripping in such a setting, then follow the precedent. Note that it's not only possible to strip spaces before and/or after the first '=' character, but also before name and/or after x = y. This option string is parsed by HTTPD. The API provided by HTTPD is quite bad for this use case. This is work in progress, and the above behaviour will be changed. See http://subversion.tigris.org/issues/show_bug.cgi?id=4113 In the long term, the argument to the SVNHooksEnv option will be changed anyway. It will become a path to a file which contains a list of environment variables and their values. That file will then be parsed by our own config parsing code.
svn commit: r1297633 - in /subversion/trunk/subversion/tests/cmdline: svnsync_tests.py svntest/main.py svntest/sandbox.py
Author: hwright
Date: Tue Mar 6 18:27:05 2012
New Revision: 1297633
URL: http://svn.apache.org/viewvc?rev=1297633view=rev
Log:
Remove a couple of custom Python methods which are now part of the standard
library.
* subversion/tests/cmdline/svnsync_tests.py
(setup_and_sync, copy_delete_unreadable_child): Use urllib versions.
* subversion/tests/cmdline/svntest/main.py
(pathname2url, url2pathname): Remove.
(_create_parser, execute_tests): Use urllib versions.
* subversion/tests/cmdline/svntest/sandbox.py
(Sandbox): Same.
Modified:
subversion/trunk/subversion/tests/cmdline/svnsync_tests.py
subversion/trunk/subversion/tests/cmdline/svntest/main.py
subversion/trunk/subversion/tests/cmdline/svntest/sandbox.py
Modified: subversion/trunk/subversion/tests/cmdline/svnsync_tests.py
URL:
http://svn.apache.org/viewvc/subversion/trunk/subversion/tests/cmdline/svnsync_tests.py?rev=1297633r1=1297632r2=1297633view=diff
==
--- subversion/trunk/subversion/tests/cmdline/svnsync_tests.py (original)
+++ subversion/trunk/subversion/tests/cmdline/svnsync_tests.py Tue Mar 6
18:27:05 2012
@@ -28,7 +28,7 @@
import sys, os
# Test suite-specific modules
-import locale, re
+import locale, re, urllib
# Our testing module
import svntest
@@ -191,14 +191,16 @@ def setup_and_sync(sbox, dump_file_conte
repo_url = sbox.repo_url
cwd = os.getcwd()
if is_src_ra_local:
-repo_url = svntest.main.file_scheme_prefix +
svntest.main.pathname2url(os.path.join(cwd, sbox.repo_dir))
+repo_url = svntest.main.file_scheme_prefix + \
+urllib.pathname2url(os.path.join(cwd, sbox.repo_dir))
if subdir:
repo_url = repo_url + subdir
dest_repo_url = dest_sbox.repo_url
if is_dest_ra_local:
-dest_repo_url = svntest.main.file_scheme_prefix +
svntest.main.pathname2url(os.path.join(cwd, dest_sbox.repo_dir))
+dest_repo_url = svntest.main.file_scheme_prefix + \
+urllib.pathname2url(os.path.join(cwd, dest_sbox.repo_dir))
run_init(dest_repo_url, repo_url, source_prop_encoding)
run_sync(dest_repo_url, repo_url,
@@ -1029,7 +1031,7 @@ def copy_delete_unreadable_child(sbox):
% (authz, authz))
dest_url = svntest.main.file_scheme_prefix \
- + svntest.main.pathname2url(os.path.abspath(dest_sbox.repo_dir))
++ urllib.pathname2url(os.path.abspath(dest_sbox.repo_dir))
run_init(dest_url, sbox.repo_url)
run_sync(dest_url)
Modified: subversion/trunk/subversion/tests/cmdline/svntest/main.py
URL:
http://svn.apache.org/viewvc/subversion/trunk/subversion/tests/cmdline/svntest/main.py?rev=1297633r1=1297632r2=1297633view=diff
==
--- subversion/trunk/subversion/tests/cmdline/svntest/main.py (original)
+++ subversion/trunk/subversion/tests/cmdline/svntest/main.py Tue Mar 6
18:27:05 2012
@@ -132,23 +132,6 @@ wc_author2 = 'jconstant' # use the same
# Set C locale for command line programs
os.environ['LC_ALL'] = 'C'
-# This function mimics the Python 2.3 urllib function of the same name.
-def pathname2url(path):
- Convert the pathname PATH from the local syntax for a path to the form
- used in the path component of a URL. This does not produce a complete URL.
- The return value will already be quoted using the quote() function.
-
- # Don't leave ':' in file://C%3A/ escaped as our canonicalization
- # rules will replace this with a ':' on input.
- return urllib_parse_quote(path.replace('\\', '/')).replace('%3A', ':')
-
-# This function mimics the Python 2.3 urllib function of the same name.
-def url2pathname(path):
- Convert the path component PATH from an encoded URL to the local syntax
- for a path. This does not accept a complete URL. This function uses
- unquote() to decode PATH.
- return os.path.normpath(urllib_parse_unquote(path))
-
##
# The locations of the svn, svnadmin and svnlook binaries, relative to
# the only scripts that import this file right now (they live in ../).
@@ -1551,7 +1534,8 @@ def _create_parser():
# most of the defaults are None, but some are other values, set them here
parser.set_defaults(
server_minor_version=SVN_VER_MINOR,
-url=file_scheme_prefix + pathname2url(os.path.abspath(os.getcwd())),
+url=file_scheme_prefix + \
+urllib.pathname2url(os.path.abspath(os.getcwd())),
http_library=_default_http_library)
return parser
@@ -1719,7 +1703,8 @@ def execute_tests(test_list, serial_only
or function '%s'\n % arg)
# Calculate pristine_greek_repos_url from test_area_url.
- pristine_greek_repos_url = options.test_area_url + '/' +
pathname2url(pristine_greek_repos_dir)
+ pristine_greek_repos_url = options.test_area_url + '/' + \
+
svn commit: r1297661 - /subversion/trunk/subversion/tests/cmdline/svntest/main.py
Author: hwright Date: Tue Mar 6 19:30:59 2012 New Revision: 1297661 URL: http://svn.apache.org/viewvc?rev=1297661view=rev Log: Fix test failures (I hope) on the windows bot as a result of r1297633 by adjusting the file scheme prefix the tests use on Windows. * subversion/tests/cmdline/svntest/main.py (file_scheme_prefix): Drop trailing '///' which are provided by urllib. Modified: subversion/trunk/subversion/tests/cmdline/svntest/main.py Modified: subversion/trunk/subversion/tests/cmdline/svntest/main.py URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/tests/cmdline/svntest/main.py?rev=1297661r1=1297660r2=1297661view=diff == --- subversion/trunk/subversion/tests/cmdline/svntest/main.py (original) +++ subversion/trunk/subversion/tests/cmdline/svntest/main.py Tue Mar 6 19:30:59 2012 @@ -105,7 +105,7 @@ class SVNRepositoryCreateFailure(Failure # Windows specifics if sys.platform == 'win32': windows = True - file_scheme_prefix = 'file:///' + file_scheme_prefix = 'file:' _exe = '.exe' _bat = '.bat' os.environ['SVN_DBG_STACKTRACES_TO_STDERR'] = 'y'
[Subversion Wiki] Update of InheritedProperties by pburba
Dear Wiki user, You have subscribed to a wiki page or wiki category on Subversion Wiki for change notification. The InheritedProperties page has been changed by pburba: http://wiki.apache.org/subversion/InheritedProperties?action=diffrev1=32rev2=33 Comment: Clarification on limits of automatic merging of inherited props. A child path that inherits a property from its parent may not have ready access to that parent in the working copy (e.g. the root of the working copy is a subtree of the parent path). To ensure that traditionally disconnected operations (i.e. those that require no access to the repository, like 'svn add') remain disconnected, we will maintain a cache of properties inherited by the root of the working copy. Whenever a new working copy is checked out, any properties inherited by the root of the working copy will be cached in the working copy. If a subtree within a working copy is switched, a separate cache will be created for the root of that subtree. Whenever an update occurs the cache(s) will be refreshed. The cache will be stored in a new wc-ng table: - ||tablewidth=978px tableheight=324pxstyle=font-weight:bold; ;text-align:centerTABLE: INHERITABLE_PROPS || + ||tablewidth=978px tableheight=324pxstyle=font-weight:bold; ;text-align:centerTABLE: INHERITABLE_PROPS || ||style=font-weight:bold;Name ||style=font-weight:bold;Data Type ||style=font-weight:bold;Primary Key ||style=font-weight:bold;Foreign Key ||style=font-weight:bold;Notes || ||wc_id ||integer ||Yes ||References NODES.WC_ID || || ||local_relpath ||text ||Yes ||References NODES.LOCAL_RELPATH || || @@ -104, +104 @@ === Merging Inherited Properties === - This proposal purposefully avoids any concept of how inherited properties might be merged together with explicit properties. In some cases if a path has an inheritable property set on it, then it might make sense to consider that property's value as the complete and full value for that path, end of story (e.g. svn:mergeinfo). On the other hand, it's easy to imagine use-cases where it might be useful to merge a path's explicit and inherited properties. However, both cases depend on the semantics of the particular inherited property. There is no way to develop a one-size-fits-all approach to merging inheritable properties. So while the suggested API changes below support the ability to get a path's explicit and inherited properties, how to merge these values (if at all) will be handled on a case-by-case basis. + This proposal purposefully avoids any concept of how inherited properties (possibly from multiple parents) might be merged together with explicit properties. In some cases if a path has an inheritable property set on it, then it might make sense to consider that property's value as the complete and full value for that path, end of story (e.g. svn:mergeinfo). On the other hand, it's easy to imagine use-cases where it might be useful to merge a path's explicit and inherited properties. However, both cases depend on the semantics of the particular inherited property. There is no way to develop a one-size-fits-all approach to merging inheritable properties. So while the suggested API changes below support the ability to get a path's explicit and inherited properties, how to merge these values (if at all) will be handled on a case-by-case basis. === Subcommand Changes === In general inherited properties will be treated like any other versioned property, so most subcommands will only notice the paths on which explicit properties are set, regardless of whether these are inheritable or not. For example, if we have an unmodified working copy and then make a local change to a parent path's explicit inheritable property:
svn commit: r1297676 - /subversion/trunk/subversion/tests/cmdline/svntest/main.py
Author: hwright
Date: Tue Mar 6 20:18:16 2012
New Revision: 1297676
URL: http://svn.apache.org/viewvc?rev=1297676view=rev
Log:
Start trying to use the Python logging framework in our testsuite.
Right now, this just dumps everything unaltered to stdout (even output of
commands that previously went to stderr). Further formatting and usage
improvements will hopefully follow.
* subversion/tests/cmdline/svntest/main.py
(logger): New.
(wait_on_pipe, spawn_process, run_command_stdin, copy_repos) : Use the
logger, rather than writing to stdout.
(_create_parser): Use the '--verbose' switch to change the level of logging.
(_parse_options): Remove some sanity checking.
Modified:
subversion/trunk/subversion/tests/cmdline/svntest/main.py
Modified: subversion/trunk/subversion/tests/cmdline/svntest/main.py
URL:
http://svn.apache.org/viewvc/subversion/trunk/subversion/tests/cmdline/svntest/main.py?rev=1297676r1=1297675r2=1297676view=diff
==
--- subversion/trunk/subversion/tests/cmdline/svntest/main.py (original)
+++ subversion/trunk/subversion/tests/cmdline/svntest/main.py Tue Mar 6
20:18:16 2012
@@ -35,6 +35,7 @@ import threading
import optparse # for argument parsing
import xml
import urllib
+import logging
try:
# Python =3.0
@@ -78,6 +79,10 @@ SVN_VER_MINOR = 8
default_num_threads = 5
+# Set up logging
+logger = logging.getLogger(__name__)
+logger.addHandler(logging.StreamHandler(sys.stdout))
+
class SVNProcessTerminatedBySignal(Failure):
Exception raised if a spawned process segfaulted, aborted, etc.
pass
@@ -427,22 +432,17 @@ def wait_on_pipe(waiter, binary_mode, st
exit_signal = exit_code
if stdout_lines is not None:
- sys.stdout.write(.join(stdout_lines))
- sys.stdout.flush()
+ logger.info(.join(stdout_lines))
if stderr_lines is not None:
- sys.stderr.write(.join(stderr_lines))
- sys.stderr.flush()
+ logger.warning(.join(stderr_lines))
if options.verbose:
# show the whole path to make it easier to start a debugger
- sys.stderr.write(CMD: %s terminated by signal %d\n
+ logger.warning(CMD: %s terminated by signal %d
% (command_string, exit_signal))
- sys.stderr.flush()
raise SVNProcessTerminatedBySignal
else:
-if exit_code and options.verbose:
- sys.stderr.write(CMD: %s exited with %d\n
- % (command_string, exit_code))
- sys.stderr.flush()
+if exit_code:
+ logger.info(CMD: %s exited with %d % (command_string, exit_code))
return stdout_lines, stderr_lines, exit_code
def spawn_process(command, bufsize=0, binary_mode=0, stdin_lines=None,
@@ -460,10 +460,9 @@ def spawn_process(command, bufsize=0, bi
raise TypeError(stdin_lines should have list type)
# Log the command line
- if options.verbose and not command.endswith('.py'):
-sys.stdout.write('CMD: %s %s\n' % (os.path.basename(command),
- ' '.join([_quote_arg(x) for x in
varargs])))
-sys.stdout.flush()
+ if not command.endswith('.py'):
+logger.info('CMD: %s %s' % (os.path.basename(command),
+ ' '.join([_quote_arg(x) for x in varargs])))
infile, outfile, errfile, kid = open_pipe([command] + list(varargs), bufsize)
@@ -494,8 +493,7 @@ def run_command_stdin(command, error_exp
If ERROR_EXPECTED is None, any stderr output will be printed and any
stderr output or a non-zero exit code will raise an exception.
- if options.verbose:
-start = time.time()
+ start = time.time()
exit_code, stdout_lines, stderr_lines = spawn_process(command,
bufsize,
@@ -503,18 +501,17 @@ def run_command_stdin(command, error_exp
stdin_lines,
*varargs)
- if options.verbose:
-stop = time.time()
-print('TIME = %.6f' % (stop - start))
-for x in stdout_lines:
- sys.stdout.write(x)
-for x in stderr_lines:
- sys.stdout.write(x)
+ stop = time.time()
+ logger.info('TIME = %.6f' % (stop - start))
+ for x in stdout_lines:
+logger.info(x[:-1])
+ for x in stderr_lines:
+logger.info(x)
if (not error_expected) and ((stderr_lines) or (exit_code != 0)):
if not options.verbose:
for x in stderr_lines:
-sys.stdout.write(x)
+logger.warning(x[:-1])
raise Failure
return exit_code, \
@@ -856,11 +853,10 @@ def copy_repos(src_path, dst_path, head_
if ignore_uuid:
load_args = load_args + ['--ignore-uuid']
- if options.verbose:
-sys.stdout.write('CMD: %s %s | %s %s\n' %
+
+ logger.info('CMD: %s %s | %s %s' %
(os.path.basename(svnadmin_binary), ' '.join(dump_args),
os.path.basename(svnadmin_binary), ' '.join(load_args)))
[Subversion Wiki] Update of InheritedProperties by pburba
Dear Wiki user,
You have subscribed to a wiki page or wiki category on Subversion Wiki for
change notification.
The InheritedProperties page has been changed by pburba:
http://wiki.apache.org/subversion/InheritedProperties?action=diffrev1=33rev2=34
Comment:
Adjust propget and proplist examples to reflect that any versioned property can
be inherited.
A child path that inherits a property from its parent may not have ready
access to that parent in the working copy (e.g. the root of the working copy is
a subtree of the parent path). To ensure that traditionally disconnected
operations (i.e. those that require no access to the repository, like 'svn
add') remain disconnected, we will maintain a cache of properties inherited by
the root of the working copy. Whenever a new working copy is checked out, any
properties inherited by the root of the working copy will be cached in the
working copy. If a subtree within a working copy is switched, a separate cache
will be created for the root of that subtree. Whenever an update occurs the
cache(s) will be refreshed.
The cache will be stored in a new wc-ng table:
- ||tablewidth=978px tableheight=324pxstyle=font-weight:bold;
;text-align:centerTABLE: INHERITABLE_PROPS ||
+ ||tablewidth=978px tableheight=324pxstyle=font-weight:bold;
;text-align:centerTABLE: INHERITABLE_PROPS ||
||style=font-weight:bold;Name ||style=font-weight:bold;Data Type
||style=font-weight:bold;Primary Key ||style=font-weight:bold;Foreign
Key ||style=font-weight:bold;Notes ||
||wc_id ||integer ||Yes ||References NODES.WC_ID || ||
||local_relpath ||text ||Yes ||References NODES.LOCAL_RELPATH || ||
@@ -107, +107 @@
This proposal purposefully avoids any concept of how inherited properties
(possibly from multiple parents) might be merged together with explicit
properties. In some cases if a path has an inheritable property set on it,
then it might make sense to consider that property's value as the complete and
full value for that path, end of story (e.g. svn:mergeinfo). On the other
hand, it's easy to imagine use-cases where it might be useful to merge a path's
explicit and inherited properties. However, both cases depend on the semantics
of the particular inherited property. There is no way to develop a
one-size-fits-all approach to merging inheritable properties. So while the
suggested API changes below support the ability to get a path's explicit and
inherited properties, how to merge these values (if at all) will be handled on
a case-by-case basis.
=== Subcommand Changes ===
- In general inherited properties will be treated like any other versioned
property, so most subcommands will only notice the paths on which explicit
properties are set, regardless of whether these are inheritable or not. For
example, if we have an unmodified working copy and then make a local change to
a parent path's explicit inheritable property:
+ Since inherited properties are really just a new way of looking at
versioned properties, most subcommands will only notice the paths on which
explicit properties are set. For example, if we have an unmodified working
copy and then make a local change to a parent path's explicit property:
* 'svn status' won't show any property mods on the parent's children paths.
* 'svn diff' will only show the property difference on the parent path.
@@ -119, +119 @@
* update (up): Updates the inherited properties cache(s).
* upgrade: Creates an empty inherited properties cache, but doesn't populate
it since that would require contacting the repository which upgrade shouldn't
need to do.
- Two subcommands, propget (pget, pg) and proplist (plist, pl), will support a
new option enabling users to find inherited properties on a path:
'--show-inherited-props'. When used, any inherited properties for the target of
the subcommand will be displayed. Inherited properties for subtrees of the
target are not shown. For example: Given this repository structure with the
explicit properties noted:
+ Two subcommands, propget (pget, pg) and proplist (plist, pl), will support a
new option enabling users to view inherited properties on a path:
'--show-inherited-props'. When used, any properties inherited by the target of
the subcommand will be displayed. Properties inherited by subtrees of the
target are not shown. For example: Given this repository structure with the
explicit properties noted:
{{{
- /svn:inheritable:foo=bar
+ /prop:foo=bar
- branch/ svn:inheritable:foo=bat, prop:baz=qux
+ branch/ prop:foo=bat
- branch/src/ svn:inheritable:foo=bax
+ branch/src/ prop:foo=bax
branch/src/pop.c
branch/inc
branch/inc/pop.h
@@ -139, +139 @@
{{{
svn pl -vR ^/branch
Properties on '%ROOT_URL%/branch':
- svn:inheritable:foo
+ prop:foo
bat
- prop:baz
- qux
Properties on '%ROOT_URL%/branch/src':
-
svn commit: r1297724 - /subversion/trunk/subversion/tests/cmdline/svntest/main.py
Author: hwright Date: Tue Mar 6 21:17:29 2012 New Revision: 1297724 URL: http://svn.apache.org/viewvc?rev=1297724view=rev Log: Use the root logger in the python tests, rather than a custom one. * subversion/tests/cmdline/svntest/main.py (logger): Grab the root logger. Modified: subversion/trunk/subversion/tests/cmdline/svntest/main.py Modified: subversion/trunk/subversion/tests/cmdline/svntest/main.py URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/tests/cmdline/svntest/main.py?rev=1297724r1=1297723r2=1297724view=diff == --- subversion/trunk/subversion/tests/cmdline/svntest/main.py (original) +++ subversion/trunk/subversion/tests/cmdline/svntest/main.py Tue Mar 6 21:17:29 2012 @@ -80,7 +80,7 @@ SVN_VER_MINOR = 8 default_num_threads = 5 # Set up logging -logger = logging.getLogger(__name__) +logger = logging.getLogger() logger.addHandler(logging.StreamHandler(sys.stdout)) class SVNProcessTerminatedBySignal(Failure):
svn commit: r1297725 - /subversion/trunk/subversion/tests/cmdline/svntest/sandbox.py
Author: hwright
Date: Tue Mar 6 21:18:37 2012
New Revision: 1297725
URL: http://svn.apache.org/viewvc?rev=1297725view=rev
Log:
Followup to r1297676 by using the logger in the sandbox module of the tests.
* subversion/tests/cmdline/svntest/sandbox.py
(_cleanup_test_path): Use the logger, rather than directly printing to stdout.
Modified:
subversion/trunk/subversion/tests/cmdline/svntest/sandbox.py
Modified: subversion/trunk/subversion/tests/cmdline/svntest/sandbox.py
URL:
http://svn.apache.org/viewvc/subversion/trunk/subversion/tests/cmdline/svntest/sandbox.py?rev=1297725r1=1297724r2=1297725view=diff
==
--- subversion/trunk/subversion/tests/cmdline/svntest/sandbox.py (original)
+++ subversion/trunk/subversion/tests/cmdline/svntest/sandbox.py Tue Mar 6
21:18:37 2012
@@ -25,9 +25,12 @@ import os
import shutil
import copy
import urllib
+import logging
import svntest
+logger = logging.getLogger('svntest')
+
class Sandbox:
Manages a sandbox (one or more repository/working copy pairs) for
@@ -307,14 +310,13 @@ def cleanup_deferred_test_paths():
def _cleanup_test_path(path, retrying=False):
- if svntest.main.options.verbose:
-if retrying:
- print(CLEANUP: RETRY: %s % path)
-else:
- print(CLEANUP: %s % path)
+ if retrying:
+logger.info(CLEANUP: RETRY: %s, path)
+ else:
+logger.info(CLEANUP: %s, path)
+
try:
svntest.main.safe_rmtree(path)
except:
-if svntest.main.options.verbose:
- print(WARNING: cleanup failed, will try again later)
+logger.info(WARNING: cleanup failed, will try again later)
_deferred_test_paths.append(path)
svn commit: r1297735 - /subversion/trunk/subversion/tests/cmdline/svntest/main.py
Author: hwright Date: Tue Mar 6 21:40:24 2012 New Revision: 1297735 URL: http://svn.apache.org/viewvc?rev=1297735view=rev Log: * subversion/tests/cmdline/svntest/main.py (TestRunner): Always print progress. Modified: subversion/trunk/subversion/tests/cmdline/svntest/main.py Modified: subversion/trunk/subversion/tests/cmdline/svntest/main.py URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/tests/cmdline/svntest/main.py?rev=1297735r1=1297734r2=1297735view=diff == --- subversion/trunk/subversion/tests/cmdline/svntest/main.py (original) +++ subversion/trunk/subversion/tests/cmdline/svntest/main.py Tue Mar 6 21:40:24 2012 @@ -1251,7 +1251,7 @@ class TestRunner: # If there is no filter or this test made if through # the filter then print it! if options.milestone_filter is None or len(issues): -if options.verbose and self.pred.inprogress: +if self.pred.inprogress: tail += [[%s]] % self.pred.inprogress else: print( %3d%-5s %s%s % (self.index,
svn commit: r1297736 - in /subversion/trunk/subversion/tests/cmdline: checkout_tests.py tree_conflict_tests.py
Author: hwright Date: Tue Mar 6 21:42:51 2012 New Revision: 1297736 URL: http://svn.apache.org/viewvc?rev=1297736view=rev Log: Remove the last references to options.verbose(). * subversion/tests/cmdline/checkout_tests.py (test_stderr): Use the log, instead of printing output. * subversion/tests/cmdline/tree_conflict_tests.py (verbose_print, verbose_printlines): Remove. (ensure_tree_conflict): Use the log, rather than another selective printer. Modified: subversion/trunk/subversion/tests/cmdline/checkout_tests.py subversion/trunk/subversion/tests/cmdline/tree_conflict_tests.py Modified: subversion/trunk/subversion/tests/cmdline/checkout_tests.py URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/tests/cmdline/checkout_tests.py?rev=1297736r1=1297735r2=1297736view=diff == --- subversion/trunk/subversion/tests/cmdline/checkout_tests.py (original) +++ subversion/trunk/subversion/tests/cmdline/checkout_tests.py Tue Mar 6 21:42:51 2012 @@ -31,6 +31,7 @@ import sys, re, os, time, subprocess # Our testing module import svntest from svntest import wc, actions +import logging # (abbreviation) Skip = svntest.testcase.Skip_deco @@ -41,6 +42,8 @@ Issue = svntest.testcase.Issue_deco Wimp = svntest.testcase.Wimp_deco Item = wc.StateItem +logger = logging.getLogger() + #-- # Helper function for testing stderr from co. # If none of the strings in STDERR list matches the regular expression @@ -50,10 +53,9 @@ def test_stderr(re_string, stderr): for line in stderr: if exp_err_re.search(line): return - if svntest.main.options.verbose: -for x in stderr: - sys.stdout.write(x) -print(Expected stderr reg-ex: ' + re_string + ') + for x in stderr: +logger.debug(x[:-1]) +logger.info(Expected stderr reg-ex: ' + re_string + ') raise svntest.Failure(Checkout failed but not in the expected way) #-- Modified: subversion/trunk/subversion/tests/cmdline/tree_conflict_tests.py URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/tests/cmdline/tree_conflict_tests.py?rev=1297736r1=1297735r2=1297736view=diff == --- subversion/trunk/subversion/tests/cmdline/tree_conflict_tests.py (original) +++ subversion/trunk/subversion/tests/cmdline/tree_conflict_tests.py Tue Mar 6 21:42:51 2012 @@ -38,6 +38,7 @@ from svntest.actions import run_and_veri from svntest.actions import run_and_verify_info from svntest.actions import get_virginal_state import shutil +import logging # (abbreviation) Skip = svntest.testcase.Skip_deco @@ -49,16 +50,7 @@ Wimp = svntest.testcase.Wimp_deco Item = svntest.wc.StateItem AnyOutput = svntest.verify.AnyOutput -# If verbose mode is enabled, print the LINE and a newline. -def verbose_print(line): - if main.options.verbose: -print(line) - -# If verbose mode is enabled, print the (assumed newline-terminated) LINES. -def verbose_printlines(lines): - if main.options.verbose: -for line in lines: - sys.stdout.write(line) +logger = logging.getLogger() ## # Tests @@ -388,14 +380,14 @@ def ensure_tree_conflict(sbox, operation def url_of(repo_relative_path): return sbox.repo_url + '/' + repo_relative_path - verbose_print() - verbose_print(=== Starting a set of ' + operation + ' tests.) + logger.debug() + logger.debug(=== Starting a set of ' + operation + ' tests.) # Path to source branch, relative to wc_dir. # Source is where the incoming mods are made. source_br = branch1 - verbose_print(--- Creating changes in repos) + logger.debug(--- Creating changes in repos) source_wc_dir = os.path.join(wc_dir, source_br) source_left_rev, source_right_rev = set_up_repos(wc_dir, source_wc_dir, incoming_scenarios) @@ -430,9 +422,9 @@ def ensure_tree_conflict(sbox, operation source_url = url_of(source_br + '/' + scen_name) target_path = os.path.join(target_br, scen_name) - verbose_print(=== + str(inc_action) + onto + str(loc_action)) + logger.debug(=== + str(inc_action) + onto + str(loc_action)) - verbose_print(--- Making local mods) + logger.debug(--- Making local mods) for modaction in loc_action: modify(modaction, localmod_paths(., target_path), is_init=False) if commit_local_mods: @@ -444,7 +436,7 @@ def ensure_tree_conflict(sbox, operation # For update, verify the pre-condition that WC is out of date. # For switch/merge, there is no such precondition. if operation == 'update': -verbose_print(--- Trying to commit (expecting 'out-of-date' error)) +logger.debug(--- Trying to commit
svn commit: r1297739 - /subversion/trunk/subversion/tests/cmdline/svntest/main.py
Author: gstein
Date: Tue Mar 6 21:44:51 2012
New Revision: 1297739
URL: http://svn.apache.org/viewvc?rev=1297739view=rev
Log:
Add a timestamp to the log output.
* subversion/tests/cmdline/svntest/main.py:
(...): create the handler, then add a Formatter to it with our new
format, and then add the handler to the root logger.
Modified:
subversion/trunk/subversion/tests/cmdline/svntest/main.py
Modified: subversion/trunk/subversion/tests/cmdline/svntest/main.py
URL:
http://svn.apache.org/viewvc/subversion/trunk/subversion/tests/cmdline/svntest/main.py?rev=1297739r1=1297738r2=1297739view=diff
==
--- subversion/trunk/subversion/tests/cmdline/svntest/main.py (original)
+++ subversion/trunk/subversion/tests/cmdline/svntest/main.py Tue Mar 6
21:44:51 2012
@@ -81,7 +81,12 @@ default_num_threads = 5
# Set up logging
logger = logging.getLogger()
-logger.addHandler(logging.StreamHandler(sys.stdout))
+handler = logging.StreamHandler(sys.stdout)
+formatter = logging.Formatter('%(asctime)s [%(levelname)s] %(message)s',
+ '%Y-%m-%d %H:%M:%S')
+handler.setFormatter(formatter)
+logger.addHandler(handler)
+
class SVNProcessTerminatedBySignal(Failure):
Exception raised if a spawned process segfaulted, aborted, etc.
svn commit: r1297742 - /subversion/trunk/subversion/tests/cmdline/checkout_tests.py
Author: hwright Date: Tue Mar 6 21:52:13 2012 New Revision: 1297742 URL: http://svn.apache.org/viewvc?rev=1297742view=rev Log: Fix indentation misstep in r1297736. * subversion/tests/cmdline/checkout_tests.py (test_stderr): Don't spew forth an extra log message ad nauseum. Modified: subversion/trunk/subversion/tests/cmdline/checkout_tests.py Modified: subversion/trunk/subversion/tests/cmdline/checkout_tests.py URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/tests/cmdline/checkout_tests.py?rev=1297742r1=1297741r2=1297742view=diff == --- subversion/trunk/subversion/tests/cmdline/checkout_tests.py (original) +++ subversion/trunk/subversion/tests/cmdline/checkout_tests.py Tue Mar 6 21:52:13 2012 @@ -55,7 +55,7 @@ def test_stderr(re_string, stderr): return for x in stderr: logger.debug(x[:-1]) -logger.info(Expected stderr reg-ex: ' + re_string + ') + logger.info(Expected stderr reg-ex: ' + re_string + ') raise svntest.Failure(Checkout failed but not in the expected way) #--
svn commit: r1297795 - in /subversion/trunk/tools/server-side/svnpubsub: svnwcsub.py svnwcsub.tac
Author: gstein
Date: Tue Mar 6 23:41:40 2012
New Revision: 1297795
URL: http://svn.apache.org/viewvc?rev=1297795view=rev
Log:
Move the working copy review and preparation into a synchronous
process in the (new) BDEC.start() method. The start method will be
called on the main thread before starting Twisted; if running under
the Twisted daemon (potentially along with other services), then the
start method will be invoked on a background thread.
Also ensured that we pass ENV to all subprocess invocations.
* tools/server-side/svnpubsub/svnwcsub.py:
(check_output): accept an ENV parameter in the back-compat code
(svn_info): accept an ENV param and pass it along
(WorkingCopy.__init__): call _get_match synchronously, apply the
results to self, and notify the BDEC that the wc is ready. log any
errors processing the working copy.
(WorkingCopy._set_match): removed
(WorkingCopy._get_match): accept SVNBIN and ENV rather than looking
in our instance vars. pass the ENV to the subprocess calls.
simplify the data returned.
(BigDoEverythingClasss.__init__): save the list of working copies
that we want to track. stop setting up working copies.
(BigDoEverythingClasss.start): new method to process all the WCs for
tracking. create all of them synchronously.
(BackgroundWorker._update): pass ENV to svn_info()
(...): call BDEC.start() just before starting Twisted
* tools/server-side/svnpubsub/svnwcsub.tac:
(get_service): tell Twisted to call BDEC.start() on a background
thread, once the service is up and running (ie. post-daemonize)
Modified:
subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py
subversion/trunk/tools/server-side/svnpubsub/svnwcsub.tac
Modified: subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py
URL:
http://svn.apache.org/viewvc/subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py?rev=1297795r1=1297794r2=1297795view=diff
==
--- subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py (original)
+++ subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py Tue Mar 6
23:41:40 2012
@@ -54,8 +54,8 @@ from twisted.internet import protocol
try:
check_output = subprocess.check_output
except AttributeError:
-def check_output(args): # note: we don't use anything beyond args
-pipe = subprocess.Popen(args, stdout=subprocess.PIPE)
+def check_output(args, env): # note: we only use these two args
+pipe = subprocess.Popen(args, stdout=subprocess.PIPE, env=env)
output, _ = pipe.communicate()
if pipe.returncode:
raise subprocess.CalledProcessError(pipe.returncode, args)
@@ -65,10 +65,10 @@ except AttributeError:
### note: this runs synchronously. within the current Twisted environment,
### it is called from ._get_match() which is run on a thread so it won't
### block the Twisted main loop.
-def svn_info(svnbin, path):
+def svn_info(svnbin, env, path):
Run 'svn info' on the target path, returning a dict of info data.
args = [svnbin, info, --non-interactive, --, path]
-output = check_output(args).strip()
+output = check_output(args, env=env).strip()
info = { }
for line in output.split('\n'):
idx = line.index(':')
@@ -78,20 +78,14 @@ def svn_info(svnbin, path):
class WorkingCopy(object):
def __init__(self, bdec, path, url):
-self.bdec = bdec
self.path = path
self.url = url
-self.repos = None
-self.match = None
-d = threads.deferToThread(self._get_match)
-d.addCallback(self._set_match)
-
-def _set_match(self, value):
-self.match = str(value[0])
-self.url = value[1]
-self.repos = value[2]
-self.uuid = value[3]
-self.bdec.wc_ready(self)
+
+try:
+self.match, self.uuid = self._get_match(bdec.svnbin, bdec.env)
+bdec.wc_ready(self)
+except:
+logging.exception('problem with working copy: %s', path)
def update_applies(self, uuid, path):
if self.uuid != uuid:
@@ -114,23 +108,24 @@ class WorkingCopy(object):
return True
return False
-def _get_match(self):
+def _get_match(self, svnbin, env):
### quick little hack to auto-checkout missing working copies
if not os.path.isdir(self.path):
logging.info(autopopulate %s from %s % (self.path, self.url))
-subprocess.check_call([self.bdec.svnbin, 'co', '-q',
+subprocess.check_call([svnbin, 'co', '-q',
'--non-interactive',
'--config-dir',
'/home/svnwc/.subversion',
- '--', self.url, self.path])
+ '--', self.url, self.path],
+ env=env)
# Fetch the info for matching
svn commit: r1297805 - /subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py
Author: gstein
Date: Wed Mar 7 00:10:13 2012
New Revision: 1297805
URL: http://svn.apache.org/viewvc?rev=1297805view=rev
Log:
Strip all of the reconnect logic. Joe reports that it doesn't work
very well, and we're eventually going to rely on the logic in
client.py anyways.
* tools/server-side/svnpubsub/svnwcsub.py:
(...): remove some unused imports
(StreamHandler.startElement): ignore the keepalive elements
(XMLHTTPStream.__init__): no need to store ALIVE or BDEC
(XMLHTTPStream.pageStart, .pageEnd): remove these methods overrides;
we don't need to inform BDEC
(BigDoEverythingClasss.__init__): no need for the CHECKER, or to
record FAILURES or ALIVE
(BigDoEverythingClasss.pageStart, .pageEnd, ._checkalive,
.stillalive, .streamDead): removed. unneeded.
(BigDoEverythingClasss._restartStream): no need to record ALIVE or
to def a streamDead() call.
Modified:
subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py
Modified: subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py
URL:
http://svn.apache.org/viewvc/subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py?rev=1297805r1=1297804r2=1297805view=diff
==
--- subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py (original)
+++ subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py Wed Mar 7
00:10:13 2012
@@ -40,13 +40,11 @@ import logging.handlers
import Queue
import optparse
-from twisted.internet import reactor, task, threads
-from twisted.internet.utils import getProcessOutput
+from twisted.internet import reactor
from twisted.application import internet
from twisted.web.client import HTTPClientFactory, HTTPPageDownloader
from urlparse import urlparse
from xml.sax import handler, make_parser
-from twisted.internet import protocol
# check_output() is only available in Python 2.7. Allow us to run with
@@ -159,7 +157,6 @@ class StreamHandler(handler.ContentHandl
self.text_value = None
def startElement(self, name, attrs):
-#print start element: %s % (name)
commit revision=7
dirs_changedpath//path/dirs_changed
@@ -167,8 +164,7 @@ class StreamHandler(handler.ContentHandl
if name == commit:
self.rev = Revision(attrs['repository'], int(attrs['revision']))
-elif name == stillalive:
-self.bdec.stillalive(self.stream)
+
def characters(self, data):
if self.text_value is not None:
self.text_value = self.text_value + data
@@ -176,7 +172,6 @@ class StreamHandler(handler.ContentHandl
self.text_value = data
def endElement(self, name):
-#print end element: %s % (name)
if name == commit:
self.bdec.commit(self.stream, self.rev)
self.rev = None
@@ -188,20 +183,13 @@ class StreamHandler(handler.ContentHandl
class XMLHTTPStream(HTTPStream):
def __init__(self, url, bdec):
HTTPStream.__init__(self, url)
-self.alive = 0
-self.bdec = bdec
self.parser = make_parser(['xml.sax.expatreader'])
self.handler = StreamHandler(self, bdec)
self.parser.setContentHandler(self.handler)
-def pageStart(self, parital):
-self.bdec.pageStart(self)
-
def pagePart(self, data):
self.parser.feed(data)
-def pageEnd(self):
-self.bdec.pageEnd(self)
def connectTo(url, bdec):
u = urlparse(url)
@@ -228,65 +216,19 @@ class BigDoEverythingClasss(object):
self.tracking = config.get_track()
self.worker = BackgroundWorker(self.svnbin, self.env)
self.service = service
-self.failures = 0
-self.alive = time.time()
-self.checker = task.LoopingCall(self._checkalive)
self.transports = {}
self.streams = {}
for u in self.urls:
self._restartStream(u)
self.watch = []
-self.checker.start(CHECKBEAT_TIME)
def start(self):
for path, url in self.tracking.items():
# working copies auto-register with the BDEC when they are ready.
WorkingCopy(self, path, url)
-def pageStart(self, stream):
-logging.info(Stream %s Connection Established % (stream.url))
-self.failures = 0
-
-def pageEnd(self, stream):
-logging.info(Stream %s Connection Dead % (stream.url))
-self.streamDead(stream.url)
-
def _restartStream(self, url):
(self.streams[url], self.transports[url]) = connectTo(url, self)
-self.streams[url].deferred.addBoth(self.streamDead, url)
-self.streams[url].alive = time.time()
-
-def _checkalive(self):
-n = time.time()
-for k in self.streams.keys():
- s = self.streams[k]
- if n - s.alive CHECKBEAT_TIME:
-logging.info(Stream %s is dead, reconnecting % (s.url))
-#self.transports[s.url].disconnect()
-
svn commit: r1297815 - /subversion/trunk/tools/server-side/svnpubsub/daemonize.py
Author: gstein
Date: Wed Mar 7 00:50:52 2012
New Revision: 1297815
URL: http://svn.apache.org/viewvc?rev=1297815view=rev
Log:
We need to turn svnwcsub into a daemon, rather than only running in
the foreground. Add this module to provide that functionality.
Copied from gstein.googlecode.com/svn/trunk/python/daemonize.py, r15.
* tools/server-side/svnpubsub/daemonize.py: new file
Added:
subversion/trunk/tools/server-side/svnpubsub/daemonize.py
Added: subversion/trunk/tools/server-side/svnpubsub/daemonize.py
URL:
http://svn.apache.org/viewvc/subversion/trunk/tools/server-side/svnpubsub/daemonize.py?rev=1297815view=auto
==
--- subversion/trunk/tools/server-side/svnpubsub/daemonize.py (added)
+++ subversion/trunk/tools/server-side/svnpubsub/daemonize.py Wed Mar 7
00:50:52 2012
@@ -0,0 +1,262 @@
+# ---
+#
+# Copyright (c) 2005, Greg Stein
+#
+# Licensed under the Apache License, Version 2.0 (the License);
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an AS IS BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# ---
+#
+# This software lives at:
+#http://gstein.googlecode.com/svn/trunk/python/daemonize.py
+#
+
+import os
+import signal
+import sys
+import time
+
+
+# possible return values from Daemon.daemonize()
+DAEMON_RUNNING = 'The daemon is running'
+DAEMON_NOT_RUNNING = 'The daemon is not running'
+DAEMON_COMPLETE = 'The daemon has completed its operations'
+DAEMON_STARTED = 'The daemon has been started'
+
+
+class Daemon(object):
+
+ def __init__(self, logfile, pidfile):
+self.logfile = logfile
+self.pidfile = pidfile
+
+ def foreground(self):
+Run in the foreground.
+### we should probably create a pidfile. other systems may try to detect
+### the pidfile to see if this daemon is running.
+self.setup()
+self.run()
+### remove the pidfile
+
+ def daemonize_exit(self):
+try:
+ result = self.daemonize()
+except (ChildFailed, DaemonFailed), e:
+ # duplicate the exit code
+ sys.exit(e.code)
+except (ChildTerminatedAbnormally, ChildForkFailed,
+DaemonTerminatedAbnormally, DaemonForkFailed), e:
+ sys.stderr.write('ERROR: %s\n' % e)
+ sys.exit(1)
+except ChildResumedIncorrectly:
+ sys.stderr.write('ERROR: continued after receiving unknown signal.\n')
+ sys.exit(1)
+
+if result == DAEMON_STARTED or result == DAEMON_COMPLETE:
+ sys.exit(0)
+elif result == DAEMON_NOT_RUNNING:
+ sys.stderr.write('ERROR: the daemon exited with a success code '
+ 'without signalling its startup.\n')
+ sys.exit(1)
+
+# in original process. daemon is up and running. we're done.
+
+ def daemonize(self):
+# fork off a child that can detach itself from this process.
+try:
+ pid = os.fork()
+except OSError, e:
+ raise ChildForkFailed(e.errno, e.strerror)
+
+if pid 0:
+ # we're in the parent. let's wait for the child to finish setting
+ # things up -- on our exit, we want to ensure the child is accepting
+ # connections.
+ cpid, status = os.waitpid(pid, 0)
+ assert pid == cpid
+ if os.WIFEXITED(status):
+code = os.WEXITSTATUS(status)
+if code:
+ raise ChildFailed(code)
+return DAEMON_RUNNING
+
+ # the child did not exit cleanly.
+ raise ChildTerminatedAbnormally(status)
+
+# we're in the child.
+
+# decouple from the parent process
+os.chdir('/')
+os.umask(0)
+os.setsid()
+
+# remember this pid so the second child can signal it.
+thispid = os.getpid()
+
+# register a signal handler so the SIGUSR1 doesn't stop the process.
+# this object will also record whether if got signalled.
+daemon_accepting = SignalCatcher(signal.SIGUSR1)
+
+# if the daemon process exits before sending SIGUSR1, then we need to see
+# the problem. trap SIGCHLD with a SignalCatcher.
+daemon_exit = SignalCatcher(signal.SIGCHLD)
+
+# perform the second fork
+try:
+ pid = os.fork()
+except OSError, e:
+ raise DaemonForkFailed(e.errno, e.strerror)
+
+if pid 0:
+ # in the parent.
+
+ # we want to wait for the daemon to signal that it has created and
+ # bound the socket, and is (thus) ready for connections. if the
+ # daemon improperly exits before serving, we'll see SIGCHLD and the
+ # .pause will
svn commit: r1297822 - /subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py
Author: gstein
Date: Wed Mar 7 01:17:06 2012
New Revision: 1297822
URL: http://svn.apache.org/viewvc?rev=1297822view=rev
Log:
Add the ability to run as a daemon (specified by the --daemon command
line option).
* tools/server-side/svnpubsub/svnwcsub.py:
(class Daemon): new subclass of daemonize.Daemon to run the server,
once it has become a daemon
(handle_options): don't write out a pidfile if we're going to run as
a daemon. gotta wait until after the double-fork.
(main): add a new --daemon option. require a logfile and pidfile
when running as a daemon. start up the daemon as appropriate.
Modified:
subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py
Modified: subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py
URL:
http://svn.apache.org/viewvc/subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py?rev=1297822r1=1297821r2=1297822view=diff
==
--- subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py (original)
+++ subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py Wed Mar 7
01:17:06 2012
@@ -46,6 +46,7 @@ from twisted.web.client import HTTPClien
from urlparse import urlparse
from xml.sax import handler, make_parser
+import daemonize
# check_output() is only available in Python 2.7. Allow us to run with
# earlier versions
@@ -392,6 +393,22 @@ class ReloadableConfig(ConfigParser.Safe
return str(option)
+class Daemon(daemonize.Daemon):
+def __init__(self, logfile, pidfile, bdec):
+daemonize.Daemon.__init__(self, logfile, pidfile)
+
+self.bdec = bdec
+
+def setup(self):
+# There is no setup which the parent needs to wait for.
+pass
+
+def run(self):
+# Start the BDEC (on the main thread), then start up twisted
+self.bdec.start()
+reactor.run()
+
+
def prepare_logging(logfile):
Log to the specified file, or to stdout if None.
@@ -420,7 +437,9 @@ def handle_options(options):
# Set up the logging, then process the rest of the options.
prepare_logging(options.logfile)
-if options.pidfile:
+# In daemon mode, we let the daemonize module handle the pidfile.
+# Otherwise, we should write this (foreground) PID into the file.
+if options.pidfile and not options.daemon:
pid = os.getpid()
open(options.pidfile, 'w').write('%s\n' % pid)
logging.info('pid %d written to %s', pid, options.pidfile)
@@ -465,6 +484,8 @@ def main(args):
help='switch to this GID before running')
parser.add_option('--umask',
help='set this (octal) umask before running')
+parser.add_option('--daemon', action='store_true',
+ help='run as a background daemon')
options, extra = parser.parse_args(args)
@@ -472,15 +493,26 @@ def main(args):
parser.error('CONFIG_FILE is required')
config_file = extra[0]
+if options.daemon and not options.logfile:
+parser.error('LOGFILE is required when running as a daemon')
+if options.daemon and not options.pidfile:
+parser.error('PIDFILE is required when running as a daemon')
+
# Process any provided options.
handle_options(options)
c = ReloadableConfig(config_file)
bdec = BigDoEverythingClasss(c)
-# Start the BDEC on the main thread, then start up twisted
-bdec.start()
-reactor.run()
+# We manage the logfile ourselves (along with possible rotation). The
+# daemon process can just drop stdout/stderr into /dev/null.
+d = Daemon('/dev/null', options.pidfile, bdec)
+if options.daemon:
+# Daemonize the process and call sys.exit() with appropriate code
+d.daemonize_exit()
+else:
+# Just run in the foreground (the default)
+d.foreground()
if __name__ == __main__:
svn commit: r1297823 - in /subversion/trunk/tools/server-side/svnpubsub/rc.d: svnwcsub svnwcsub.debian svnwcsub.solaris
Author: gstein
Date: Wed Mar 7 01:19:13 2012
New Revision: 1297823
URL: http://svn.apache.org/viewvc?rev=1297823view=rev
Log:
Adjust the rc.d scripts to invoke svnwcsub.py directly (rather than
via the twistd daemon), and to request daemon mode.
* tools/server-side/svnpubsub/rc.d/svnwcsub.solaris:
* tools/server-side/svnpubsub/rc.d/svnwcsub:
* tools/server-side/svnpubsub/rc.d/svnwcsub.debian:
(...): adjust invocation and add --daemon
Modified:
subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub
subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub.debian
subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub.solaris
Modified: subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub
URL:
http://svn.apache.org/viewvc/subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub?rev=1297823r1=1297822r2=1297823view=diff
==
--- subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub (original)
+++ subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub Wed Mar 7
01:19:13 2012
@@ -18,22 +18,19 @@ load_rc_config $name
svnwcsub_enable=${svnwcsub_enable-NO}
svnwcsub_user=${svnwcsub_user-svnwc}
svnwcsub_group=${svnwcsub_group-svnwc}
-svnwcsub_reactor=${svnwcsub_reactor-poll}
svnwcsub_pidfile=${svnwcsub_pidfile-/var/run/svnwcsub/svnwcsub.pub}
-svnwcsub_program=${svnwcsub_program-/usr/local/bin/twistd}
svnwcsub_env=PYTHON_EGG_CACHE
svnwcsub_cmd_int=${svnwcsub_cmd_int-python}
pidfile=${svnwcsub_pidfile}
export PYTHON_EGG_CACHE=/var/run/svnwcsub
-command=/usr/local/bin/twistd
+command=/usr/local/svnpubsub/svnwcsub.py
command_interpreter=/usr/local/bin/${svnwcsub_cmd_int}
-command_args=-y /usr/local/svnpubsub/svnwcsub.tac \
---logfile=/var/log/svnwcsub.log \
---pidfile=${pidfile} \
---uid=${svnwcsub_user} --gid=${svnwcsub_group} \
---umask=002 -r${svnwcsub_reactor}
+command_args=--daemon \
+ --logfile=/var/log/svnwcsub.log \
+ --pidfile=${pidfile} \
+ --uid=${svnwcsub_user} --gid=${svnwcsub_group} \
+ --umask=002
run_rc_command $1
-
Modified: subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub.debian
URL:
http://svn.apache.org/viewvc/subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub.debian?rev=1297823r1=1297822r2=1297823view=diff
==
--- subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub.debian (original)
+++ subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub.debian Wed Mar
7 01:19:13 2012
@@ -14,21 +14,20 @@
svnwcsub_user=${svnwcsub_user-svnwc}
svnwcsub_group=${svnwcsub_group-svnwc}
-svnwcsub_reactor=${svnwcsub_reactor-poll}
svnwcsub_pidfile=${svnwcsub_pidfile-/var/run/svnwcsub.pid}
pidfile=${svnwcsub_pidfile}
-TWSITD_CMD=/usr/bin/twistd -y /opt/svnpubsub/svnwcsub.tac \
---logfile=/var/bwlog/svnpubsub/svnwcsub.log \
---pidfile=${pidfile} \
---uid=${svnwcsub_user} --gid=${svnwcsub_group} \
--r${svnwcsub_reactor}
+SVNWCSUB_CMD=/opt/svnpubsub/svnwcsub.py \
+ --daemon \
+ --logfile=/var/bwlog/svnpubsub/svnwcsub.log \
+ --pidfile=${pidfile} \
+ --uid=${svnwcsub_user} --gid=${svnwcsub_group}
RETVAL=0
start() {
echo Starting SvnWcSub Server:
-$TWSITD_CMD
+$SVNWCSUB_CMD
RETVAL=$?
[ $RETVAL -eq 0 ] echo ok || echo failed
return $RETVAL
Modified: subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub.solaris
URL:
http://svn.apache.org/viewvc/subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub.solaris?rev=1297823r1=1297822r2=1297823view=diff
==
--- subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub.solaris
(original)
+++ subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub.solaris Wed Mar
7 01:19:13 2012
@@ -5,22 +5,21 @@
svnwcsub_user=${svnwcsub_user-svnwc}
svnwcsub_group=${svnwcsub_group-other}
-svnwcsub_reactor=${svnwcsub_reactor-poll}
svnwcsub_pidfile=${svnwcsub_pidfile-/var/run/svnwcsub/svnwcsub.pid}
pidfile=${svnwcsub_pidfile}
-TWSITD_CMD=/opt/python/2.6.2/bin/twistd -y /usr/local/svnpubsub/svnwcsub.tac \
---logfile=/x1/log/svnwcsub.log \
---pidfile=${pidfile} \
---umask=002 \
---uid=${svnwcsub_user} --gid=${svnwcsub_group} \
--r${svnwcsub_reactor}
+SVNWCSUB_CMD=/usr/local/svnpubsub/svnwcsub.py \
+ --daemon \
+ --logfile=/x1/log/svnwcsub.log \
+ --pidfile=${pidfile} \
+ --umask=002 \
+ --uid=${svnwcsub_user} --gid=${svnwcsub_group}
RETVAL=0
start() {
echo Starting SvnWcSub Server:
-$TWSITD_CMD
+$SVNWCSUB_CMD
RETVAL=$?
[ $RETVAL -eq 0 ] echo ok || echo failed
return
svn commit: r1297824 - in /subversion/trunk/tools/server-side/svnpubsub/rc.d: svnwcsub svnwcsub.debian svnwcsub.solaris
Author: joes
Date: Wed Mar 7 01:26:25 2012
New Revision: 1297824
URL: http://svn.apache.org/viewvc?rev=1297824view=rev
Log:
specify conf file
Modified:
subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub
subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub.debian
subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub.solaris
Modified: subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub
URL:
http://svn.apache.org/viewvc/subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub?rev=1297824r1=1297823r2=1297824view=diff
==
--- subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub (original)
+++ subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub Wed Mar 7
01:26:25 2012
@@ -31,6 +31,6 @@ command_args=--daemon \
--logfile=/var/log/svnwcsub.log \
--pidfile=${pidfile} \
--uid=${svnwcsub_user} --gid=${svnwcsub_group} \
- --umask=002
+ --umask=002 /etc/svnwcsub.conf
run_rc_command $1
Modified: subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub.debian
URL:
http://svn.apache.org/viewvc/subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub.debian?rev=1297824r1=1297823r2=1297824view=diff
==
--- subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub.debian (original)
+++ subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub.debian Wed Mar
7 01:26:25 2012
@@ -21,7 +21,7 @@ SVNWCSUB_CMD=/opt/svnpubsub/svnwcsub.py
--daemon \
--logfile=/var/bwlog/svnpubsub/svnwcsub.log \
--pidfile=${pidfile} \
- --uid=${svnwcsub_user} --gid=${svnwcsub_group}
+ --uid=${svnwcsub_user} --gid=${svnwcsub_group}
/etc/svnwcsub.conf
RETVAL=0
Modified: subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub.solaris
URL:
http://svn.apache.org/viewvc/subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub.solaris?rev=1297824r1=1297823r2=1297824view=diff
==
--- subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub.solaris
(original)
+++ subversion/trunk/tools/server-side/svnpubsub/rc.d/svnwcsub.solaris Wed Mar
7 01:26:25 2012
@@ -13,7 +13,7 @@ SVNWCSUB_CMD=/usr/local/svnpubsub/svnwc
--logfile=/x1/log/svnwcsub.log \
--pidfile=${pidfile} \
--umask=002 \
- --uid=${svnwcsub_user} --gid=${svnwcsub_group}
+ --uid=${svnwcsub_user} --gid=${svnwcsub_group}
/etc/svnwcsub.conf
RETVAL=0
svn commit: r1297842 - /subversion/trunk/tools/server-side/svnpubsub/svnwcsub.tac
Author: gstein Date: Wed Mar 7 02:46:34 2012 New Revision: 1297842 URL: http://svn.apache.org/viewvc?rev=1297842view=rev Log: We no longer use twistd, preferring direct invocation/daemonization. Thus, we can lose the twisted application setup script thingymajig. * tools/server-side/svnpubsub/svnwcsub.tac: removed Removed: subversion/trunk/tools/server-side/svnpubsub/svnwcsub.tac
svn commit: r1297843 - /subversion/trunk/tools/server-side/svnpubsub/svnpubsub/client.py
Author: gstein Date: Wed Mar 7 02:53:15 2012 New Revision: 1297843 URL: http://svn.apache.org/viewvc?rev=1297843view=rev Log: The revision should be an integer (rather than unicode). * tools/server-side/svnpubsub/client.py: (XMLStreamHandler.startElement): convert the revnum to an int Modified: subversion/trunk/tools/server-side/svnpubsub/svnpubsub/client.py Modified: subversion/trunk/tools/server-side/svnpubsub/svnpubsub/client.py URL: http://svn.apache.org/viewvc/subversion/trunk/tools/server-side/svnpubsub/svnpubsub/client.py?rev=1297843r1=1297842r2=1297843view=diff == --- subversion/trunk/tools/server-side/svnpubsub/svnpubsub/client.py (original) +++ subversion/trunk/tools/server-side/svnpubsub/svnpubsub/client.py Wed Mar 7 02:53:15 2012 @@ -117,7 +117,7 @@ class XMLStreamHandler(xml.sax.handler.C def startElement(self, name, attrs): if name == 'commit': - self.rev = Revision(attrs['repository'], attrs['revision']) + self.rev = Revision(attrs['repository'], int(attrs['revision'])) # No other elements to worry about. def characters(self, data):
svn commit: r1297845 - /subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py
Author: gstein
Date: Wed Mar 7 02:57:06 2012
New Revision: 1297845
URL: http://svn.apache.org/viewvc?rev=1297845view=rev
Log:
Switch the client over to the new client code based on asyncore.
This is a minimal-impact change to make it more obvious what has
changed. A future revision will clear out all of the Twisted code, and
then tighten things up.
* tools/server-side/svnpubsub/svnwcsub.py:
(BigDoEverythingClasss._restartStream): disable twisted-based
connection setup.
(run_client): new function to construct a MultiClient and then start
it up, looking for updates.
(_commit): wrapper around the BDEC's commit callback. we need to
remap the stream and tweak the Revision object
(_event): basic callback to log various events from the MultiClient
(Daemon.run): use run_client() rather than the Twisted reactor
Modified:
subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py
Modified: subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py
URL:
http://svn.apache.org/viewvc/subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py?rev=1297845r1=1297844r2=1297845view=diff
==
--- subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py (original)
+++ subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py Wed Mar 7
02:57:06 2012
@@ -39,7 +39,9 @@ import time
import logging.handlers
import Queue
import optparse
+import functools
+### these will go away shortly
from twisted.internet import reactor
from twisted.application import internet
from twisted.web.client import HTTPClientFactory, HTTPPageDownloader
@@ -47,6 +49,7 @@ from urlparse import urlparse
from xml.sax import handler, make_parser
import daemonize
+import svnpubsub.client
# check_output() is only available in Python 2.7. Allow us to run with
# earlier versions
@@ -229,6 +232,7 @@ class BigDoEverythingClasss(object):
WorkingCopy(self, path, url)
def _restartStream(self, url):
+return
(self.streams[url], self.transports[url]) = connectTo(url, self)
def wc_ready(self, wc):
@@ -264,6 +268,33 @@ class BigDoEverythingClasss(object):
self.worker.add_work(OP_UPDATE, wc)
+### hack in support for the MultiClient.
+def run_client(bdec):
+hostports = [ ]
+for url in bdec.urls:
+parsed = urlparse(url)
+hostports.append((parsed.hostname, parsed.port))
+
+mc = svnpubsub.client.MultiClient(hostports,
+ functools.partial(_commit, bdec),
+ _event)
+mc.run_forever()
+
+def _commit(bdec, host, port, rev):
+class _stream(object):
+url = '%s:%s' % (host, port)
+rev.repos = rev.uuid ### quick little hack
+bdec.commit(_stream, rev)
+
+def _event(host, port, event_name):
+if event_name == 'error':
+logging.exception('from %s:%s', host, port)
+elif event_name == 'ping':
+logging.debug('ping from %s:%s', host, port)
+else:
+logging.info('%s from %s:%s', event_name, host, port)
+
+
# Start logging warnings if the work backlog reaches this many items
BACKLOG_TOO_HIGH = 20
OP_UPDATE = 'update'
@@ -406,7 +437,8 @@ class Daemon(daemonize.Daemon):
def run(self):
# Start the BDEC (on the main thread), then start up twisted
self.bdec.start()
-reactor.run()
+#reactor.run()
+run_client(self.bdec)
def prepare_logging(logfile):
svn commit: r1297846 - /subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py
Author: gstein
Date: Wed Mar 7 03:04:36 2012
New Revision: 1297846
URL: http://svn.apache.org/viewvc?rev=1297846view=rev
Log:
Strip now-unused code.
* tools/server-side/svnpubsub/svnwcsub.py:
(class HTTPStream, class Revision, class StreamHandler, class
XMLHTTPStream): removed entire classes
(connectTo): removed
(BigDoEverythingClasss.__init__): remove SERVICE param. remove
unused instance variable initialization.
(BigDoEverythingClasss._restartStream): removed
(run_client): update reference to urlparse()
Modified:
subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py
Modified: subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py
URL:
http://svn.apache.org/viewvc/subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py?rev=1297846r1=1297845r2=1297846view=diff
==
--- subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py (original)
+++ subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py Wed Mar 7
03:04:36 2012
@@ -40,13 +40,7 @@ import logging.handlers
import Queue
import optparse
import functools
-
-### these will go away shortly
-from twisted.internet import reactor
-from twisted.application import internet
-from twisted.web.client import HTTPClientFactory, HTTPPageDownloader
-from urlparse import urlparse
-from xml.sax import handler, make_parser
+import urlparse
import daemonize
import svnpubsub.client
@@ -130,100 +124,15 @@ class WorkingCopy(object):
return str(relpath), uuid
-class HTTPStream(HTTPClientFactory):
-protocol = HTTPPageDownloader
-
-def __init__(self, url):
-self.url = url
-HTTPClientFactory.__init__(self, url, method=GET,
agent=SvnWcSub/0.1.0)
-
-def pageStart(self, partial):
-pass
-
-def pagePart(self, data):
-pass
-
-def pageEnd(self):
-pass
-
-class Revision:
-def __init__(self, repos, rev):
-self.repos = repos
-self.rev = rev
-self.dirs_changed = []
-
-class StreamHandler(handler.ContentHandler):
-def __init__(self, stream, bdec):
-handler.ContentHandler.__init__(self)
-self.stream = stream
-self.bdec = bdec
-self.rev = None
-self.text_value = None
-
-def startElement(self, name, attrs):
-
-commit revision=7
-dirs_changedpath//path/dirs_changed
- /commit
-
-if name == commit:
-self.rev = Revision(attrs['repository'], int(attrs['revision']))
-
-def characters(self, data):
-if self.text_value is not None:
-self.text_value = self.text_value + data
-else:
-self.text_value = data
-
-def endElement(self, name):
-if name == commit:
-self.bdec.commit(self.stream, self.rev)
-self.rev = None
-if name == path and self.text_value is not None and self.rev is not
None:
-self.rev.dirs_changed.append(self.text_value.strip())
-self.text_value = None
-
-
-class XMLHTTPStream(HTTPStream):
-def __init__(self, url, bdec):
-HTTPStream.__init__(self, url)
-self.parser = make_parser(['xml.sax.expatreader'])
-self.handler = StreamHandler(self, bdec)
-self.parser.setContentHandler(self.handler)
-
-def pagePart(self, data):
-self.parser.feed(data)
-
-
-def connectTo(url, bdec):
-u = urlparse(url)
-port = u.port
-if not port:
-port = 80
-s = XMLHTTPStream(url, bdec)
-if bdec.service:
- conn = internet.TCPClient(u.hostname, u.port, s)
- conn.setServiceParent(bdec.service)
-else:
- conn = reactor.connectTCP(u.hostname, u.port, s)
-return [s, conn]
-
-
-CHECKBEAT_TIME = 60
PRODUCTION_RE_FILTER = re.compile(/websites/production/[^/]+/)
class BigDoEverythingClasss(object):
-def __init__(self, config, service = None):
+def __init__(self, config):
self.urls = [s.strip() for s in config.get_value('streams').split()]
self.svnbin = config.get_value('svnbin')
self.env = config.get_env()
self.tracking = config.get_track()
self.worker = BackgroundWorker(self.svnbin, self.env)
-self.service = service
-self.transports = {}
-self.streams = {}
-for u in self.urls:
- self._restartStream(u)
self.watch = []
def start(self):
@@ -231,10 +140,6 @@ class BigDoEverythingClasss(object):
# working copies auto-register with the BDEC when they are ready.
WorkingCopy(self, path, url)
-def _restartStream(self, url):
-return
-(self.streams[url], self.transports[url]) = connectTo(url, self)
-
def wc_ready(self, wc):
# called when a working copy object has its basic info/url,
# Add it to our watchers, and trigger an svn update.
@@ -272,7 +177,7 @@ class BigDoEverythingClasss(object):
def
svn commit: r1297847 - /subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py
Author: joes
Date: Wed Mar 7 03:18:01 2012
New Revision: 1297847
URL: http://svn.apache.org/viewvc?rev=1297847view=rev
Log:
set gid first before dropping uid to non-root user
Modified:
subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py
Modified: subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py
URL:
http://svn.apache.org/viewvc/subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py?rev=1297847r1=1297846r2=1297847view=diff
==
--- subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py (original)
+++ subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py Wed Mar 7
03:18:01 2012
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python2.7
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
@@ -380,15 +380,6 @@ def handle_options(options):
open(options.pidfile, 'w').write('%s\n' % pid)
logging.info('pid %d written to %s', pid, options.pidfile)
-if options.uid:
-try:
-uid = int(options.uid)
-except ValueError:
-import pwd
-uid = pwd.getpwnam(options.uid)[2]
-logging.info('setting uid %d', uid)
-os.setuid(uid)
-
if options.gid:
try:
gid = int(options.gid)
@@ -398,6 +389,15 @@ def handle_options(options):
logging.info('setting gid %d', gid)
os.setgid(gid)
+if options.uid:
+try:
+uid = int(options.uid)
+except ValueError:
+import pwd
+uid = pwd.getpwnam(options.uid)[2]
+logging.info('setting uid %d', uid)
+os.setuid(uid)
+
if options.umask:
umask = int(options.umask, 8)
os.umask(umask)
svn commit: r1297848 - /subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py
Author: joes Date: Wed Mar 7 03:18:57 2012 New Revision: 1297848 URL: http://svn.apache.org/viewvc?rev=1297848view=rev Log: oops local mod shouldnt have been in last commit Modified: subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py Modified: subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py URL: http://svn.apache.org/viewvc/subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py?rev=1297848r1=1297847r2=1297848view=diff == --- subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py (original) +++ subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py Wed Mar 7 03:18:57 2012 @@ -1,4 +1,4 @@ -#!/usr/bin/env python2.7 +#!/usr/bin/env python # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with
svn propchange: r1297848 - svn:log
Author: gstein Revision: 1297848 Modified property: svn:log Modified: svn:log at Wed Mar 7 03:30:25 2012 -- --- svn:log (original) +++ svn:log Wed Mar 7 03:30:25 2012 @@ -1 +1,2 @@ -oops local mod shouldnt have been in last commit +* tools/server-side/svnpubsub/svnwsub.py: + (...): revert accidential change to the #! line
svn commit: r1297853 - /subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py
Author: joes Date: Wed Mar 7 03:38:12 2012 New Revision: 1297853 URL: http://svn.apache.org/viewvc?rev=1297853view=rev Log: * svnwcsub.py - drop --config-dir options as we now properly pass env Modified: subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py Modified: subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py URL: http://svn.apache.org/viewvc/subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py?rev=1297853r1=1297852r2=1297853view=diff == --- subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py (original) +++ subversion/trunk/tools/server-side/svnpubsub/svnwcsub.py Wed Mar 7 03:38:12 2012 @@ -110,8 +110,6 @@ class WorkingCopy(object): logging.info(autopopulate %s from %s % (self.path, self.url)) subprocess.check_call([svnbin, 'co', '-q', '--non-interactive', - '--config-dir', - '/home/svnwc/.subversion', '--', self.url, self.path], env=env) @@ -261,7 +259,6 @@ class BackgroundWorker(threading.Thread) ### still specific to the ASF setup. args = [self.svnbin, 'update', '--quiet', -'--config-dir', '/home/svnwc/.subversion', '--non-interactive', '--trust-server-cert', '--ignore-externals', @@ -278,7 +275,6 @@ class BackgroundWorker(threading.Thread) ### we need to move some of these args into the config. these are ### still specific to the ASF setup. args = [self.svnbin, 'cleanup', -'--config-dir', '/home/svnwc/.subversion', '--non-interactive', '--trust-server-cert', wc.path]
