Re: SVN working copy split DB vs. working area

[Doug Robinson]

Philip:

Thank you for responding!

On Wed, Apr 11, 2018 at 4:08 PM, Philip Martin 
wrote:

> Doug Robinson  writes:
>
> > The general setup is that they want to have a working copy on a
> > Distributed File System (DFS, e.g. Lustre) and the DFS either is
> > very slow (when mounted with sufficient support for SQLite) or just
> > does not work due to a lack of support for SQLite - likely locking).
>
> If SQLite locking works at all then the client-side config option
>
>--config-option config:working-copy:exclusive-locking=true
>
> may help.  It makes SQLite take less granular locks which improves
> performance, particularly on network filesystems, by reducing the SQLite
> overhead.  The performance gain can be substantial.
>
>   https://sqlite.org/pragma.html#pragma_locking_mode
>
> Exclusive locking also reduces the concurrency of Subversion operations:
> generally only one Subversion operation can run on a working copy at any
> one time, but lots of users never notice this.
>

Neat!  I'll mention this to the folks where DFS was just too slow.
Won't help the folks where their DFS just didn't work at all.


> > Has this subject come up before?  Is there a way to link a ".svn"
> > area to the rest of the working copy?  In other words, keeping the
> > housekeeping area separate/split from the rest of the working copy?
>
> It's a bit tricky.  The .svn area also includes .svn/tmp/ where most
> files are created before being moved to their final destination.  Some
> of the moves are to .svn/pristine/, within the .svn/ area, while other
> moves are into the working copy itself.  Moves don't generally cross
> filesystem boundaries.
>
> If we start splitting a working copy across multiple filesystems then we
> may need per-filesystem temporary directories:
>
>  /local/wc/.svn/
>  /local/wc/.svn/pristine/
>  /local/wc/.svn/tmp-local/
>  /local/wc/.svn/tmp-remote -> /remote/wc/.svn-tmp/
>
>  /remote/wc/
>  /remote/wc/.svn -> /local/wc/.svn/
>  /remote/wc/.svn-tmp-remote/
>
> and then we would need to make every use of the temporary directory use
> the correct one based on the intended destination of the file.
>

Assuming that atomicity on rename() is required, good point.

I hope all is going well with you.

Cheers.

Doug
-- 
*DOUGLAS B ROBINSON* SENIOR PRODUCT MANAGER

T +1 925 396 1125
*E* doug.robin...@wandisco.com

-- 


* *

**The LIVE DATA Company
*Find out more 
*wandisco.com *



 

*


THIS MESSAGE 
AND ANY ATTACHMENTS ARE CONFIDENTIAL, PROPRIETARY AND MAY BE PRIVILEGED

If 
this message was misdirected, WANdisco, Inc. and its subsidiaries, 
("WANdisco") does not waive any confidentiality or privilege. If you are 
not the intended recipient, please notify us immediately and destroy the 
message without disclosing its contents to anyone. Any distribution, use or 
copying of this email or the information it contains by other than an 
intended recipient is unauthorized. The views and opinions expressed in 
this email message are the author's own and may not reflect the views and 
opinions of WANdisco, unless the author is authorized by WANdisco to 
express such views or opinions on its behalf. All email sent to or from 
this address is subject to electronic storage and review by WANdisco. 
Although WANdisco operates anti-virus programs, it does not accept 
responsibility for any damage whatsoever caused by viruses being passed.

Re: SVN working copy split DB vs. working area

[Doug Robinson]

Brane:

Good to hear from you!

On Wed, Apr 11, 2018 at 2:57 PM, Branko Čibej  wrote:

> On 11.04.2018 20:27, Doug Robinson wrote:
> > I've now seen a request for this twice in as many days.  Once from
> > a customer and once from someone posting to a Subversion forum
> > here:
> >
> > https://www.svnforum.org/forum/opensource-subversion-
> forums/apache-subversion-1-8-support/80218-subversion-
> checkout-on-distrubuted-filesytem
> >
> > The general setup is that they want to have a working copy on a
> > Distributed File System (DFS, e.g. Lustre) and the DFS either is
> > very slow (when mounted with sufficient support for SQLite) or just
> > does not work due to a lack of support for SQLite - likely locking).
> >
> > One set of folks want to accelerate by doing parallel checkouts the
> > way they could do using SVN 1.6 (prior to the consolidated ".svn"
> > tree).  But SQLite locking will prevent that (unless I'm missing
> > something).
> >
> > Another set of folks is ok for having the ".svn" area on a local
> > file system (e.g. ext4) but want the rest of the working copy out
> > there on the DFS.
> >
> > NOTE: both sets of folks are experienced SVN users.  Both know and
> > have rejected "exporting" - they want a working copy.
> >
> > Has this subject come up before?  Is there a way to link a ".svn"
> > area to the rest of the working copy?  In other words, keeping the
> > housekeeping area separate/split from the rest of the working copy?
>
> One of the original design goals for the SQLite-based working copy was
> that the database could be hoisted out of the working copy proper and
> that multiple working copies could share the same database. However,
> there has been no real work done toward that goal.
>
> It's possible to move the .svn/wc.db database elsewhere and replace it
> with a symbolic link to the original database. However, I'm not too sure
> how that will help, since the SQLite journal files will still be created
> in .svn/. Also, this would have to be done manually for every external
> directory (which currently has its own, separate .svn/ directory).
>

Simple enough experiment for them to try - I'll suggest it to them.

Especially when I combine it with an observation made by Julian that a
WC with the .svn and zero other files can be tar'd up and copied and
then re-populated.  If the payback is large enough then perhaps...

Cheers!

Doug
-- 
*DOUGLAS B ROBINSON* SENIOR PRODUCT MANAGER

T +1 925 396 1125
*E* doug.robin...@wandisco.com

-- 


* *

**The LIVE DATA Company
*Find out more 
*wandisco.com *



 

*


THIS MESSAGE 
AND ANY ATTACHMENTS ARE CONFIDENTIAL, PROPRIETARY AND MAY BE PRIVILEGED

If 
this message was misdirected, WANdisco, Inc. and its subsidiaries, 
("WANdisco") does not waive any confidentiality or privilege. If you are 
not the intended recipient, please notify us immediately and destroy the 
message without disclosing its contents to anyone. Any distribution, use or 
copying of this email or the information it contains by other than an 
intended recipient is unauthorized. The views and opinions expressed in 
this email message are the author's own and may not reflect the views and 
opinions of WANdisco, unless the author is authorized by WANdisco to 
express such views or opinions on its behalf. All email sent to or from 
this address is subject to electronic storage and review by WANdisco. 
Although WANdisco operates anti-virus programs, it does not accept 
responsibility for any damage whatsoever caused by viruses being passed.

Wiki migration - remaining issues before final migration

[Johan Corveleyn]

I've continued tweaking the Wiki conversion. See the latest attempt at:

https://cwiki.apache.org/confluence/display/SVNTEST/

Compare with https://wiki.apache.org/subversion if you want to see the
old MoinMoin wiki.

Below are the issues I still noticed. I'll try to fix these the coming
weekend, so I can do the really really final migration early next
week. If anyone finds other things, feel free to add them to the list.
Some I will try to fix in the conversion (so the conversion itself
becomes more robust too, as it will be reused for other ASF projects).
Some I will leave for manual fixing in the new wiki later.


The home page (of which all the other pages are children) is
FrontPage2 now (with no useful content), but I intend to put FrontPage
there.

MeasuringRepositoryPerformance
- Link created where there was no link: RepoPerf (probably don't need
to detect WikiNames inside a link already, i.e. inside []).

KeepingReintegratedBranchAlive
- Link created where there was no link: A_COPY (probably "_" can't be part of a
WikiName)

InRepoAuthz
- Link created where there was no link: AuthzSVNAccessFile etc
(probably no consecutive uppercase letters allowed in WikiNames)

InheritedProperties
- link TortoiseSVN
- too long bold in "Differentiating 'Inheritable' Vs. 'Normal'
Properties" (regex inside *XXX* too greedy)

Meetings
- link
- Separately: migrate old page of old tigris wiki that's linked from
here? Someday?
http://subversion.tigris.org/wiki/Subconf2009Hackathon

MergeDev/DataModel
- Weird indentation numbered list beginning of page

MergeTheoryTopics
- links: PlasticSCM, http://wiki.monotone.ca/MarkMerge/

MergeTrackingIdeas
- at the end: A->B->C, A->B->A ... letters are gone (only ->->A etc). Weird ...

MoveDev (misschien manueel fixen)
- link to MoveDev/How to Add Moves to Svn
- link to [MoveDev/MovesInFSFS]

MoveDev/Ev2MovesDesign
- Wrong indentation "Notation: " because of * Notation:* (boldness).

MoveDev/MoveTrackingUpdateEditor
- Wrong indentation in Many-to-Many Move (because of "###"). Escape #
in beginning of line or everywhere? Interesting

MtimePreservation
- italics at the beginning and the end failed
- italics failed inside the list under Rules. Not sure why.

SupportedMergeScenarios
- Wrong indentation in Merging Scenarios for Subversion 1.5 through 1.7
(###) and in  Multiple Branches

Svn18Changes
- indentation by '.' -> shouldn't use a bullet (can use {indent}) ??

SVNVideoProject
- incorrect indentation of numbered list + list met 1, b, c, d, e, f

Walkthrough+directory+TODO
- link http://moinmo.in/FeatureRequests/Templates

-- 
Johan

[ANNOUNCE] Apache Subversion 1.10.0 released

[Julian Foad]

I'm happy to announce the release of Apache Subversion 1.10.0.
Please choose the mirror closest to you by visiting:

https://subversion.apache.org/download.cgi#recommended-release

This is a stable feature release of the Apache Subversion open source
version control system.

The SHA1 checksums are:

345bc84edc5fe72f70997d247230fd8d3a18d83b subversion-1.10.0.tar.bz2
383e69c077a985bf5a091494d45655714180d5cb subversion-1.10.0.tar.gz
9facdb5a5652b5d90fe9afaebf97ed6b94cf657c subversion-1.10.0.zip

SHA-512 checksums are available at:

https://www.apache.org/dist/subversion/subversion-1.10.0.tar.bz2.sha512
https://www.apache.org/dist/subversion/subversion-1.10.0.tar.gz.sha512
https://www.apache.org/dist/subversion/subversion-1.10.0.zip.sha512

PGP Signatures are available at:

https://www.apache.org/dist/subversion/subversion-1.10.0.tar.bz2.asc
https://www.apache.org/dist/subversion/subversion-1.10.0.tar.gz.asc
https://www.apache.org/dist/subversion/subversion-1.10.0.zip.asc

For this release, the following people have provided PGP signatures:

   Julian Foad [4096R/1FB064B84EECC493] with fingerprint:
6011 63CF 9D49 9FD7 18CF  582D 1FB0 64B8 4EEC C493
   Philip Martin [2048R/76D788E1ED1A599C] with fingerprint:
A844 790F B574 3606 EE95  9207 76D7 88E1 ED1A 599C
   Stefan Sperling [2048R/4F7DBAA99A59B973] with fingerprint:
8BC4 DAE0 C5A4 D65F 4044  0107 4F7D BAA9 9A59 B973
   Stefan Hett (CODE SIGNING KEY) [4096R/376A3CFD110B1C95] with fingerprint:
7B8C A7F6 451A D89C 8ADC  077B 376A 3CFD 110B 1C95
   Branko Čibej [4096R/1BCA6586A347943F] with fingerprint:
BA3C 15B1 337C F0FB 222B  D41A 1BCA 6586 A347 943F
   Johan Corveleyn [4096R/B59CE6D6010C8AAD] with fingerprint:
8AA2 C10E EAAD 44F9 6972  7AEA B59C E6D6 010C 8AAD

Release notes for the 1.10.x release series may be found at:

https://subversion.apache.org/docs/release-notes/1.10.html

You can find the list of changes between 1.10.0 and earlier versions at:

https://svn.apache.org/repos/asf/subversion/tags/1.10.0/CHANGES

Questions, comments, and bug reports to us...@subversion.apache.org.

Thanks,
- The Subversion Team

--
To unsubscribe, please see:

https://subversion.apache.org/mailing-lists.html#unsubscribing