Re: future of our "experimental" features

[Paul Hammant]

My bad. I misunderstood 'the "v1" implementation fails to shelve many kinds
of WC modifications other than plain text changes' statement Julian made.

Re: future of our "experimental" features

[Paul Hammant]

I think y'all should promote something to mainstream, even if the
performance isn't what you want it to be (for this feature). Better slow
IMO, than "does the wrong thing".  If y'all promote neither it'll just
decay away from trunk and then risk never going live. As it doesn't affect
any existing part of Subversion its case for merging into trunk is stronger.

I cobbled together a "VCS Nirvana" blog entry some days ago -
https://paulhammant.com/2020/01/19/vcs-nirvana.

-ph

On Fri, Jan 24, 2020 at 12:18 PM Julian Foad  wrote:

> Stefan Sperling wrote:
> > There are a few features in 1.13 still declared "experimental".
> >
> > Experimental subcommands:
> > x-shelf-diff
> > x-shelf-drop
> > x-shelf-list (x-shelves)
> > x-shelf-list-by-paths
> > x-shelf-log
> > x-shelf-save
> > x-shelve
> > x-unshelve
> > x-wc-copy-mods
> >
> > It sounds like Julian won't have time to keep working on them. So unless
> > someone else picks them up, will they stay in experimental status
> forever?
> >
> > Did the concept of "experimental feature" have the desired results?
> > As far as I remember the goal was to gather feedback from users before
> > finalizing on-disk formats and APIs. Did this work out as planned?
> >
> > Is there any reason why we should not simply move these features to an
> > officially supported status on trunk now, and in the future 1.14?
> > Alternatively, should these features be excluded from 1.14 because they
> > aren't finished? Perhaps even be removed from trunk?
> >
> > I had good results with incrementally developing the conflict resolver by
> > gathering feedback from users and fixing issues and adding improvements
> > in future releases. So I don't really see a need to be extra careful by
> > labeling features as "experimental" and I would be in favour of declaring
> > these commands as officially supported, unless anyone can present a
> reason
> > not to do so. (Disclaimer: I have not yet used these commands and have
> not
> > reviewed their design and implementation myself.)
>
> Thanks for bringing this up.
>
> Shelving:
>
> The current incarnation of shelving ("v3") is feature-wise better than
> the previous incarnations but performance-wise unusably slow on medium
> to large WCs, because it uses a crude, complete "svn checkout" to create
> a shelf storage base state before (reasonably efficiently) shelving the
> changes into it.
>
> Frankly, in this state, probably the older implementation (shelving "v1"
> from svn 1.10) is more useful in practice to more users.  Based on
> saving patch files created by "svn diff" and "svn patch", the "v1"
> implementation fails to shelve many kinds of WC modifications other than
> plain text changes, but its performance characteristics are in line with
> expectations, proportional to size of changes.
>
> The current implementation does prove and exercise the new APIs I put in
> place to neatly copy changes into and out of a WC, so it has a
> development value.
>
> I am not sure what to do about this.  Abandon, revert to "v1", keep as
> is, put both versions in?
>
>
> svn x-wc-copy-mods:
>
> The copy-mods feature is a reasonably good implementation of a
> feature-complete upgrade for "svn diff in dir X | apply patch in dir Y".
>   The main reason why I left it "experimental" is because it doesn't fit
> neatly into the existing command-line UI functions and command names,
> and other corresponding.
>
> Even though it's a niche feature, I suggest we could promote it to a
> non-experimental status, on the basis that the svn command-line client
> has a valid secondary role to play in exposing a basic UI onto all the
> available client API functionalities, as well as its role in providing a
> UI tailored to users' most common tasks.
>
>
> svn info --x-viewspec=
>
> This is not ready to be promoted to non-experimental.  This command
> itself is not too bad; what is missing is a proper efficient
> implementation of "apply a viewspec".  I consider that forces us to keep
> this "report the current viewspec" command as "experimental".
>
> - Julian
>

Re: OpenBSD buildbot

[Paul Hammant]

Just change it to retries=5 on some fine-grained basis and stop worrying
about it as long as that fixes it?

Re: Svn server images / appliances / packages

[Paul Hammant]

Here's one for AWS -
https://aws.amazon.com/marketplace/pp/Bitnami-Subversion-Certified-by-Bitnami/B00NN8NOAE
- but it does't meet your at home requirement.

I've run several Svn's for various at home projects. Always into Ubuntu or
Raspbian - following the setup scripts. Ten years back I put one one an
AppleTV 1.0 (after backdooring it). None particularly enterprise grade.

Re: GIT/Perforce like .svnignore

[Paul Hammant]

Of course.

Re: GIT/Perforce like .svnignore

[Paul Hammant]

All of which can be played out in the issues and the PRs of the most viable
GitHub repo for this idea.

Re: GIT/Perforce like .svnignore

[Paul Hammant]

> ... "pointing at GitHub not going to solve anything" ...

I was pointing at a GitHub repo in order for the OP to use that, and
perhaps contribute to it there. It can easily be perfected there in (say)
Python, and in a couple of years time re-implemened in C in for Svn on
Apache's canonical Svn 'upstream'.

I get that the implementation cost is (say) 4x higher in C with tests, and
think prototyping things in higher level languages is a great idea.

So I'd say it is solving the OPs problem imperfectly.

Re: Re: GIT/Perforce like .svnignore

[Paul Hammant]

This is a fantastic feature. At least, it is very popular in Git and
Mercurial.

In lieu of the Svn dev team agreeing, there's at least two implementations
on GitHub - https://github.com/search?q=subversion+ignore

On Fri, Nov 29, 2019 at 11:12 AM Krzysztof Siewiorek 
wrote:

> Hi,
> Thank you all for the feedback. You are right. I was not too precise on
> what is the problem, and what is used already.
> We are using the latest version of Subversion (1.13.0), and the latest
> TortoiseSVN client (1.13.1), so we take advantage of svn:global-ignores
> properties - and that feature helps a lot already :)
> The problem we have, is that we handle big repositories with the games we
> develop, including modified UnrealEngine dedicated for each one of the
> games. There are some cases that apply using path based rules ( not only
> file based ones, like *.obj *.pdb and so on ). We want them to be applied
> to all of our repositories, so going down the tree, and adding the
> properties in certain directories on each one of them is time consuming and
> hard to maintain.
> I'll give you some example of our old ignore rules we had, and that are
> hard to manage in svn:
>
> Engine\Engine\Plugins\**\XboxOne\*.lib
> Engine\Engine\Plugins\**\PS4\*.a
> Game\Plugins\**\XBoxOne\*.lib
> Game\Plugins\**\PS4\*.a
> Engine\Engine\Source\Programs\**\obj\*
>
> There are similar, but those should give the idea of what is the problem.
> Ideally, I would like to set exactly those rules as a svn:global-ignores in
> the main directory of the repository, so it's easy to change them any time
> if needed.
> Basically the idea of having all the rules in one place is a key of
> successful and easy going maintaince - and this is the goal for me here.
>
> If that was possible, putting the rules in a file and updating it on hook
> with "svn propset svn:global-ignores -F .svnignore ." is easy already. But
> that would be actually optional and not needed.
>
> I hope that clears out all of the questions :)
>
> Thanks!
> Krzysztof Siewiorek-Pieniążek
>
> Dnia 28 listopada 2019 21:24 Nathan Hartman 
> napisał(a):
>
> On Thu, Nov 28, 2019 at 2:20 AM Krzysztof Siewiorek < 
> krzych...@tlen.pl> wrote:
>
> Hi!
> We've started to move from Perforce to SVN in my company for some reasons.
> We moved quite a few big projects that we have or we had been working in
> the past. Working with perforce for years gave us quite a big and precise
> ignore rules list. The problem is that SVN's approach to that does not
> quite scale up and also makes managing ignored files a pain - especially
> when working on many projects in same time.
> I was trying to dig for some piece of information, why actually SVN
> doesn't have implemented something simillar to GIT's or Perforce's ignore
> file that contains extended rules including full directories in the rules,
> !mark to not apply the rules for some files/dir, and so on.
>
>
> Hello,
>
> Since you mentioned .svnignore in the subject line, I'd like to point out
> that Subversion doesn't require you to clutter your version-controlled
> directories with such dotfiles.
>
> Subversion offers versioned properties. These are pieces of metadata that
> can be associated to files and directories, and are version-controlled
> alongside them. Subversion has various built-in properties, whose names
> begin with "svn:". In addition, you can create any other properties you
> wish for your own purposes (e.g., to support custom tooling) so long as you
> don't start their names with "svn:" as that is reserved for the built-in
> properties.
>
> When it comes to ignore rules, there are two kinds of properties:
>
> svn:ignore - ignores files matching a pattern in the same directory.
>
> svn:global-ignores - like svn:ignore, but recursive.
>
> In my company's Subversion repository we have quite a few of these
> properties set up and to date they have covered all of our needs.
>
> Although there is currently no '!' to ignore a rule for a particular file,
> be aware that once a file is added to version control, ignore rules no
> longer apply to it. The ignore patterns apply only to files that Subversion
> is not tracking, for the purpose of not cluttering up the output of 'svn
> status'.
>
> Hopefully my message is helpful for you and not merely a regurgitation of
> things you already know. :-)
>
> We're glad to hear from you. Feel free to write anytime!
>
> Also, as Brane points out this is a volunteer run open source project so
> we're always happy to meet enthusiastic new contributors. If you'd like a
> cool new feature and are willing to invest some effort, anything is
> possible. Let us know if you're interested...
>
> Cheers,
> Nathan
>
>
>

Re: Python3 work [was: The run up to Subversion 1.13.0]

[Paul Hammant]

One more thing re Python3 - there's Svn Homebrew fu for Mac folks -
https://github.com/Homebrew/homebrew-core/blob/master/Formula/subversion.rb
- that work admirably for Python 2.7.x but has no code for Python 3.  I
don't think this brew formula is maintained in the svn dev team. Of course,
pull-requests are processed as you'd expected for a project hosted on
GitHub.

Re: What comes after 1.12?

[Paul Hammant]

My DevOps consulting life (around how to get to Trunk-Based Development
from some often ClearCase inspired branching model), starts with what
release cadence are you at now, and what do you want to get to.  Clients
who are quartly want to get to monthly (and have less unplanned releases
following each). Clients that are monthly want to get to weekly (same
elimination on unplanned). Aaand clients that are weekky eye daily.  Sure
that's dev teams of 20-1000 in one repo, where change would firehose into
the repo if it could,

*I've not encountered an enterprise team that wanted to slow down release
cadence.*

Seems to me Svn's problems are 1) lack of committers, perhaps because 2)
patch contributors don't feel it matches their smoother experience
elsewhere, and because 3) running "the build" from zero is hard, and 4)
tests in the build isn't a super uniform thing.

If you all developed on trunk (patchsets for trunk, or direct to it), and
had an automated CI build setup that tested all permutations in parallel,
and a merging bot that attempted to cherry-pick commits marked as [BUGFIX]
back to all supported release branches without human intervention (and
Apache-TM votes), then you might have streamlined things. Such a bot
shouldn't shit the bed if the cherry-pick fails *

Of course you're stymied by a lack of a integrated patch consumption
technology. Gerrit and Rietveld were made in Mondrian's image, ten years
ago. That said GitHub and pull requests a few months later stole the
spotlight and have not relinquished it since.  Julian's stash/shelve tech
is the start of a standardized Pull-Request system for Svn, but the vision
need to be expanded.

Meanwhile, Mononoke ** is pushing forward to monorepo-nirvana even if
Atlassian today declare that Mercurial support in BitBucket has an end
date.

* oh, the back-applying merge bot will work fine as long as no rename/move
refactorings happen in the Subversion tree. That's because Subversion can't
merge through rename (unlike Git and Mercurial - seamless). Perforce could
do it if you muck around with branch-specs, but nobody does.

** Mononoke (a Rust backend for Mercurial) is hosted on GitHub (Git). The
dev eam feels no shame from that, perhaps because it's a periodical copy
from the same source-tree inside Facebook (and isn't currently buildable).
Git made a Svn compatibility for GitHub's Git repos - I wonder if future
contributors to Mononoke could too. Repo:
https://github.com/facebookexperimental/mononoke

Re: Subversion 2.0

[Paul Hammant]

The "shelve" functionality in Subversion may be grown into a "continuous
review" system in the future. If you can't wait that long Rhodecode and
Assembla both give Subversion a code review capability today and would be
able to migrate your existing repo to their tech/service. Various CollabNet
products may do so too.

- Paul

Monorepos: Facebook's Mononoke and Eden (file system)

[Paul Hammant]

Monorepo tech: https://github.com/facebookexperimental/mononoke (Mercurial
rewritten in Rust)
File system: https://github.com/facebookexperimental/eden

Xoogers (and friends) at Facebook toiling towards Google's in-house
capability - formerly Perforce, since 2012 'Piper'.  This -
https://trunkbaseddevelopment.com/expanding-contracting-monorepos/ - is a a
goal (among others).

I wonder if they have alleviated the push/pull bottleneck and allow
effective direct PUTs to resurces without a clone operation first (as
Subversion, Perforce and PlasticSVM do)

Anyway, the code is being maintained on GitHub (or at least mirrored to
GH), so they're not feeling shame from that. Nor should they, say I.
Subversion's rejuvenation could happen from the same pragmatic move.

- Paul

Re: Subversion 2.0

[Paul Hammant]

> [..] How can we leverage Subversion's centralized structure to give
> something _better_than modern workflows?

I'm the guy behind http://trunkbaseddevelopment.com and am predictably
going to say the workflow Svn needs to ace is trunk-based development.
That includes (since 2008) some mechanism for pull-requests. I think
the Svn team should only concentrate on command line capability for
that, and leave web UIs to the commercial teams - Assembla, RhodeCode,
CollabNet.

I also wrote something about Subversion on my blog a week or so ago -
lesser known strengths and weaknesses -
https://paulhammant.com/2019/06/14/merkle-trees-and-source-control/.
Sure, there's some other attempt at a comparison with
https://svnvsgit.com/, but that's not me.

-ph

Re: Subversion 2.0

[Paul Hammant]

> building those ideas on the Subversion 1.x code base

+1

> Rust or Go

+1. Rust doesn't yet target all the platforms that Subversion already
targets. It will do though, there's something unstoppable about the
Rust community. I commissioned Rust ports of Python pieces on UpWorks
for fixed prices and have always been impressed.  If you choose a
"most depended on, and least depending, FIRST" strategy for flipping
to Rust, you could methodically transition to Rust from C/C++ and keep
shipping.

If anything feels 2.x worthy, it's addressing the chatty nature of the
HTTP wire api. I don't think there'd be any appetite for that though,
as Greg Stein et al participated with the W3C to make HTTP 1.1
(including WebDAV) in order to polish Svn's over-HTTP sid. And given
that the there's zero community interest in revisiting the WebDAV
spec, you'd be talking about a non-standard extension, and I'll be
there's no interest in that here.

-ph

Re: GitHub pull requests

[Paul Hammant]

The gaping hole in Subversion's capability, 11 years on from when
GitHub launched with it, and 13 or so years from Google
operational-izing the functional equivalent for themselves is
patch-review in a way that each is re-creatable in working-copy by
reviewers from the command line.  GitHub made this as a Pull-Requests
from forks that are essentially short lived feature branches.  They
later talked of "GitHub Flow" (not to be confused with GitFlow) as a
branching model.  Perforce couldn't do SLFBs so Google implemented it
as a patch management and review system that was effectively the same.
The UI was called Mondrian (Guido tech-talked about that himself in
2006). Opensource-land received Rietveld and the Gerrit in the image
of Mondrian, and indeed Subversion was supported at the outset.

This needs to be built in to Subversion IMO, and an extension of
Julian's shelve tech. No web-ui, just sve.ee compatibility.

I'm saying this because dogfooding requires minimal comparable features.

-ph

Re: GitHub pull requests

[Paul Hammant]

>   * include the patch (as an attachment?) in the PR mails

https://coderwall.com/p/6aw72a/creating-patch-from-github-pull-request
<- turns out GitHub has a secret URL that aids in that workflow.

Remaining: a pledge for speedy processing of these.  If standards for
code donations are made clear, and CLA obligations streamlined, then
you have a possible way forward. If an item starts in GH-PR though,
does it get entered into Jira too?  I would leave it there, and
complete (prosefully at least) the workflow there - 1) code-review
commentary in GH, 2) conclusion statement with a reminder that the
changes in question will reappear in GitHub later aftr a sync from
upstream Svn@theASF, 3) close of PR.

Re: GitHub pull requests

[Paul Hammant]

One of the major GitHub social contributions to the community is the
ease of forking. I don't mean as part of a "contribution workflow",
but as a tool for getting an upstream team to change their attitude. A
famous case was Node.js and how it ended well (merged again -
http://anandmanisankar.com/posts/nodejs-iojs-why-the-fork/) but was a
smack of sorts for the maintaining company. There are others, too. I'm
talking folks that elicit changes in maintainer policy.

Generally, any viable F/OSS thing is going to end up on GitHub if it
fits (Git has size limits; GitHub have a page on limits -
https://help.github.com/en/articles/what-is-my-disk-quota). It is
going to exist there with Pull-Requests and Issues enabled by some
group.

If a team that "owns" a compelling F/OSS thing on GitHub and attempt
to not have Pull-Requests or GH issues - the thing effectively gets
taken away from them in a fork, that has that turned on. Sure, they
may insist the other "team" doesn't use their trademarks, but they
used and OSI approved license so that's all they can do.

The only defense against that is to NOT have a compelling viable project.

Well that's not absolutely true - https://github.com/mackyle/sqlite is
a fork of something that's canonical on Fossil but
https://github.com/mackyle/sqlite/network shows there's no
maintained-divergence on GH. SqLite is super viable as a project
though - things are happening to it at enough of a pace (2 or 3
commits a day) to make you overcome objections to contribute at the
source in Fossil if you want to get something into upstream. Maybe
after you tried to build it from a git-clone (assuming up to date) and
prove your patch there. However if D Richard Hipp went on a 6mo
vacation, Sqlite would have relocated to GitHub without him.

Forks are going to happen on GitHub regardless with PRs enabled. You
might as well embrace that, and play ball by leaving PRs enabled for
your repo too. Subversion's not in a shameful place if its source is
in Git. Instead it is boosted, IMO.  Time was when your wire protocol
(at least WebDAV) was lingua franca. Now it's Gits: Perforce,
PlasticSCM and Mercurial all speak Git.

- Paul

Re: Docker Svn build environment [was: Subversion's community health]

[Paul Hammant]

People wanting to donate a feature or fix a bug with Subversion are
increasingly going to be happy with a Docker-based quick start. Most
F/OSS teams want tests too, and y'all are very strict there in that
regard, and someone who can write tests for your test-base covering
code for your codebase is most likely going to be super-OK with
Docker.  Where that incentivized groups is left with pause for thought
is on the consumption of contributions and the schedule for seeing
consequential releases.

Now, there's another group who want to image machines and shove in
"latest released" Subversion. Their problem is downstream maintainers
of operating systems for one, and the lack of comprehensive
Svn-dev-team install instructions to overcome maintainer choices
secondarily.  Ubuntu's at 19.04 now, and it's shipping Svn 1.10.  The
Mac's homebrew is at 1.12 presently, but it is quite often months
behind. This group is happy to build from source (if that is reliable)
and would use any on-platform or cross-compilation script to target
the machine/OS they're interested in. They may not care to edit
source. They may even be happy to skip tests - they just want the
binaries in situ.

How much of the first group would not want to wait for releases, and
instead deploy unreleased versions of Subversion?  Meaning they've
donated a patch, have some assurances that it is being accepted as it
meets standards, but don't want to wait any longer before
productionizing it to some level?

Re: Subversion's community health

[Paul Hammant]

Dockerized build templates would be great for a quickstart. I might have
one or two for Svn and if I de-hack them might be shareable. They'd be
biased towards Linux for build/test/standup of course, but would be better
than nothing.

Re: Subversion's community health

[Paul Hammant]

Nobody is going to roll up to contribute to 1.x or 2.x Subversion if there
is no copy-pasteable instructions for building on Mac, Win and Linux. And
there's not - https://svn.apache.org/repos/asf/subversion/trunk/INSTALL -
has no apt, nuget or homebrew advice for people wanting to get into
development of Svn.

My comment from 4 days ago about unconsumed pull requests?

- Paul

Re: SvnMerkleizer: announcement

[Paul Hammant]

> 404 Not Found

Yeah, my bad. Was a 'private' repo in GitHub, now public.

Re: Subversion's community health

[Paul Hammant]

Three PRs out on GitHub for Subversion spanning years -
https://github.com/apache/subversion/pulls

^ "join in" versus "don't join" in is communicated via team welcoming,
curating, and consuming of PRs in this age. Sure, Julian's shelve tech is
the starting point of a PR system for Subversion itself, but that doesn't
mean Svn development can't take advantage of a workflow that millions of
devs have adopted today on GitHub. To say no to that *feels like* an
arbitrary policy.

Towards that in GitHub-land, people dread bug reports remaining unattended.
Issues in GH is turned off, of course. They also are put off by incomplete
READMEs. Ref https://www.makeareadme.com/ (I have made contribs to that)
and a few similar efforts. Lots to say on Svn's README.

SvnMerkleizer: announcement

[Paul Hammant]

URL: https://github.com/paul-hammant/SvnMerkleizer

It's a Java process that runs alongside Apache and MOD_DAV_SVN to make a
full Merkle tree of the repo. Implicitly too, any sub-directory within. It
attempts to maintain different trees for people with different read
permissions/groups. Coming with it, a Docker image that sets it all up
together. Really that last is a demo, but it does place the SvnMerkleizer
within the same directory structure as the Subversion repo itself. See
within the line of code for that
<https://github.com/paul-hammant/SvnMerkleizer-all-in-one/blob/master/vh-davsvn.conf#L8>

While Svn keeps SHA1 for each file resource within the repo, it does not do
that for directories. Hence I made this tech.

Along the way I created a "service virtualization" tech called Servirtium
that is more interesting to enterprise Java teams than anyone else, but
some of the tests for SvnMerkleizer use Servirtium to record 15K PROPFINDS
and OPTIONS operations and allow them to be used in playback.

Pull Requests accepted.

- Paul

Re: SHA1 collisions became cheaper to create.

[Paul Hammant]

Why a merkle tree? One of Subversion's strengths is its linear revision
history. You could use blockchain and get financial strength audit ability.

Blockchain is so over sold. There's a whole class of application where the
bastard child of Git and Subversion would be a perfect non-repudiable and
tamper evident store. Ones where distributed consensus that's vulnerable to
51% attacks isn't actually needed.


>

Re: SHA1 collisions became cheaper to create.

[Paul Hammant]

The Git folks moved to a hardened SHA1 function as an interim measure
on the way to SHA-256 -
https://github.com/git/git/blob/master/Documentation/technical/hash-function-transition.txt

I think you're generally right. While I might think that an auditor
would simply be advised of the root hash for a Merkle tree that for a
branch at a moment in time, or a tag, Subversion doesn't have a a
Merkle tree under the hood. I coded something niche to retrofit
Subversion with that, but it's not core and far from perfect as it
relies on an LRU cache and keeps no history itself. Git's merkle tree
would be perfect if it didn't blow up when repos get too big, and if
allowed clone from nodes other than root (branches and tags are in
respect of root of course).  So, ignore me here.

Re: SHA1 collisions became cheaper to create.

[Paul Hammant]

Yes, Subversion would remain good a keeping versions of honest development work.

Problem I'm trying to solve: In an audit situation where prior commits
were to be analyzed the owner of the repo that was motivated enough
could tell the auditor that black was while in respect of a certain
historical commit. Assuming the auditor had prior SHA1s (in lieu of a
full Merkle tree), for the resources at a historical revision under
audit.

Granted PDF payloads (& other large encoded-stream binaries like
movies) are susceptible for such retroactive fakery, whereas
CR-delimited text files with plausible content are not retroactively
fake-able without that being clear to the eye: "Hey, that's not a C
source file".

Feel free to ignore - this can wait a number of years.

Re: SHA1 collisions became cheaper to create.

[Paul Hammant]

I'm suggesting phasing out SHA1, and during a v1.x to v1.x+1 upgrade
do a migration script for all content to gain (say) BLAKE2 hashes
*instead*, and for that install, client's with incompatible hashing
are rejected.

There are alternates too, where up to a moment in time a repo has
SHA1s, and thence after has some other algo.

SHA1 collisions became cheaper to create.

[Paul Hammant]

Article: 
https://www.zdnet.com/article/sha-1-collision-attacks-are-now-actually-practical-and-a-looming-danger/

Subversion makes a SHA1 hash for each resource held. It is certainly
available as part of the detail for a file/resource, but I don't know
to what extend the PUT logic relies on it.

The ZDNet article talks of better algorithms, but perhaps isn't an
authority on which one is best. I wonder if a pluggable design would
work. Separately a mechanism for the server to reject a Subversion
client as too old may be needed.

- Paul

Re: [ANNOUNCE] Apache Subversion 1.12.0 released

[Paul Hammant]

The last note on 'shelve' is in 1.10 and that was to suggest it is
experimental. Is that still the case in 1.12, or should it graduate to
a different adjective now?

Re: Spack - package manager for Linux+Mac

[Paul Hammant]

In praise of that community, they appear to be "on it" with respect to
problem reports - https://github.com/spack/spack/pull/10335 being an
example that coming from an issue I'd had with their Python as I ran
through the 1.9.7 install of Subversion with it.  That after them
quickly solving an issue where my Mac's file-system barfed with a
case-insensitive naming collision during initial clone-centric setup
for Spack - https://github.com/spack/spack/issues/10318

Spack - package manager for Linux+Mac

[Paul Hammant]

Spack currently has 1.9.7 as the most recent Svn that it can deliver:

https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/subversion/package.py

Someone may be interested in dipping their toe into that world and
updating their script for them.

- Paul

Re: Common header(s) for C and C++ API

[Paul Hammant]

My bad - I assumed the contents of #1850344 were all new, but of
course the tests pre-existed that commit.

Re: Common header(s) for C and C++ API

[Paul Hammant]

http://svn.apache.org/viewvc?view=revision=1850344

^ Any chance of example code being checked in too?  Short
how-to-embed-subversion for C++ and C, I mean. Others elsewhere could
go on to do examples for Rust, Python, Java
(https://github.com/bytedeco/javacpp), etc.

Re: How to join in to dev@

[Paul Hammant]

Hi Nathan,

Some Apache mail lists won't like HTML email or "top-posting". I think the
subversion dev mail list is one of those, but I could be wrong.

You subscribe by emailing dev-subscr...@subversion.apache.org :)

On GitHub Pull-Requests - yes that'd be super useful. Julian has been
developing "shelve" and "checkpoint" which would doubtless be the basis for
light-weight pull-requests.  At least "patch sets" was in Google some 12-14
years ago (for their Perforce configuration of their monorepo -
https://trunkbaseddevelopment.com/game-changers/#mondrian-2006)

- Paul

Re: Suggestion: linkify revnums and issues in the CHANGES file

[Paul Hammant]

Markdown seems ubiquitous nowadays.  In its raw form it is human readable,
as well as being machine-parsable into HTML / PDF, etc.

Git's README is markdown (https://github.com/git/git/blob/master/README.md)
but the releases notes are still plain text (
https://raw.githubusercontent.com/git/git/master/Documentation/RelNotes/2.19.0.txt).
I realize those are two contradictory datapoints. Ho hum :-(

- Paul

Re: Shelving / Checkpointing thoughts

[Paul Hammant]

[going back in time to August 29th]



> Apache products aren't allowed to depend on GPL'd code for mandatory /
> core features.
>
>
This is academic now as Julian is making great progress with his shelve
tech that's built in to Subversion and idiomatic, but there are MIT
licensed versions of Git:

https://github.com/go-gitea/git/blob/master/LICENSE (MIT)

You could link that in instead of LibGit2 (GPL) for storage and
dissemination of patch sets.

Sure, a Go run-time library is not as bytespace/ram/CPU efficient as C or
Rust, but it's closer by a mile than Java or Python (for example).

- Paul

Re: Subversion clients following HTTP 302 response codes.

[Paul Hammant]

Re https://issues.apache.org/jira/browse/SVN-4738

If I paid for this to be developed (say on UpWork), what criteria would
that work have to meet to be consumed into Subversion's trunk for later
release?  I ask because I'd not want to pay for its development then have
it rejected for a long list of reasons. I'd rather bake those into the
acceptance criteria.  As part of the paid development work I'd get
copyright assigned to me by the contractor, and being an Apache member with
a CLA on file from 2002 or so, I'd then grant copyright to Apache for the
work.

Regards,

- Paul

Re: Subversion clients following HTTP 302 response codes.

[Paul Hammant]

My bad - I pasted/typed into the env field rather than the desc one.  Fixed.

Re: Subversion clients following HTTP 302 response codes.

[Paul Hammant]

https://issues.apache.org/jira/browse/SVN-4738 filed :)

Re: Subversion clients following HTTP 302 response codes.

[Paul Hammant]

> I'm still wondering what you're aiming for though. An obvious example
> would be offloading resource content storage from the repository server
> to some other storage. But if you want to do that, relying on redirects
> is hardly a good option, since you'd be increasing the number of
> requests and hence the latency. Transparent load balancing on the server
> side is probably a much better solution.
>


Overall system throughput is what I'm aiming at - large payloads for
thousands of users concurrently (and spread worldwide) are a focus for me -
remember the timing info I was positing (sometime inaccurately I grant you)
some months ago. Latency increasing (before a single GET starts streaming)
isn't a worry for me.


>
> Other than that consideration, I don't see any technical barrier to
> supporting redirects within a live session ... the DAV/HTTPv2 experts
> should chime in here.
>

I think DAV's enhancements to HTTP came for v1.1 not 2.0.

- Paul

Re: Subversion clients following HTTP 302 response codes.

[Paul Hammant]

The main conversation - PROPFIND, OPTIONS etc with a canonical centralized
server, *but* select GETs (among many) being redirected to other URLs.

My testing didn't show that that was supported. As a good XPer, I should be
able to conjure up a script to show that but I can't, so I mean manual
testing for now.

Subversion clients following HTTP 302 response codes.

[Paul Hammant]

It would be cool if svn.exe (the client) could follow HTTP return code '302'
during svn-co and svn-up operations.

I'm thinking that this is just for GET* of resources, and that someone
who's managed to *front* their Mod_Dav_Svn with something that can do
redirects for select resources. Say to resources in S3.

(request)

GET /repos/asf/!svn/rvr/1234/path/to/movie.mp4 HTTP/1.1
Host: svn.example.com
User-Agent: SVN/1.9.7 (x86_64-apple-darwin17.3.0) serf/1.3.9
Accept-Encoding: gzip

(response)

302 Found
Location https://foobar.s3.amazonaws.com/1234/path/to/movie.mp4

(subsequent request)

GET 1234/path/to/movie.mp4 HTTP/1.1
Host: foobar.s3.amazonaws.com
User-Agent: SVN/1.9.7 (x86_64-apple-darwin17.3.0) serf/1.3.9
Accept-Encoding: gzip

(Amazon itself does more 302's here, probably)

* You could make a general case that any (or more than just GET) of the
HTTP methods could be redirectable but discussing GET is a narrow case for
the sake of a debate.

Also, you could make a case for 307 responses too -
https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/307

I'd go ahead a raise a Jira feature request, depending on the outcome of
this debate, per dev-team rules.

Thoughts?

- Paul

Re: Potential regression: high server-side memory consumption during import

[Paul Hammant]

> Yes, the problem seems to be that mod_authz_svn has no way of storing
per-connection state at present.

Please excuse the interjection here: My eyes spied "per connection" and I
wondered if something I'm doing elsewhere that is closer (but not exactly)
"per user" might help in terms of thinking.

I've a server-side 'wrapper' around Svn that's calculating authz
permutations for each *user*. That transcends any sessions and connections
for the user.  Two users in the same groups(s) with no other variance in
authz config will use the same cached Svn entries everywhere.  I've speedy
unit tests too for it, but I've a ways to go because of the exclude grammar
of authz's little language.

If the thing works well enough, but eats too much of the
disk-backed-ram-cache  I'm using, I could extend it
to do the same per directory. Because right now it is effectively a 'from
root of repo' determination for a user.

Of course, you don't want the code I've done because it is not compatible
with the current Svn toolchain. I'm thinking about a Rust rewrite, but
that's a year away, even if it was compatible with the way Subversion is
today.

Regards,

- Paul

Re: Building Subversion on Debian stretch with serf support

[Paul Hammant]

> I'll have an updated Debian package uploaded to experimental in the next
few days.

Which will be great, Philip.  I look forward to it :)

And thanks for the `apt-get build-dep subversion` tip off.

Re: apt-get installs for the Svn build on Raspbian 'Stretch' (Nov, 2017)

[Paul Hammant]

Another SD card, another permutation to get the basic build working for
Subversion:

  apt-get install apache2-dev libsqlite3-dev libserf-dev liblz4-dev
libutf8proc-dev sqlite3 subversion libapache2-mod-svn

This is getting quite easy now :)

Someone should make a webapp that translates rpm, apt-get and brew package
names for posterity, though!

-ph

Subversion build ... but just client commands?

[Paul Hammant]

I'm probably never going to deploy a Svn server on the Mac or Windows10(S)
- keep up the good work Satya!

What are the options for ./configure and make to focus on the client
executables only, and skip svnserve and mod_dav_svn?

- Paul

Re: Building Subversion on the Mac

[Paul Hammant]

The homebrew install does a source compile rather than pluck of a binary
and compare to a SHA256 ?

-Paul

Re: Building Subversion on the Mac

[Paul Hammant]

This seems to work for me, based on your advice:

./configure --with-apr=/usr/local/opt/apr
--with-apr-util=/usr/local/opt/apr-util
--with-apxs=/usr/local/opt/apache2/bin/apxs

I'm thinking those are solid Homebrew paths for most people.

Building Subversion on the Mac

[Paul Hammant]

Based on Brane's email from yesterday, I did a build for Subversion on the
Mac, with Homebrew and XCode installed already.

brew install apr apr-util httpd serf zlib berkeley-db swig lz4 utf8proc

# These two were necessary for me, for some reason
# Although handing flags to ./configure was an alternative
brew link --force apr
brew link --force apr-util

./autogen.sh

./configure

make && make install

Exhaustive test suites and language-binding builds are another thing
altogether - this was basic build of binaries.

The list of brew installs were the ones needed by me to get over the line.
There's other that may have been needed for noobs, that I already had
installed.

Versions after Brew's actions:

apr 1.6.3
apr-util 1.6.1_1
httpd 2.4.29_1
serf 0.8.1
zlib 1.2.11
berkeley-db 6.2.32
swig 3.0.12
lz4 1.8.1.2
utf8proc 2.1

- ph

apt-get installs for the Svn build on Raspbian 'Stretch' (Nov, 2017)

[Paul Hammant]

On that Pi3 again - the joy of having many microSD Cards :)

apt-get install apache2-devel
apt-get install libsqlite3-dev
apt-get install libserf-dev
apt-get install liblz4-dev
apt-get install libutf8proc-dev
apt-get install sqlite3

Everything else was installed already. Installation goes to expected dirs.

- Paul

Subversion openSUSE build pre-requisites

[Paul Hammant]

OpenSUSE is interesting to me because it has a 64bit image for the
RspberryPi3 , and there is no
64 bit version of Raspbian, presently.

If anyone is interested, the 1.10 branch (if not more) builds on the Pi3
after these additional package installs:

zypper install libapr-util1-devel
zypper install libtool
zypper install sqlite
zypper install gcc
zypper install sqlite3-devel
zypper install libserf-devel
zypper install liblz4-devel
zypper install apache2-devel
zypper install utf8proc-devel
zypper install make

'make install' is putting libraries in /usr/local/lib64/ and binaries in
/usr/local/bin
It is putting mod_dav_svn.so into /usr/local/lib/

- Paul

Re: Tree conflict 'svn merge' message

[Paul Hammant]

Here's one from the past - https://issues.apache.org/jira/browse/SVN-3495

- A famous airline was the client back then and they were strangling their
older (but accomplished) cgi-bin/C++ stack. We were doing concurrent
development of consecutive releases. Different teams from two consultancies
(ThoughtWorks and Globant) were working on different branches, and about
four major releases were being developed at the same time. Each "following"
release team was supposed to merge the prior release teams weekly changes.
A cascade branching model.  We tried to hunt around for someone to fix that
big for hard cash, but could not.

We hacked our way through it at the time, and a couple of months later
moved every one to Trunk-Based Development (including a love of build
flags, and boot flags). Flags are 'toggles' in Martin Fowler's lexicon.
Branch by Abstraction figured too, as did a lot of Jenkins infrastructure
and a policy of rolling back commits if they broke in any of the pipelines
that Jenkins was running in parallel.  Builds were still batches of commits
though, a we didn't have the infra to run one build per commit. We (I) had
put too much Selenium tests in the pipeline and they really took a while
(even if run parallelized). It wasn't a patch on Google's per-commit fu
though.  Oh, with Trunk-Based Development (and flags and
branch-by-abstraction) we were never going to have a merge conflict again,
of course.

- Paul

Feature suggestion: determination of the subversion commit number for a directory (over DAV without svn clients involved)

[Paul Hammant]

One can use WebDAV's PROPFIND to list the contents of a directory in
Subversion. You won't be told the Subversion revision for any of the
directory results however, but you will for the file items in the result
set.

You have to do an OPTIONS and a PROPFIND call for each directory one at a
time to get the revision numbers for directories - see here:
https://github.com/paul-hammant/SvnMerkleizer/blob/master/src/main/java/com/paulhammant/svnmerkleizer/App.java#L330
(method get getSvnRevision).

Can the XML response of PROPFIND be enhanced to have an extra element that
reports the revision number to the caller? That way one could do a single
PROPFIND (even a depth=infinity one, subject to config) and get back all
the Subversion revision numbers for all the directories with the result set.

Some operations report a revision number that appears a first glance to be
for the directory, but it is actually the revision number of the root of
the repo. While that is useful too, it isn't what I'm after here.

Permission to add a Jira issue to track this, after due discussion here of
course.

- Paul

-- 
Paul Hammant DevOps <https://devops.paulhammant.com> Let me give your
enterprise a step by step plan to get out of the hell of crazy branching
models (ClearCase maybe?) and into the world of high-throughput CD on
DevOps foundations.

Re: Authz suggestion

[Paul Hammant]

Filed - https://issues.apache.org/jira/browse/SVN-4710

Re: Authz suggestion

[Paul Hammant]

Jira feature request needed to capture anything from this thread?  Maybe
not, if plans were already in action anyway...

Re: Authz suggestion

[Paul Hammant]

>
>
> Specifically, by "currently" I mean that this is the state on trunk. :)
> I don't believe anyone is working on adding explicit traversal
> permission on trunk in time for 1.10. It would require some rework of
> the way the authz info is used within the core libraries, it's not just
> a question of teaching the authz parser a new trick.
>

I myself am never for holding up releases in order to add an extra feature,
if everything else about the work to date says 'release it' :)

This stuff is a corner case for me, and only topical as I've a very naive
parser for Authz files in Java. Test driven of course.

- Paul

Re: Authz suggestion

[Paul Hammant]

> Currently it's implied in 'r' and 'rw' modes.

Great news. Specifically by currently you mean in 1.9.7 right?  And that 
further enhancements are Coming in 1.10.

You also said you’ve a plan for further enhancements :)

Authz suggestion

[Paul Hammant]

Consider:

[/]
harry=rw

[dataset:/A]
sally=rw

[dataset:/Z]
sally=rw


If I had directories B through Y, I am pretty sure Sally cannot see them
let along change anything in them. Cool that's what I want.

What I don't have though is the ability for Sally to checkout from root and
recieve A/* and B/* in one operation.  I could grant 'r' for the root for
sally, but I'd have to do this for all of B through Y which would be overly
verbose:

[dataset:/B]
sally=


So I think I'm asking for a feature, but I'm not sure what would be best
for it.

Choice 1:

[/]
harry=rw
sally=dironly


Choice 2:


DAV svn
AuthzSVNParentDirsIfChildrenPermitted


Thoughts?


- Paul

Re: Authz - paths with trailing slash isn't supported - right?

[Paul Hammant]

>
>
> > Should I raise this in https://issues.apache.org/jira/browse/SVN
> > or not ?
>
> We could clarify the error message by having it refer the admin to the
> server log.  We might also have the error message state "The authz file
> failed to parse" (without details; we'd consider the authz file's path
> and section names to be confidential).
>
> Is that what you had in mind?  Or are you thinking of a larger change,
> e.g., detecting the invalid authz file even before a request is made to
> a repository that uses it (= the invalid authz file)?
>

No, a small informational change, IMO.

Eight years ago in the Selenium team we sprinkled helpful URLs the in
exception messages. Selenium was Java+Python+C#+Ruby+JavaScript back then
and a hard tech to get going with in the view of many. It uses many more
languages since V2. We had and still have lots of noobs wanting to get
running with it, and driving them to a help page if they could see it in
exception messages and logs was a way of reducing questions on mail-lists.

- Paul

Re: Authz - paths with trailing slash isn't supported - right?

[Paul Hammant]

>
> The cause of the 403 should be logged on the server side.
>>
>
> It was:
>
> [Sat Dec 09 22:24:47.803767 2017] [authz_svn:error] [pid 13] [client
> 172.17.0.1:35066] Failed to load the mod_authz_svn config: Section name
> '/foo/a/' contains non-canonical fspath '/foo/a/'
> [Sat Dec 09 22:24:47.803817 2017] [authz_svn:error] [pid 13] [client
> 172.17.0.1:35066] Access denied: 'harry' GET foo:/a
>

Should an unparsable authz file be communicated in a clearer way than some
URLs working and some 403ing?  Should I raise this in
https://issues.apache.org/jira/browse/SVN or not ?

- Paul

Re: Authz - paths with trailing slash isn't supported - right?

[Paul Hammant]

>
>
> No, it's intentional:
>
> https://subversion.apache.org/docs/release-notes/1.8.html#
> authz-fspath-syntax
>
> The cause of the 403 should be logged on the server side.
>

It was:

[Sat Dec 09 22:24:47.803767 2017] [authz_svn:error] [pid 13] [client
172.17.0.1:35066] Failed to load the mod_authz_svn config: Section name
'/foo/a/' contains non-canonical fspath '/foo/a/'
[Sat Dec 09 22:24:47.803817 2017] [authz_svn:error] [pid 13] [client
172.17.0.1:35066] Access denied: 'harry' GET foo:/a



> >.. would be great if it were slurped into the SvnBook somehow. Anyone
> > from Collabnet care to weigh in?
>
> Bug reports against the book should go to svnbook-...@red-bean.com
>

Wil do.

Re: Authz - paths with trailing slash isn't supported - right?

[Paul Hammant]

>
>
> The / on the root is not a trailing slash; it's a leading slash. Paths
> in the authz file must /start/ with a slash.
>

Obvious really, I guess.

Authz - paths with trailing slash isn't supported - right?

[Paul Hammant]

In authz files, [/] is often mentioned as a cross-cutting repo root that
has permissions for users and groups.

There's no other references to a path in square brackets with a trailing
slash. At least, not
in
http://svnbook.red-bean.com/nightly/en/svn.serverconfig.pathbasedauthz.html.

That was context, and here's the question or perhaps bug report...

So, administrators of Svn installs should *not* create paths in authz files
with *trailing slashes* - like [/foo/] - right?  I say that because my
attempts to do so yields plenty of unexplainable 403 responses in ordinary
clients like web browsers. Making me think it's a bug.

Or there could be a dev-team view that this is a documentation improvement
issue at this point. Specifically some advice  like "Paths that are
directories don't have trailing slashes in square brackets other than [/]
for the root" added to the above page.

One more thing - Mike Pilato's
http://blogs.collab.net/subversion/authz_and_anon_ article on CollabNet's
blog ... would be great if it were slurped into the SvnBook somehow. Anyone
from Collabnet care to weigh in?

- Paul

A server-side merkle-tree capability for ordinary Subversion* installs.

[Paul Hammant]

Implementation here - https://github.com/paul-hammant/SvnMerkleizer. It is
written In Java in about 600 lines of code. There's unit and integration
tests too (coverage between those two is 92%).

In operation it is pretty slow to build caches, but performs really well
after that when there's not many changes hitting the repo.

Enjoy.

* those over HTTP/DAV that is.

-- 
Paul Hammant DevOps <https://devops.paulhammant.com> Let me give your
enterprise a step by step plan to get out of the hell crazy branching
models (ClearCase maybe?) and into the world of high-throughput CD on
DevOps foundations.

Re: Ideas for tracking Authz changes in a repo

[Paul Hammant]

s/Marc/Mark/ sorry ... I've been dealing with a fellow with the other
spelling today in work.

Re: Ideas for tracking Authz changes in a repo

[Paul Hammant]

Thanks for this Marc. I'll go ahead and play with it to learn it
capabilities and (hopefully few) snags.

- Paul

Ideas for tracking Authz changes in a repo

[Paul Hammant]

One thing I need to be able to do in the near term is track when authz
settings change for a user (or groups).

It'd be great if Svn had the authz file (optionally) under source control.
I'd be happy with

1) a robot user performs a *copy* of the file from its canonical location
and automatic commit of the authz file into /.authz. And perhaps that
would have permissions such that ordinary users cannot see it.

2) the configuration of the authz side of Svn (optionally) can be
*canonically* located at /.authz (same notes as to hidden, itself)

3) something else - advice welcome!

Re #1, I could engineer a Jenkins job to perform the commits, but the
config changing isn't in the classic trigger system.

Regards,

- Paul

Re: Augmenting the WebDAV side of Subversion with merkle hashes for directories.

[Paul Hammant]

OK, not a straight handler, but 'ScriptAliasMatch'. Like so:

ScriptAliasMatch \.foo*$ "/path/to/helloworld.py$1"


I should be able to hit /a/b/d/e/f.foo in a browser and have helloworld.py
execute, right?

Well, it works just fine when declared at root level (not in a 
or  directive) and for when the request URL is anything outside
the Subversion mounted dir/location:

But when I change the URL to be inside the Svn location, then mod_dav_svn
intercepts the request and reports 404 - the wildcard on .foo suffixed URLs
is ignored.

Specifically:

   http://myserver/a/b/d/dfgw/wewerwer/anthing.foo  --> works fine
   http://myserver/svn/anthing.foo
  --> does not work

Unless there's a way through this (other nesting techniques), I'll have to
mount the Merkle directory summaries on a parallel directory tree:

   http://myserver/svn/path/to/department_salaries.xls  --> Svn via DAV
   http://myserver/svn/path/to/imminent_layoffs.doc  --> Svn via DAV

   http://myserver/meta/path/to/.indexSHA1s.csv  --> a GET-centric app
   http://myserver/meta/path/.indexSHA1s.csv  --> a GET-centric app
   http://myserver/meta/.indexSHA1s.csv  --> a GET-centric app

Regards,

- Paul

Re: Augmenting the WebDAV side of Subversion with merkle hashes for directories.

[Paul Hammant]

Well the code from that blog entry works as reported for 
declarations but not for  ones where mod_dav_svn is the handler.
I recreate both of those in a single server implementation if I mount the a
svn/ folder inside the canonical docroot (as is common), then play with
URLs that should be rewritten in the browser (or Wget) that are inside and
outside of that svn/ sub directory.

For extra fun I moved the four rewrite lines before, inside, and after the
Svn , but it didn't make any difference.

I'll try a straight handler next, in the hope that it can wholly intercept
requests that would otherwise be routed to mod_dav_svn.

- Paul

Re: Subversion 1.10 RC1?

[Paul Hammant]

I think you're misunderstanding me. I noted this in the conversation above:
"One conflict option that's *not done yet* is one that merges a textual
change from a path ^/trunk/B to a path ^/branch/A, after a move A -> B on
trunk. I don't consider this a release blocker. We can always add this
option in a patch release."

I interpreted that comment to be about something merged/integrated that
should not go ahead just yet, and deserving of some consideration.

I felt I had a suggestion that may be helpful to allow the release of
something alpha/beta/RC-ish to go ahead even with something the team felt
wasn't ready for release.

To reiterate: I agree the feature is useful and I have nothing to say that
I would wish to have interpreted as a concern, was trying to help speed
things up not slow things down.



On Thu, Nov 23, 2017 at 4:04 PM, Stefan Sperling <s...@apache.org> wrote:

> On Thu, Nov 23, 2017 at 03:53:02PM -0500, Paul Hammant wrote:
> > >
> > >
> > > > Is that a regression versus v1.9 and before?
> > >
> > > Far from it.
> > > This discussion is about the new conflict resovler added in 1.10.
> > >
> > > We can and always could perform such merges by manually specifying
> > > the paths, i.e. running a merge from ^/trunk/B to branch/A.
> > >
> > > The goal of the resolver is to make it easier to perform such merges
> > > in the straightforward 'svn merge ^/trunk' style.
> > >
> >
> > OK, so at the risk of representing *the department of unsolicited
> advice*,
> > I don't know why you would not ship with as 'off' but able to be toggled
> > 'on' client-side with a new option
> > --with-experimental-1-10-conflict-enhancement-technology
> > (or a better name). The 0.1% of users that want to play with it before it
> > is ready can do.
> >
> > - Paul
>
> Why? There's a lot of benefits in this feature. It has been worked on
> sicnce May 2015 and is already pretty useful. There's a high-level
> description in the 1.10 release notes, have you seen that?
> http://subversion.apache.org/docs/release-notes/1.10.html#
> conflict-resolver
>
> You're a bringing your concerns up a little late :)
> But I'm not even sure why you would have any concerns about this.
>



-- 
Paul Hammant DevOps <https://devops.paulhammant.com> Let me give your
enterprise a step by step plan to get out of the hell crazy branching
models (ClearCase maybe?) and into the world of high-throughput CD on
DevOps foundations.

Re: Augmenting the WebDAV side of Subversion with merkle hashes for directories.

[Paul Hammant]

Yup. I'm able to go down a rabbit hole on things, though. I've just spent
an hour trying to find a docker image to start with so that I was able to
ship something that's easily run as an an example for this group. Who knew
there's a *few thousand* ways of configuring Apache in docker images, heh?

Re: Subversion 1.10 RC1?

[Paul Hammant]

>
>
> > Is that a regression versus v1.9 and before?
>
> Far from it.
> This discussion is about the new conflict resovler added in 1.10.
>
> We can and always could perform such merges by manually specifying
> the paths, i.e. running a merge from ^/trunk/B to branch/A.
>
> The goal of the resolver is to make it easier to perform such merges
> in the straightforward 'svn merge ^/trunk' style.
>

OK, so at the risk of representing *the department of unsolicited advice*,
I don't know why you would not ship with as 'off' but able to be toggled
'on' client-side with a new option
--with-experimental-1-10-conflict-enhancement-technology
(or a better name). The 0.1% of users that want to play with it before it
is ready can do.

- Paul

Re: Augmenting the WebDAV side of Subversion with merkle hashes for directories.

[Paul Hammant]

"Two docroots" might be what I'm looking for:  http://madbean.com/2007/two-
docroots/

The author, Matt Quail, has been Jira lead for a decade, I think.

Augmenting the WebDAV side of Subversion with merkle hashes for directories.

[Paul Hammant]

OK, so most of ye know I am on this quest.

*Context*: I've a working merkle-ish client tracking changes in a Svn
server over DAV, and noting when files directories change and replicating
that change on the client side. For files it gets the SHA1 for the file,
stores it, and updates local file copies when the SHA1 changes. For
directories there's not SHA1 (yet), and it tracks directory revisions.
They'll change too fast, so I need to server-side create the SHA1s for
directories and cache them appropriately.

I know how to calculate the (presently missing) SHA1s for directories in
Svn. I even know how to do that for different auth settings for different
users safely.

*Intention:*

I want to mount (over DAV only) '.indexSHA1s.csv' (or something similar) in
every directory, but I don't want those checked in (race condition
avoidance, and commit noise).  Thus, while those files are visible over
DAV, subversion does not know about them, even if they do follow the ACLs
apparent in the authz files. Specifically, if you are not permitted to read
/foo/bar/ then you cannot read /foo/bar/.indexSHA1s.csv either.

*Question:*

What Apache2 technique to need to read about, and find docker-style
examples for to facilitate this? There's a huge amount of Apache modules,
and I can't glean whether Mod_filter or any Apache module is the right tool
for the job.  Or even if this isn't delivered in modules at all (build in).

*More info: *I've prototyped my Merkle hash calc (and recalc) logic in
Rust, with tests of course. I may end up writing an apache module for this
(and donating back) but right now I'm happy with any Apache2-centric
technique (regardless of speed or hackyness) that can acclimate me to
augmenting the DAV available Svn representation with files that are
hot-calculated (and LRU cached).

Can someone point me in the right direction?

- Paul

Re: Subversion 1.10 RC1?

[Paul Hammant]

>
>
> One conflict option that's not done yet is one that merges a textual change
> from a path ^/trunk/B to a path ^/branch/A, after a move A -> B on trunk.
> I don't consider this a release blocker. We can always add this option in
> a patch release.
>
>
Is that a regression versus v1.9 and before?

- Paul

Re: Checkpointing v1 design -- terminology

[Paul Hammant]

>
> $ svn save --revert -v2 "add hair color to person page"
>>
>> .. does the save and THEN drops the CL and it's changes in working copy -
>> back to no changes but not necessarily up to date.
>>
>
> That sounds like you're proposing an alternative syntax for what the
> 'shelve' command does -- except I'm not sure what you want the 'v2' option
> to do here; was that accidental?
>
>
Maybe alternate, sure.

$ svn savelist

add hair color to person page; v1
add hair color to person page; v1.2
add hair color to person page; whatever_passes_fora_versn_string

$ svn save -v v2 -m "add hair color to person page"
# Overwrites v2, perhaps would need a --force.

$ svn save -v three
# Created a new 'three' version of (implicit) "add hair color to person
page"

$ svn save -v three --revert
# As above but also deletes the changelist and any changes to working copy
versus last 'svn up'

$ svn restore -v three
# Creates a CL with all those changes on them.

Oh and shelve/save/checkpoint would all reuse impl as they are pretty
close. As does "Pull-request" (PR).

$ svn pull-request -m "add hair color to person page"
# Goes into codereview for Rietveld, Assembla, RhodeCode (etc),
# Possibly needing a plugin, or CURL configuration in .subversion/

- Paul

Re: Checkpointing v1 design -- terminology

[Paul Hammant]

I'd be happier with save-cl rather than ci-save.  Or just 'save', as I'm
not sure what you're saving if not a change list.  Incidentally CL looks
pretty close to CI in lower case and that's already used.

Spaces allowed in FOO name?

$ svn save -v2 "add hair color to person page"

Also..

$ svn save --revert -v2 "add hair color to person page"

.. does the save and THEN drops the CL and it's changes in working copy -
back to no changes but not necessarily up to date.

> Revert files in (active) changelist FOO:
>   svn revert --cl=FOO  # existing syntax

I don't know what this one does Julian?

Re: Subversion Design Contribution Question

[Paul Hammant]

I'm also interested in standardized server-side handling of such things for
a different reason to the one you state - code reviews. Well outside the
purview of vanilla Subversion for sure, but a feature that the portal
vendors have or are coding themselves. I've a love of Trunk-Based
Development and got to see the code review system that Google built for
themselves around Perforce (Subversion was partially inspired by Perforce
back in 2000). Developers at their workstations would declare 'done', and
initiate code review. The changelist and some metainfo would be zipped up
and pulled to somewhere central. The bulk of the workflow is in the UI
Guido Van Rossum led at Google and showcases in
https://www.youtube.com/watch?v=sMql3Di4Kgc (2006). Post code review (and
CI added metrics) the change-list could be reconstituted and committed
(submitted in Perforce lang).  It is the humble little save point that
facilitates the 25,000 developers co-existing in one trunk with Piper
(their 2012 replacement to Perforce) and ultimately bots integrating
(Martin Fowler's preferred language for merge to trunk/master/mainline when
practicing CI) change sets ever few seconds.

I'm much less interested in workflows where I'm sharing something by that
mechanism for others to work on, as that's not Trunk-Based Development.

-ph

Re: Building Subversion on the Mac

[Paul Hammant]

>
> I sent you my build script for the Mac, that uses Homebrew, a while ago.
> Why are you still having problems?
>
>
You're right, you did. I'll take another look, Brane.

Meanwhile I wrote:
https://github.com/subsyncit/subsyncit/wiki/Building-Subversion-on-the-Mac

Contributing to the "how to build" documentation

[Paul Hammant]

There's a file tools/dev/unix-build/README that has some information. Is it
up to date?

It's not referred to in other files. Most importantly, not in/from
/README.

- Paul

Re: Building Subversion on the Mac

[Paul Hammant]

Yup, that builds through to completion, now. Thanks Troy.

Re: Building Subversion on the Mac

[Paul Hammant]

Yup, glibtool is needed on the Mac (brew install libtool  makes
/usr/local/bin/glibtool). That and a corresponding shim source:

#!/bin/sh

/usr/local/bin/glibtool "$@"

Re: Building Subversion on the Mac

[Paul Hammant]

Thank you, that allows configure to complete w/o error.  Next up, libtool


$ make
/bin/sh "/scm/oss/subversion/libtool" --tag=CC --silent --mode=compile gcc
-std=c90  -DDARWIN -DSIGPROCMASK_SETS_THREAD_MASK -DDARWIN_10
 -Werror=unknown-warning-option -g -O2  -g -O2   -I./subversion/include
-I./subversion -I/usr/local/opt/apr/libexec/include/apr-1
 -I/usr/local/opt/apr-util/libexec/include/apr-1
-I/usr/local/opt/openssl/include -I/usr/local/Cellar/lz4/1.8.0/include   -o
subversion/libsvn_delta/branch.lo -c subversion/libsvn_delta/branch.c
/bin/sh: /scm/oss/subversion/libtool: No such file or directory
make: *** [subversion/libsvn_delta/branch.lo] Error 127

So I can find a /usr/bin/libtool and one in
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/
but both are binary, and the failing make step is an attempt to hand a
shell script to /bin/sh to execute.  This feels like a wrapper-like thing,
and I've googled a little, but can't see anything clear like "copy this
script to folder/libtool".

- Paul

On Sat, Oct 28, 2017 at 8:07 AM, Troy Curtis Jr <troycurti...@gmail.com>
wrote:

>
>
> On Sat, Oct 28, 2017, 7:02 AM Paul Hammant <p...@hammant.org> wrote:
>
>> OK, so I can get so far ...
>>
>> svn co http://svn.apache.org/repos/asf/subversion/trunk subversion
>> cd subversion
>> brew install libtool apr
>> cp /usr/local/Cellar/libtool/2.4.6_1/share/libtool/build-aux/config.sub
>> build/
>> autoconf
>> ./configure --build=x86_64 --prefix=/usr/local --with-openssl --with-ssl
>> --with-zlib --with-apr=/usr/local/Cellar/apr/1.6.3/
>> --with-apr-util=/usr/local/Cellar/apr-util/1.6.1/
>>
>>
>> (Note that I had to config.sub in my Homebrew folders and copy it into
>> the build/ directory - barfs without it)
>>
>> .. but it barfs after during the configure step:
>>
>>
>> 
>>
>> checking how to use output level for Ruby bindings tests... normal
>> checking for ctypesgen.py... none
>> checking stdbool.h usability... yes
>> checking stdbool.h presence... yes
>> checking for stdbool.h... yes
>> checking for stdint.h... (cached) yes
>> configure: creating ./config.status
>> config.status: creating Makefile
>> config.status: error: cannot find input file: `subversion/libsvn_*/*.
>> pc.in'
>>
>> I can't see any files ending in .pc.in in any subversion/libsvn_
>> prefixed folder (in the checkout). Therefore I'm not surprised the
>> configure step fails when it's looking for those files.
>>
>> Any advice would be appreciated.
>>
>> - Paul
>>
>
> Since you are building from a checkout and not from a dist tarball, you
> need to run "autogen.sh" before configure.
>
> Troy
>



-- 
Paul Hammant DevOps <https://devops.paulhammant.com> Let me give you a step
by step plan to get out of the hell of ClearCase and crazy branching models
and into the world of high-throughput CD on DevOps foundations.

Building Subversion on the Mac

[Paul Hammant]

OK, so I can get so far ...

svn co http://svn.apache.org/repos/asf/subversion/trunk subversion
cd subversion
brew install libtool apr
cp /usr/local/Cellar/libtool/2.4.6_1/share/libtool/build-aux/config.sub
build/
autoconf
./configure --build=x86_64 --prefix=/usr/local --with-openssl --with-ssl
--with-zlib --with-apr=/usr/local/Cellar/apr/1.6.3/
--with-apr-util=/usr/local/Cellar/apr-util/1.6.1/


(Note that I had to config.sub in my Homebrew folders and copy it into the
build/ directory - barfs without it)

.. but it barfs after during the configure step:




checking how to use output level for Ruby bindings tests... normal
checking for ctypesgen.py... none
checking stdbool.h usability... yes
checking stdbool.h presence... yes
checking for stdbool.h... yes
checking for stdint.h... (cached) yes
configure: creating ./config.status
config.status: creating Makefile
config.status: error: cannot find input file: `subversion/libsvn_*/*.pc.in'

I can't see any files ending in .pc.in in any subversion/libsvn_ prefixed
folder (in the checkout). Therefore I'm not surprised the configure step
fails when it's looking for those files.

Any advice would be appreciated.

- Paul

Subsyncit - file sync that uses Subversion as a backend.

[Paul Hammant]

http://subsyncit.com/

If you're prepared to make a few additions to httpd.conf you get running
with this Python3 technology to do file-sync using Subversion as the
backing store.  It doesn't require any client side Subversion install, but
does Python and some pips. At least up until I make an installer.

And I've yet to make a Window-tray or Mac-menu icon thingamy like every
other background app does. Meaning this is still beta quality, really - but
it does its business and doesn't eat all available CPU.

I'm hoping to integrate with various other applications/services. Trello,
Slack, etc. That would be for workflow-ish functionality.  Fred assigns
something to Wilma in Trello by changing the 'assigned to' aspect of a
card, and a few seconds later it appears on her C: drive. A spreadsheet say.

I've tested it with both Assembla's and RhodeCode's Subversion platforms.
I've also tested it with a bunch of my own Subversion deployments. One - a
$10 Pi Zero W with a $70 200GB SdCard in it. Another a EC2 deployment of
Subversion/Apache2.

Questions?

- Paul

Re: How to manage PR's (was: [GitHub] subversion pull request #6: VC should be 14.1 and not 15.0)

[Paul Hammant]

>
>
> Anybody knows why asfgit closed the PR?
>
> I've clicked on the above hyperlink, and saw some comments made by
> Julian (11 hours ago) and by the submitter (8 hours ago), but these
> comments are not visible on dev@. Shouldn't those be synced back here
> somehow? Or how should we handle PR's, preferably with a process
> that's still centered around the dev@ mailinglist?
>
> If we can't really handle PR's well, maybe we shouldn't allow them
> (i.e. indicate this on github somehow, that it's just a mirror and we
> won't accept PR's, but expect patch submissions on dev@ instead)?
>
>
If you ask people comfortable with donating code via PR to email patches
(or similar) to dev@ instead, you'll get a 1/100th contribution rate. Or
worse.

There's no easy answer.

Partially related - I'd previously whined about GitHub's lock-in for some
data and wrote - https://github.com/paul-hammant/pull-request-scraper -
which probably needs refreshing since last using it.

Re: MKCOL question

[Paul Hammant]

Well pipelining into Svn's HTTP 1.1 interface does work as you suggested.

There is a library in Python called 'hyper' that does it -
http://hyper.readthedocs.io/en/latest/quickstart.html#streams though the
example get_response signature is not current (I raised a bug).  Also, if I
change HTTPConnection to HTTP11Connection it works, but not if I change to
HTTP20Connection (ConnectionResetError) so I think I'll just leave my
implementation as it is - mechanically recursing through MKCOL operations
over a single reused connection.

Re: Merkle trees in svn [was: Quick question about the sha1-checksum for directories in svn.]

[Paul Hammant]

Yikes! I'll have to run through the build setup for Subversion - 52 pages
of requirements *before* launching ./config, right? - including
wind-of-newt, left handed screwdriver, *two* four-leaf clovers ;) :-P

Seriously: sounds good - and I appreciate it :)

Re: Merkle trees in svn [was: Quick question about the sha1-checksum for directories in svn.]

[Paul Hammant]

As my forthcoming multi-user app that uses Subversion as a backing store is
going to kill Svn it with Depth-∞ PROPFINDs from root, I really want to see
this implemented.

Because I can't wait I will implement something that calculates SHA1s for
the directory in question, and drops it into a '.sha1' file in the folder
in question. It'll be a job that runs perpetually, and keeps committing as
things change.

To cut through the permissions based quandary that y'all are having, how
about ignore the subsetting that's implicit from permissions, and just calc
the SHA1 for the 'all' situation, requiring the client-side end-user of the
this setup to correlate a localally calculated SHA1 for the permitted set
and the remote canonical SHA1 for the potential superset. If the same
algorithm is used on the client and the server, then all the end user can
determine is that they are not (for some directories) permitted to read all
files.

Crude example, using command line unix tools:

$ cat foo
2013.json 1674790a70b984c9041ab86c370f942861ead004
2016.json 194f6519cd60b773a82857cf1aeba8dad4a223ed
2020.json 20e3ff1ade2385c593f73fd44fd157391d2424e7
2050.json 19b2da433a273840deddb7a46b16891acab16e3f
2060.json 45418423999c155abc434e175d42ccf6534bee6d
2068.json 69576d3632c7ce8b0b2a42d87e9e75049bdaff9d
$ cat foo | sha1sum
c1874a0ba80cb51245cb78f567dbbd46271d7ba1  -
$ head -3 foo | sponge foo
$ cat foo
2013.json 1674790a70b984c9041ab86c370f942861ead004
2016.json 194f6519cd60b773a82857cf1aeba8dad4a223ed
2020.json 20e3ff1ade2385c593f73fd44fd157391d2424e7
$ cat foo | sha1sum
1cbbec9747ab817dac26892cfd437f72d7366858  -

^ the server side says c1874a... for the 'directory' that foo represents
and passes that to all clients that ask (provided they are permitted to the
directory).  The client side holds that ref, *and* 1cbbec... maybe in a
database of some sort. This way, it can track whether it has kept abreast
of the Svn server or not. It may determine that in needs to do a depth-1
PROPFIND and because the server side SHA1 for the directory changed, and in
the not-permitted-to-see-all situation may determine that that was
ultimately unnecessary.

Of course I can alternatively correlate subversion revision numbers right
now with locally calc'd SHA1, but it doesn't feel very Merkle-like.

MKCOL question

[Paul Hammant]

Hi gang,

*(non subversion-client usage warning, also BDD ahead warning)*

Given I have a directory /path/to/missing/directory/ that does not exist on
the Svn server at all
When I want to put a file in there (say foo.mp3)
Then I have to MKCOL path/
And I have to MKCOL path/to/
And I have to MKCOL path/to/missing
And I have to MKCOL path/to/missing/directory/
And only the can I then PUT path/to/missing/directory/foo.mp3


I'd love to be able to make the directories in a oner. Like Unix's mkdir -p
option.

Or something better still in the PUT operation, if there were extra header
params I was not aware of.

- Paul

Re: Merkle trees in svn [was: Quick question about the sha1-checksum for directories in svn.]

[Paul Hammant]

Agree.

For those that are super interested in this, I did a series of four blog
entries on Merkle trees and Blockchains.


   - Sept 28th » Choosing Between Blockchains And Vanilla Merkle Trees
   

   - Sept 28th » Blockchains in pictures
   
   - Sept 17th » 'Old-School' Merkle Trees Rock
   
   - Sept 17th » Merkle Trees In Pictures
   

Re: Merkle trees in svn [was: Quick question about the sha1-checksum for directories in svn.]

[Paul Hammant]

> "Obeying read permissions" means that the directory hashes would have to
> be computed dynamically for each user.

Yes I know.

I've a frankenstein project over here -
https://github.com/paul-hammant/merkle-rust - There's a solid Rust merkle
tree that watches for changes at nodes then re-calcs the SHA1 of the parent
dir (and the parent of that, and all the way back to root).

Because I can't actually write Rust yet, I've made a Python webserver, and
Python test that shows what nodes should report on the wire (plain text in
this demo). Here's is the GET result for node A/AK/A/Alaska/ ( a list of
resources and their SHA1s) ->

96b55f350ccb1911d7f9d7e7645d6d11178ad753
2013.json 1674790a70b984c9041ab86c370f942861ead004
2016.json 194f6519cd60b773a82857cf1aeba8dad4a223ed
2020.json 20e3ff1ade2385c593f73fd44fd157391d2424e7
2050.json 19b2da433a273840deddb7a46b16891acab16e3f
2060.json 45418423999c155abc434e175d42ccf6534bee6d
2068.json 69576d3632c7ce8b0b2a42d87e9e75049bdaff9d
2070.json 2d50d3100744fb7c4c5ead72a2896909fbe2ba6a
2090.json 434e72590bdaa03176ebabc18e52dc0a24918da9
2100.json a884cf405af5a20276b3b1cc72885833905ffa97
2105.json f7bfd6be756c919e4ea69cb466f31ae0b2fd213d
2110.json c88ed14784907db94b11941b760892742bd043f3
2122.json e5247216a7a1f32851807d4b4b9cd1cf152cd57d
2130.json 375fc259d26710afe31e1367a59de3d5dea729b1
2150.json b5c17ce941a9647fbc179cb1b769348cfdaebb98
2164.json ab2f71a47910463bdde92a77313f7e5edba00063
2170.json c1d0842e2c77d53bee5c684e0e8ad580a2fff05f
2180.json 9759646b5a2811efc6bfaee84a2b813f85cb1e1a
2185.json e2beddc3f245ca79908d9a1590aa177151e66e4d
2188.json 5881cdc3a15eac0cba15a3e07ad54176cabeeedf
2195.json b7085173d8685deecf70c6efb63d92ec6db2cf21
2198.json 78a399df3eeb4f696812d6d46882e4029190cf67
2220.json 594a20ec3d550eae2ad848fa8c0e08d50bd4e7fa
2230.json ec4c2c8dfc3cf4c1e6719de0544aaeba7731fc9c
2240.json 798b13c1c4aa7ee2e5744f8b8a2b553c1447229b
2261.json 0cd460d99cc7a871230cbe0f6f2ab703339e1630
2270.json 4e77074d4adc2ac7bb486408b00bda26f98820ea
2275.json 91ce27fc8b643540dda88afdc5b5b62d97a026fa
2282.json 3bec0f1f9e3ae818dd096303f796ce28e2fe6d08
2290.json fd4014808dd77351b939060f1283d395deb0cea5

Here's the GET result for the SHA1 of that A/AK/A/Alaska/.sha1 ->


96b55f350ccb1911d7f9d7e7645d6d11178ad753


You know already, Brane, but others might not: I take the first text (the
list) run it through 'sha1sum' to make the second text (the single SHA1).
That the relatively dynamic SHA1s that would obey read permissions for the
user in question, only need to SHA1 a small amount of data, not whole files
again. Of course that is recursive so it would need to be maintained once
per user group, or user where there's different permissions.

- Paul

Re: Merkle trees in svn [was: Quick question about the sha1-checksum for directories in svn.]

[Paul Hammant]

Not that my vote counts for much, but I'd prefer w/o props, obeying read
permissions.

Re: Quick question about the sha1-checksum for directories in svn.

[Paul Hammant]

> Correct.

Thanks Julian.

Observation: If it did then Subversion would qualify as a *full* Merkle
tree implementation. Albeit through APIs designed for other purposes :-p ;)

Quick question about the sha1-checksum for directories in svn.

[Paul Hammant]

While a _directory_ has a revision number, it does not have a SHA1 does it?

If so there's no point asking how it's calculated, is there ?

- Paul

Re: Sparse checkouts suggestion

[Paul Hammant]

>
>
> I'm not sure if you agree or not, but I think the fine-grained globbing
> include/exclude capability as Perforce has it. is required.
>
>
> "Required" implies "working copy format change". That's always hard.
>
> "Optional" means "get some of the features sooner."
>

I'm a big advocate for "ship-it now, add features later".

I'll try again:

"I'm not sure if you agree or not, but I think the fine-grained globbing
include/exclude capability as Perforce has it, is *ultimately* required."

Re: Sparse checkouts suggestion

[Paul Hammant]

>
>
>
> 2. Previously I forked a medium-sized Monorepo on Github, and did the the
> complete expand/contract work for it - https://github.com/paul-hamm
> ant-fork/jooby-monorepo-experiment - in Python.
>
>
LOGIBALL's Tim Krüger has just written about a deployment of an
expanding/contracting monorepo for Git and Maven based on the above proof
of concept: https://timkrueger.me/a-maven-git-monorepo/ (I'd helped him
with the article).

There's are two Git realities that have to be closely monitored for teams
wanting to do trunk-based development in a monorepo:

1) the clone-size has an upper limit (accounts differ as to where that is)
2) there's a push-pull bottleneck when the team starts to attempt to push
at the same time (Git makes you pull stuff you don't have before you push)

Subversion doesn't have to consider those year-on-year. It does of course
have merge snafus that Git doesn't have. Then again Git has merge snafus
that Svn doesn't have - at least for creative branch use, IMO.

- Paul

Re: Sparse checkouts suggestion

[Paul Hammant]

>
> [...]
>
> if /foo does not exist at checkout time -- and if we'd want /* to mean
> "include anything new that appears in the repository", not "include
> everything else that's in the repository at the time of the checkout".
>

I'm not sure if you agree or not, but I think the fine-grained globbing
include/exclude capability as Perforce has it. is required.

- Paul

Re: Sparse checkouts suggestion

[Paul Hammant]

> On 19.09.2017 00:59, Paul Hammant wrote:
> > I like the `svn co svn://vcs/trunk --view foo`. Well maybe if in a
> > following `svn up` it would *remember the current view*, it would be
> good.
>
>
> I don't think there's any need to remember the "current view": a sparse
> working copy already maintains its topology, so the view spec is only
> needed on checkout to construct the sparse WC (and then update to
> explicitly change it, etc.)


Yes, you're right.

Re: Sparse checkouts suggestion

[Paul Hammant]

I like the `svn co svn://vcs/trunk --view foo`. Well maybe if in a
following `svn up` it would *remember the current view*, it would be good.


> I'm not sure about listing the "available views" in a default
> property, but okay, it's a possibility. I think it would be nice to be
> able to read it from a file, a url or a property (chosen by the user).
> In theory I'd say it would be nice to read it from a pipe, so you can
> 'cat', 'svn cat' or 'svn propget' the viewspec as input.
>

Google's 9-million* source files at HEAD revision: When checking out the
trunk, and going on to subset down to your expected checkout, there's so
many applications and services in the trunk that their `gcheckout`
technology makes a custom include/exclude set for each developer need based
on a traversal of the Blaze BUILD files.  You're mentioning "list" as a sub
command, yet it's not needed at all.

I never saw the source for gcheckout, and it's long gone given Perforce was
replaced by Piper in 2012, and a FUSE took over developer working copy, but
it is still worth understanding to a deeper level.

> gcheckout adsense:tests,adwords:tests,others

The last line ofthe script would invoke

p4 client -i < "$P4_CLIENTSPEC_TEMP"

And you would do `p4 sync` on the command line following that.

The number of permutations ==  factorial for (num of modules * packages or
namespaces * build targets)

Don't aim for a concise list of view-specs, or "available" views. Leave it
completely open for globbing include and excludes (as Perforce/Google) to
be generated by smart script techs, and only verify the spec for
correctness on ingestion, IMO.

- Paul