Re: Shelving / Checkpointing thoughts

[Julian Foad]

Forgot the conclusion...

Julian Foad wrote:
> Johan Corveleyn wrote:
>> Still only one shelf per WC (*the* shelf)? Grouping them through
>> naming ("savepoint-1", "savepoint-2" are two shelved patches belonging
>> to the same series, but "featureA" (which was reverted) is separate
>> because it doesn't have the same prefix)?

Absolutely yes! I thought I had previously indicated so.

>> Or do we need multiple shelves with some name too?

Absolutely not. Not for any reason that I can see.

- Julian

Re: Shelving / Checkpointing thoughts

[Julian Foad]

Johan Corveleyn wrote:
>>   * "Shelving" or "to shelve" means putting something on a shelf. There
>> is one "shelf" per WC.
>>
>>   * The thing we put on the shelf is called a "patch" or a "shelved
>> change", and is analogous to a book or a paper placed on the shelf. A
>> numbered version of a patch can be called a "checkpoint".
>>
>>   * A series of checkpoint patches is a series of "patch versions" or a
>> "checkpoint series". I think this is simpler than introducing a new term.
> 
> Ah yes, of course. Sorry, no need to invent a new term.
> 
> Just wondering then, when we create a "series of patches" that belong
> together, that have some ordering, how do we organize that?

Let's clarify. We can mean two possible things when we say 'a series of
patches':

  1. "patch versions": a series of successively better patches, all
attempting the same logical thing, all from the same base, and only one
of which is applied at any time;

  2. a series of patches, each providing a different logical change,
where each patch is based on the result of applying the previous one.
("quilt" is a tool for managing path series of this kind. My 'option 3'
(local repository) design for checkpointing could also be used in this
way, in a primitive way, but would not support revising earlier patches
in the series which is a key strength of what "quilt" can do.)

I am talking about definition 1 ("patch versions").

I propose patches in a series of patch versions be named "featureA-1",
"featureA-2", ... (This is what I do already, manually, in my own work.)

I propose that we should not attempt to provide any special support for
definition 2 within this "shelving" feature; users can manage that
themselves by simply remembering which feature names depend on which
other ones, or by including some other numbering system within the names.

> Still only one shelf per WC (*the* shelf)? Grouping them through
> naming ("savepoint-1", "savepoint-2" are two shelved patches belonging
> to the same series, but "featureA" (which was reverted) is separate
> because it doesn't have the same prefix)? Or do we need multiple
> shelves with some name too?
> 
> Just one more thought: in the namespace of shelved changes, we might
> want to reserve "svn:" or some such prefix, for internal use, to give
> us possibilities for features built upon the shelving infrastructure.

Good thought.

- Julian

Re: Shelving / Checkpointing thoughts

[Johan Corveleyn]

On Fri, Aug 25, 2017 at 4:59 PM, Julian Foad  wrote:
> Johan Corveleyn wrote:
>> On Fri, Aug 25, 2017 at 3:33 PM, Julian Foad  wrote:
> [...]>> The Checkpoint feature could add the copy-and-modify facility
> for the
>>> log message.
>>
>> Yes, maybe we'll need to have some grouping structure / namespacing in
>> the shelves for this. A "rack" or something :-). The rack carries a
>> name ("savepoints", "feature A"); a single shelf in a rack is just
>> 'svn shelve --rack "feature A"'; If I add more shelves to a rack, they
>> get numbered. [...]
> I think the terminology works best, and most in line with other tools
> (p4, hg, bzr) like this:
>
>   * "Shelving" or "to shelve" means putting something on a shelf. There
> is one "shelf" per WC.
>
>   * The thing we put on the shelf is called a "patch" or a "shelved
> change", and is analogous to a book or a paper placed on the shelf. A
> numbered version of a patch can be called a "checkpoint".
>
>   * A series of checkpoint patches is a series of "patch versions" or a
> "checkpoint series". I think this is simpler than introducing a new term.

Ah yes, of course. Sorry, no need to invent a new term.

Just wondering then, when we create a "series of patches" that belong
together, that have some ordering, how do we organize that?

Still only one shelf per WC (*the* shelf)? Grouping them through
naming ("savepoint-1", "savepoint-2" are two shelved patches belonging
to the same series, but "featureA" (which was reverted) is separate
because it doesn't have the same prefix)? Or do we need multiple
shelves with some name too?

Just one more thought: in the namespace of shelved changes, we might
want to reserve "svn:" or some such prefix, for internal use, to give
us possibilities for features built upon the shelving infrastructure.

-- 
Johan

Re: Tip of the day: Cheatsheet for common cmdline client operations

[C. Michael Pilato]

I like either idea, though of course I lean towards not duplicating
information that's already maintained elsewhere unless, of course, there's
reason to believe that it *won't* be maintained elsewhere.

Just for kicks, I did some Googling around:


   - "Subversion cheat sheet":  returns a number of third-party (largely
   PDF) resources
   - "Subversion quick start":  the project's quick-start page wins the
   day, with the Book's "High-Speed Tutorial" second
   - "getting started with Subversion":  the project's quick-start page
   again is the top hit, with the book nearly not even making the top-10 hits

So, yeah, I think linking from http://subversion.apache.org/quick-start is
a wise move.

-- Mike

On Fri, Aug 25, 2017 at 7:03 AM, Johan Corveleyn  wrote:

> [ cc -= svnbook-dev, as I guess this will not become book-material.
> More below ... ]
>
> On Thu, Jul 13, 2017 at 6:34 PM, C. Michael Pilato
>  wrote:
> > On Thu, Jul 13, 2017 at 11:21 AM, Daniel Shahaf 
> > wrote:
> >>
> >> [moving from users@ ]
> >>
> >> Daniel Shahaf wrote on Fri, 30 Jun 2017 09:15 +:
> >> > I just ran into the following cheatsheet:
> >> >
> >> > http://www.chim.unifi.it/~signo/did/etc/subversion/neat.html
> >> >
> >> > It covers the normal multi-user workflow, branching, etc..
> >> >
> >> > (Kudos to the author, Giorgio Signorini, not to me.)
> >>
> >> As some of you know, the author gave us permission to incorporate that
> >> cheatsheet into the official documentation.  Any volunteers to start the
> >> process?
> >>
> >> Cheers,
> >>
> >> Daniel
> >>
> > With respect to the author and the work he's done, I'm not really
> interested
> > in us maintaining yet another collection of the same information already
> > covered -- in some cases multiple times, when you factor in the reference
> > sections -- by the book.  At best this cheatsheet would be an appendix.
> I'm
> > happy to link to the cheatsheet from from the book website, though.
> >
> > -- Mike
>
> FWIW, I think this cheatsheet is quite good and valuable (though I
> agree there is a lot of overlap with existing documentation),
> especially for newcomers. Just to have a good summary / reminder of
> common things you'll encounter.
>
> We have our own quickstart page: http://subversion.apache.org/quick-start
>
> How about:
>
> - Putting a link at the bottom of that page (there is already a link
> to the quickstart section of the book), linking to the original
> webpage of the author.
>
> or
>
> - Incorporating (some of) the content of that cheatsheet directly on
> http://subversion.apache.org/quick-start, so it's right there in front
> of you ...
>
>
> The latter option would be my personal preference (putting my user /
> admin hat on) -- I like having short info right in front of me in the
> right place -- but obviously imposes some amount of doc-maintenance
> work on "us".
>
> Maybe someone on this list would be willing to take this doc-task
> (migrating the current content, and keep an eye on keeping it up to
> date)?
>
> --
> Johan
>

Towards a Shelving MVP

[Julian Foad]

It seems Shelving in the current form is working out quite nicely. I'm
thinking about what more we need so that any of us or our friends would
be comfortable testing it on non-trivial data and calling the feature
set a "minimum viable product". I thought of:


Hardening before being safe to use:

  * Help text should contain a short introduction to how to use; set the
user's expectations; state the limitations (see "Extensions / Not
Supported Initially" section [1] in design doc.).

  * Output should be verbose and clear about what is happening.

  * The prototype should not delete patch files when unshelving, in case
it goes wrong; instead rename/move them.

  * Must handle unsupported scenarios gracefully (e.g. kinds of changes
that we can't yet save and restore in a patch file).

  * If applying a patch fails in any way, or produces conflicts, it
should notify the user and keep the patch (moved somewhere is OK).

  * When 'unshelve' would touch an already modified file, consider
aborting instead of patching. Maybe only patch it if '--force' given.


Some easy UI improvements for a better first impression:

  * Let 'unshelve' choose the most recent shelved patch by default (like
'git stash pop' does).

  * Let 'shelve' default to 'this working directory' and an automatic
name, so that command-line arguments are not required.

  * Tidy up the '--list' output. e.g. show age as minutes/hours/days;
remove file size in bytes.


There remain many limitations listed in [1], mainly of svn diff/patch,
which we will certainly need to address, but I don't think any of them
are essential for the MVP stage. And we have today been talking about
extending the shelving UI to facilitate simple checkpointing, but that
isn't within MVP scope either.

Any thoughts please?

- Julian


[1]
https://docs.google.com/document/d/1PVgw0BdPF7v67oxIK7B_Yjmr3p28ojabP5N1PfZTsHk/edit#heading=h.ysnjkpcjx9ee

RE: svn commit: r1806017 - /subversion/trunk/subversion/libsvn_ra_serf/merge.c

[Bert Huijben]

> -Original Message-
> From: kot...@apache.org [mailto:kot...@apache.org]
> Sent: donderdag 24 augustus 2017 11:43
> To: comm...@subversion.apache.org
> Subject: svn commit: r1806017 -
> /subversion/trunk/subversion/libsvn_ra_serf/merge.c
> 
> Author: kotkov
> Date: Thu Aug 24 09:43:28 2017
> New Revision: 1806017
> 
> URL: http://svn.apache.org/viewvc?rev=1806017=rev
> Log:
> ra_serf: Prevent the server from generating and sending the full MERGE
> response in cases when we don't require it.
> 
> The full response is not required when working over HTTPv2 protocol.
> When working over HTTPv1, it's only required when the new working copy
> properties need to be stored as part of a commit (indicated by a non-null
> svn_ra_push_wc_prop_func_t callback).

Nice catch!

Does this affect performance enough that we should backport this fix?

Bert

Re: Problem with non-conflicting merges of moves

[Daniel Shahaf]

Stefan Sperling wrote on Thu, 24 Aug 2017 13:44 +0200:
>  5) Make the server send copyfrom args again (as it did in 1.5 times for
> files during 'svn update'), but only during merges/(diffs?) and for
> files and directories, with a heuristic that sends the correct copyfrom
> source to the client. Allows 'svn merge' to record "correct" copyfrom
> in the WC without overhead and conflicts, but implies additional
> server-side processing. Also requires RA protocol changes.
> My impression is that some of the code which was reverted in r1597989
> could be used for making this work.
> This is probably a lot of effort but could be worth it.

Doesn't directly answer your question, but: If we do end up making the
server send heuristic guesses to the client, could we please have them
*explicitly* marked as such, so API consumers don't mistaken the
heuristic guess for an accurate answer.

Cheers,

Daniel

Re: Shelving / Checkpointing thoughts

[Julian Foad]

Johan Corveleyn wrote:
> On Fri, Aug 25, 2017 at 3:33 PM, Julian Foad  wrote:
[...]>> The Checkpoint feature could add the copy-and-modify facility
for the
>> log message.
> 
> Yes, maybe we'll need to have some grouping structure / namespacing in
> the shelves for this. A "rack" or something :-). The rack carries a
> name ("savepoints", "feature A"); a single shelf in a rack is just
> 'svn shelve --rack "feature A"'; If I add more shelves to a rack, they
> get numbered. [...]
I think the terminology works best, and most in line with other tools
(p4, hg, bzr) like this:

  * "Shelving" or "to shelve" means putting something on a shelf. There
is one "shelf" per WC.

  * The thing we put on the shelf is called a "patch" or a "shelved
change", and is analogous to a book or a paper placed on the shelf. A
numbered version of a patch can be called a "checkpoint".

  * A series of checkpoint patches is a series of "patch versions" or a
"checkpoint series". I think this is simpler than introducing a new term.

- Julian

Re: Shelving / Checkpointing thoughts

[Johan Corveleyn]

On Fri, Aug 25, 2017 at 3:33 PM, Julian Foad  wrote:
...
>> == Shelving ==
>>
>> Looks great so far. Of course a lot of challenges remain for all the
>> cases which are not yet (correctly) covered by 'svn diff' and 'svn
>> patch' (property changes, tree operations, binary files, unresolved
>> conflicts, etc.). Attaching a log message to a shelf is key, and the
>> association with changelists looks like a good approach.
>>
>> - Shelves should (eventually) support directories as "versioned
>> items". Changelists currently don't support directories.
>
> I agree with all this too. A log message is supported in the prototype
> (but not yet integrated with changelists, as discussed in another thread).

We'll need to have the discussion about what changelists should do
with directories though. Not sure what prior discussions were already
held about this, when the changelist feature was designed ...

>> - Suggestion: 'svn shelve --keep', to create a shelf (patch) in the
>> "shelf storage" but not revert it. That would enable some crude way of
>> checkpointing your work, through simple patches (which can be applied
>> later by fuzzy patching, or ...):
>>
>> work on feature A
>> svn shelve --keep --name "feature A"
>> continue work on feature A
>> turns out badly, lets go back
>> svn revert; svn unshelve "feature A"
>
> Yes; implemented now in r1806168.

Great!

> In fact this usage aligns very well with how I use patch files myself. I
> often save a series of patches to checkpoint my development of a
> feature, with names like 'foo-1.patch' ... 'foo-N.patch'.
>
> One thing I like to do in that case is to copy the log message from the
> top of the previous patch into the top of the new patch and update it.
> If Subversion could make that step easier, that would help.
>
> At present the prototype 'svn shelve' accepts a log message with '-m'
> or'-F' but doesn't provide an easy way to invoke an editor, assuming you
> don't want a log message if you don't specify one. One way to solve that
> would be by adopting the same convention as 'commit': pop up an editor
> by default and allow quitting with an empty message if desired. Another
> way could be a command-line option if we think no-log-message use cases
> should be the default. But that's a UI detail that we can leave till later.
>
> The Checkpoint feature could add the copy-and-modify facility for the
> log message.

Yes, maybe we'll need to have some grouping structure / namespacing in
the shelves for this. A "rack" or something :-). The rack carries a
name ("savepoints", "feature A"); a single shelf in a rack is just
'svn shelve --rack "feature A"'; If I add more shelves to a rack, they
get numbered.

svn shelve --keep-local --rack savepoints -> shelf "savepoints-0" (or
just "savepoints"?)
svn shelve --keep-local --rack savepoints -> shelf "savepoints-1"
svn revert
svn unshelve savepoints-0

Just thinking out loud here ...

>
>> == Checkpointing ==
>>
>> I think only "option 3" looks viable / interesting in the long run
>> (option 1, storing patches, looks a lot like simple shelving, so not
>> much more value imho). Or even a completely different approach which
>> is implemented in working copy storage (option 4? I haven't thought
>> this through, but I'm afraid of the disk space requirements, and init
>> I/O cost, of option 3 for large multi-GB working copies).
>>
>> It's very hard for me to not think of checkpoints as local branches of
>> some sort. And my users will immediately want to use them in that way.
>> In all honesty, I think we should aim for powerful local branches (and
>> think of an architecture / design / ui for that), and then think about
>> how we can perhaps start with something simpler and more limited as a
>> first step, but which goes in that direction. I.e. a more holistic
>> design around "local branches", "local commits", "checkpointing".
>> What's the big picture?
>
> Good question. While I am continuing to think about ways in which some
> larger scope towards "local branching" could play out, I don't currently
> see that ever working well for Subversion, at least not the current
> generation 1.x.
>
> Given your comments about shelving being almost enough, I now wonder if
> we should design simple checkpointing as a simple extension to shelving,
> whereby we:
>
>   * automatically increment the patch suffix number,
> if the latest shelved patch was made with '--keep-local';
>
>   * also transfer the log message transfer from the previous numbered
> patch and offer it for editing;
>
>   * offer a UI shortcut to revert to the previous version in the series;
>
> and any other similar things.
>
> WDYT?

Yes! I think that would be enough for a good release 1.11, with nice
"shelving" and basic checkpointing (actually mainly UI lipstick with
options / conventions on the "shelve / unshelve" commands) on top of
it. Leaves the door open for more fundamental things in the future ...

>> - After 

Re: Shelving / Checkpointing thoughts

[Julian Foad]

Johan, thank you very much for these considered thoughts!

Johan Corveleyn wrote:
> First of all: thanks for working on shelving and checkpointing,
> Julian. These could become very important and big features. Daunting
> to take them on, but it's good to see someone having a go at it.
> 
> I've tried to read through all the docs and recent mail threads. Below
> are a couple of thoughts.
> 
> TL;DR: I'd suggest first going for a very good Shelving
> implementation, and not rushing for Checkpointing. Shelving can
> already solve some checkpointing use-cases (see below), and
> Checkpointing as a full-blown feature needs very careful thought (even
> if limited, it's risky to paint ourselves into a corner), and will
> quickly raise expectations of "local commits" or "local branching"
> (which still seem far away).

Agreed.

> == Shelving ==
> 
> Looks great so far. Of course a lot of challenges remain for all the
> cases which are not yet (correctly) covered by 'svn diff' and 'svn
> patch' (property changes, tree operations, binary files, unresolved
> conflicts, etc.). Attaching a log message to a shelf is key, and the
> association with changelists looks like a good approach.
> 
> - Shelves should (eventually) support directories as "versioned
> items". Changelists currently don't support directories.

I agree with all this too. A log message is supported in the prototype
(but not yet integrated with changelists, as discussed in another thread).

> - Suggestion: 'svn shelve --keep', to create a shelf (patch) in the
> "shelf storage" but not revert it. That would enable some crude way of
> checkpointing your work, through simple patches (which can be applied
> later by fuzzy patching, or ...):
> 
> work on feature A
> svn shelve --keep --name "feature A"
> continue work on feature A
> turns out badly, lets go back
> svn revert; svn unshelve "feature A"

Yes; implemented now in r1806168.

In fact this usage aligns very well with how I use patch files myself. I
often save a series of patches to checkpoint my development of a
feature, with names like 'foo-1.patch' ... 'foo-N.patch'.

One thing I like to do in that case is to copy the log message from the
top of the previous patch into the top of the new patch and update it.
If Subversion could make that step easier, that would help.

At present the prototype 'svn shelve' accepts a log message with '-m'
or'-F' but doesn't provide an easy way to invoke an editor, assuming you
don't want a log message if you don't specify one. One way to solve that
would be by adopting the same convention as 'commit': pop up an editor
by default and allow quitting with an empty message if desired. Another
way could be a command-line option if we think no-log-message use cases
should be the default. But that's a UI detail that we can leave till later.

The Checkpoint feature could add the copy-and-modify facility for the
log message.


> == Checkpointing ==
> 
> I think only "option 3" looks viable / interesting in the long run
> (option 1, storing patches, looks a lot like simple shelving, so not
> much more value imho). Or even a completely different approach which
> is implemented in working copy storage (option 4? I haven't thought
> this through, but I'm afraid of the disk space requirements, and init
> I/O cost, of option 3 for large multi-GB working copies).
> 
> It's very hard for me to not think of checkpoints as local branches of
> some sort. And my users will immediately want to use them in that way.
> In all honesty, I think we should aim for powerful local branches (and
> think of an architecture / design / ui for that), and then think about
> how we can perhaps start with something simpler and more limited as a
> first step, but which goes in that direction. I.e. a more holistic
> design around "local branches", "local commits", "checkpointing".
> What's the big picture?

Good question. While I am continuing to think about ways in which some
larger scope towards "local branching" could play out, I don't currently
see that ever working well for Subversion, at least not the current
generation 1.x.

Given your comments about shelving being almost enough, I now wonder if
we should design simple checkpointing as a simple extension to shelving,
whereby we:

  * automatically increment the patch suffix number,
if the latest shelved patch was made with '--keep-local';

  * also transfer the log message transfer from the previous numbered
patch and offer it for editing;

  * offer a UI shortcut to revert to the previous version in the series;

and any other similar things.

WDYT?


> - After reading the "Terminology" section of
> https://wiki.apache.org/subversion/SavePoints, I agree with that
> document that "Savepoints" might be a better name. But don't want to
> bikeshed over it ...

Could be. (Also, 'checkpoint' is similar to 'checkout' which makes
command-line completion not work well, and the possible abbreviation
'cp' clashes with 'copy'.)

> - 

How should shelving interact with checkpoints?

[Julian Foad]

I have been thinking about, and written [1], the following:


How should shelving interact with checkpoints?

In other words, what do we expect to happen if we start working, make
some checkpoints, and then shelve "the work" to work on something else?
Do we usually want to shelve the whole series of checkpoints, or just
the latest (uncheckpointed) working change and start new work against
the latest checkpoint?

If the checkpointing design shows WC state relative to the latest
checkpoint (as in option 3), then shelving should:

  * Shelve the (uncheckpointed) working change, leaving all checkpoints
as they are.

This returns the WC to showing an unmodified state relative to the
latest checkpoint.

If the checkpointing design shows WC state relative to the original base
(as in option 1), and checkpoints exist, then shelving should:

  * Shelve the working change and restore the latest checkpoint.

In this case, since the working change is based on the original base,
the resulting patch includes the changes made in all the earlier
checkpoints. The effect is like making a more permanent kind of
checkpoint that is tagged with a name and will survive rollbacks.

Other possibilities that I thought about and rejected were:

  * Warn and/or require user input; then discard (squash) checkpoints
and shelve a single patch.

  * Save the whole series of (N) checkpoints and working state, e.g. as
'foo-1.patch' ... 'foo-N.patch', 'foo.patch'. In that case, unshelving
while some (M) checkpoints already exist would perhaps add 'foo-1.patch'
as checkpoint(M+1) ... 'foo-N.patch' as checkpoint(M+N), 'foo.patch' as
the new working state. But then what? Too messy.


That seems good to me. Any other thoughts?

- Julian


[1]
https://docs.google.com/document/d/1PVgw0BdPF7v67oxIK7B_Yjmr3p28ojabP5N1PfZTsHk/edit#heading=h.ilq1gee589ue

Shelving / Checkpointing thoughts

[Johan Corveleyn]

First of all: thanks for working on shelving and checkpointing,
Julian. These could become very important and big features. Daunting
to take them on, but it's good to see someone having a go at it.

I've tried to read through all the docs and recent mail threads. Below
are a couple of thoughts.

TL;DR: I'd suggest first going for a very good Shelving
implementation, and not rushing for Checkpointing. Shelving can
already solve some checkpointing use-cases (see below), and
Checkpointing as a full-blown feature needs very careful thought (even
if limited, it's risky to paint ourselves into a corner), and will
quickly raise expectations of "local commits" or "local branching"
(which still seem far away).


== Shelving ==

Looks great so far. Of course a lot of challenges remain for all the
cases which are not yet (correctly) covered by 'svn diff' and 'svn
patch' (property changes, tree operations, binary files, unresolved
conflicts, etc.). Attaching a log message to a shelf is key, and the
association with changelists looks like a good approach.

- Shelves should (eventually) support directories as "versioned
items". Changelists currently don't support directories.

- Suggestion: 'svn shelve --keep', to create a shelf (patch) in the
"shelf storage" but not revert it. That would enable some crude way of
checkpointing your work, through simple patches (which can be applied
later by fuzzy patching, or ...):

work on feature A
svn shelve --keep --name "feature A"
continue work on feature A
turns out badly, lets go back
svn revert; svn unshelve "feature A"


== Checkpointing ==

I think only "option 3" looks viable / interesting in the long run
(option 1, storing patches, looks a lot like simple shelving, so not
much more value imho). Or even a completely different approach which
is implemented in working copy storage (option 4? I haven't thought
this through, but I'm afraid of the disk space requirements, and init
I/O cost, of option 3 for large multi-GB working copies).

It's very hard for me to not think of checkpoints as local branches of
some sort. And my users will immediately want to use them in that way.
In all honesty, I think we should aim for powerful local branches (and
think of an architecture / design / ui for that), and then think about
how we can perhaps start with something simpler and more limited as a
first step, but which goes in that direction. I.e. a more holistic
design around "local branches", "local commits", "checkpointing".
What's the big picture?

- After reading the "Terminology" section of
https://wiki.apache.org/subversion/SavePoints, I agree with that
document that "Savepoints" might be a better name. But don't want to
bikeshed over it ...

- In a prior mail-thread between you and Nathan Hartman the "rebasing"
problem was mentioned. In [1] you said:

> Performing an 'update' with a checkpoint series is a bigger ask than it
> might at first seem. In effect, it requires rebasing the series of
> checkpoints on the new base, which gets ugly because of the need to
> handle conflicts (which is ugly enough already in the existing
> single-depth WC).

However, that seems to only sane way to go for me. Rebasing the
checkpoints one by one, and resolving conflicts along the way. Don't
know how you'd do that though, if the checkpoints are revisions 1, 2,
3 in a local repository (with immutable history etc). This is really
something where the "local repository" technique breaks down IMHO. In
contract with DVCS's, in SVN history is immutable. But mutability is
quite important for local branches / commits.

In that sense, a series of patches is more flexible: you can still
apply them with fuzz even if applying them to a different BASE state,
and often that will "just work" (and conflicts "just" need to be
resolved).

In that light, Nathan's latest post in that thread ([2]) was also
interesting, where he suggested to store the BASE together with the
WORK for every checkpoint. I'm not sure if that's the way to go (maybe
you only need a "description of BASE", not the pristines etc), but it
made me realize: don't expect checkpoints to work for "undo 'svn
update'" if you can't restore the original BASE tree (down to the
exact mixed-revisionness).

[1] https://svn.haxx.se/dev/archive-2017-07/0302.shtml
[2] https://svn.haxx.se/dev/archive-2017-08/0064.shtml

-- 
Johan

Re: 'svn list --search' feature complete?

[Julian Foad]

Johan Corveleyn wrote:
> On Mon, Aug 21, 2017 at 8:14 PM, Daniel Shahaf  
> wrote:
>> Julian Foad wrote on Mon, 21 Aug 2017 14:09 +0100:
>>> Also the search term seems to be required to match the entire path, so I
>>> need to write "co*" rather than just "co" to match "configure.ac". That
>>> is inconsistent with "svn log --search" which looks for the pattern as a
>>> substring even when matching paths. I think differences such as this are
>>> bad.
>>
>> From the peanut gallery, I'm not sure about this.  When one greps log
>> messages, I think it is far more common to search for a _substring_ of
>> the log message than to ask for all revisions whose log messages are
>> strcmp()-equal to a particular string.
>>
>> On the other hand, with paths, it's plausible that one would know the
>> exact basename being sought, and wouldn't be interested in filenames
>> that contain that basename as a substring.
>>
>> Further, with glob patterns, if the CLI implements exact matching, users
>> can achieve either exact matching (default) or substring matching (by
>> adding * before and after the pattern), whereas if the client implemented
>> substring matching, there would be no way for users to request exact
>> matching: fnmatch() doesn't have an equivalent of regexes' ^ and $
>> anchors.
>>
>> But I haven't thought much about this.  There may well be logic to the
>> current behaviour that I'm overlooking.
> 
> FWIW, I agree with Daniel's argumentation here about doing exact
> matching with --list (and requiring *'s for substring matching).

So do I.

I also think, for consistency and for the reasons above, we should
change 'svn log --search' so that when searching in the changed paths it
works this way too. Then 'foo' and '*foo*' would have different meanings
when searching in the changed paths. When searching in fields such as
the log message, it would keep the existing behaviour whereby both of
those search terms would mean a substring search.

That would be a backwards-incompatible change, but to a non-prominent
and undocumented detail of the UI, and for good reasons. Does that make
sense?

- Julian

Re: Tip of the day: Cheatsheet for common cmdline client operations

[Johan Corveleyn]

[ cc -= svnbook-dev, as I guess this will not become book-material.
More below ... ]

On Thu, Jul 13, 2017 at 6:34 PM, C. Michael Pilato
 wrote:
> On Thu, Jul 13, 2017 at 11:21 AM, Daniel Shahaf 
> wrote:
>>
>> [moving from users@ ]
>>
>> Daniel Shahaf wrote on Fri, 30 Jun 2017 09:15 +:
>> > I just ran into the following cheatsheet:
>> >
>> > http://www.chim.unifi.it/~signo/did/etc/subversion/neat.html
>> >
>> > It covers the normal multi-user workflow, branching, etc..
>> >
>> > (Kudos to the author, Giorgio Signorini, not to me.)
>>
>> As some of you know, the author gave us permission to incorporate that
>> cheatsheet into the official documentation.  Any volunteers to start the
>> process?
>>
>> Cheers,
>>
>> Daniel
>>
> With respect to the author and the work he's done, I'm not really interested
> in us maintaining yet another collection of the same information already
> covered -- in some cases multiple times, when you factor in the reference
> sections -- by the book.  At best this cheatsheet would be an appendix.  I'm
> happy to link to the cheatsheet from from the book website, though.
>
> -- Mike

FWIW, I think this cheatsheet is quite good and valuable (though I
agree there is a lot of overlap with existing documentation),
especially for newcomers. Just to have a good summary / reminder of
common things you'll encounter.

We have our own quickstart page: http://subversion.apache.org/quick-start

How about:

- Putting a link at the bottom of that page (there is already a link
to the quickstart section of the book), linking to the original
webpage of the author.

or

- Incorporating (some of) the content of that cheatsheet directly on
http://subversion.apache.org/quick-start, so it's right there in front
of you ...


The latter option would be my personal preference (putting my user /
admin hat on) -- I like having short info right in front of me in the
right place -- but obviously imposes some amount of doc-maintenance
work on "us".

Maybe someone on this list would be willing to take this doc-task
(migrating the current content, and keep an eye on keeping it up to
date)?

-- 
Johan

Re: Shelving and Checkpointing support log messages

[Stefan Sperling]

On Fri, Aug 25, 2017 at 10:08:04AM +0100, Julian Foad wrote:
> Example workflow:
> 
> 1. There exists a shelved patch 'foo' with a verbose log message
> 2. 'svn unshelve foo'

There can only be one changelist per file.
What if at this point there is another changelist 'bar' which includes a
file modified by the shelved patch? Would 'unshelve' error out or change
that file's changelist to 'foo'?

I suppose it should error out and ask the user to commit or remove
the conflicting changelist first.

> 3. 'make check'
> 4. 'svn commit --changelist foo'
> 
> In this example, the unshelve creates & populates changelist 'foo', and
> the commit takes its log msg from changelist foo.
> 
> To me this seems very appealing.

Yes, indeed!

Re: Shelving and Checkpointing support log messages

[Johan Corveleyn]

On Fri, Aug 25, 2017 at 11:08 AM, Julian Foad  wrote:
> Daniel Shahaf wrote:
>> Julian Foad wrote on Thu, 24 Aug 2017 19:46 +0100:
>>> One rough edge is that "unshelving" a patch or finishing/squashing a
>>> checkpoint series simply discards the log message(s). [...]
>>
>> [...] to enable the "long log message" use-case, there should be an easy
>> "upgrade path" for the shelf's log message to become a commit log message;
>> optionally with editing first.
>>
>> Example workflow:
>>
>> 1. There exists a shelf with a verbose log message
>> 2. Unshelve that patch
>> 3. 'make check'
>> 4. 'svn commit'
>>
>> In this workflow, it would be nice for $EDITOR to be pre-filled with the
>> shelf's log message.
>>
>> The catch is, what happens if the patch is reverted [...] step #4
>> doesn't know whether it's committing the patch that had been
>> unshelved, or something else [...]
>
> A neat solution could be to integrate Shelving with Changelists in such
> a way that Unshelving means converting a patch to a changelist, and a
> changelist grows support for storing a log message.
>
> This would be a two-way integration: shelving a changelist would then
> mean converting the changelist to a patch and reverting the changelist
> from the WC.
>
> This would be more in line with how, for example, Perforce defines a
> changelist: there it is a change, whether committed or shelved or in one
> or two other states; and includes a log message and some other metadata.
>
> (In fact, right from when we introduced "changelists", I have wanted
> them to support storing a log message, without even thinking about the
> possibility of shelving.)
>
> Example workflow:
>
> 1. There exists a shelved patch 'foo' with a verbose log message
> 2. 'svn unshelve foo'
> 3. 'make check'
> 4. 'svn commit --changelist foo'
>
> In this example, the unshelve creates & populates changelist 'foo', and
> the commit takes its log msg from changelist foo.
>
> To me this seems very appealing.
>
> I outlined this idea in the section "Integrate Shelving with
> Changelists" in the "Shelving-Checkpointing Dev" doc:
> https://docs.google.com/document/d/1PVgw0BdPF7v67oxIK7B_Yjmr3p28ojabP5N1PfZTsHk/edit#heading=h.ay5h5pfumpv8
>
> What do you think?

+1, that looks like a good approach.

In the google doc you mentioned some issues with this:

* a changelist can include unmodified paths
  should a shelved patch also include unmodified paths?

-> I guess that would be the best solution for this. The shelve should
keep (in metadata) a list of paths that were part of the changelist.

* a path can appear in multiple shelves, but only in one changelist
  what to do when unshelving if a path clashes?

-> Hmm, that's a hard one. No idea for now, except perhaps extending
changelist to make it possible for paths to be part of multiple
changelists too.

FWIW: IntelliJ IDEA gives you a warning when you start editing a file
that's part of some changelist, while your default changelist is set
to another changelist ("File from non-active changelist is modified").
The user can "Move changes", "Switch changelist" or "Ignore" (though
it's not clear to me what "Ignore" does ... the file doesn't become
part of the "active changelist" suddenly, or of both, so this seems
equal to "Move changes").

* any extensions should apply uniformly to both shelving and changelists
  if a shelved change supports a log msg, a changelist should support
a log msg too; this would be a good enhancement for changelists anyway
even without shelving

-> Yup, sounds good to me :)


I see one more important caveat: changelists currently don't support
directories as members.

I have some more thoughts on shelves and checkpoints, but I'll put
them in a separate thread.

-- 
Johan

Re: Shelving and Checkpointing support log messages

[Julian Foad]

Daniel Shahaf wrote:
> Julian Foad wrote on Thu, 24 Aug 2017 19:46 +0100:
>> One rough edge is that "unshelving" a patch or finishing/squashing a
>> checkpoint series simply discards the log message(s). [...]
> 
> [...] to enable the "long log message" use-case, there should be an easy
> "upgrade path" for the shelf's log message to become a commit log message;
> optionally with editing first.
> 
> Example workflow:
> 
> 1. There exists a shelf with a verbose log message
> 2. Unshelve that patch
> 3. 'make check'
> 4. 'svn commit'
> 
> In this workflow, it would be nice for $EDITOR to be pre-filled with the
> shelf's log message.
> 
> The catch is, what happens if the patch is reverted [...] step #4
> doesn't know whether it's committing the patch that had been
> unshelved, or something else [...]

A neat solution could be to integrate Shelving with Changelists in such
a way that Unshelving means converting a patch to a changelist, and a
changelist grows support for storing a log message.

This would be a two-way integration: shelving a changelist would then
mean converting the changelist to a patch and reverting the changelist
from the WC.

This would be more in line with how, for example, Perforce defines a
changelist: there it is a change, whether committed or shelved or in one
or two other states; and includes a log message and some other metadata.

(In fact, right from when we introduced "changelists", I have wanted
them to support storing a log message, without even thinking about the
possibility of shelving.)

Example workflow:

1. There exists a shelved patch 'foo' with a verbose log message
2. 'svn unshelve foo'
3. 'make check'
4. 'svn commit --changelist foo'

In this example, the unshelve creates & populates changelist 'foo', and
the commit takes its log msg from changelist foo.

To me this seems very appealing.

I outlined this idea in the section "Integrate Shelving with
Changelists" in the "Shelving-Checkpointing Dev" doc:
https://docs.google.com/document/d/1PVgw0BdPF7v67oxIK7B_Yjmr3p28ojabP5N1PfZTsHk/edit#heading=h.ay5h5pfumpv8

What do you think?

- Julian