Re: [RFC] Shelving and Checkpointing

[Julian Foad]

Hi Nathan. Thanks for your thoughts.

Nathan Hartman wrote:
> Julian Foad wrote:
> Diffing checkpoints is quite hard (well, needs a patch-arithmetic
> library) in this design. Even just determining whether two patches
> are "identical". That might be a good reason for me to try the
> alternative design "store checkpoints in a local repository"
> ("option 3") next.
> 
> My thinking is that options 1 or 2 will eventually grow into a redundant
> implementation of a VCS within svn to handle various revisions of the
> working copy. That will most likely be faster to get off the ground, but
> results in redundant code, or refactoring existing code to do the patch
> arithmetic, etc. Option 3 requires reinventing the client outright but
> gives more flexibility in the long run.

Yes, this is a serious consideration. Re-using the repository design
gives not only flexibility but the reliability and maintainability that
comes from sharing a single implementation of the functionality (DRY). I
am going to look further into this approach next.

> I was thinking that svn switch needs to be handled, or chaos would ensue
> if someone does some work, makes some shelves and checkpoints, and then
> does svn switch to a different path. A solution could be to remember the
> checkout path and revision with each shelve/checkpoint. If the user
> performs svn switch and later wants to reinstate a shelved working copy,
> the client will notice that it can't be unshelved until the workspace is
> switched back to the correct path and error out with a descriptive
> message. The user will then svn switch to the correct branch and reissue
> the unshelve command.

It's not extremely unusual to want to apply some changes onto a
different branch, but I agree it's probably best to record the base path
used, and for the 'svn' CLI to assume this might be a mistake and so
warn or error and require 'force' to unshelve to a different path. (A
more interactive client could offer a better UX.)

I have now listed this suggestion under 'extensions', and promoted
'recording the base state' into the 'good usability' column of the table
rather than 'no'.

> The normal rules apply, which mean that if there
> are uncommitted changes in the working copy, the user will have to
> either commit them or shelve them before performing the svn switch.
> [...]

Actually, 'switch' normally allows uncommitted changes in just the same
way that 'update' does.

- Julian

Re: [RFC] Shelving and Checkpointing

[Nathan Hartman]

On Fri, Jul 14, 2017 at 7:56 AM, Julian Foad  wrote:
> Diffing checkpoints is quite hard (well, needs a patch-arithmetic library) in 
> this design. Even just determining whether two patches are "identical". That 
> might be a good reason for me to try the alternative design "store 
> checkpoints in a local repository" ("option 3") next.

My thinking is that options 1 or 2 will eventually grow into a redundant 
implementation of a VCS within svn to handle various revisions of the working 
copy. That will most likely be faster to get off the ground, but results in 
redundant code, or refactoring existing code to do the patch arithmetic, etc. 
Option 3 requires reinventing the client outright but gives more flexibility in 
the long run.

I was thinking that svn switch needs to be handled, or chaos would ensue if 
someone does some work, makes some shelves and checkpoints, and then does svn 
switch to a different path. A solution could be to remember the checkout path 
and revision with each shelve/checkpoint. If the user performs svn switch and 
later wants to reinstate a shelved working copy, the client will notice that it 
can't be unshelved until the workspace is switched back to the correct path and 
error out with a descriptive message. The user will then svn switch to the 
correct branch and reissue the unshelve command. The normal rules apply, which 
mean that if there are uncommitted changes in the working copy, the user will 
have to either commit them or shelve them before performing the svn switch. A 
--force option could allow the user to unshelve the work into the wrong path, 
possibly with conflicts (but maybe this is intentional).

>> 
>>> References:
>>>[1] Shelving-Checkpointing Dev doc. (J Foad)
>>> https://docs.google.com/document/d/1PVgw0BdPF7v67oxIK7B_Yjmr3p28ojabP5N1PfZTsHk/edit#
>>>  

Re: [RFC] Shelving and Checkpointing

[Julian Foad]

I committed an initial prototype for checkpointing on the 
"shelve-checkpoint" branch.  http://svn.apache.org/r1801942


This prototype is of the "store snapshot patches" ("option 1") design.

Here is the help output:
[[[
$ svn checkpoint --help
checkpoint: Checkpoint the local changes.
usage: 1. checkpoint save
   2. checkpoint revert
   3. checkpoint rollback NUMBER
   4. checkpoint list|--list

  1. Save the working state as a new checkpoint.
  2. Revert the working state to the current checkpoint.
  3. Roll back the working state to checkpoint NUMBER.
  4. List all checkpoints. A synonym for 'svn checkpoints'.
[...]
]]]

Please try it and let me know your thoughts on the directions we should 
take it in, if you have a chance.


I think the next most important things it needs are:

  * recognize when there are no further changes relative to the last 
checkpoint, and do nothing in that case;

  * see the status and diff of the WC against the current checkpoint;
  * see a summary and diff of each checkpoint against the previous one.

Diffing checkpoints is quite hard (well, needs a patch-arithmetic 
library) in this design. Even just determining whether two patches are 
"identical". That might be a good reason for me to try the alternative 
design "store checkpoints in a local repository" ("option 3") next.


- Julian


On 12/07/17, Julian Foad wrote:
I committed an initial prototype for shelving on the "shelve-checkpoint" 
branch.


Here is the help output:
[[[
$ svn shelve --help
shelve: Shelve changes.
usage: 1. shelve NAME PATH...
2. shelve --delete NAME
3. shelve --list

   1. Shelve as NAME the local changes in the given PATHs.
   2. Delete the shelved patch NAME.
   3. List shelved patches.
[...]

$ svn unshelve --help
unshelve: Unshelve changes.
usage: unshelve [--keep] NAME

]]]

Here is an example session:
[[[
$ # create and commit some files

$ mkdir doc; echo hello > doc/hello.txt; echo config > config

$ svn add --force .; svn ci -q -m ""
A config
A doc
A doc/hello.txt

$ # start making some changes to docs

$ echo new > doc/new.txt; svn add doc/new.txt
A doc/new.txt

$ echo more >> config

$ svn st
M   config
A   doc/new.txt

$ # shelve these documentation changes

$ svn shelve docs .
U config
D doc/new.txt
shelved 'docs'

$ svn st

$ svn shelve --list
docs.patch

$ # work on a quick fix and commit it

$ echo Hello > doc/hello.txt

$ svn ci -q -m "Fix capitalization."

$ # unshelve to continue work on docs

$ svn unshelve docs
U config
A doc/new.txt
unshelved 'docs'

$ svn st
M   config
A   doc/new.txt

$ svn shelve --list

]]]

If you have a chance to try it yourself, I'd love to hear your first 
impressions.


- Julian



References:
[1] Shelving-Checkpointing Dev doc. (J Foad)
https://docs.google.com/document/d/1PVgw0BdPF7v67oxIK7B_Yjmr3p28ojabP5N1PfZTsHk/edit# 

Re: [RFC] Shelving and Checkpointing

[Julian Foad]

Bert Huijben wrote:

Julian Foad wrote:

Mark Phippard wrote:

[...]  does patch just create conflicts that you
resolve like anything else?


It is 'svn patch' -- so it raises conflicts.


Svn patch creates reject files. It doesn't create conflicts (yet).


Gosh, so it does! How primitive! :-)

I will expand on that aspect.

- Julian

RE: [RFC] Shelving and Checkpointing

[Bert Huijben]

> -Original Message-
> From: Julian Foad [mailto:julianf...@apache.org]
> Sent: woensdag 12 juli 2017 16:38
> To: Mark Phippard <markp...@gmail.com>
> Cc: Subversion Developers <dev@subversion.apache.org>
> Subject: Re: [RFC] Shelving and Checkpointing
> 
> Mark Phippard wrote:
> > Nice to see you have gotten this far.
> >
> > I am not even sure how this behaves with git stash but what happens if
> > the patch cannot be applied cleanly?  Is the path to "fixing things"
> > relatively clear?  Like does patch just create conflicts that you
> > resolve like anything else?
> 
> It is 'svn patch' -- so it raises conflicts.

Svn patch creates reject files. It doesn't create conflicts (yet).

Bert

Re: [RFC] Shelving and Checkpointing

[Julian Foad]

Mark Phippard wrote:

Nice to see you have gotten this far.

I am not even sure how this behaves with git stash but what happens if 
the patch cannot be applied cleanly?  Is the path to "fixing things" 
relatively clear?  Like does patch just create conflicts that you 
resolve like anything else?


It is 'svn patch' -- so it raises conflicts.

- Julian

Re: [RFC] Shelving and Checkpointing

[Julian Foad]

Johan Corveleyn wrote:

I've quickly scanned your google docs, but have to go through them in
some more detail. I'll try to dig into them and give some feedback if
I can.


Thanks you Johan! I look forward to hearing your comments when you have 
a chance.



One thing that crossed my mind: a nice additional feature that
probably could be built on top of shelving would be "partial commit"
(committing only some hunks out of a modified file). I've recently
looked up past discussions about this, in response to a post on
users@. See my response with a lot of references here:

 https://svn.haxx.se/users/archive-2017-06/0004.shtml


Yes, I would find that useful myself. A good future extension.

That's what I mean by the 'patch hunks' feature listed in the 'Role 
Models' table in the 'UI' doc which is linked from the 'Dev' doc. Bazaar 
'bzr shelve' does that interactively by default, and both Mercurial and 
Git do it with 'hg shelve --interactive' and 'git stash --patch'.


- Julian

Re: [RFC] Shelving and Checkpointing

[Mark Phippard]

On Wed, Jul 12, 2017 at 4:18 AM, Julian Foad  wrote:

> I committed an initial prototype for shelving on the "shelve-checkpoint"
> branch.
>
> Here is the help output:
> [[[
> $ svn shelve --help
> shelve: Shelve changes.
> usage: 1. shelve NAME PATH...
>2. shelve --delete NAME
>3. shelve --list
>
>   1. Shelve as NAME the local changes in the given PATHs.
>   2. Delete the shelved patch NAME.
>   3. List shelved patches.
> [...]
>
> $ svn unshelve --help
> unshelve: Unshelve changes.
> usage: unshelve [--keep] NAME
>
> ]]]
>
> Here is an example session:
> [[[
> $ # create and commit some files
>
> $ mkdir doc; echo hello > doc/hello.txt; echo config > config
>
> $ svn add --force .; svn ci -q -m ""
> A config
> A doc
> A doc/hello.txt
>
> $ # start making some changes to docs
>
> $ echo new > doc/new.txt; svn add doc/new.txt
> A doc/new.txt
>
> $ echo more >> config
>
> $ svn st
> M   config
> A   doc/new.txt
>
> $ # shelve these documentation changes
>
> $ svn shelve docs .
> U config
> D doc/new.txt
> shelved 'docs'
>
> $ svn st
>
> $ svn shelve --list
> docs.patch
>
> $ # work on a quick fix and commit it
>
> $ echo Hello > doc/hello.txt
>
> $ svn ci -q -m "Fix capitalization."
>
> $ # unshelve to continue work on docs
>
> $ svn unshelve docs
> U config
> A doc/new.txt
> unshelved 'docs'
>
> $ svn st
> M   config
> A   doc/new.txt
>
> $ svn shelve --list
>
> ]]]
>
> If you have a chance to try it yourself, I'd love to hear your first
> impressions.
>
>
>

Nice to see you have gotten this far.

I am not even sure how this behaves with git stash but what happens if the
patch cannot be applied cleanly?  Is the path to "fixing things" relatively
clear?  Like does patch just create conflicts that you resolve like
anything else?

-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

Re: [RFC] Shelving and Checkpointing

[Johan Corveleyn]

On Mon, Jul 10, 2017 at 2:59 PM, Julian Foad  wrote:
> Dear Subversion Developers,
>
> I am delighted to announce that I am working with Assembla to develop
> shelving and checkpointing functionality in Subversion. These have
> been on the wish list for many years, and are becoming ever more in
> demand since the popularity of git is making more users appreciate the
> benefits of local operations.
>
> Based on our many discussions in the past, along with a fresh look at
> how other VCS's implement these features, I have written down the
> requirements and started working on a design in the document
> Shelving-Checkpointing Dev [1]. I will follow up with an HTML email
> containing a copy of today's version. I invite you to leave comments
> directly in Google Docs, or send them by reply here, as you prefer.
>
> I will be the project lead on this within Assembla. I will regularly
> demonstrate small increments of functionality, which will also act as
> calls for feedback. We want to keep the scope quite small to get it
> done as quickly as possible.
>
> Please read through and if you are able to contribute with any
> suggestions or practical help, that would be wonderful. Thanks!
>
> - Julian
>
>
> References:
>[1] Shelving-Checkpointing Dev doc. (J Foad)
> https://docs.google.com/document/d/1PVgw0BdPF7v67oxIK7B_Yjmr3p28ojabP5N1PfZTsHk/edit#

Hi Julian,

That's great news! I hope this will give Subversion some much needed
extra momentum (together with the tree conflict resolution
improvements of 1.10). Indeed, these "local / offline" features have
been on many people's wish list for a long time.

I've quickly scanned your google docs, but have to go through them in
some more detail. I'll try to dig into them and give some feedback if
I can.

One thing that crossed my mind: a nice additional feature that
probably could be built on top of shelving would be "partial commit"
(committing only some hunks out of a modified file). I've recently
looked up past discussions about this, in response to a post on
users@. See my response with a lot of references here:

https://svn.haxx.se/users/archive-2017-06/0004.shtml
(For the "partial commit" feature, start reading below "Is there an
option to inspect each file further line-by-line for lines that have
changed to either be selected or excluded from the commit?")

As I explained in that post, TortoiseSVN has implemented this in some
way with what they call "restore after commit" [1].

So I imagine we could do the same by (1) making the user select some
lines he wants to commit; (2) shelve the modifications that were not
selected; (3) commit; (4) unshelve. We even have a command line UI
(sort of) for selecting lines out of a diff, namely the UI stsp
created for handling text conflicts interactively.

Just a thought ... Of course fleshing out the fundamentals of the
Shelving and Checkpointing features in the first place will be most
important.

[1] 
https://tortoisesvn.net/docs/nightly/TortoiseSVN_en/tsvn-dug-commit.html#tsvn-dug-commit-restore

-- 
Johan

Re: [RFC] Shelving and Checkpointing

[Julian Foad]

I committed an initial prototype for shelving on the "shelve-checkpoint" 
branch.


Here is the help output:
[[[
$ svn shelve --help
shelve: Shelve changes.
usage: 1. shelve NAME PATH...
   2. shelve --delete NAME
   3. shelve --list

  1. Shelve as NAME the local changes in the given PATHs.
  2. Delete the shelved patch NAME.
  3. List shelved patches.
[...]

$ svn unshelve --help
unshelve: Unshelve changes.
usage: unshelve [--keep] NAME

]]]

Here is an example session:
[[[
$ # create and commit some files

$ mkdir doc; echo hello > doc/hello.txt; echo config > config

$ svn add --force .; svn ci -q -m ""
A config
A doc
A doc/hello.txt

$ # start making some changes to docs

$ echo new > doc/new.txt; svn add doc/new.txt
A doc/new.txt

$ echo more >> config

$ svn st
M   config
A   doc/new.txt

$ # shelve these documentation changes

$ svn shelve docs .
U config
D doc/new.txt
shelved 'docs'

$ svn st

$ svn shelve --list
docs.patch

$ # work on a quick fix and commit it

$ echo Hello > doc/hello.txt

$ svn ci -q -m "Fix capitalization."

$ # unshelve to continue work on docs

$ svn unshelve docs
U config
A doc/new.txt
unshelved 'docs'

$ svn st
M   config
A   doc/new.txt

$ svn shelve --list

]]]

If you have a chance to try it yourself, I'd love to hear your first 
impressions.


- Julian



References:
[1] Shelving-Checkpointing Dev doc. (J Foad)
https://docs.google.com/document/d/1PVgw0BdPF7v67oxIK7B_Yjmr3p28ojabP5N1PfZTsHk/edit#

Re: [RFC] Shelving and Checkpointing

[Julian Foad]

Daniel Shahaf wrote:

Julian Foad wrote on Tue, 11 Jul 2017 21:53 +0100:

2. What I was thinking there is to rewrite as much of our libs as needed
to implement deeply integrated local branching in Svn client. The full
extent of what that might entail or look like is unknown.


I don't understand the distinction apparent here between 'checkpointing'
and 'local branching'.  Are these two terms not synonymous?


Sorry, I didn't mean to make a distinction. "... implement deeply 
integrated ". Such as extra trees in the WC DB, as 
envisioned in Reference 5. Svn Wiki page "Design: SavePoints" 
, for example.


- Julian

Re: [RFC] Shelving and Checkpointing

[Branko Čibej]

On 11.07.2017 10:53, Julian Foad wrote:
> Thanks for your suggestion, Nathan.
>
> Nathan Hartman wrote:
>> [...] What if, instead of just a pristine copy, it actually created a
>> private local repository. Revision 1 of this repository would be the
>> pristine copy. [...] if you type some other command instead of
>> commit, or maybe prepend the word "local" or something, the commit
>> would go into the local repository [...] Shelving, stashing, etc.,
>> all become local operations against this local souped up pristine
>> copy replacement.
>
> That is exactly what I was thinking about when I wrote "Option 3...
> Checkpoints are commits in a local repository embedded in the WC"
> towards the end of that document. My feeling is that both the amount
> of effort required just to get that far, and the explosion of further
> possibilities it would open up, make it too heavy-weight for my
> current project, but I would definitely like to explore that
> possibility further.

Ben (Reser) and I discussed this to death at one of the hackathons. It
turns out that checkpoints / local commits are, while not a
pre-requisite, definitely a logical first step on the road to stashes
and local branches. We even had a pretty good architectural model and UI
feature list written down ... I wonder where that's gone now.

The problem we encountered at the time is that the current working copy
schema is too limited to implement them effectively; it needs another
dimension of change tracking.

-- Brane

Re: [RFC] Shelving and Checkpointing

[Daniel Shahaf]

Sorry, pressed [Send] too early.

Thanks for the explanations.  More below.

Daniel Shahaf wrote on Tue, 11 Jul 2017 22:02 +:
> Julian Foad wrote on Tue, 11 Jul 2017 21:53 +0100:
> > Daniel Shahaf wrote:
> > > Julian Foad wrote:
> > >> Checkpointing
> > >>
> > >> Options:
> > >>
> > >> 1. further patch management built on a series of shelved changes
> > >> 2. local commits tightly integrated
> > >> 3. checkpoints are commits in a local repository
> > > 
> > > Can you explain these three options in more words?  AIUI #1 is
> > > "syntactic sugar to manage a patch series", like quilt(1), but I'm not
> > > sure I understand #2 and #3.
> > 
> > 2. What I was thinking there is to rewrite as much of our libs as needed 
> > to implement deeply integrated local branching in Svn client. The full 
> > extent of what that might entail or look like is unknown.

I don't understand the distinction apparent here between 'checkpointing'
and 'local branching'.  Are these two terms not synonymous?

> > 3. To store a series of checkpoints, create a temporary repos inside 
> > .svn/ and "relocate" the WC base to it. Then use it for all operations 
> > until the user comes to the point they want to commit to the real repo.

I think when wc-ng was invented, the idea was that a 'checkpoint' would
simply be another tree, alongside BASE and WORKING.  I suppose this
is option #4.

In general, we have at least five ways of representing a tree: as a
dumpstream, as an editor stream, as a FSFS revision file, as a wc.db
(+ pristine store), as a diff against an empty tree.  The 'temporary
repository' could store the changes in any of these forms.  I suppose the
question is which representation is best amenable to being replayed and to
resolving conflicts (when rebasing a checkpoint onto a younger start
revision).

Cheers,

Daniel

Re: [RFC] Shelving and Checkpointing

[Nathan Hartman]

On Tue, Jul 11, 2017 at 4:53 AM, Julian Foad  wrote:

> Nathan Hartman wrote:
>
>> [...] What if, instead of just a pristine copy, it actually created a
>> private local repository. Revision 1 of this repository would be the
>> pristine copy. [...]
>>
>
> That is exactly what I was thinking about when I wrote "Option 3...
> Checkpoints are commits in a local repository embedded in the WC" towards
> the end of that document. My feeling is that both the amount of effort
> required just to get that far, and the explosion of further possibilities
> it would open up, make it too heavy-weight for my current project, but I
> would definitely like to explore that possibility further.


The explosion of possibilities, to borrow your words, is IMO a disadvantage
of Option 3. I'm saying this despite having suggested this same idea.

I'd like to emphasize that I don't advocate turning Subversion into a DVCS.
Proponents of DVCSs seem to think that Subversion's centralized nature is a
disadvantage. I think the exact opposite. With centralized, you get Single
Source of Truth, a system that is easy to learn, easy to teach, programmers
and non programmers can use it successfully. I like that history is
immutable, never rewritten, because I think the primary responsibility is
to never lose information. You only have two places to know about: the
repository and your working copy.

What I do like is the idea of remaining centralized while having some local
capabilities. The stashing / checkpointing wishlist items are valid. I
mentioned the ability to roll back a svn update if something unexpected
happens (by doing an implied local commit of your current state as the
first step of svn update). Local feature branches could be useful for
one-off experiments, short-lived work done by one person, etc. In any case,
these features should all be optional. They'd be there if you want them,
but you never have to know about them if you're not interested.

In any event, I wouldn't go full blown DVCS where the concept of repository
vs. working copy goes away. I wouldn't turn a checkout into a full clone,
or a working copy into a repository.

Other than the addition of new features in the client, all else would
remain the same.

Why put a hidden repository in the working copy, then? The reason is that
while the client could use a different "lighter weight" method to manage
various versions of its working copy, that would reinvent the wheel. You'd
end up with a second incompatible version control system within Subversion.
If I understand correctly, the client already has server code in it,
because when you use the file:// URL scheme, the client is the server. So
why reinvent a new version control system within Subversion when it already
has everything it needs?

But here's the key: The difference between a normal repository and the
hidden one in the working copy is that the latter is a hidden
implementation detail, not something the user can access or control. For
example you can't checkout a working copy of it. Everything that concerns
that hidden repo, its creation, the directory and file layout within it...
all of that is controlled by Subversion itself. It's more of an internal
client-side filing system than an actual repository, although it uses a
repository as the underlying implementation. The user gets some new local
features, but not the dreaded "explosion of possibilities."

Kind regards,
Nathan

Re: [RFC] Shelving and Checkpointing

[Daniel Shahaf]

Julian Foad wrote on Tue, 11 Jul 2017 21:53 +0100:
> Daniel Shahaf wrote:
> > Julian Foad wrote:
> >> Checkpointing
> >>
> >> Options:
> >>
> >> 1. further patch management built on a series of shelved changes
> >> 2. local commits tightly integrated
> >> 3. checkpoints are commits in a local repository
> > 
> > Can you explain these three options in more words?  AIUI #1 is
> > "syntactic sugar to manage a patch series", like quilt(1), but I'm not
> > sure I understand #2 and #3.
> 
> 2. What I was thinking there is to rewrite as much of our libs as needed 
> to implement deeply integrated local branching in Svn client. The full 
> extent of what that might entail or look like is unknown.

I don't understand the distinction apparent here between 'checkpointing'
and 'local branching'.

> 3. To store a series of checkpoints, create a temporary repos inside 
> .svn/ and "relocate" the WC base to it. Then use it for all operations 
> until the user comes to the point they want to commit to the real repo.

I think when wc-ng was invented, the thinking was that a 'checkpoint'
would simply be another tree, alongside BASE and WORKING.  None


> 
> >> SHELVING
> >>
> >> git stashing 
> > 
> > For git, 'stashing' can also be implemented by a temporary branch («git
> > checkout -b foo && git commit -amm && git checkout master»), which
> > changes some of the values of the table.  I assume the same is true for hg.
> 
> Yes, I should note that mode of working as well. Thanks.
> 
> - Julian

Re: [RFC] Shelving and Checkpointing

[Julian Foad]

Daniel Shahaf wrote:

Julian Foad wrote:

Checkpointing

Options:

1. further patch management built on a series of shelved changes
2. local commits tightly integrated
3. checkpoints are commits in a local repository


Can you explain these three options in more words?  AIUI #1 is
"syntactic sugar to manage a patch series", like quilt(1), but I'm not
sure I understand #2 and #3.


2. What I was thinking there is to rewrite as much of our libs as needed 
to implement deeply integrated local branching in Svn client. The full 
extent of what that might entail or look like is unknown.


3. To store a series of checkpoints, create a temporary repos inside 
.svn/ and "relocate" the WC base to it. Then use it for all operations 
until the user comes to the point they want to commit to the real repo.




SHELVING

git stashing 


For git, 'stashing' can also be implemented by a temporary branch («git
checkout -b foo && git commit -amm && git checkout master»), which
changes some of the values of the table.  I assume the same is true for hg.


Yes, I should note that mode of working as well. Thanks.

- Julian

Re: [RFC] Shelving and Checkpointing

[Daniel Shahaf]

On Mon, Jul 10, 2017 at 02:03:41PM +0100, Julian Foad wrote:
> Checkpointing
> 
> Options:
> 
> 
>1.
> 
>further patch management built on a series of shelved changes
>2.
> 
>local commits tightly integrated
>3.
> 
>checkpoints are commits in a local repository

Can you explain these three options in more words?  AIUI #1 is
"syntactic sugar to manage a patch series", like quilt(1), but I'm not
sure I understand #2 and #3.

> SHELVING
> 
> git stashing 
> 

For git, 'stashing' can also be implemented by a temporary branch («git
checkout -b foo && git commit -amm && git checkout master»), which
changes some of the values of the table.  I assume the same is true for hg.

> hg shelving 
> 
> bzr shelving
> 
> 
> p4 shelving
> 
> 
> IntelliJ
> 
> 
> NetBeans
> 

Unfortunately the text/plain rendering of that table is quite lossy.
The part I quoted are the column headings; the row headings are
"general" "stored where?" "stored how?" "changelist integration" "pop
vs. apply" "access in other cmds" "merge & conflicts".

Cheers,

Daniel

Re: [RFC] Shelving and Checkpointing

[Julian Foad]

Thanks for your suggestion, Nathan.

Nathan Hartman wrote:

[...] What if, instead of just a pristine copy, it actually created a private local 
repository. Revision 1 of this repository would be the pristine copy. [...] if you type 
some other command instead of commit, or maybe prepend the word "local" or 
something, the commit would go into the local repository [...] Shelving, stashing, etc., 
all become local operations against this local souped up pristine copy replacement.


That is exactly what I was thinking about when I wrote "Option 3... 
Checkpoints are commits in a local repository embedded in the WC" 
towards the end of that document. My feeling is that both the amount of 
effort required just to get that far, and the explosion of further 
possibilities it would open up, make it too heavy-weight for my current 
project, but I would definitely like to explore that possibility further.


- Julian


References:
   [1] Shelving-Checkpointing Dev doc. (J Foad)
https://docs.google.com/document/d/1PVgw0BdPF7v67oxIK7B_Yjmr3p28ojabP5N1PfZTsHk/edit#

Re: [RFC] Shelving and Checkpointing

[Nathan Hartman]

On Jul 10, 2017, at 8:59 AM, Julian Foad  wrote:
> 
> Dear Subversion Developers,
> 
> I am delighted to announce that I am working with Assembla to develop
> shelving and checkpointing functionality in Subversion. These have
> been on the wish list for many years, and are becoming ever more in
> demand since the popularity of git is making more users appreciate the
> benefits of local operations.

Hi all,

I just wanted to chime in and say that I'm very happy to hear that these local 
features are planned for Subversion. After ten years of using Subversion, I can 
say that it has been a joy to use from day one, owing in large part to its 
simplicity and reliability.

It occurred to me recently that in the distant past Subversion's purpose was to 
be a better CVS, that that goal has been achieved long ago, and that perhaps 
it's time for a Subversion 2.0, which could take it to the next level. I don't 
know much about future plans, such as what FSX is intended to achieve, but it 
occurred to me that perhaps Subversion could gain the ability to do local 
commits, which could be an optional step before doing the real commit back to 
the repository. As I thought about that, I realized that it might not have to 
require too much new machinery to achieve that.

I had this idea: When you checkout a working copy, Subversion creates the 
hidden .svn directory which contains the pristine copies among other things. 
What if, instead of just a pristine copy, it actually created a private local 
repository. Revision 1 of this repository would be the pristine copy. The user 
need not know that anything has changed and can continue working as before, 
committing back to the server. But, if you type some other command instead of 
commit, or maybe prepend the word "local" or something, the commit would go 
into the local repository rather than the remote one. Now this makes it 
possible to do locally anything you could do remotely such as creating multiple 
local feature branches. It also means that when you do an update, there could 
be an implied local commit first, which allows you to undo an update which 
perhaps messed something up. Shelving, stashing, etc., all become local 
operations against this local souped up pristine copy replacement.

Since you're only checking out one revision just as before, as opposed to 
cloning a whole repository, this idea comes with some big advantages that you 
don't get with git or hg, such as that you need not clone an entire gigantic 
repository. This is one of the big reasons that those systems are not a good 
choice for housing lots of big files that change infrequently, as every user 
would have to clone potentially gigabytes of data they're not interested in. In 
other words a svn checkout still does what it always did before, except that 
you have the option to work locally if you want to.

I didn't mention any of this before because:

(1) there are obviously a lot of holes in this idea, such as, what happens to 
this private local repo when you do a svn switch? Or the pesky issue of how to 
add such a major new feature without inheriting a ton of obscure commands a la 
git?

And (2) although I am a developer and very fond of Subversion, I cannot take 
this upon myself because of other obligations now and in the foreseeable 
future. So I felt it would be rude to dump a crazy idea like this on everyone 
else. Julian's post prompted me to break my silence, not knowing if what I've 
suggested is remotely realistic.

I realize that this suggests a major backwards-incompatible change to the 
working copy format and indeed the entire client side. But this was a request 
for comment. :-) Hopefully someone finds my input valuable.

Kind regards,
Nathan 

> References:
>   [1] Shelving-Checkpointing Dev doc. (J Foad)
> https://docs.google.com/document/d/1PVgw0BdPF7v67oxIK7B_Yjmr3p28ojabP5N1PfZTsHk/edit#