Re: Hidden Commits in 4.3

[Pierre-Yves David]

On 04/18/2017 10:51 PM, Durham Goode wrote:

I respond below, but I believe Jun has sent you a innovative proposal
that may solve both of our needs and render this discussion irrelevant.
So take a look at his proposal at your earliest convenience, since it
may let us put this behind us.


My first round of exchange with Jun on this was promising, but there was 
some area that needed further investigation before getting too 
enthusiastic. Jun sent me details on this further investigation, but I 
need a good chunk of time to look at them. Something which proved 
challenging to find this week so far. I'll look at them when possible, I 
was hoping this week but it is friday night already.


Cheers,

--
Pierre-Yves David
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Pierre-Yves David]

On 04/12/2017 12:16 AM, Jun Wu wrote:

Excerpts from Pierre-Yves David's message of 2017-04-11 22:29:15 +0200:

[...]

Mixing it with local only elements will not work. They are tons of
simple case where we won't be able to determine a simple and consistent
behavior for it. Even the local case can quickly raise shortcoming to
the above proposal:

   When an orphan changeset gets hidden by evolution, what should we do
   with its obsolete parent? Was it visible because of the orphan or
   because the user actively "unhide" it some time ago?


This is not hard to answer. "rebase" with some auto flag will hide the
parent if it has no visible children, recursively.  "rebase" with explicit
"-s" and "-d" could choose to not be smart about parents (it may also
choose to be smart with some non-default config set).


Rebase is maybe an easy case, but what about all the other actions 
adding markers ? hg push, hg pull, hg evolve, hg unbundle, etc…?


To dig further around impact of new proposals on changeset evolution, 
one can also checks and evaluate the concrete example on the Concept 
wiki page[1]


I'm not trying to revive this branch on the discussion. And I think we 
should focus on smaller steps first. However, since multiple people has 
pointed out confusion around concrete examples, I wanted the highlight 
and clarify the one already on the table to help people think about them 
on their own.


(and yes, I'll look at your email in my inbox soon™)

[1] https://www.mercurial-scm.org/wiki/CEDConcept#concrete_examples


--
Pierre-Yves David
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Pierre-Yves David]

note: I recommend people interrested in this discussion to subscribe to 
the related wiki pages:


https://www.mercurial-scm.org/wiki/CategoryEvolution
https://www.mercurial-scm.org/wiki/HideUnhidePlan

On 04/14/2017 01:12 AM, Durham Goode wrote:



On 4/13/17 2:43 PM, Pierre-Yves David wrote:



On 04/13/2017 11:37 PM, Pierre-Yves David wrote:

On 04/12/2017 04:23 PM, Ryan McElroy wrote:

I think the next step is for the community to officially figure out
if this is a good direction to go in, however that happens.


I had productive face to face discussion with multiple people in the
past couple a day.  Let us put all technical details aside and look at
the situation at the high level.

The current tentacular discussions are the *gathering of three
different goals*:


There a was a secondary point I wanted to make. But I did not wanted to
inflate the the size of the previous email too much.

The recent discussion *mixes multiple complex topics*. Complex both at
the technical and UI level. It results in a *back and forth of
interleaved discussion* *with people caring about different aspects*.

As a good symptom of that, I've read maybe 3 summaries of the situation
and existing consensus by three different people in the last 48h. To me
they seems to reach sightly different conclusions and outline different
consensus. I'm not even sure how to interpret some ambiguity in them.

After discussion this further with Gregory, I think *we really needs to
introduce more focus this discussion*. I know everybody are itching to
bring up some topics, but we should agree on what is the *next most
important small step we can take and focus on that*. Without trying to
alter other aspects in the same go. Having a*small group of neutral
moderators* to lead the discussion would help.

From the current state of the discussion, I think we should starts with
a *restricted discussion about an alternative to strip*, /mutually
exclusive with evolution/. I personally though local-hiding and
obsolescence hiding could coexist until I read Gregory email. He seems
to be raising valid concerns there.  When that topic is sorted out, we
can move the discussion to something else, for example how they could
coexist.

What do you think?


If we delete the "Interaction With Obsmarkers" paragraph of my proposal,
it's effectively a proposal for an alternative to strip. Would that be
an appropriate starting place for a alternative-to-strip discussion?  We
could leave out it's interaction with evolve for now.


(Yes, I'll look at Jun email shortly, but I had this draft 80% written 
and it can help define a smooth way forward, so I polished and sent it).


Focusing on a simple replacement for strip would probably be a good 
start and your proposal provides a good base for that. However we needs 
a deeper path at the proposed UI and behavior.


When I was chatting with Gregory Szorc, he highlighted that solution we 
ship earlier should had a "good compatibility story" for the time we 
ship completed changeset evolution. Some part of the proposal are 
currently unclear to me in that regard (eg: unhiding mechanism/command)


On a general basis, we needs to design a UI where user can reach new 
features with a reasonable amount of complexity (in regard with the 
feature reached). Lets take an example of a user slowly using more and 
more of the features,


1) making commit,
2) pushing commit
3) rewrite commits,
4) rewrite stack,
5) use feature branches,
6) use long lived branches,
7) exchange draft changesets,
8) collaborate with other on draft branch.

Each time the user takes one of this steps, he should just have to add a 
small overlay to what he already know about Mercurial. For example 
moving from (1) to (3) just means learning about the '--amend' flag, the 
`hg commit` commands do not needs to be replaced by something else to 
deal with (1). Or when it starts exchanging changeset with other, it 
just needs to learn about `hg pull` and `hg pull` not replace the other 
command he already use locally.
In the same way, when moving from local only rewrite to distributed 
rewrite, we should just adds the minimum amount of complexity (eg: hg 
evolve) and be able to leverage all already known command from the local 
workflow. So if "reviving" and editing commits needs a different command 
set for the local and distributed case we would be on the wrong track.
Such smooth transition in the UI also allows member of the same team to 
collaborate at their level, since no simpler command used by simpler 
user will conflict with some other more advance usages.


What I'm trying to say here is that the delta of complexity between each 
step is probably as important as the punctual complexity of each steps.


(I'll spent my next chunk of time reading the Jun proposal (that might 
obsolete the current proposal), I'll have a deeper look at the current 
proposal after it.)


Cheers,

--
Pierre-Yves David
___

Re: Hidden Commits in 4.3

[Augie Fackler]

(So it’s not buried: Pierre-yves, I’d like to grab a 30 minute VC to get a 
summary of your thoughts on this, since I know you’ve spent a lot of time on 
this, assuming you can make time Friday (the 21st).)

> On Apr 20, 2017, at 7:37 PM, Sean Farley  wrote:
> 
>>> In my example above, once "hg undo" is upstreamed, we are able to build
>>> a variant without hash preservation[1] and ship that the user by default
>>> without delay.
>> 
>> I'm sure you don't mean it this way, but this sounds a lot like "let's 
>> ship my thing and we'll figure your thing out later". The moment evolve 
>> ships, it becomes a lot harder to make the types of changes I'm talking 
>> about (both from a user experience point of view and community 
>> momentum), so I'd rather have the discussion now while it's still 
>> possible to make important changes.  If we flipped the sentence around 
>> (ship mine, figure yours out later) I'm sure you wouldn't be happy with 
>> that outcome either.
> 
> For what it's worth, I did not read it that way. Also, I've tried to
> avoid this discussion (across all the various threads) because it has
> been draining.
> 
>>> On a general principle, decoupling implementation discussion from UI
>>> polishing is usually a win.
>> 
>> If we decoupled UI from implementation here, we'd just go ahead and ship 
>> "hg unhide" with hash preservation, since that's objectively the more 
>> intuitive UI for unhide. But since the implementation of the obsolete 
>> graph doesn't allow that UI, implementation therefore affects our 
>> discussion.
> 
> At first reading of this, I find this kind of hostile from Facebook. I'm
> sure you didn't mean it that way but eh.
> 
> I found it hard to reply to this thread because of all the different
> derailings. For every non-Facebook dev here, there are so many more
> Facebook devs to reply to. That is exhausting. *I* am exhausted.

Looking at the participant list on this message (including me), it’s more 
non-FB people than FB. That said, all of the following are true:

 * Facebook is employing a *ton* of people to work on their tooling, including 
Mercurial

 * They have a will to upstream some really great work they’ve done to improve 
the Mercurial user experience, with many hundreds of happy users to back up the 
utility of their offerings, which means we could be the benefactors of their 
large group

 * The community is generally being difficult about these offerings, reducing 
that will[0]

I’m pretty frustrated with how this has gone (both on this thread, and 
elsewhere). We’ve got *real wins* available, but the conversation has been so 
fraught and verbose that I have no sense of who wants what. I can be available 
for video chats most of tomorrow (the 21st, America/New_York "typical waking 
hours") if that’s of interest and people are willing. Could a couple of people 
jump in and help me make sense of the state of things (without adding to the 
book’s worth of email already on this topic).

Thanks,
Augie
(more below as well)

0: It’s not my intent to cast blame. There’s plenty to go around, and I’ll say 
I’ve not infrequently been a source of difficulty. It’s my belief that we 
should, as a community, do better at taking advantage of the large user bases 
that are offering us pre-tested innovative ideas, with contributions of 
engineering time to land complicated work in core. Every monster thread like 
this one reduces *everyone’s* willingness to participate in the community, 
including my own.

> I'd love to discuss the pro's and con's of these two ideas purely from a 
>> UX standpoint, but I haven't seen any concrete examples of situations 
>> where evolve is affected by this for us to have a tangible discussion 
>> around it. If we had a concrete example, we could more productively 
>> evaluate the pro's and con's of these ideas.
> 
> There's so much convolution in this thread. For instance, I think (fast)
> local hiding would be cool. A version of that is possible today and I've
> even had my hand at it:
> 
> https://bitbucket.org/seanfarley/hgremotenames/commits/69cda2b8353ab1b108ca120d08eb253941f9ae70?at=hideable
> 
> That's remarkably simple and having a bitmap (or whatever) api would be
> good.

Yes, I think we’re all agreed on that, and it should preferably land (someone’s 
time allowing) very soon after the freeze ends so we can experiment with it and 
roll it back if it’s painful.

> I'm surprised there has been so much misinformation and changing
> of the goal posts. For instance,
> 

[...]

> None of the above, in my mind, requires the unhiding / cycle business.
> To me, getting the above points in core first (just making the happy
> path working, basically) would be a win. If that's not the goal, then
> I'm not really enthused about your proposals.

I’m starting to think that the unhiding without changing revision hash is more 
ergonomically reasonable. But as I said above, I think that decision can wait 
until later. There are real 

Re: Hidden Commits in 4.3

[Sean Farley]

Durham Goode  writes:

> I respond below, but I believe Jun has sent you a innovative proposal 
> that may solve both of our needs and render this discussion irrelevant. 
> So take a look at his proposal at your earliest convenience, since it 
> may let us put this behind us.
>
> On 4/18/17 11:03 AM, Pierre-Yves David wrote:
>> On 04/14/2017 01:04 AM, Durham Goode wrote:
>>> On 4/13/17 2:37 PM, Pierre-Yves David wrote:
 On 04/12/2017 04:23 PM, Ryan McElroy wrote:
> […]
 *Practical example* (/simplified/):

Situation:

  * Facebook has a useful: hg undo command.
  * Facebook cares about hg undo, preserving the hash,
  * this hash preservation conflict with current constraint of
changeset-evolution.

Way forward:

  * Facebook can upstream hg undo with hash preservation,
  * We introduces a variant that changes the hash and makes it
available to all users with BC,
  * Facebook can set the config for hash preservation internally.

Result: the code is upstream, user has new feature and we can
discuss hash preservation later.
>>>
>>> I think this example is missing the step where we discuss what we should
>>> ship to users. My goal is not to enable Facebook to have a feature (we
>>> already have these features), my goal is to create a good user
>>> experience for Mercurial users. If we do the config knob route, we must
>>> at some point have the discussion about what experience we want to give
>>> to users, before we actually ship it to them.
>>>
>>> So in your proposal, when will it become appropriate to have that
>>> discussion? And what can we do between now and then to make that
>>> discussion more productive? I think we're blocked on getting bits into
>>> the hands of users by default until then.
>>
>> (note: I've purposely delay more "discussion" oriented part in my other
>> email (that you also replied to) I'll reply to it too shortly after this
>> one.)
>>
>>
>> In my example above, once "hg undo" is upstreamed, we are able to build
>> a variant without hash preservation[1] and ship that the user by default
>> without delay.
>
> I'm sure you don't mean it this way, but this sounds a lot like "let's 
> ship my thing and we'll figure your thing out later". The moment evolve 
> ships, it becomes a lot harder to make the types of changes I'm talking 
> about (both from a user experience point of view and community 
> momentum), so I'd rather have the discussion now while it's still 
> possible to make important changes.  If we flipped the sentence around 
> (ship mine, figure yours out later) I'm sure you wouldn't be happy with 
> that outcome either.

For what it's worth, I did not read it that way. Also, I've tried to
avoid this discussion (across all the various threads) because it has
been draining.

>> On a general principle, decoupling implementation discussion from UI
>> polishing is usually a win.
>
> If we decoupled UI from implementation here, we'd just go ahead and ship 
> "hg unhide" with hash preservation, since that's objectively the more 
> intuitive UI for unhide. But since the implementation of the obsolete 
> graph doesn't allow that UI, implementation therefore affects our 
> discussion.

At first reading of this, I find this kind of hostile from Facebook. I'm
sure you didn't mean it that way but eh.

I found it hard to reply to this thread because of all the different
derailings. For every non-Facebook dev here, there are so many more
Facebook devs to reply to. That is exhausting. *I* am exhausted.

> I'd love to discuss the pro's and con's of these two ideas purely from a 
> UX standpoint, but I haven't seen any concrete examples of situations 
> where evolve is affected by this for us to have a tangible discussion 
> around it. If we had a concrete example, we could more productively 
> evaluate the pro's and con's of these ideas.

There's so much convolution in this thread. For instance, I think (fast)
local hiding would be cool. A version of that is possible today and I've
even had my hand at it:

https://bitbucket.org/seanfarley/hgremotenames/commits/69cda2b8353ab1b108ca120d08eb253941f9ae70?at=hideable

That's remarkably simple and having a bitmap (or whatever) api would be
good. I'm surprised there has been so much misinformation and changing
of the goal posts. For instance,

* obs marker are global state

  I didn't think this was even a point to be contested. Distributed
  systems is difficult and evolve is solving a hard problem of shared
  rebase, histedit, etc.

* hash preservation

  Sure, that's neat. But that's not a blocker. Like, not even a little
  bit. If I hear Jun say that obs-based hiding is broken by design one
  more time, I might roll my eyes so hard that they'll fall out the back
  of my skull.

> Given that evolve has been around for years and people still aren't able 
> to reason about concrete examples in these 

Re: Hidden Commits in 4.3

[Durham Goode]

I respond below, but I believe Jun has sent you a innovative proposal 
that may solve both of our needs and render this discussion irrelevant. 
So take a look at his proposal at your earliest convenience, since it 
may let us put this behind us.


On 4/18/17 11:03 AM, Pierre-Yves David wrote:

On 04/14/2017 01:04 AM, Durham Goode wrote:

On 4/13/17 2:37 PM, Pierre-Yves David wrote:

On 04/12/2017 04:23 PM, Ryan McElroy wrote:

[…]

*Practical example* (/simplified/):

   Situation:

 * Facebook has a useful: hg undo command.
 * Facebook cares about hg undo, preserving the hash,
 * this hash preservation conflict with current constraint of
   changeset-evolution.

   Way forward:

 * Facebook can upstream hg undo with hash preservation,
 * We introduces a variant that changes the hash and makes it
   available to all users with BC,
 * Facebook can set the config for hash preservation internally.

   Result: the code is upstream, user has new feature and we can
   discuss hash preservation later.


I think this example is missing the step where we discuss what we should
ship to users. My goal is not to enable Facebook to have a feature (we
already have these features), my goal is to create a good user
experience for Mercurial users. If we do the config knob route, we must
at some point have the discussion about what experience we want to give
to users, before we actually ship it to them.

So in your proposal, when will it become appropriate to have that
discussion? And what can we do between now and then to make that
discussion more productive? I think we're blocked on getting bits into
the hands of users by default until then.


(note: I've purposely delay more "discussion" oriented part in my other
email (that you also replied to) I'll reply to it too shortly after this
one.)


In my example above, once "hg undo" is upstreamed, we are able to build
a variant without hash preservation[1] and ship that the user by default
without delay.


I'm sure you don't mean it this way, but this sounds a lot like "let's 
ship my thing and we'll figure your thing out later". The moment evolve 
ships, it becomes a lot harder to make the types of changes I'm talking 
about (both from a user experience point of view and community 
momentum), so I'd rather have the discussion now while it's still 
possible to make important changes.  If we flipped the sentence around 
(ship mine, figure yours out later) I'm sure you wouldn't be happy with 
that outcome either.



On a general principle, decoupling implementation discussion from UI
polishing is usually a win.


If we decoupled UI from implementation here, we'd just go ahead and ship 
"hg unhide" with hash preservation, since that's objectively the more 
intuitive UI for unhide. But since the implementation of the obsolete 
graph doesn't allow that UI, implementation therefore affects our 
discussion.


I'd love to discuss the pro's and con's of these two ideas purely from a 
UX standpoint, but I haven't seen any concrete examples of situations 
where evolve is affected by this for us to have a tangible discussion 
around it. If we had a concrete example, we could more productively 
evaluate the pro's and con's of these ideas.


Given that evolve has been around for years and people still aren't able 
to reason about concrete examples in these discussions, I don't think 
the strategy of 'give it more time' is going to help move this 
discussion forward.

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Pierre-Yves David]

On 04/14/2017 01:04 AM, Durham Goode wrote:

On 4/13/17 2:37 PM, Pierre-Yves David wrote:

On 04/12/2017 04:23 PM, Ryan McElroy wrote:

[…]

*Practical example* (/simplified/):

   Situation:

 * Facebook has a useful: hg undo command.
 * Facebook cares about hg undo, preserving the hash,
 * this hash preservation conflict with current constraint of
   changeset-evolution.

   Way forward:

 * Facebook can upstream hg undo with hash preservation,
 * We introduces a variant that changes the hash and makes it
   available to all users with BC,
 * Facebook can set the config for hash preservation internally.

   Result: the code is upstream, user has new feature and we can
   discuss hash preservation later.


I think this example is missing the step where we discuss what we should
ship to users. My goal is not to enable Facebook to have a feature (we
already have these features), my goal is to create a good user
experience for Mercurial users. If we do the config knob route, we must
at some point have the discussion about what experience we want to give
to users, before we actually ship it to them.

So in your proposal, when will it become appropriate to have that
discussion? And what can we do between now and then to make that
discussion more productive? I think we're blocked on getting bits into
the hands of users by default until then.


(note: I've purposely delay more "discussion" oriented part in my other 
email (that you also replied to) I'll reply to it too shortly after this 
one.)



In my example above, once "hg undo" is upstreamed, we are able to build 
a variant without hash preservation[1] and ship that the user by default 
without delay. Of course "hg undo" with hash salting is less nice than 
"hg undo" with hash preservation, but already much better than no "hg 
undo" at all.



To generalize, I've already heard about multiple interesting UI bits 
from Facebook that the community could benefit from (undo, ui focused on 
evolving orphans, etc). These UI does not strictly depends on 
contentious bits like hash preservation.


We can upstream and ship these bit while discussing more complex UI 
changes on the side[2]. These discussion will be simpler since less work 
will be "blocked" behind their conclusion. Having the feature variants 
in the wild will also gather data on user response and help

other developer to be more familiar with the topic of the discussion.
On a general principle, decoupling implementation discussion from UI 
polishing is usually a win.



I know you have the experience of external user in mind. But having this 
code upstream will also directly benefit Facebook operations. Your 
maintenance load would be lowered (one of the topic at the past sprint) 
and your divergence from the Core UI reduced.


Cheers,

[1] (For the sake of this example, we assume a world with 
hash-preservation blocking changeset-evolution plans)


[2] while keeping that discussion focused.

--
Pierre-Yves David
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Denis Laxalde]

Durham Goode wrote:

On 4/13/17 2:37 PM, Pierre-Yves David wrote:

On 04/12/2017 04:23 PM, Ryan McElroy wrote:

I think the next step is for the community to officially figure out if
this is a good direction to go in, however that happens.


I had productive face to face discussion with multiple people in the
past couple a day.  Let us put all technical details aside and look at
the situation at the high level.

The current tentacular discussions are the *gathering of three different
goals*:

 * *upstreaming* more of Facebook work (yeah!),
 * *completing changeset evolution* and shipping it to all users,
 * *shipping improvement to users* sooner than later.

All these goals are *important, good, realistic *and*compatible* if done
carefully.
They interact with each others, so we have to make sure we *achieves
each goal without blocking the others*. Something not guaranteed so far
in the current discussions.

So what is our way forward in practice? After discussions with Ryan,
Siddharth and Gregory, I think *we could use the  following principle*:

   When If *someone raise concerns* about the impact of a feature on
   other goals, get the feature in, but *keep it under a config flag
   while we resolve the situation* in one of the following ways:

 * more thinking *lift the concerns*,
 * a *small variant* that can be on by default is introduced,
 * an *equivalent, alternative featured be *on by default in added,
 * an alternative path to the concepts is found that.

As a result:

 * *Facebook can upstream and share code* more easily. Having to live
   with a config knob for a while is not a big deal for them,
 * The surface on which we guarantee backward compatibility *leaves the
   path to complete changeset-evolution open*,
 * In many cases, *community can take advantage of the upstreamed code*
   to offer better capability to the user with just an extra bit of
   code to implement a small variant,
 * As a bonus we can experiment with alternative path and get data
   about them.


I'm happy to introduce the initial version of this under a config flag.

However, I think this just means we delay having this same discussion to
a later date. And it's not clear how having that config flag for some
time will improve people's understanding to make the discussion more
productive (since presumably the community isn't using the flag).



I think that exposing the feature to more users behind a config knob
might help moving forwards to more practical considerations, and
eventually let the opportunity to users missing abstract knowledge to
participate to the discussion (by "users", I mean: readers of this
thread or list in general, so probably not the whole community but more
than a handful of developers).
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Durham Goode]

On 4/13/17 2:43 PM, Pierre-Yves David wrote:



On 04/13/2017 11:37 PM, Pierre-Yves David wrote:

On 04/12/2017 04:23 PM, Ryan McElroy wrote:

I think the next step is for the community to officially figure out
if this is a good direction to go in, however that happens.


I had productive face to face discussion with multiple people in the
past couple a day.  Let us put all technical details aside and look at
the situation at the high level.

The current tentacular discussions are the *gathering of three
different goals*:


There a was a secondary point I wanted to make. But I did not wanted to
inflate the the size of the previous email too much.

The recent discussion *mixes multiple complex topics*. Complex both at
the technical and UI level. It results in a *back and forth of
interleaved discussion* *with people caring about different aspects*.

As a good symptom of that, I've read maybe 3 summaries of the situation
and existing consensus by three different people in the last 48h. To me
they seems to reach sightly different conclusions and outline different
consensus. I'm not even sure how to interpret some ambiguity in them.

After discussion this further with Gregory, I think *we really needs to
introduce more focus this discussion*. I know everybody are itching to
bring up some topics, but we should agree on what is the *next most
important small step we can take and focus on that*. Without trying to
alter other aspects in the same go. Having a*small group of neutral
moderators* to lead the discussion would help.

From the current state of the discussion, I think we should starts with
a *restricted discussion about an alternative to strip*, /mutually
exclusive with evolution/. I personally though local-hiding and
obsolescence hiding could coexist until I read Gregory email. He seems
to be raising valid concerns there.  When that topic is sorted out, we
can move the discussion to something else, for example how they could
coexist.

What do you think?



If we delete the "Interaction With Obsmarkers" paragraph of my proposal, 
it's effectively a proposal for an alternative to strip. Would that be 
an appropriate starting place for a alternative-to-strip discussion?  We 
could leave out it's interaction with evolve for now.

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Durham Goode]

On 4/13/17 2:37 PM, Pierre-Yves David wrote:

On 04/12/2017 04:23 PM, Ryan McElroy wrote:

I think the next step is for the community to officially figure out if
this is a good direction to go in, however that happens.


I had productive face to face discussion with multiple people in the
past couple a day.  Let us put all technical details aside and look at
the situation at the high level.

The current tentacular discussions are the *gathering of three different
goals*:

 * *upstreaming* more of Facebook work (yeah!),
 * *completing changeset evolution* and shipping it to all users,
 * *shipping improvement to users* sooner than later.

All these goals are *important, good, realistic *and*compatible* if done
carefully.
They interact with each others, so we have to make sure we *achieves
each goal without blocking the others*. Something not guaranteed so far
in the current discussions.

So what is our way forward in practice? After discussions with Ryan,
Siddharth and Gregory, I think *we could use the  following principle*:

   When If *someone raise concerns* about the impact of a feature on
   other goals, get the feature in, but *keep it under a config flag
   while we resolve the situation* in one of the following ways:

 * more thinking *lift the concerns*,
 * a *small variant* that can be on by default is introduced,
 * an *equivalent, alternative featured be *on by default in added,
 * an alternative path to the concepts is found that.

As a result:

 * *Facebook can upstream and share code* more easily. Having to live
   with a config knob for a while is not a big deal for them,
 * The surface on which we guarantee backward compatibility *leaves the
   path to complete changeset-evolution open*,
 * In many cases, *community can take advantage of the upstreamed code*
   to offer better capability to the user with just an extra bit of
   code to implement a small variant,
 * As a bonus we can experiment with alternative path and get data
   about them.


I'm happy to introduce the initial version of this under a config flag.

However, I think this just means we delay having this same discussion to 
a later date. And it's not clear how having that config flag for some 
time will improve people's understanding to make the discussion more 
productive (since presumably the community isn't using the flag).



*Practical example* (/simplified/):

   Situation:

 * Facebook has a useful: hg undo command.
 * Facebook cares about hg undo, preserving the hash,
 * this hash preservation conflict with current constraint of
   changeset-evolution.

   Way forward:

 * Facebook can upstream hg undo with hash preservation,
 * We introduces a variant that changes the hash and makes it
   available to all users with BC,
 * Facebook can set the config for hash preservation internally.

   Result: the code is upstream, user has new feature and we can
   discuss hash preservation later.


I think this example is missing the step where we discuss what we should 
ship to users. My goal is not to enable Facebook to have a feature (we 
already have these features), my goal is to create a good user 
experience for Mercurial users. If we do the config knob route, we must 
at some point have the discussion about what experience we want to give 
to users, before we actually ship it to them.


So in your proposal, when will it become appropriate to have that 
discussion? And what can we do between now and then to make that 
discussion more productive? I think we're blocked on getting bits into 
the hands of users by default until then.

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Pierre-Yves David]

On 04/13/2017 11:37 PM, Pierre-Yves David wrote:

On 04/12/2017 04:23 PM, Ryan McElroy wrote:
I think the next step is for the community to officially figure out 
if this is a good direction to go in, however that happens.


I had productive face to face discussion with multiple people in the 
past couple a day.  Let us put all technical details aside and look at 
the situation at the high level.


The current tentacular discussions are the *gathering of three 
different goals*:


There a was a secondary point I wanted to make. But I did not wanted to 
inflate the the size of the previous email too much.


The recent discussion *mixes multiple complex topics*. Complex both at 
the technical and UI level. It results in a *back and forth of 
interleaved discussion* *with people caring about different aspects*.


As a good symptom of that, I've read maybe 3 summaries of the situation 
and existing consensus by three different people in the last 48h. To me 
they seems to reach sightly different conclusions and outline different 
consensus. I'm not even sure how to interpret some ambiguity in them.


After discussion this further with Gregory, I think *we really needs to 
introduce more focus this discussion*. I know everybody are itching to 
bring up some topics, but we should agree on what is the *next most 
important small step we can take and focus on that*. Without trying to 
alter other aspects in the same go. Having a*small group of neutral 
moderators* to lead the discussion would help.


From the current state of the discussion, I think we should starts with 
a *restricted discussion about an alternative to strip*, /mutually 
exclusive with evolution/. I personally though local-hiding and 
obsolescence hiding could coexist until I read Gregory email. He seems 
to be raising valid concerns there.  When that topic is sorted out, we 
can move the discussion to something else, for example how they could 
coexist.


What do you think?

--
Pierre-Yves David

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Pierre-Yves David]

On 04/12/2017 04:23 PM, Ryan McElroy wrote:
I think the next step is for the community to officially figure out if 
this is a good direction to go in, however that happens.


I had productive face to face discussion with multiple people in the 
past couple a day.  Let us put all technical details aside and look at 
the situation at the high level.


The current tentacular discussions are the *gathering of three different 
goals*:


 * *upstreaming* more of Facebook work (yeah!),
 * *completing changeset evolution* and shipping it to all users,
 * *shipping improvement to users* sooner than later.

All these goals are *important, good, realistic *and*compatible* if done 
carefully.
They interact with each others, so we have to make sure we *achieves 
each goal without blocking the others*. Something not guaranteed so far 
in the current discussions.


So what is our way forward in practice? After discussions with Ryan, 
Siddharth and Gregory, I think *we could use the  following principle*:


   When If *someone raise concerns* about the impact of a feature on
   other goals, get the feature in, but *keep it under a config flag
   while we resolve the situation* in one of the following ways:

 * more thinking *lift the concerns*,
 * a *small variant* that can be on by default is introduced,
 * an *equivalent, alternative featured be *on by default in added,
 * an alternative path to the concepts is found that.

As a result:

 * *Facebook can upstream and share code* more easily. Having to live
   with a config knob for a while is not a big deal for them,
 * The surface on which we guarantee backward compatibility *leaves the
   path to complete changeset-evolution open*,
 * In many cases, *community can take advantage of the upstreamed code*
   to offer better capability to the user with just an extra bit of
   code to implement a small variant,
 * As a bonus we can experiment with alternative path and get data
   about them.

*Practical example* (/simplified/):

   Situation:

 * Facebook has a useful: hg undo command.
 * Facebook cares about hg undo, preserving the hash,
 * this hash preservation conflict with current constraint of
   changeset-evolution.

   Way forward:

 * Facebook can upstream hg undo with hash preservation,
 * We introduces a variant that changes the hash and makes it
   available to all users with BC,
 * Facebook can set the config for hash preservation internally.

   Result: the code is upstream, user has new feature and we can
   discuss hash preservation later.

--
Pierre-Yves David

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Durham Goode]

On 4/12/17 10:44 PM, Gregory Szorc wrote:


I just read this entire thread and am trying to wrap my head around
different sides of the argument. I clearly see where Durham, Ryan, and
others are coming from with "strip-based operations are bad." I think we
can all agree on that. I can also see how the new hiding mechanism is
effectively the same as strip-based operations in terms of behavior, but
without the poor performance and hacks. So it seems like a generally
good idea.

But I'm having difficulty comprehending Pierre-Yves's opposition to this
seemingly good proposal. Pierre-Yves has thought about evolution
concepts more than anyone, so when he raises an alarm, it is worth
taking the time to listen and understand his concerns so we don't
jeopardize the future of evolve. However, despite the vast amount of
words written, it still isn't clear to me exactly what the main
objection is. But from seeing "global state" and "monotonic" enough
times and reading through https://www.mercurial-scm.org/wiki/CEDConcept
,
I think I have an idea. (Pierre-Yves: please confirm or correct what I'm
about to say.)

I think most of Pierre-Yves's concerns are rooted at a *potential*
intrinsic incompatibility between a) a generic, non-versioned,
non-monotonic hiding mechanism (the proposal) and b) what is essentially
obsolescence markers of today. For example, if the new hiding mechanism
isn't designed with consideration for evolve, evolve/Mercurial would see
an instruction to hide a changeset but evolve would be powerless to make
any kind of reasonable judgement based on the "why" and "how" that
changeset became hidden because that data just isn't recorded. In other
words, generic hiding lacks the "global state" and "monotonic"
properties of the obsolescence store and its exchange, making evolve
less effective. This *would* undermine the ability to ship a high
quality product: so Pierre-Yves isn't using hyperbole when he says he
fears that he could deliver evolve depending on how this hiding feature
is implemented.


I think your analysis is correct. There are trade offs: A completely 
mutable hidden store would hinder certain types of analysis in the 
future. A append-only-obsmarker based hidden would prevent certain types 
of user interactions in the future (ex: unhiding).


I believe optimizing for the 'certain types of analysis' case is 
optimizing for an edge case of an edge case workflow (mutable history 
conflicts within mutable history collaboration) at the expense of the 
common case workflows (local history mutation, commit revival, commit 
hiding mental model, and simple client/server workflows).


In my understanding, the actual user experience impact on evolve of this 
proposal (vs the theory oriented impact) is that *if* the user is doing 
mutable history collaboration, and *if* they have a mutable history 
conflict of some type, and *if* they've messed with visibility 
themselves, *then* evolve would be unable to determine the correct 
result and would have to prompt the user. I might be over simplifying, 
so I agree that concrete examples would help understand the impact of 
this change.


So, in my head having 'hg hide/unhide' be a dead simple, understandable 
(for both users and extensions) primitive in all cases is highly 
desirable, and is worth the trade off.



Regardless of whether this is actually Pierre-Yves's fundamental
concern, there is a conflict here that needs addressed. And I /think/
that conflict can be summarized as "how do we ensure that evolve is the
sole source of truth for hidden/obsolescence data?" Because if evolve
isn't the sole source of truth for hidden/obsolescence data, then evolve
has to essentially guess at what to do with changesets hidden for
"unknown" reasons and that severely undermines the ability of evolve to
work well.


As mentioned above, it only undermines evolve's ability if the user 
actually did manual visibility manipulations. If they didn't, then 
things work exactly as they do today. And the remediation in the failure 
case is to just tell the user "hey, you have to choose between action A 
or action B". This doesn't sound so bad, especially if this prompt is 
going to be part of certain evolve workflows anyway (since there will 
almost certainly exist ambiguous situations in even the most well 
designed evolve world).



What I think this means is that we need writes to a generic hiding
mechanism and evolve/obsolescence markers to be mutually exclusive. I.e.
if obsolescence markers are active, we can't be hiding things via
not-obsolescence-markers because doing so would deprive evolve of
critical knowledge.


We'll want obsmarker data even in a non-evolve world. See my restack 
proposal for how obs information can be useful as 

Re: Hidden Commits in 4.3

[Gregory Szorc]

On Fri, Apr 7, 2017 at 4:29 AM, Pierre-Yves David <
pierre-yves.da...@ens-lyon.org> wrote:

> On 04/06/2017 12:02 AM, Durham Goode wrote:
>
>> On 4/5/17 4:06 AM, Pierre-Yves David wrote:
>>
>>> […]
>>> The change proposed to evolution is problematic. As explain in my reply
>>> to the other thread[1], at minimum, removing the link between
>>> obsolescence and visibility is destroying the "global-state" property we
>>> currently have. This put in disarray the whole ability to use evolution
>>> to exchange and collaborate on mutable history, the very thing evolution
>>> has been built for. I've not seen strong-enough rationals for derailing
>>> the current evolution plan and design.
>>>
>>
>> The simpler implementation model, the simpler mental model, the simpler
>> ability for commands to use hiding, and the lack of tie-in with other
>> more divisive features is the reason for this proposal. With this
>> proposal we can have the majority of the benefits quickly, without
>> notable changes to the existing core UI, and in a intuitive form for
>> users.
>>
>
> I agree that having a local-only hiding mechanism would be a win.
>
> However obsolescence and the hiding computed from it must stay fully
> independent from it. Changeset-evolution relies on building a global
> state[1] mixing it with local-only elements breaks this property.
>
> I believe these benefits outweigh the downside to evolve's current goal,
>> […]
>> I think other members of the community would have to weigh in on whether
>> this trade off is worth it, since it is a subjective decision.
>>
>
> This is not a "small downside" to evolution goal. This is a full stop to
> our ability to ship changeset-evolution.
>
> The founding goal of changeset-evolution is to allow exchange and
> collaboration around mutable history[2]. The local benefit are "just" a
> nice side effect, but the concept is fully design around distribution. This
> is a problem no other DVCS solves and we have a plan, on track, to complete
> the concept and solve it. That will give Mercurial clear edge over others.
>
> The current proposal is destroying at least one pillard currently holding
> the changeset-evolution plan (global state), without replacement. As of
> today, this kills our ability to complete and ship changeset-evolution to
> all Mercurial user in the next year or two.
>
> This is not a small subjective adjustment, this is a full discontinuation
> of concept critical to the future of Mercurial.
>

I just read this entire thread and am trying to wrap my head around
different sides of the argument. I clearly see where Durham, Ryan, and
others are coming from with "strip-based operations are bad." I think we
can all agree on that. I can also see how the new hiding mechanism is
effectively the same as strip-based operations in terms of behavior, but
without the poor performance and hacks. So it seems like a generally good
idea.

But I'm having difficulty comprehending Pierre-Yves's opposition to this
seemingly good proposal. Pierre-Yves has thought about evolution concepts
more than anyone, so when he raises an alarm, it is worth taking the time
to listen and understand his concerns so we don't jeopardize the future of
evolve. However, despite the vast amount of words written, it still isn't
clear to me exactly what the main objection is. But from seeing "global
state" and "monotonic" enough times and reading through
https://www.mercurial-scm.org/wiki/CEDConcept, I think I have an idea.
(Pierre-Yves: please confirm or correct what I'm about to say.)

I think most of Pierre-Yves's concerns are rooted at a *potential*
intrinsic incompatibility between a) a generic, non-versioned,
non-monotonic hiding mechanism (the proposal) and b) what is essentially
obsolescence markers of today. For example, if the new hiding mechanism
isn't designed with consideration for evolve, evolve/Mercurial would see an
instruction to hide a changeset but evolve would be powerless to make any
kind of reasonable judgement based on the "why" and "how" that changeset
became hidden because that data just isn't recorded. In other words,
generic hiding lacks the "global state" and "monotonic" properties of the
obsolescence store and its exchange, making evolve less effective. This
*would* undermine the ability to ship a high quality product: so
Pierre-Yves isn't using hyperbole when he says he fears that he could
deliver evolve depending on how this hiding feature is implemented.

Regardless of whether this is actually Pierre-Yves's fundamental concern,
there is a conflict here that needs addressed. And I /think/ that conflict
can be summarized as "how do we ensure that evolve is the sole source of
truth for hidden/obsolescence data?" Because if evolve isn't the sole
source of truth for hidden/obsolescence data, then evolve has to
essentially guess at what to do with changesets hidden for "unknown"
reasons and that severely undermines the ability of evolve to work well.

What I think this means is 

Re: Hidden Commits in 4.3

[Pierre-Yves David]

TL;DR;

I had a productive video call with Siddharth Agarwal today. We should be 
careful to not block the current path for completing 
changeset-evolution, but they are way forward that would allow to make 
progress on upstreaming Facebook work and improving local workflow while 
avoiding introducing BC problematic to changeset-evolution completion.


I saw Ryan sent a longer an email on this thread. I'll check it up and 
reply with more details on these way forward. (that won't happen today)


(yes this TL;DR; is a bit long in itself).

On 04/12/2017 12:20 AM, Kevin Bullock wrote:

It sounds like there are no objections to the immediate 'hg hide' proposal, and 
the only points of contention are around the expected future interactions 
between hiding and exchanged obsolete markers. Is that an accurate assessment?


¿I'm quite confused? how do you end up interpreting:

  "this kills our ability to complete and ship changeset-evolution
   to all Mercurial user in the next year or two."

as:

  "there are no objections"?





It's not at all clear to me that having a single changeset hiding mechanism in
core prevents completing and shipping evolve. My understanding from discussion
at the sprint is that obsolescence markers could be used as one source of
hidden nodes, with no external change in behavior. Let's not engage in
hyperbolic statements about the future of any given feature; let's discuss the
tradeoffs of various approaches.


I'm trying to underline a reality here, not making hyperbolic statement. 
We currently have:


1) a prototype for evolve, successfully used by actual people. (to solve 
the target goal)

2) a Roadmap to smooth the rough edges and ship it to all users[1],
3) funding for that Roadmap,
4) available coder-s- to turn this funding into code (me and a colleague).

Items (3) and (4) above are new, and (finally!) make is possible to 
envision changeset-evolution being completed in a not so distant future. 
(I'm not aware of alternative plan to ship evolution)


There are proposals in this thread and related ones that breaks concept 
heavily relied on by changeset-evolution (eg: global state, monotic 
rewrite[3]), without providing alternative to the solutions these 
concepts bring.


Moving forward with simply breaking this pillar-concepts holding 
changeset-evolution blocks our path forward to complete and ship 
evolution. This moves us,

* from: "I know how to ship evolution and have the resource to do so"
* to: "I do not know how to ship evolution".

Since I'm not aware of any alternative plan to complete and 
ship-changeset evolution, that effectively kills our current ability to 
bring changeset-evolution to completion in the foreseeable future.



[1] https://www.mercurial-scm.org/wiki/CEDRoadMap
[2] recent activity in the evolve extension:
* last 45 days: 286 new changesets,
* full year before that: 188 new changesets.
[3] https://www.mercurial-scm.org/wiki/CEDConcept

As pointed in the TL;DR; I had a productive discussion with Siddharth 
Agarwal today. We see ways to move forward with upstreaming (yeah!) work 
from Facebook to improves rewrite related workflows while being careful 
about their impact on changeset evolution. In short some of the behavior 
variants Facebook needs could be kept outside of Mercurial core backward 
compatibility envelope (eg: with configuration) until we are sure we 
have a way to completing-evolution with them.


I'll follow up more on these way forward once I've catch up with the 
Ryan email that arrived while writing this one.


--
Pierre-Yves David
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Denis Laxalde]

Augie Fackler a écrit :

On Fri, Apr 07, 2017 at 12:20:44PM +0200, Pierre-Yves David wrote:

On 04/06/2017 02:46 AM, Jun Wu wrote:

obsstore-based hidden is broken by design, since it cannot unhide commits:

 $ hg export . > patch.txt
 $ hg prune . # could also be done by pulling obsmarkers from a remote
  # peer, or it could be amend / metaedit etc.
 $ hg import --exact --bypass patch.txt # cannot really import the patch


The case you describe above work as intended.


I know it's the original intent, but I'm not sure that the original intent is 
actually a desirable set of outcomes. I've actually tripped over this usability 
wart a decent number of times, usually some variation on this theme:

hg pull marmoute-wip -r some-feature
hg co some-feature
hg rebase -d @

hg strip 'ancestors(some-feature) and not ancestors(@)'

hg pull marmoute-wip -r some-feature # which has not moved since the last time 
I pulled


This briefly confuses *me* when I trip over it, and I've helped work out some of the 
design of the feature, which strongly suggests to me that the "is listed as a 
precursor in an obsmarker means its always hidden" behavior is suboptimal in the 
real world. As a rough cut of how we might reconcile usability and theory, I wonder if 
the correct path is to have obsmarkers immediately hide things (if possible) only when 
they're first introduced (likely through pull), but if the user takes an action that 
unhides, we just do that and then there's some advisory mechanism by which they can be 
told that things have been obsoleted.


I'm also confused by this kind of things. Another example is:

* pull a changeset to publish (hg phase -p)
* rebase it on top of the public head
* change your mind (because, for instance, tests fail)
* from this point, there's no simple way to get the original changeset
  and, more importantly, remove the obsmarker that'd get exchanged upon
  next push.

If I understood correctly, this issue would be solved by improving "hg
strip" to remove the obsmarker.



So far, I'm not seen a rational for hash-preservation been a -requirement-
nor a group of people agreeing it is a -requirement-.

Sure it would be nice, but "nice" is different from -required-.

This is probably a core point we need to iron out in this discussion.


I'm starting (based on the above rough edges that I've tripped on more than 
once) to be of the opinion that supporting hash preservation is effectively 
non-optional because of workflows people already have in their brains. It also 
makes a bunch of cases in tools like histedit and commit --amend easier to 
write (and harder for future extension authors to screw up!) to get that 
working in a sensible way.



I also think that hash preservation would be nice when it comes local
operations. This is important when exchanging with others to prevent
information that should remain local to propagate. Basically, in my
example above or yours, we don't want to exchange our local round-trip
operations.
But if one happens to have exchanged (pushed at least, not sure about
pull) the information, I think it does not sound unreasonable that
unhiding a changeset changes its hash (unless there is another possible
mechanism, like the "layer on top of obsmarkers" in the plan document).

Apart from this case of local-only operations, it's not very clear to me
why hash preservation would so important.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Ryan McElroy]

On 4/5/17 2:11 AM, Durham Goode wrote:
I would like to formally propose a new pattern for dealing with hidden 
commits, along with the concrete steps to getting it enabled in core 
by default by the August release.


The proposal is quite concise, so check out this 1-page Google doc for 
the details and to comment:


https://goo.gl/7DJ9AI



Much of this thread (and various subthreads and VC chats) has gone down 
the path of rehashing the same arguments for and against one type of 
hiding vs another. After chatting with most of the people expressing 
strong opinions here, I believe we will not resolve these disagreements 
with more replies and examples. While I happen to still believe that 
this proposal and changeset evolution are not mutually exclusive, some 
people seem to disagree with me on this.


The limited specific feedback on the proposed plan was incorporated into 
the above-linked Google Doc. Since it has been unchanged for a while, I 
turned it into a Plan wiki page:


https://www.mercurial-scm.org/wiki/HideUnhidePlan

Note that I cleaned up a few things:

* s/commit/changeset (except when referring to the commit command)
* Trivial cleanups to unclear language
* Small softening of some language that I felt was too strong given the 
data we have.


I think the next step is for the community to officially figure out if 
this is a good direction to go in, however that happens.


Cheers,

~Ryan
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Pierre-Yves David]

On 04/12/2017 12:20 AM, Kevin Bullock wrote:

I'm not responding to any technical points in this e-mail; I only want to 
address the current tone of the discussion.


On Apr 11, 2017, at 15:29, Pierre-Yves David  
wrote:

On 04/11/2017 12:43 AM, Augie Fackler wrote:


This briefly confuses *me* when I trip over it,

and I've helped work out some of the design of the feature,


I'm sorry, but no. you did not helped work out some of the design of the 
feature. You have barely contributed code touching evolution and not 
contributed to its concept. You attended some meeting and discussion, that does 
not automatically makes you an evolution expert.


Please keep these discussions polite and focused. There's no need to minimize 
the time and effort that others, including Augie, have put into making evolve 
something that we, and all our users, can all live with.


Let me clarify this point, I did not meant to minimize the time and 
effort put by Augie here. As you, I'm craving for a community respecting 
people time and involvement. If seems this part of my previous message 
came across harshly, I apologize for any arms it might have created.


(I will address the other part of that message independently later).

Cheers,

--
Pierre-Yves David
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Jun Wu]

Excerpts from Jun Wu's message of 2017-04-11 14:11:55 -0700:
> Excerpts from Pierre-Yves David's message of 2017-04-11 22:29:15 +0200:
> > The issue you face here is related to known limitation of `hg strip`. 
> > When it removes the changesets, strip should also remove the 
> > obsolescence markers leading to them.
> > 
> > That will be fixed soon. Having strip able to remove obsmarker has been 
> > close to the top of the evolve roadmap[1] for some times and it is on 
> > the funded path.
> 
> strip is just one case. precursor (i.e. len(successors) > 0) hiding is also
> un-hide-able. And I think that's also a problem. So fixing strip alone does
> not solve all of them.
> 
> > [...]
> > I'm going to repeat myself here: hiding is an important feature of 
> > changeset-evolution and part of the global state it allows people to 
> > propagate[2].
> 
> I think the hiding itself does not conflict with evolve directly. The fact
> that the new "unhide" command could introduce cycles could conflict with the
> current evolve design. That is a problem of evolve itself to solve, not a
> problem of core hg hidden store.

To add clarification, I think the obsmarker cycle problem (which has been
there for many years) is solvable. As I understand it, there are 3 solutions:

  a) The "auto rebase" proposal [1]
  b) The "node versions" approach I had sent before
  c) Build an obsmarker DAG and introduce new concepts to do marker merges
 and conflict resolution (with some unknown UI)

Currently I prefer "a" because it's much simpler, just works, and I think it
has better behavior in some cases (ex. a stack: A - B - C; amend B -> B',
strip B', then B gets revived automatically. strip B' means the amended
revision B' is a failure, it does not mean both B and B').

[1]: https://goo.gl/wuP2Vl

> > Mixing it with local only elements will not work. They are tons of 
> > simple case where we won't be able to determine a simple and consistent 
> > behavior for it. Even the local case can quickly raise shortcoming to 
> > the above proposal:
> 
> Could you give a practical example (i.e. hg commands) that things do not
> work? That will be more helpful than repeating the abstract concepts.
> 
> As said above, I'm fully aware of that evolve cannot handle cycles. So
> please avoid giving examples that create obsmarker cycles. They're known
> already.
> 
> > [...]
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Kevin Bullock]

I'm not responding to any technical points in this e-mail; I only want to 
address the current tone of the discussion.

> On Apr 11, 2017, at 15:29, Pierre-Yves David  
> wrote:
> 
> On 04/11/2017 12:43 AM, Augie Fackler wrote:
> 
>> This briefly confuses *me* when I trip over it,
> >
>> and I've helped work out some of the design of the feature,
> 
> I'm sorry, but no. you did not helped work out some of the design of the 
> feature. You have barely contributed code touching evolution and not 
> contributed to its concept. You attended some meeting and discussion, that 
> does not automatically makes you an evolution expert.

Please keep these discussions polite and focused. There's no need to minimize 
the time and effort that others, including Augie, have put into making evolve 
something that we, and all our users, can all live with.

>> It sounds like there are no objections to the immediate 'hg hide' proposal, 
>> and the only points of contention are around the expected future 
>> interactions between hiding and exchanged obsolete markers. Is that an 
>> accurate assessment?
> 
> ¿I'm quite confused? how do you end up interpreting:
> 
>   "this kills our ability to complete and ship changeset-evolution
>to all Mercurial user in the next year or two."
> 
> as:
> 
>   "there are no objections"?

It's not at all clear to me that having a single changeset hiding mechanism in 
core prevents completing and shipping evolve. My understanding from discussion 
at the sprint is that obsolescence markers could be used as one source of 
hidden nodes, with no external change in behavior. Let's not engage in 
hyperbolic statements about the future of any given feature; let's discuss the 
tradeoffs of various approaches.

As I read it, there is consensus, even violent agreement, that having a 
mechanism to hide changesets that is separate from (and at a lower level than) 
obsolescence markers is desirable for a wide variety of use cases.

As I read it, there is some agreement that this should be a single unified 
mechanism which evolve should also eventually use. How it should use it is not 
agreed-on. If there is a reason that a different internal mechanism is required 
(or preferable), we should discuss the specific reasons why that's the case.

We all share the goal of making Mercurial even more flexible while keeping it 
easy-to-use for a broad spectrum of users. Doing that is tricky, so it requires 
clear, focused discussion.

pacem in terris / мир / शान्ति / ‎‫سَلاَم‬ / 平和
Kevin R. Bullock

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Jun Wu]

Excerpts from Pierre-Yves David's message of 2017-04-11 22:29:15 +0200:
> [...]
> 
> Mixing it with local only elements will not work. They are tons of 
> simple case where we won't be able to determine a simple and consistent 
> behavior for it. Even the local case can quickly raise shortcoming to 
> the above proposal:
> 
>When an orphan changeset gets hidden by evolution, what should we do
>with its obsolete parent? Was it visible because of the orphan or
>because the user actively "unhide" it some time ago?

This is not hard to answer. "rebase" with some auto flag will hide the
parent if it has no visible children, recursively.  "rebase" with explicit
"-s" and "-d" could choose to not be smart about parents (it may also
choose to be smart with some non-default config set). In either case, if the
user is not satisfied with the behavior, they can always override it
manually by a follow-up hide or unhide command.

I don't think this qualifies as an example about how things are broken
because the behavior is fairly reasonable. I see it as a general inquiry
about the new behavior. I'm happy to answer more inquires like this.

Besides, it's impossible to hide a commit when one of its children is
children is visible. Ryan didn't make this very clear.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Jun Wu]

Excerpts from Pierre-Yves David's message of 2017-04-11 22:29:15 +0200:
> The issue you face here is related to known limitation of `hg strip`. 
> When it removes the changesets, strip should also remove the 
> obsolescence markers leading to them.
> 
> That will be fixed soon. Having strip able to remove obsmarker has been 
> close to the top of the evolve roadmap[1] for some times and it is on 
> the funded path.

strip is just one case. precursor (i.e. len(successors) > 0) hiding is also
un-hide-able. And I think that's also a problem. So fixing strip alone does
not solve all of them.

> [...]
> I'm going to repeat myself here: hiding is an important feature of 
> changeset-evolution and part of the global state it allows people to 
> propagate[2].

I think the hiding itself does not conflict with evolve directly. The fact
that the new "unhide" command could introduce cycles could conflict with the
current evolve design. That is a problem of evolve itself to solve, not a
problem of core hg hidden store.

> Mixing it with local only elements will not work. They are tons of 
> simple case where we won't be able to determine a simple and consistent 
> behavior for it. Even the local case can quickly raise shortcoming to 
> the above proposal:

Could you give a practical example (i.e. hg commands) that things do not
work? That will be more helpful than repeating the abstract concepts.

As said above, I'm fully aware of that evolve cannot handle cycles. So
please avoid giving examples that create obsmarker cycles. They're known
already.

> [...]
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Pierre-Yves David]

On 04/11/2017 12:43 AM, Augie Fackler wrote:

On Fri, Apr 07, 2017 at 12:20:44PM +0200, Pierre-Yves David wrote:

On 04/06/2017 02:46 AM, Jun Wu wrote:

obsstore-based hidden is broken by design, since it cannot unhide commits:

 $ hg export . > patch.txt
 $ hg prune . # could also be done by pulling obsmarkers from a remote
  # peer, or it could be amend / metaedit etc.
 $ hg import --exact --bypass patch.txt # cannot really import the patch


The case you describe above work as intended.


I know it's the original intent, but I'm not sure that the original intent is 
actually a desirable set of outcomes. I've actually tripped over this usability 
wart a decent number of times, usually some variation on this theme:

hg pull marmoute-wip -r some-feature
hg co some-feature
hg rebase -d @

hg strip 'ancestors(some-feature) and not ancestors(@)'

hg pull marmoute-wip -r some-feature # which has not moved since the last time 
I pulled



The issue you face here is related to known limitation of `hg strip`. 
When it removes the changesets, strip should also remove the 
obsolescence markers leading to them.


That will be fixed soon. Having strip able to remove obsmarker has been 
close to the top of the evolve roadmap[1] for some times and it is on 
the funded path.


Once Strip is fixed, we'll be back in a world without natural way to 
locally have an obsmarkers chain ending on a latest successors we do not 
have locally. In other words, if a changset is obsolete in your repo, it 
either has a visible latest successors or it is pruned.


[1] https://www.mercurial-scm.org/wiki/CEDRoadMap


This briefly confuses *me* when I trip over it,

>

and I've helped work out some of the design of the feature,


I'm sorry, but no. you did not helped work out some of the design of the 
feature. You have barely contributed code touching evolution and not 
contributed to its concept. You attended some meeting and discussion, 
that does not automatically makes you an evolution expert.



which strongly suggests to me that the "is listed as a precursor in an
obsmarker means its always hidden" behavior is suboptimal in the real world.
As a rough cut of how we might reconcile usability and theory, I wonder if
the correct path is to have obsmarkers immediately hide things (if possible)
only when they're first introduced (likely through pull), but if the user
takes an action that unhides, we just do that and then there's some advisory
mechanism by which they can be told that things have been obsoleted.


I'm going to repeat myself here: hiding is an important feature of 
changeset-evolution and part of the global state it allows people to 
propagate[2].


Mixing it with local only elements will not work. They are tons of 
simple case where we won't be able to determine a simple and consistent 
behavior for it. Even the local case can quickly raise shortcoming to 
the above proposal:


  When an orphan changeset gets hidden by evolution, what should we do
  with its obsolete parent? Was it visible because of the orphan or
  because the user actively "unhide" it some time ago?

Looking at the distributed case is worse, the global state is a critical 
part of changeset evolution. This proposal is not getting through the 
basic example[3] (pulling the markers before the changesets). It also 
break the branch replacement code[4], a bit we have concrete data that 
users cares a lot about.


The issue with mixing local data with the global state is at the 
theoretical level. This we can probably keep finding new problematic 
cases over and over.


[2] https://www.mercurial-scm.org/wiki/CEDConcept#Global_State
[3] https://www.mercurial-scm.org/wiki/CEDConcept#concrete_examples
[4] 
https://www.mercurial-scm.org/wiki/CEDConcept#Finding_Branch_Replacement_During_Push



By using "hg prune ." you globally marks the changeset as an undesirable
dead-end (obsolete-pruned)[1]. re-obtaining the same changesets in this
repository or in another will not affect this global state. We are just
adding global informations together:

 The changeset exists + a prune marker exists == this changeset is
obsolete.

This global state is a critical part of changeset evolution, and the
visibility rules computed from obsolescence are fully parts of this
state[2].


I'm starting to get nervous about the "global state" business, because it's 
starting to remind me of bookmarks.


Can you elaborate? bookmarks are a very different concept, without 
history, I do not see what relation you see between the two.



Sometimes (often?) user-local state is super important. I think it's feasible 
to have a global obsolescence state as an advisory mechanism for hiding


I've already written multiple documents pointing at why allowing local 
data to permanently mud with hidden-ness from obsolescence is 
problematic. Please go back to them and if needed point out precises 
elements that needs clarification.



(for example "caution: the revision you've checked out 

Re: Hidden Commits in 4.3

[Augie Fackler]

On Fri, Apr 07, 2017 at 12:20:44PM +0200, Pierre-Yves David wrote:
> On 04/06/2017 02:46 AM, Jun Wu wrote:
>> obsstore-based hidden is broken by design, since it cannot unhide commits:
>> 
>>  $ hg export . > patch.txt
>>  $ hg prune . # could also be done by pulling obsmarkers from a remote
>>   # peer, or it could be amend / metaedit etc.
>>  $ hg import --exact --bypass patch.txt # cannot really import the patch
> 
> The case you describe above work as intended.

I know it's the original intent, but I'm not sure that the original intent is 
actually a desirable set of outcomes. I've actually tripped over this usability 
wart a decent number of times, usually some variation on this theme:

hg pull marmoute-wip -r some-feature
hg co some-feature
hg rebase -d @

hg strip 'ancestors(some-feature) and not ancestors(@)'

hg pull marmoute-wip -r some-feature # which has not moved since the last time 
I pulled


This briefly confuses *me* when I trip over it, and I've helped work out some 
of the design of the feature, which strongly suggests to me that the "is listed 
as a precursor in an obsmarker means its always hidden" behavior is suboptimal 
in the real world. As a rough cut of how we might reconcile usability and 
theory, I wonder if the correct path is to have obsmarkers immediately hide 
things (if possible) only when they're first introduced (likely through pull), 
but if the user takes an action that unhides, we just do that and then there's 
some advisory mechanism by which they can be told that things have been 
obsoleted.

> By using "hg prune ." you globally marks the changeset as an undesirable
> dead-end (obsolete-pruned)[1]. re-obtaining the same changesets in this
> repository or in another will not affect this global state. We are just
> adding global informations together:
> 
>  The changeset exists + a prune marker exists == this changeset is
> obsolete.
> 
> This global state is a critical part of changeset evolution, and the
> visibility rules computed from obsolescence are fully parts of this
> state[2].

I'm starting to get nervous about the "global state" business, because it's 
starting to remind me of bookmarks. Sometimes (often?) user-local state is 
super important. I think it's feasible to have a global obsolescence state as 
an advisory mechanism for hiding  (for example "caution: the revision you've 
checked out has been rebased by $user to $revision - you may not want to do any 
work here", or the automatic hiding I mentioned above that's easy to undo) and 
still retain enough of the "magic" that evolution is still a huge win over the 
status quo in other history editing tooling.

It sounds like there are no objections to the immediate 'hg hide' proposal, and 
the only points of contention are around the expected future interactions 
between hiding and exchanged obsolete markers. Is that an accurate assessment?

> 
> The uses can change their mind and bring the changes back to life. We
> currently have a basic command `hg touch` doing so (more useful friendly
> alternative are in project),
> 
> To "bring" this changeset back to life, new information needs to be added.
> In addition, currently the changesets is "brought back" with a different
> hash.  A new changeset node "NEW" and an obsolescence markers marking "NEW
> as a replacement for OLD" are created.
> 
> This hash changes is necessary as the current implementation of evolution
> relies on "monotonic rewrite"[3] that ensure different actions will always
> creates a new nodes. These monotonic rewrites comes with many useful
> property to simplify the problems evolution as to solves[4].  The hash
> change is not the best thing from user experience perspective, but it
> practice the impact is very minor[5].
> 
> They -might- be ways to introduce new mechanism that would lift the need for
> monotonic rewrites while still solving the same problems. So far, such
> mechanism are unknown and concrete plan does not exists. The current roadmap
> focus on being able to ship evolve to all with monotonic rewrites.
> 
> [1]
> https://www.mercurial-scm.org/wiki/CEDConcept#Obsolescence_markers_semantic
> [2] https://www.mercurial-scm.org/wiki/CEDConcept#Global_State
> [3] https://www.mercurial-scm.org/wiki/CEDConcept#Monotonic_rewrite
> [4] https://www.mercurial-scm.org/wiki/CEDConcept#Problem_this_Easily_Solve
> [5] https://www.mercurial-scm.org/wiki/CEDConcept#User_Impact
> 
>> The new ui requires the ability to unhide commits. So hidden-ness should not
>> be affected by obsstore.  That means extra work is needed to make sure
>> commands like amend also write to the new hidden store if they write
>> obsolete markers to hide commits today.
>> 
>> I think most people agree on this. I just want to make it clear as
>> Pierre-Yves does not mention this aspect in his long text.
> 
> So far, I'm not seen a rational for hash-preservation been a -requirement-
> nor a group of people agreeing it is a -requirement-.
> 
> Sure it 

Re: Hidden Commits in 4.3

[Ryan McElroy]

I don't see as big of a conflict between evolve and locally hidden as 
others on this thread. I have a couple of specific responses below. But 
first, I wanted provide some specific examples of how this propsoal 
could/would work (I think a few people [martinvonz?] wanted this to 
ground the discussion). I think the examples will make it clear how this 
hiding proposal would make things a lot better for Mercurial in the 
short term.


First, a confession: My previous replies to this thread were tainted by 
the fact that I didn't know that obsolescence-based hiding was already 
in core. I thought it was added-on by evolve. Thanks to Pierre-Yves, 
Jun, and Durham for pointed out to me privately that I was missing that 
key fact.


Now, some examples of how this simple version of hidden-ness could work 
in practice. The key point is that this new hiddenness can simply 
replace all stripping that happens today, and so we can make it behave 
in the same way that things work today, but without the slow and fragile 
bundling and stripping process.


---

EXAMPLE 1: `hg commit --amend`

* Instead of bundling and stripping the old changeset, we simply create 
a new changeset and hide the old changeset.
* If we have exchanged the old changeset, we would get the old changeset 
back when pulling from the peer we exchanged with (this is exactly what 
happens today with strip-based amend).
* If we unhide the old changeset, we will see the old changeset again 
(this is exactly what happens if we unbundle the old changeset today).
* If obsolescence markers are enabled, we also create one, so that more 
advanced exchange workflows can work well.
* Note that we no longer need to use obsmarkers to hide the "temporary 
amend changeset" as we currently do. We can use the new hiding mechanism 
for this, which is more correct.


In summary, all behaviors are exactly the same as today, but they are 
faster, safer, and easier to recover from.



EXAMPLE 2: `hg shelve`

* Instead of creating a partial changeset, bundling, and aborting, we 
can create a real changeset and then hide it. This is faster and far 
safer than the janky aborted transaction that shelve uses now.
* If the same changeset is pulled from another server, we will show the 
changeset (this is exactly what happens today with abort-based shelve).
* If the same changeset is unbundled, we will show it again (this is 
exactly what happens today with abort-based shelve)
* No need to create obsmarkers for this case; there is no history 
rewriting going on that needs to be globally understood.


In summary, all behaviors are exactly the same as today, but they are 
faster, safer, and it is easier to recover from errors.



EXAMPLE 3: `hg rebase`

* Instead of bundling and stripping the old changesets, we simply mark 
them as hidden.
* If any of the old changesets had ever been exchanged or bundled, 
pulling from a peer with those changesets or unbundling would unhide the 
old changesets. This is the same behavior as with strip-based rebase today.
* If we unhide the old changesets, it's the same as if we unbundle the 
changesets from strip-based rebase today.
* If obsolescence markers are enabled, we also create those for the old 
changesets, so that more advanced exchange workflows can work well.


In summary, all behaviors are exactly the same as today, but they are 
faster and safer, and it is easier to recover from mistakes.


There's an obvious theme here. I'm not trying to overly simplify 
anything here, so let me know if you see issues, but basically I only 
see positives.


Again, the key here is that this provides a much better speed and safety 
win compared the the current strip-based approach. The downsides are 
exactly the same as with current strip-based workflows: old changesets 
can show back up and be confusing. Future extensions like evolve can 
solve those problems, but this proposal is generally only better along 
any axis you compare it to the current way hg works.


---

I strongly feel that we should move in this direction -- it would be 
huge win for everyone who uses mercurial over the workflows that they 
have today. I hope these examples can help everyone see how this new 
hiddenness concept can make all these workflows a lot better inside core 
within the next 3 months. I also hope that the interactions with 
obsolescence markers I've outlined show why this proposal doesn't 
interact badly with evolve's global state paradigm either. It also 
nicely solves concerns like those that come up with Kostia's obsshelve 
series as well -- we will have a way to hide that is separate from 
obsolescence, so everyone should be a little more happy.



The final topic to discuss is how does this new hidden concept interact 
with evolve's hiddenness concepts? I've convered this below in my 
response to Durham and Pierre-Yves's email thread. The short version is 
that I believe both concepts can co-exist with no issues. Hiddenness 
doesn't need to affect 

Re: Hidden Commits in 4.3

[Pierre-Yves David]

On 04/06/2017 12:02 AM, Durham Goode wrote:

On 4/5/17 4:06 AM, Pierre-Yves David wrote:

[…]
The change proposed to evolution is problematic. As explain in my reply
to the other thread[1], at minimum, removing the link between
obsolescence and visibility is destroying the "global-state" property we
currently have. This put in disarray the whole ability to use evolution
to exchange and collaborate on mutable history, the very thing evolution
has been built for. I've not seen strong-enough rationals for derailing
the current evolution plan and design.


The simpler implementation model, the simpler mental model, the simpler
ability for commands to use hiding, and the lack of tie-in with other
more divisive features is the reason for this proposal. With this
proposal we can have the majority of the benefits quickly, without
notable changes to the existing core UI, and in a intuitive form for users.


I agree that having a local-only hiding mechanism would be a win.

However obsolescence and the hiding computed from it must stay fully 
independent from it. Changeset-evolution relies on building a global 
state[1] mixing it with local-only elements breaks this property.



I believe these benefits outweigh the downside to evolve's current goal,
[…]
I think other members of the community would have to weigh in on whether
this trade off is worth it, since it is a subjective decision.


This is not a "small downside" to evolution goal. This is a full stop to 
our ability to ship changeset-evolution.


The founding goal of changeset-evolution is to allow exchange and 
collaboration around mutable history[2]. The local benefit are "just" a 
nice side effect, but the concept is fully design around distribution. 
This is a problem no other DVCS solves and we have a plan, on track, to 
complete the concept and solve it. That will give Mercurial clear edge 
over others.


The current proposal is destroying at least one pillard currently 
holding the changeset-evolution plan (global state), without 
replacement. As of today, this kills our ability to complete and ship 
changeset-evolution to all Mercurial user in the next year or two.


This is not a small subjective adjustment, this is a full 
discontinuation of concept critical to the future of Mercurial.


Cheers,

--
Pierre-Yves David

[1] https://www.mercurial-scm.org/wiki/CEDConcept#Global_State
[2] https://www.mercurial-scm.org/wiki/CEDConcept#Changeset_Evolution_Goal
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Pierre-Yves David]

On 04/06/2017 02:47 PM, Ryan McElroy wrote:
> […]

That's why I want to focus on a scalable hidden storage solution that
everyone can use (including evolve),


To clarify this point. They are no need to plug evolution to a new 
storage system, the current approach is scalable[1]. In addition, as 
explain in my reply to Jun[2] hiding from obs-markers is a static 
computation from a global-state[3] and we cannot let other mechanism 
affects it. So we should not mix its implementation with something user 
manipulable.


Both hiding source can peacefully coexists, and I agree that local 
hiding is useful. So let us implement local hiding independently from 
obsolescence.


[1] current implementation is slow, but adding a trivial "hasmarkers" 
cache will solve this. This will happens in the next month as side 
effect of my work on obsmarkers discovery.
[2] 
https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-April/096360.html

[3] https://www.mercurial-scm.org/wiki/CEDConcept#Global_State

--
Pierre-Yves David
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Pierre-Yves David]

On 04/06/2017 02:46 AM, Jun Wu wrote:

obsstore-based hidden is broken by design, since it cannot unhide commits:

  $ hg export . > patch.txt
  $ hg prune . # could also be done by pulling obsmarkers from a remote
   # peer, or it could be amend / metaedit etc.
  $ hg import --exact --bypass patch.txt # cannot really import the patch


The case you describe above work as intended.

By using "hg prune ." you globally marks the changeset as an undesirable 
dead-end (obsolete-pruned)[1]. re-obtaining the same changesets in this 
repository or in another will not affect this global state. We are just 
adding global informations together:


  The changeset exists + a prune marker exists == this changeset is 
obsolete.


This global state is a critical part of changeset evolution, and the 
visibility rules computed from obsolescence are fully parts of this 
state[2].


The uses can change their mind and bring the changes back to life. We 
currently have a basic command `hg touch` doing so (more useful friendly 
alternative are in project),


To "bring" this changeset back to life, new information needs to be 
added. In addition, currently the changesets is "brought back" with a 
different hash.  A new changeset node "NEW" and an obsolescence markers 
marking "NEW as a replacement for OLD" are created.


This hash changes is necessary as the current implementation of 
evolution relies on "monotonic rewrite"[3] that ensure different actions 
will always creates a new nodes. These monotonic rewrites comes with 
many useful property to simplify the problems evolution as to solves[4]. 
 The hash change is not the best thing from user experience 
perspective, but it practice the impact is very minor[5].


They -might- be ways to introduce new mechanism that would lift the need 
for monotonic rewrites while still solving the same problems. So far, 
such mechanism are unknown and concrete plan does not exists. The 
current roadmap focus on being able to ship evolve to all with monotonic 
rewrites.


[1] 
https://www.mercurial-scm.org/wiki/CEDConcept#Obsolescence_markers_semantic

[2] https://www.mercurial-scm.org/wiki/CEDConcept#Global_State
[3] https://www.mercurial-scm.org/wiki/CEDConcept#Monotonic_rewrite
[4] https://www.mercurial-scm.org/wiki/CEDConcept#Problem_this_Easily_Solve
[5] https://www.mercurial-scm.org/wiki/CEDConcept#User_Impact


The new ui requires the ability to unhide commits. So hidden-ness should not
be affected by obsstore.  That means extra work is needed to make sure
commands like amend also write to the new hidden store if they write
obsolete markers to hide commits today.

I think most people agree on this. I just want to make it clear as
Pierre-Yves does not mention this aspect in his long text.


So far, I'm not seen a rational for hash-preservation been a 
-requirement- nor a group of people agreeing it is a -requirement-.


Sure it would be nice, but "nice" is different from -required-.

This is probably a core point we need to iron out in this discussion.

On 04/05/2017 11:22 PM, Jun Wu wrote:

[…]
There are some questions that I'm waiting for your answers:

  - Define "user intention" about adding a obsmarker formally [1]


I've started formally writing some of the evolution concept.
Does this section fits your need:

* 
https://www.mercurial-scm.org/wiki/CEDConcept#obs-marker:_General_Principle



  - Why "an unified hidden store" does not work [2]


Changeset evolution is based on a global state created by the 
obsolescence markers. The hiding from evolution is a computed property 
of this global state. It -cannot- be delegated to an independent, local- 
only, user manipulable hiding mechanism. Hiding from obsolescence 
markers is a "static" property that -must- remains independent from the 
local hiding mechanism (See previous [2] for details).


Local-hiding has its own merit and would help various usecase. The two 
mechanism can perfectly exists side by side.


On 04/05/2017 11:50 PM, Jun Wu wrote:

I think it's unnecessary to have "internal changeset" concept separated from
the general-purposed hidden.


They are some case where having explicit internal changeset are useful:

Case 1: looking at hidden changeset,

  When the user want to look or query the set of hidden changeset (eg: 
user remember he hided something he now needs). Commands will be run to 
see `hg log --hidden` or even select `hg log --hidden --rev 'hidden() 
and file('foo/bar') and date(-7)'`.
  Without an internal changesets, this command will display and select 
internal changesets that are irrelevant/scary to the user. Having 
explicit "internal" changesets allow to filter them out of such search 
from the user.


Case 2: History of hiding

  I can expect user to request a command displaying the history of what 
they hide/archived[6]. They could use it to undo things. Without 
distinction between archived and real changesets, this history will list 
hiding and changesets from internal operation, 

Re: Hidden Commits in 4.3

[Durham Goode]

On 4/5/17 2:28 AM, Simon Farnsworth wrote:

On 05/04/2017 02:11, Durham Goode wrote:

There's been a lot of discussion about how to hide and unhide commits
lately [0][1], and I feel the complexity of our current approach is
hurting our ability to reason about it, making it impossible to make
progress.

I would like to formally propose a new pattern for dealing with hidden
commits, along with the concrete steps to getting it enabled in core by
default by the August release.

The proposal is quite concise, so check out this 1-page Google doc for
the details and to comment:

https://urldefense.proofpoint.com/v2/url?u=https-3A__goo.gl_7DJ9AI=DwIGaQ=5VD0RTtNlTh3ycd41b3MUw=mEgSWILcY4c4W3zjApBQLA=cTl6eEfd0SyhW5M6ZM65eJ2ZldOrwdJ_KM0pKS9tZ5M=T30BYaK4BkpV_DBoFTcWZm0_VGBJG0FEHA5jUJqcxXw=




Is there any reason why this is a Google Doc, not a HiddenCommitsPlan on
the mercurial-scm.org wiki?


I was hoping people would comment on specific sections, so we could have 
threaded discussions before committing it to a more permanent wiki page.

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Ryan McElroy]

(text re-arranged for clarity)


On 4/6/17 1:46 AM, Jun Wu wrote:

Excerpts from Pierre-Yves David's message of 2017-04-06 01:01:19 +0200:

(important bit about how we -already- have a generic hiding API is at
the end of the email).

Thanks. I used that and had a working general-purposed, phase-based hidden
demo at the sprint [1]. So I think we are pretty familiar with the API
already.

[1]: https://www.mercurial-scm.org/wiki/4.2sprint#Enhanced_repoview


To be fair to Pierre-Yves, I had forgotten about it (though once he 
mentioned it I remember that he had mentioned it to me before some years 
ago).


However, if it's so easy to add new forms of hidden commits, why do we 
care how evolve happens to implement hiddenness? Why not just add a 
bitmap based hiddenness in core (so it's fast and scalable), let evolve 
be a user of that system, and let other tools that want to do 
hiding/unhiding use that system in a different way?


I guess what I fundamentally don't understand is why evolve (an 
extension not yet in core) is intertwined with the discussion of a 
hidden concept that it could use. Let evolve do things in its more 
advanced way. Let core tools do things in a 
better-than-today-but-not-as-advanced-way (particular since all of the 
history rewriting tools are *already* extensions!)


On 4/6/17 1:46 AM, Jun Wu wrote:

obsstore-based hidden is broken by design, since it cannot unhide commits:

   $ hg export . > patch.txt
   $ hg prune . # could also be done by pulling obsmarkers from a remote
# peer, or it could be amend / metaedit etc.
   $ hg import --exact --bypass patch.txt # cannot really import the patch

The new ui requires the ability to unhide commits. So hidden-ness should not
be affected by obsstore.  That means extra work is needed to make sure
commands like amend also write to the new hidden store if they write
obsolete markers to hide commits today.

I think most people agree on this. I just want to make it clear as
Pierre-Yves does not mention this aspect in his long text.



I think this is too harsh and not a fair assessment of what evolve does. 
It is possible to unhide commits -- by putting a bookmark there, by 
checking it out, by adding a local tag, by creating an extension like 
inhibit, etc.


The import case above could have a better UI build around it that makes 
it clear what happened: "imported patch already has a newer version 
available", for example. That's not unreasonable.


However, I agree that it's not the easiest user experience to master -- 
when I import a patch, or unbundle, or whatever, I want to see the patch 
show up in my log! That's what I expect.


Jun, I think I agree with you that "evolve doesn't behave exactly how we 
want". But it's unfair to call it "broken by design". It's designed to 
do something different (and more ambitious) than where we are aiming 
right now. That's fine -- as should be clear from the extensions we 
(Facebook) have made available [1], we don't expose the full evolve UI 
internally anyway.


That's why I want to focus on a scalable hidden storage solution that 
everyone can use (including evolve), and then we can improve the current 
extensions (in and out of core) to use this new hidden storage. Evolve 
can continue to do it's thing that will enable safe exchange of 
rewritten draft commits. In my opinions, there's no need to try to 
derail evolve's current roadmap as part of building that. I think evolve 
can remain an interesting experience on top of whatever we're building 
with this proposal. I don't see direct conflict.



[1] for example, 'hg rebase --restack' inside of the 'fbamend' extension
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Jun Wu]

obsstore-based hidden is broken by design, since it cannot unhide commits:

  $ hg export . > patch.txt
  $ hg prune . # could also be done by pulling obsmarkers from a remote
   # peer, or it could be amend / metaedit etc.
  $ hg import --exact --bypass patch.txt # cannot really import the patch

The new ui requires the ability to unhide commits. So hidden-ness should not
be affected by obsstore.  That means extra work is needed to make sure
commands like amend also write to the new hidden store if they write
obsolete markers to hide commits today.

I think most people agree on this. I just want to make it clear as
Pierre-Yves does not mention this aspect in his long text.

Excerpts from Pierre-Yves David's message of 2017-04-06 01:01:19 +0200:
> (important bit about how we -already- have a generic hiding API is at 
> the end of the email).

Thanks. I used that and had a working general-purposed, phase-based hidden
demo at the sprint [1]. So I think we are pretty familiar with the API
already.

[1]: https://www.mercurial-scm.org/wiki/4.2sprint#Enhanced_repoview
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Pierre-Yves David]

(important bit about how we -already- have a generic hiding API is at 
the end of the email).


On 04/05/2017 04:06 PM, Ryan McElroy wrote:

On 4/5/17 12:06 PM, Pierre-Yves David wrote:

On 04/05/2017 03:11 AM, Durham Goode wrote:



I think we really needs to take a step back here. Before thinking
about unification, I think we needs a clean definition of what we are
doing here. As mentioned in another thread[2], they are at least three
identified categories of things we want to hide. obsolete, internal,
and some local only hiding. The first two are quite well defined now,
the last one less so.


I disagree. I don't think we need to understand all the ways that things
might be hidden before coming up with a good way to hide things that
core hg and extensions can use in whatever way they wish.
Generic hiding allows the deeply needed improvements to core hg the Greg 
described in
the other thread, without waiting for evolve to be complete (which is
still a ways off).


I agree, that non-user facing bits of the API do not needs to wait. And 
by chances we already have that reusable hiding ability. (I'm addressing 
this lower in this email). What we lack is the storage story for the 
local-only use-case. (that should not be quite easy to do)



I think we should currently refocus the discussion on the local-only
hiding. What are its usecases, what are the wished behavior. A large
part of what Durham just wrote can directly be re-used for that. But
re-centered on the local usecase, with the changes to evolution core
behavior.


No, please let's not require solving a potential use case before
building the mechanism that will allow us to build use cases we already
know we want and need.


* UI to view and manipulate it (because we probably want one and Durham 
proposal includes one).


UI is the harder part and I think it is important to gather use-cases 
before designing the UI. We already know we want it to replace the 
current usage of strip. is there other known planned usage we should 
takes in account?



Once we have a clear view of the three concepts usecase, behavior and
constraint we can start looking for synergy between them.


We already know the synergy -- we want ways to hide things. Let's build
that and let each new concept use it in appropriate ways without trying
to solve every potential problem first. Everybody wants hiding. So let's
add generic hiding that can be exposed in various ways, including by
evolve.


By chances, 4.25 years ago[1], I've already implemented a generic way to 
hide revision. As mentioned before, the current hiding mechanism is 
fully extensible and anything can plugs into it. All is controlled by 
this function here:


  https://www.mercurial-scm.org/repo/hg/file/tip/mercurial/repoview.py#l24

Currently only obsolescence related data are feed into this, but it is 
trivial to provides data from other mechanisms.


For example adding hiding from an "internal" phases is as simple as 
adding this line to the function:


   revs = revs | repo._phasecache._phasesets[phases.internal]

Enforcement of the current behavior we need around unhiding (working 
copy parents, non-hideable descendant, bookmark, etc) is made with some 
some small extra logic above this function. So we already have all the 
exception behavior we need for internal changesets.


Data from a local-hiding mechanism could also be fed directly into this 
function. We computes hidden revs from whatever storage we pick[2] and 
we adds that to the set and voila! We could have a proof of concept in 
an extension in a couple of hours and a full featured solution under a 
couple of days.


Conclusion
==

Existing API is already fully extensible and usable by other mechanisms. 
We just need to define a storage for local-hiding. They are no needed 
for a full rework of the area. Touching evolution is also unnecessary to 
get local hiding feature.


Speed-wise, they are many easy optimization available around the hiding 
code waiting to happesn. So if they are any performance concerns with 
the current naive implementation, they should be trivial to lift.


--
Pierre-Yves David


[1] https://www.mercurial-scm.org/repo/hg/rev/1f35d6737ed8
[2] Currently I suspect a storing "root-head" ranges would be the best 
option we have.

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Martin von Zweigbergk via Mercurial-devel]

On Wed, Apr 5, 2017 at 3:02 PM, Durham Goode  wrote:
> On 4/5/17 4:06 AM, Pierre-Yves David wrote:
>>
>> On 04/05/2017 03:11 AM, Durham Goode wrote:
>>>
>>> There's been a lot of discussion about how to hide and unhide commits
>>> lately [0][1], and I feel the complexity of our current approach is
>>> hurting our ability to reason about it, making it impossible to make
>>> progress.
>>>
>>> I would like to formally propose a new pattern for dealing with hidden
>>> commits, along with the concrete steps to getting it enabled in core by
>>> default by the August release.
>>>
>>> The proposal is quite concise, so check out this 1-page Google doc for
>>> the details and to comment:
>>>
>>>
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__goo.gl_7DJ9AI=DwICaQ=5VD0RTtNlTh3ycd41b3MUw=nuarHzhP1wi1T9iURRCj1A=Yj1C1AtQvsolF0hlwM1JkxxKSLbV6vXg7DoMNDxaG1M=Ek8tm4MPVTL8WZOf4TuwRANa0lR_mzze8jCT5MdvXTk=
>>>
>>> If people find this approach promising, I can commit to trying to get
>>> this upstream by the August release. So if you have questions or
>>> concerns, please let me know so I can address them.
>>
>>
>> Thanks for writing this proposal.
>>
>> The change proposed to evolution is problematic. As explain in my reply
>> to the other thread[1], at minimum, removing the link between
>> obsolescence and visibility is destroying the "global-state" property we
>> currently have. This put in disarray the whole ability to use evolution
>> to exchange and collaborate on mutable history, the very thing evolution
>> has been built for. I've not seen strong-enough rationals for derailing
>> the current evolution plan and design.
>
>
> The simpler implementation model, the simpler mental model, the simpler
> ability for commands to use hiding, and the lack of tie-in with other more
> divisive features is the reason for this proposal. With this proposal we can
> have the majority of the benefits quickly, without notable changes to the
> existing core UI, and in a intuitive form for users.
>
> I believe these benefits outweigh the downside to evolve's current goal, in
> particular since the actual end experience of evolve is still under
> discussion.
>
> I think other members of the community would have to weigh in on whether
> this trade off is worth it, since it is a subjective decision.

As I said before, I like the proposal. I would be happy to have a more
thorough discussion of how various use cases would work out with your
proposal, though. The arguments I have seen have been too abstract for
me to really care about. I like more concrete points about what will
and will not work. The other arguments are just skim over and forget.
Maybe others like the abstract arguments better, so feel free to
ignore me if I'm the exception here.

>
>> I think we really needs to take a step back here. Before thinking about
>> unification, I think we needs a clean definition of what we are doing
>> here. As mentioned in another thread[2], they are at least three
>> identified categories of things we want to hide. obsolete, internal, and
>> some local only hiding. The first two are quite well defined now, the
>> last one less so.
>
>
> Others should chime in here. I think this proposal is the step back and
> provides a clean definition of what hiding is and a unified approach that
> can be applied intuitively to hiding needs.
>
> Even the current obsolete-markers-cause-things-to-be-hidden functionality
> could be implemented on top of this, if we wanted it to. We just gain the
> ability for users to unhide those commits if they want, which I think is an
> improvement for the user experience.
>
>> I think we should currently refocus the discussion on the local-only
>> hiding. What are its usecases, what are the wished behavior. A large
>> part of what Durham just wrote can directly be re-used for that. But
>> re-centered on the local usecase, with the changes to evolution core
>> behavior.
>>
>> Once we have a clear view of the three concepts usecase, behavior and
>> constraint we can start looking for synergy between them.
>
>
> I agree with Ryan here. I think the use case here is already clear: we want
> the ability to hide and unhide commits. This proposal will let us experiment
> with use cases more freely, and we should not limit the discussion here to
> avoid affecting existing behavior, nor delve down rabbit holes when there's
> a simple unified proposal that many people seem satisfied with.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Durham Goode]

On 4/5/17 4:06 AM, Pierre-Yves David wrote:

On 04/05/2017 03:11 AM, Durham Goode wrote:

There's been a lot of discussion about how to hide and unhide commits
lately [0][1], and I feel the complexity of our current approach is
hurting our ability to reason about it, making it impossible to make
progress.

I would like to formally propose a new pattern for dealing with hidden
commits, along with the concrete steps to getting it enabled in core by
default by the August release.

The proposal is quite concise, so check out this 1-page Google doc for
the details and to comment:

https://urldefense.proofpoint.com/v2/url?u=https-3A__goo.gl_7DJ9AI=DwICaQ=5VD0RTtNlTh3ycd41b3MUw=nuarHzhP1wi1T9iURRCj1A=Yj1C1AtQvsolF0hlwM1JkxxKSLbV6vXg7DoMNDxaG1M=Ek8tm4MPVTL8WZOf4TuwRANa0lR_mzze8jCT5MdvXTk=

If people find this approach promising, I can commit to trying to get
this upstream by the August release. So if you have questions or
concerns, please let me know so I can address them.


Thanks for writing this proposal.

The change proposed to evolution is problematic. As explain in my reply
to the other thread[1], at minimum, removing the link between
obsolescence and visibility is destroying the "global-state" property we
currently have. This put in disarray the whole ability to use evolution
to exchange and collaborate on mutable history, the very thing evolution
has been built for. I've not seen strong-enough rationals for derailing
the current evolution plan and design.


The simpler implementation model, the simpler mental model, the simpler 
ability for commands to use hiding, and the lack of tie-in with other 
more divisive features is the reason for this proposal. With this 
proposal we can have the majority of the benefits quickly, without 
notable changes to the existing core UI, and in a intuitive form for users.


I believe these benefits outweigh the downside to evolve's current goal, 
in particular since the actual end experience of evolve is still under 
discussion.


I think other members of the community would have to weigh in on whether 
this trade off is worth it, since it is a subjective decision.



I think we really needs to take a step back here. Before thinking about
unification, I think we needs a clean definition of what we are doing
here. As mentioned in another thread[2], they are at least three
identified categories of things we want to hide. obsolete, internal, and
some local only hiding. The first two are quite well defined now, the
last one less so.


Others should chime in here. I think this proposal is the step back and 
provides a clean definition of what hiding is and a unified approach 
that can be applied intuitively to hiding needs.


Even the current obsolete-markers-cause-things-to-be-hidden 
functionality could be implemented on top of this, if we wanted it to. 
We just gain the ability for users to unhide those commits if they want, 
which I think is an improvement for the user experience.



I think we should currently refocus the discussion on the local-only
hiding. What are its usecases, what are the wished behavior. A large
part of what Durham just wrote can directly be re-used for that. But
re-centered on the local usecase, with the changes to evolution core
behavior.

Once we have a clear view of the three concepts usecase, behavior and
constraint we can start looking for synergy between them.


I agree with Ryan here. I think the use case here is already clear: we 
want the ability to hide and unhide commits. This proposal will let us 
experiment with use cases more freely, and we should not limit the 
discussion here to avoid affecting existing behavior, nor delve down 
rabbit holes when there's a simple unified proposal that many people 
seem satisfied with.

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Fwd: Hidden Commits in 4.3

[Durham Goode]

On 4/5/17 9:58 AM, Martin von Zweigbergk wrote:

*sigh* I was confused by the gmail/inbox default reply yet again. The
below was meant not just for Greg.

Doesn't really add much value at this point :-P


-- Forwarded message --
From: Martin von Zweigbergk <martinv...@google.com>
Date: Tue, Apr 4, 2017 at 9:20 PM
Subject: Re: Hidden Commits in 4.3
To: Gregory Szorc <gregory.sz...@gmail.com>


On Tue, Apr 4, 2017 at 8:58 PM, Gregory Szorc <gregory.sz...@gmail.com> wrote:

On Tue, Apr 4, 2017 at 6:11 PM, Durham Goode <dur...@fb.com> wrote:


There's been a lot of discussion about how to hide and unhide commits
lately [0][1], and I feel the complexity of our current approach is hurting
our ability to reason about it, making it impossible to make progress.

I would like to formally propose a new pattern for dealing with hidden
commits, along with the concrete steps to getting it enabled in core by
default by the August release.

The proposal is quite concise, so check out this 1-page Google doc for the
details and to comment:

https://urldefense.proofpoint.com/v2/url?u=https-3A__goo.gl_7DJ9AI=DwIBaQ=5VD0RTtNlTh3ycd41b3MUw=nuarHzhP1wi1T9iURRCj1A=9ujoyPVTKANNXyeB-TWto89n1J_Wgrj9nn1gn38N_xE=Dol8nY676yUOBn4FDWvmMdPgoxALSIH2aLElSDOA760=

If people find this approach promising, I can commit to trying to get this
upstream by the August release. So if you have questions or concerns, please
let me know so I can address them.


[0] see: "obsolete: track node versions"
[1] see: "repo: add an ability to hide nodes in an appropriate way"



I lost track of all the developments in the various ongoing threads about
hidden changesets and obsolescence versioning. So thank you for taking the
time to write a concise summary.

I emphatically support your proposal of introducing a non-obsolescence
mechanism for hiding changesets. It solves a very real and very painful
deficiency in the storage model (strips required to rewrite history) without
forcing complicated workflows and concepts onto users (evolve). As I said at
the sprint, the horrible performance associated with history rewriting
without obsolescence enabled is one of Mozilla's biggest pain points for
large repos. I can't in good faith recommend evolve to all users because it
is too complicated and can lead to unexpected behavior.

I think this proposal for hidden changesets solves the performance problem
while not preventing obsolescence/evolve from making progress in its problem
space. It also doesn't seem to be as cognitively difficult on users.


I agree.



There are some areas that need flushed out of course. Obviously storage. But
I think the more important problems are the user-facing ones. e.g. what
happens if a user updates to a hidden changeset and makes a commit? Do we
allow that? I think there needs to be some kind of UX to discourage
accidental usage of hidden changesets or at least warnings when things
become unhidden because that may be unexpected.


Good question. Durham didn't say it explicitly, but I think you're
right that checking out a hidden commit probably unhides it (and its
ancestors).


The doc mentions 'hg checkout hash' unhides it.  Along with bookmark 
placement and the explicit 'hg unhide' command.





That's why things in the doc
like `hg checkout HASH` automatically unhiding a changeset make me a little
nervous.


I agree that some warning/note is probably appropriate to let the user
know that we unhid it.


Same with `hg pull` automatic unhiding. It seems like it is almost
too easy to create divergence without any helpful mechanism (obsolescence)
to recover from it. What I'm still trying to contemplate are the workflows
where this would most likely occur. If it is only rare scenarios or things
we don't want users doing, I think it can be tolerable.


We need concrete uses cases to discuss this. The obvious ones seem
safe and correct to me (ex: I pull from Greg and hide a branch I don't
care about, then I pull again and it reappears -- seems right to me).


This is mainly there for backwards compatibility reasons. If people have 
automation that is stripping commits and then potentially reapplying the 
commits (via unbundle or pull), then not having those commits be visible 
after the pull/unbundle in this new world would be a breaking change.


This let's us enable the feature by default with less risk to existing 
deployments.




I'm also curious if you intend to add a new repo requirement for this. If
not, what's the story for a legacy client interacting with repos with the
new hidden mechanism? IMO that experience is bad enough (legacy client would
see a bunch of visible changesets that should be hidden) that I think it
warrants locking out old clients.


Even though the hiddenness is not exchanged, it doesn't have to mean
that the hiddenness is not involved in exchange, does it? I would
think the repo would look like it didn't have the hidden nodes. I'm
sure it's trickier

Fwd: Hidden Commits in 4.3

[Martin von Zweigbergk via Mercurial-devel]

*sigh* I was confused by the gmail/inbox default reply yet again. The
below was meant not just for Greg.

Doesn't really add much value at this point :-P


-- Forwarded message --
From: Martin von Zweigbergk <martinv...@google.com>
Date: Tue, Apr 4, 2017 at 9:20 PM
Subject: Re: Hidden Commits in 4.3
To: Gregory Szorc <gregory.sz...@gmail.com>


On Tue, Apr 4, 2017 at 8:58 PM, Gregory Szorc <gregory.sz...@gmail.com> wrote:
> On Tue, Apr 4, 2017 at 6:11 PM, Durham Goode <dur...@fb.com> wrote:
>>
>> There's been a lot of discussion about how to hide and unhide commits
>> lately [0][1], and I feel the complexity of our current approach is hurting
>> our ability to reason about it, making it impossible to make progress.
>>
>> I would like to formally propose a new pattern for dealing with hidden
>> commits, along with the concrete steps to getting it enabled in core by
>> default by the August release.
>>
>> The proposal is quite concise, so check out this 1-page Google doc for the
>> details and to comment:
>>
>> https://goo.gl/7DJ9AI
>>
>> If people find this approach promising, I can commit to trying to get this
>> upstream by the August release. So if you have questions or concerns, please
>> let me know so I can address them.
>>
>>
>> [0] see: "obsolete: track node versions"
>> [1] see: "repo: add an ability to hide nodes in an appropriate way"
>
>
> I lost track of all the developments in the various ongoing threads about
> hidden changesets and obsolescence versioning. So thank you for taking the
> time to write a concise summary.
>
> I emphatically support your proposal of introducing a non-obsolescence
> mechanism for hiding changesets. It solves a very real and very painful
> deficiency in the storage model (strips required to rewrite history) without
> forcing complicated workflows and concepts onto users (evolve). As I said at
> the sprint, the horrible performance associated with history rewriting
> without obsolescence enabled is one of Mozilla's biggest pain points for
> large repos. I can't in good faith recommend evolve to all users because it
> is too complicated and can lead to unexpected behavior.
>
> I think this proposal for hidden changesets solves the performance problem
> while not preventing obsolescence/evolve from making progress in its problem
> space. It also doesn't seem to be as cognitively difficult on users.

I agree.

>
> There are some areas that need flushed out of course. Obviously storage. But
> I think the more important problems are the user-facing ones. e.g. what
> happens if a user updates to a hidden changeset and makes a commit? Do we
> allow that? I think there needs to be some kind of UX to discourage
> accidental usage of hidden changesets or at least warnings when things
> become unhidden because that may be unexpected.

Good question. Durham didn't say it explicitly, but I think you're
right that checking out a hidden commit probably unhides it (and its
ancestors).

> That's why things in the doc
> like `hg checkout HASH` automatically unhiding a changeset make me a little
> nervous.

I agree that some warning/note is probably appropriate to let the user
know that we unhid it.

> Same with `hg pull` automatic unhiding. It seems like it is almost
> too easy to create divergence without any helpful mechanism (obsolescence)
> to recover from it. What I'm still trying to contemplate are the workflows
> where this would most likely occur. If it is only rare scenarios or things
> we don't want users doing, I think it can be tolerable.

We need concrete uses cases to discuss this. The obvious ones seem
safe and correct to me (ex: I pull from Greg and hide a branch I don't
care about, then I pull again and it reappears -- seems right to me).

>
> I'm also curious if you intend to add a new repo requirement for this. If
> not, what's the story for a legacy client interacting with repos with the
> new hidden mechanism? IMO that experience is bad enough (legacy client would
> see a bunch of visible changesets that should be hidden) that I think it
> warrants locking out old clients.

Even though the hiddenness is not exchanged, it doesn't have to mean
that the hiddenness is not involved in exchange, does it? I would
think the repo would look like it didn't have the hidden nodes. I'm
sure it's trickier than that, but I'm not sure how.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Simon Farnsworth]

On 05/04/2017 02:11, Durham Goode wrote:

There's been a lot of discussion about how to hide and unhide commits
lately [0][1], and I feel the complexity of our current approach is
hurting our ability to reason about it, making it impossible to make
progress.

I would like to formally propose a new pattern for dealing with hidden
commits, along with the concrete steps to getting it enabled in core by
default by the August release.

The proposal is quite concise, so check out this 1-page Google doc for
the details and to comment:

https://urldefense.proofpoint.com/v2/url?u=https-3A__goo.gl_7DJ9AI=DwIGaQ=5VD0RTtNlTh3ycd41b3MUw=mEgSWILcY4c4W3zjApBQLA=cTl6eEfd0SyhW5M6ZM65eJ2ZldOrwdJ_KM0pKS9tZ5M=T30BYaK4BkpV_DBoFTcWZm0_VGBJG0FEHA5jUJqcxXw=



Is there any reason why this is a Google Doc, not a HiddenCommitsPlan on 
the mercurial-scm.org wiki?



If people find this approach promising, I can commit to trying to get
this upstream by the August release. So if you have questions or
concerns, please let me know so I can address them.


[0] see: "obsolete: track node versions"
[1] see: "repo: add an ability to hide nodes in an appropriate way"
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.mercurial-2Dscm.org_mailman_listinfo_mercurial-2Ddevel=DwIGaQ=5VD0RTtNlTh3ycd41b3MUw=mEgSWILcY4c4W3zjApBQLA=cTl6eEfd0SyhW5M6ZM65eJ2ZldOrwdJ_KM0pKS9tZ5M=NFt0PtRadAIuciZ1Qg6kefWsdRxaXeOt_aG7bSnFTcQ=



--
Simon Farnsworth
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Jun Wu]

Excerpts from Pierre-Yves David's message of 2017-04-05 13:06:04 +0200:
> I think we really needs to take a step back here. Before thinking about 
> unification, I think we needs a clean definition of what we are doing 
> here. As mentioned in another thread[2], they are at least three 
> identified categories of things we want to hide. obsolete, internal, and 
> some local only hiding. The first two are quite well defined now, the 
> last one less so.

Why it has to be 3 categories of hiddenness? You explained how 3 categories
work. Maybe I missed something, but I cannot find proof about why a unified
single category does not work. I also failed to find other people supporting
the 3 categories idea. If it is for some rare cases that are not useful in
common workflows, I wonder whether it is worth the additional code and
concept complexity (comparing with a single unified hidden category).
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Ryan McElroy]

On 4/5/17 12:06 PM, Pierre-Yves David wrote:

On 04/05/2017 03:11 AM, Durham Goode wrote:

There's been a lot of discussion about how to hide and unhide commits
lately [0][1], and I feel the complexity of our current approach is
hurting our ability to reason about it, making it impossible to make
progress.

I would like to formally propose a new pattern for dealing with hidden
commits, along with the concrete steps to getting it enabled in core by
default by the August release.

The proposal is quite concise, so check out this 1-page Google doc for
the details and to comment:

https://urldefense.proofpoint.com/v2/url?u=https-3A__goo.gl_7DJ9AI=DwICaQ=5VD0RTtNlTh3ycd41b3MUw=Jw8rundaE7TbmqBYd1txIQ=_EDBw1CsTCN9AalNXSST2Y62jINxpEATYeSxmKe5Brw=lcEiWk2_jcVL_lnSZF-y7QwHxlG_0__q7hK-SigQqm4= 


If people find this approach promising, I can commit to trying to get
this upstream by the August release. So if you have questions or
concerns, please let me know so I can address them.


Thanks for writing this proposal.

The change proposed to evolution is problematic. As explain in my 
reply to the other thread[1], at minimum, removing the link between 
obsolescence and visibility is destroying the "global-state" property 
we currently have. This put in disarray the whole ability to use 
evolution to exchange and collaborate on mutable history, the very 
thing evolution has been built for. I've not seen strong-enough 
rationals for derailing the current evolution plan and design.


I agree that the hiding proposal should focus on hiding instead of 
changes to evolution/obsolescence. We should be able to get the former 
without any changes to the latter.




I think we really needs to take a step back here. Before thinking 
about unification, I think we needs a clean definition of what we are 
doing here. As mentioned in another thread[2], they are at least three 
identified categories of things we want to hide. obsolete, internal, 
and some local only hiding. The first two are quite well defined now, 
the last one less so.


I disagree. I don't think we need to understand all the ways that things 
might be hidden before coming up with a good way to hide things that 
core hg and extensions can use in whatever way they wish. Generic hiding 
allows the deeply needed improvements to core hg the Greg described in 
the other thread, without waiting for evolve to be complete (which is 
still a ways off).




I think we should currently refocus the discussion on the local-only 
hiding. What are its usecases, what are the wished behavior. A large 
part of what Durham just wrote can directly be re-used for that. But 
re-centered on the local usecase, with the changes to evolution core 
behavior.


No, please let's not require solving a potential use case before 
building the mechanism that will allow us to build use cases we already 
know we want and need.




Once we have a clear view of the three concepts usecase, behavior and 
constraint we can start looking for synergy between them.


We already know the synergy -- we want ways to hide things. Let's build 
that and let each new concept use it in appropriate ways without trying 
to solve every potential problem first. Everybody wants hiding. So let's 
add generic hiding that can be exposed in various ways, including by evolve.




Cheers,

[1] 
https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-April/096239.html


___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Augie Fackler]

> On Apr 5, 2017, at 10:01, Ryan McElroy  wrote:
> 
> Side-note: +1 on Simon's request to put the doc onto the wiki as a plan.

In general, +1 on wiki pages, but only after the potential for a storm of 
comments has dwindled: discussion in doc comments is significantly less painful 
than trying to have meaningful threads of conversation in a wiki page.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Ryan McElroy]

On 4/5/17 5:29 AM, Durham Goode wrote:

On 4/4/17 8:58 PM, Gregory Szorc wrote:

On Tue, Apr 4, 2017 at 6:11 PM, Durham Goode > wrote:

There's been a lot of discussion about how to hide and unhide
commits lately [0][1], and I feel the complexity of our current
approach is hurting our ability to reason about it, making it
impossible to make progress.

I would like to formally propose a new pattern for dealing with
hidden commits, along with the concrete steps to getting it
enabled
in core by default by the August release.

The proposal is quite concise, so check out this 1-page
Google doc
for the details and to comment:

https://goo.gl/7DJ9AI



Thanks for the nice concise doc outlining a specific and achievable goal!

Side-note: +1 on Simon's request to put the doc onto the wiki as a plan.



If people find this approach promising, I can commit to
trying to
get this upstream by the August release. So if you have
questions or
concerns, please let me know so I can address them.


[0] see: "obsolete: track node versions"
[1] see: "repo: add an ability to hide nodes in an
appropriate way"


I emphatically support your proposal of introducing a 
non-obsolescence mechanism for hiding changesets. It solves a very 
real and very painful deficiency in the storage model (strips 
required to rewrite history) without forcing complicated workflows 
and concepts onto users (evolve). As I said at the sprint, the 
horrible performance associated with history rewriting without 
obsolescence enabled is one of Mozilla's biggest pain points for 
large repos. I can't in good faith recommend evolve to all users 
because it is too complicated and can lead to unexpected behavior.


I do too! And I'm glad you're on board. As I've argued in a couple of 
threads recently [1][2] I think that we should concentrate on building 
something generic for hiding without the complexity of evolution while 
evolution continues to iterate on its UX and so-forth. We can use this 
generic hiding mechanism inside of core hg with or without obsolescence 
to great effect.


[1] 
https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-April/096240.html
[2] 
https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-April/096248.html




I think this proposal for hidden changesets solves the performance 
problem while not preventing obsolescence/evolve from making progress 
in its problem space. It also doesn't seem to be as cognitively 
difficult on users.


Totally agree.



There are some areas that need flushed out of course. Obviously storage.
But I think the more important problems are the user-facing ones. e.g.
what happens if a user updates to a hidden changeset and makes a commit?
Do we allow that? I think there needs to be some kind of UX to
discourage accidental usage of hidden changesets or at least warnings
when things become unhidden because that may be unexpected. That's why
things in the doc like `hg checkout HASH` automatically unhiding a
changeset make me a little nervous. Same with `hg pull` automatic
unhiding. It seems like it is almost too easy to create divergence
without any helpful mechanism (obsolescence) to recover from it. 


I think that divergence isn't really a concept until evolution comes 
into play. Remember to compare it to what we have now. Today, 
"divergence" is just different topological branches of development that 
one person decided to rewrite. And sure, it might be a bit problematic 
but it's certainly no worse than what we have today -- now you just get 
any stripped commits back as new commits with no warning. With hidden 
commit revival, we can print a warning: "unhiding 5 hidden commits" when 
doing a pull operation, which is definitely a better place than where we 
are now.


What I like about this incremental approach is that it's clearly a step 
in the right direction without jumping all the way into the (still 
incomplete) evolution concept, but which also doesn't block evolution 
from making progress. Evolution is more work to explain today, but it's 
also more powerful.



What
I'm still trying to contemplate are the workflows where this would most
likely occur. If it is only rare scenarios or things we don't want users
doing, I think it can be tolerable.


For the purpose of the proposal, I left this scenario out since it's 
somewhat of an edge case for V1 and users could recover themselves 
pretty easily. And I feel the solution could be a layer on top of the 
proposal.


For actual concrete approaches this problem, I see us doing two things 
(both of which we're already doing within Facebook):


1. Provide tiered messaging for various actions on hidden commits. If 
you perform a safe read operation (hg log -r HIDDEN_HASH), do not 
print a warning. If you perform a write operation that unhides a 
commit, but in a 

Re: Hidden Commits in 4.3

[Pierre-Yves David]

On 04/05/2017 03:11 AM, Durham Goode wrote:

There's been a lot of discussion about how to hide and unhide commits
lately [0][1], and I feel the complexity of our current approach is
hurting our ability to reason about it, making it impossible to make
progress.

I would like to formally propose a new pattern for dealing with hidden
commits, along with the concrete steps to getting it enabled in core by
default by the August release.

The proposal is quite concise, so check out this 1-page Google doc for
the details and to comment:

https://goo.gl/7DJ9AI

If people find this approach promising, I can commit to trying to get
this upstream by the August release. So if you have questions or
concerns, please let me know so I can address them.


Thanks for writing this proposal.

The change proposed to evolution is problematic. As explain in my reply 
to the other thread[1], at minimum, removing the link between 
obsolescence and visibility is destroying the "global-state" property we 
currently have. This put in disarray the whole ability to use evolution 
to exchange and collaborate on mutable history, the very thing evolution 
has been built for. I've not seen strong-enough rationals for derailing 
the current evolution plan and design.


I think we really needs to take a step back here. Before thinking about 
unification, I think we needs a clean definition of what we are doing 
here. As mentioned in another thread[2], they are at least three 
identified categories of things we want to hide. obsolete, internal, and 
some local only hiding. The first two are quite well defined now, the 
last one less so.


I think we should currently refocus the discussion on the local-only 
hiding. What are its usecases, what are the wished behavior. A large 
part of what Durham just wrote can directly be re-used for that. But 
re-centered on the local usecase, with the changes to evolution core 
behavior.


Once we have a clear view of the three concepts usecase, behavior and 
constraint we can start looking for synergy between them.


Cheers,

[1] 
https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-April/096239.html
[2] 
https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-March/095932.html


--
Pierre-Yves David
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Durham Goode]

On 4/4/17 8:58 PM, Gregory Szorc wrote:

On Tue, Apr 4, 2017 at 6:11 PM, Durham Goode > wrote:

There's been a lot of discussion about how to hide and unhide
commits lately [0][1], and I feel the complexity of our current
approach is hurting our ability to reason about it, making it
impossible to make progress.

I would like to formally propose a new pattern for dealing with
hidden commits, along with the concrete steps to getting it enabled
in core by default by the August release.

The proposal is quite concise, so check out this 1-page Google doc
for the details and to comment:

https://goo.gl/7DJ9AI

If people find this approach promising, I can commit to trying to
get this upstream by the August release. So if you have questions or
concerns, please let me know so I can address them.


[0] see: "obsolete: track node versions"
[1] see: "repo: add an ability to hide nodes in an appropriate way"

There are some areas that need flushed out of course. Obviously storage.
But I think the more important problems are the user-facing ones. e.g.
what happens if a user updates to a hidden changeset and makes a commit?
Do we allow that? I think there needs to be some kind of UX to



discourage accidental usage of hidden changesets or at least warnings
when things become unhidden because that may be unexpected. That's why
things in the doc like `hg checkout HASH` automatically unhiding a
changeset make me a little nervous. Same with `hg pull` automatic
unhiding. It seems like it is almost too easy to create divergence
without any helpful mechanism (obsolescence) to recover from it. What
I'm still trying to contemplate are the workflows where this would most
likely occur. If it is only rare scenarios or things we don't want users
doing, I think it can be tolerable.


For the purpose of the proposal, I left this scenario out since it's 
somewhat of an edge case for V1 and users could recover themselves 
pretty easily. And I feel the solution could be a layer on top of the 
proposal.


For actual concrete approaches this problem, I see us doing two things 
(both of which we're already doing within Facebook):


1. Provide tiered messaging for various actions on hidden commits. If 
you perform a safe read operation (hg log -r HIDDEN_HASH), do not print 
a warning. If you perform a write operation that unhides a commit, but 
in a recoverable way (hg checkout HIDDEN_HASH, or hg graft HIDDEN_HASH), 
print a mild warning that you are unhiding a commit. If you attempt to 
perform a non-recoverable write operation (hg push -r HIDDEN_HASH), 
block it and require the user to explicitly unhide first.


2. In conjunction with the auto rebase proposal, when you unhide a 
commit that has an obsmarker, it will have a message on it saying that 
the commit is old ("amended as XXX"). So it will be obvious in that 
output that the commit is not the latest version.




I'm also curious if you intend to add a new repo requirement for this.
If not, what's the story for a legacy client interacting with repos with
the new hidden mechanism? IMO that experience is bad enough (legacy
client would see a bunch of visible changesets that should be hidden)
that I think it warrants locking out old clients.


In what situations would a legacy client interact with a repo created by 
a new client?  If the user downgraded their local Mercurial install? My 
initial gut feel is that this would be so rare that it would be ok to 
just let all the hidden commits become visible.  Especially since the 
only time there would be lots of previously-hidden-and-now-visible 
commits is if the user used the new client for quite a while before 
switching back to the old client. I think it might actually be a worse 
experience to put a requires file in, which would make the old client 
completely unable to use the repo at all.

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Gregory Szorc]

On Tue, Apr 4, 2017 at 6:11 PM, Durham Goode  wrote:

> There's been a lot of discussion about how to hide and unhide commits
> lately [0][1], and I feel the complexity of our current approach is hurting
> our ability to reason about it, making it impossible to make progress.
>
> I would like to formally propose a new pattern for dealing with hidden
> commits, along with the concrete steps to getting it enabled in core by
> default by the August release.
>
> The proposal is quite concise, so check out this 1-page Google doc for the
> details and to comment:
>
> https://goo.gl/7DJ9AI
>
> If people find this approach promising, I can commit to trying to get this
> upstream by the August release. So if you have questions or concerns,
> please let me know so I can address them.
>
>
> [0] see: "obsolete: track node versions"
> [1] see: "repo: add an ability to hide nodes in an appropriate way"
>

I lost track of all the developments in the various ongoing threads about
hidden changesets and obsolescence versioning. So thank you for taking the
time to write a concise summary.

I emphatically support your proposal of introducing a non-obsolescence
mechanism for hiding changesets. It solves a very real and very painful
deficiency in the storage model (strips required to rewrite history)
without forcing complicated workflows and concepts onto users (evolve). As
I said at the sprint, the horrible performance associated with history
rewriting without obsolescence enabled is one of Mozilla's biggest pain
points for large repos. I can't in good faith recommend evolve to all users
because it is too complicated and can lead to unexpected behavior.

I think this proposal for hidden changesets solves the performance problem
while not preventing obsolescence/evolve from making progress in its
problem space. It also doesn't seem to be as cognitively difficult on users.

There are some areas that need flushed out of course. Obviously storage.
But I think the more important problems are the user-facing ones. e.g. what
happens if a user updates to a hidden changeset and makes a commit? Do we
allow that? I think there needs to be some kind of UX to discourage
accidental usage of hidden changesets or at least warnings when things
become unhidden because that may be unexpected. That's why things in the
doc like `hg checkout HASH` automatically unhiding a changeset make me a
little nervous. Same with `hg pull` automatic unhiding. It seems like it is
almost too easy to create divergence without any helpful mechanism
(obsolescence) to recover from it. What I'm still trying to contemplate are
the workflows where this would most likely occur. If it is only rare
scenarios or things we don't want users doing, I think it can be tolerable.

I'm also curious if you intend to add a new repo requirement for this. If
not, what's the story for a legacy client interacting with repos with the
new hidden mechanism? IMO that experience is bad enough (legacy client
would see a bunch of visible changesets that should be hidden) that I think
it warrants locking out old clients.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Hidden Commits in 4.3

[Durham Goode]

On 4/4/17 6:11 PM, Durham Goode wrote:

There's been a lot of discussion about how to hide and unhide commits
lately [0][1], and I feel the complexity of our current approach is
hurting our ability to reason about it, making it impossible to make
progress.

I would like to formally propose a new pattern for dealing with hidden
commits, along with the concrete steps to getting it enabled in core by
default by the August release.

The proposal is quite concise, so check out this 1-page Google doc for
the details and to comment:

https://goo.gl/7DJ9AI

If people find this approach promising, I can commit to trying to get
this upstream by the August release. So if you have questions or
concerns, please let me know so I can address them.


[0] see: "obsolete: track node versions"
[1] see: "repo: add an ability to hide nodes in an appropriate way"


While I want to focus on the above proposal first, questions may come up 
about how we do evolve in a world where hidden is defined as proposed.


This next Google doc contains a draft of my next proposal. It lays out a 
auto rebase UI that I think could also make it into the August 1st 
release, and that I would also be happy to commit to getting into core.


https://goo.gl/wuP2Vl
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Hidden Commits in 4.3

[Durham Goode]

There's been a lot of discussion about how to hide and unhide commits 
lately [0][1], and I feel the complexity of our current approach is 
hurting our ability to reason about it, making it impossible to make 
progress.


I would like to formally propose a new pattern for dealing with hidden 
commits, along with the concrete steps to getting it enabled in core by 
default by the August release.


The proposal is quite concise, so check out this 1-page Google doc for 
the details and to comment:


https://goo.gl/7DJ9AI

If people find this approach promising, I can commit to trying to get 
this upstream by the August release. So if you have questions or 
concerns, please let me know so I can address them.



[0] see: "obsolete: track node versions"
[1] see: "repo: add an ability to hide nodes in an appropriate way"
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel