Re: Potential BC changes to sparse before freeze

[Martin von Zweigbergk via Mercurial-devel]

On Mon, Jul 17, 2017 at 9:57 AM, Gregory Szorc  wrote:
>
>
>> On Jul 11, 2017, at 11:01, Gregory Szorc  wrote:
>>
>> Durham, et al:
>>
>> 4.3 freeze is in a few days. While we have declared sparse as an 
>> experimental feature and not subject to BC, shipping in 4.3 means someone 
>> will inevitably use it and be impacted by future BC. (Probably Mozilla for 
>> Firefox CI). I'd like to get in as much BC before 4.3 as possible.
>>
>> Here are some changes I'm proposing. Are any of them too controversial?
>>
>> * Require a [section] in sparse config files. Right now, if there is no 
>> section, entries go in [includes]. I think less magic is better. This could 
>> break sparse profiles in old revisions in repos using sparse today. Should 
>> be easy to monkeypatch over in an extension if you need old behavior. Or we 
>> could add a function argument to control parsing strictness.
>>
>> * Remove requirement that [excludes] come after [includes]. This seems 
>> arbitrary. Requirement may stem from implicit default [includes] section. I 
>> doubt this will cause real world bustage.
>>
>> * Add a repo requirement to use sparse and add the requirement when sparse 
>> is activated on a repo. The way it is today, if you enable sparse then use 
>> an old hg or an hg without sparse enabled, the sparse-less hg sees an 
>> incomplete working directory and reports a bunch of missing files. Not cool. 
>> It feels like we need to lock out clients that aren't sparse aware from 
>> interacting with a working directory that has sparse enabled.
>
> I didn't have time for implementing the requirement last week or over the 
> weekend. But I think it is critical we ship sparse with it. I'll try to hack 
> up patches today. If I don't make it before freeze, expect patches against 
> stable.

I spent some minutes trying to see if I could make an empty sparse
config imply matching of no files, but it was unclear how to
distinguish empty config from "sparse not used". That will be clear
after you add the repo requirement. If you don't do it as part of your
series, I'll do it on top.

>
>>
>> I'm also thinking about potentially massive changes to how sparse configs 
>> work. Namely, I'd like to unify the file format with hgrc so we don't have 
>> N+1 config file formats. That's obviously a massive BC break. And I'm not 
>> sure we could land a refactor before the freeze. So we may have to deal with 
>> the BC hit in the future.
>>
>> Again, I'm trying to walk a tight rope between what is best for Mercurial 
>> and that doesn't inconvenience existing sparse users (notably Facebook). 
>> I'll try to formulate any patches that make BC changes in such a way that 
>> old behavior can be restored easily via an extension.
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Potential BC changes to sparse before freeze

[Augie Fackler]

> On Jul 17, 2017, at 12:57, Gregory Szorc  wrote:
> 
> 
> 
>> On Jul 11, 2017, at 11:01, Gregory Szorc  wrote:
>> 
>> Durham, et al:
>> 
>> 4.3 freeze is in a few days. While we have declared sparse as an 
>> experimental feature and not subject to BC, shipping in 4.3 means someone 
>> will inevitably use it and be impacted by future BC. (Probably Mozilla for 
>> Firefox CI). I'd like to get in as much BC before 4.3 as possible.
>> 
>> Here are some changes I'm proposing. Are any of them too controversial?
>> 
>> * Require a [section] in sparse config files. Right now, if there is no 
>> section, entries go in [includes]. I think less magic is better. This could 
>> break sparse profiles in old revisions in repos using sparse today. Should 
>> be easy to monkeypatch over in an extension if you need old behavior. Or we 
>> could add a function argument to control parsing strictness.
>> 
>> * Remove requirement that [excludes] come after [includes]. This seems 
>> arbitrary. Requirement may stem from implicit default [includes] section. I 
>> doubt this will cause real world bustage.
>> 
>> * Add a repo requirement to use sparse and add the requirement when sparse 
>> is activated on a repo. The way it is today, if you enable sparse then use 
>> an old hg or an hg without sparse enabled, the sparse-less hg sees an 
>> incomplete working directory and reports a bunch of missing files. Not cool. 
>> It feels like we need to lock out clients that aren't sparse aware from 
>> interacting with a working directory that has sparse enabled.
> 
> I didn't have time for implementing the requirement last week or over the 
> weekend. But I think it is critical we ship sparse with it. I'll try to hack 
> up patches today. If I don't make it before freeze, expect patches against 
> stable.

Agreed that this is important. Freeze is later today probably, but I'll 
definitely queue this afterwards if need be.

> 
>> 
>> I'm also thinking about potentially massive changes to how sparse configs 
>> work. Namely, I'd like to unify the file format with hgrc so we don't have 
>> N+1 config file formats. That's obviously a massive BC break. And I'm not 
>> sure we could land a refactor before the freeze. So we may have to deal with 
>> the BC hit in the future.
>> 
>> Again, I'm trying to walk a tight rope between what is best for Mercurial 
>> and that doesn't inconvenience existing sparse users (notably Facebook). 
>> I'll try to formulate any patches that make BC changes in such a way that 
>> old behavior can be restored easily via an extension.
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Potential BC changes to sparse before freeze

[Gregory Szorc]

> On Jul 11, 2017, at 11:01, Gregory Szorc  wrote:
> 
> Durham, et al:
> 
> 4.3 freeze is in a few days. While we have declared sparse as an experimental 
> feature and not subject to BC, shipping in 4.3 means someone will inevitably 
> use it and be impacted by future BC. (Probably Mozilla for Firefox CI). I'd 
> like to get in as much BC before 4.3 as possible.
> 
> Here are some changes I'm proposing. Are any of them too controversial?
> 
> * Require a [section] in sparse config files. Right now, if there is no 
> section, entries go in [includes]. I think less magic is better. This could 
> break sparse profiles in old revisions in repos using sparse today. Should be 
> easy to monkeypatch over in an extension if you need old behavior. Or we 
> could add a function argument to control parsing strictness.
> 
> * Remove requirement that [excludes] come after [includes]. This seems 
> arbitrary. Requirement may stem from implicit default [includes] section. I 
> doubt this will cause real world bustage.
> 
> * Add a repo requirement to use sparse and add the requirement when sparse is 
> activated on a repo. The way it is today, if you enable sparse then use an 
> old hg or an hg without sparse enabled, the sparse-less hg sees an incomplete 
> working directory and reports a bunch of missing files. Not cool. It feels 
> like we need to lock out clients that aren't sparse aware from interacting 
> with a working directory that has sparse enabled.

I didn't have time for implementing the requirement last week or over the 
weekend. But I think it is critical we ship sparse with it. I'll try to hack up 
patches today. If I don't make it before freeze, expect patches against stable.

> 
> I'm also thinking about potentially massive changes to how sparse configs 
> work. Namely, I'd like to unify the file format with hgrc so we don't have 
> N+1 config file formats. That's obviously a massive BC break. And I'm not 
> sure we could land a refactor before the freeze. So we may have to deal with 
> the BC hit in the future.
> 
> Again, I'm trying to walk a tight rope between what is best for Mercurial and 
> that doesn't inconvenience existing sparse users (notably Facebook). I'll try 
> to formulate any patches that make BC changes in such a way that old behavior 
> can be restored easily via an extension.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Potential BC changes to sparse before freeze

[Martin von Zweigbergk via Mercurial-devel]

On Mon, Jul 17, 2017 at 8:58 AM, Gregory Szorc  wrote:
>
>
>> On Jul 17, 2017, at 07:10, Augie Fackler  wrote:
>>
>>> On Sat, Jul 15, 2017 at 03:15:08PM -0700, Sean Farley wrote:
>>>
>>> Gregory Szorc  writes:
>>>
> On Sat, Jul 15, 2017 at 1:28 PM, Sean Farley  wrote:
>
>
> Martin von Zweigbergk via Mercurial-devel  scm.org> writes:
>
 It's worth noting that sparse profiles today can contain "set:" entries to
 yield the power of filesets (the matcher constructor automatically
 recognizes the type of pattern). What we can't do is change how [include]
 and [exclude] interact with each other.
>>>
>>> Perhaps this is specialized / nuanced enough that I don't quite follow;
>>> but that's fine. All I wanted to do is point out similar, existing
>>> behavior. If that's already been considered and deemed not suitable,
>>> then ok with me.
>>
>> We tried using straight filesets for narrowhg's include/exclude logic,
>> and it was thornier than we'd have liked. So far being a little more
>> restrictive has simplified both server implementations *and* not
>> caused any end-user problems.
>
> Would the concerns apply if the fileset were limited to basic operators (+ - 
> & !) and pattern matching (glob and re)?

I think we can use the same syntax for sparse and narrow but have
different restrictions on allowed patterns. Narrow should probably not
support glob and re.

Concerns that narrowhg has that sparse doesn't:

* The required parent directories needs to be well-defined. The client
and server need to agree on what parent directories are needed. This
means that patterns like "**/*.c" will work for reducing number of
filelogs, but not manifest logs. That can still be useful, depending
on how many directories you have. Importantly, the server and client
also need to not change their opinion about what directories are
needed over time. Specifically, if a visitdir() is not currently
optimized for a certain pattern, it can never be in the future either
(if it was one of the patterns we supported with narrow). Therefore, I
think we should be careful to explicitly support a very limited subset
of patterns once we add support for narrow in core. Regular
expressions and globs should probably not be allowed.

* The patterns have to be consistent across all versions (narrowhg
doesn't support having a the manifest/file at a on revision and not at
another). If we want to read patterns from versioned files, we'd
probably want to union the includes and across all versions.

>
> That's a good point on server implementations: reinventing a query language 
> parser is a bit of scope bloat.
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Potential BC changes to sparse before freeze

[Gregory Szorc]

> On Jul 17, 2017, at 07:10, Augie Fackler  wrote:
> 
>> On Sat, Jul 15, 2017 at 03:15:08PM -0700, Sean Farley wrote:
>> 
>> Gregory Szorc  writes:
>> 
 On Sat, Jul 15, 2017 at 1:28 PM, Sean Farley  wrote:
 
 
 Martin von Zweigbergk via Mercurial-devel  writes:
 
>>> It's worth noting that sparse profiles today can contain "set:" entries to
>>> yield the power of filesets (the matcher constructor automatically
>>> recognizes the type of pattern). What we can't do is change how [include]
>>> and [exclude] interact with each other.
>> 
>> Perhaps this is specialized / nuanced enough that I don't quite follow;
>> but that's fine. All I wanted to do is point out similar, existing
>> behavior. If that's already been considered and deemed not suitable,
>> then ok with me.
> 
> We tried using straight filesets for narrowhg's include/exclude logic,
> and it was thornier than we'd have liked. So far being a little more
> restrictive has simplified both server implementations *and* not
> caused any end-user problems.

Would the concerns apply if the fileset were limited to basic operators (+ - & 
!) and pattern matching (glob and re)?

That's a good point on server implementations: reinventing a query language 
parser is a bit of scope bloat.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Potential BC changes to sparse before freeze

[Sean Farley]

Gregory Szorc  writes:

> On Sat, Jul 15, 2017 at 1:28 PM, Sean Farley  wrote:
>
>>
>> Martin von Zweigbergk via Mercurial-devel > scm.org> writes:
>>
>> > On Tue, Jul 11, 2017 at 12:44 PM, Durham Goode  wrote:
>> >>
>> >>
>> >> On 7/11/17 11:01 AM, Gregory Szorc wrote:
>> >>>
>> >>> Durham, et al:
>> >>>
>> >>> 4.3 freeze is in a few days. While we have declared sparse as an
>> >>> experimental feature and not subject to BC, shipping in 4.3 means
>> >>> someone will inevitably use it and be impacted by future BC. (Probably
>> >>> Mozilla for Firefox CI). I'd like to get in as much BC before 4.3 as
>> >>> possible.
>> >>>
>> >>> Here are some changes I'm proposing. Are any of them too controversial?
>> >>>
>> >>> * Require a [section] in sparse config files. Right now, if there is no
>> >>> section, entries go in [includes]. I think less magic is better. This
>> >>> could break sparse profiles in old revisions in repos using sparse
>> >>> today. Should be easy to monkeypatch over in an extension if you need
>> >>> old behavior. Or we could add a function argument to control parsing
>> >>> strictness.
>> >>
>> >>
>> >> +1. I don't think we rely on the old behavior anywhere.
>> >>
>> >>> * Remove requirement that [excludes] come after [includes]. This seems
>> >>> arbitrary. Requirement may stem from implicit default [includes]
>> >>> section. I doubt this will cause real world bustage.
>> >>
>> >>
>> >> Sure, but the docs should be clear that order doesn't matter.
>> >
>> > I think I'm against this. I think it's clearer that excludes trump
>> > includes if they appear after. Of course, *I* can continue to write it
>> > that way and we can let anyone choose their own order, so it won't
>> > really make a difference to me. What's the reason you want to be able
>> > to put excludes before includes?
>>
>> To me, at least as a non-sparse person, I would expect the behavior to
>> match what Mercurial does for includes/excludes in other places (e.g.
>> convert, filemaps, command-line order, etc.). No?
>>
>
> And I would argue these are all not optimal because they aren't filesets.
> Attempting to shoehorn queries into CLI arguments, config sections, etc
> inevitably leads to limitations because you can't express rich primitives
> that a query language allows you to. In theory, filesets is that query
> language.
>
> That being said, filesets aren't as polished as revsets and I don't think
> they are ready to be used in this capacity. For example, we likely wouldn't
> want to expose all the wdir related filesets to sparse/ignore processing.
>
> Also, attempting to use filesets when including other files containing
> patterns is wonky. The existing [include] [exclude] model where entries are
> unioned does get the job done. We could achieve something similar if we
> invented the concept of named queries and added a fileset function to
> evaluate a named query.
>
> It's worth noting that sparse profiles today can contain "set:" entries to
> yield the power of filesets (the matcher constructor automatically
> recognizes the type of pattern). What we can't do is change how [include]
> and [exclude] interact with each other.

Perhaps this is specialized / nuanced enough that I don't quite follow;
but that's fine. All I wanted to do is point out similar, existing
behavior. If that's already been considered and deemed not suitable,
then ok with me.


signature.asc
Description: PGP signature
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Potential BC changes to sparse before freeze

[Gregory Szorc]

On Sat, Jul 15, 2017 at 1:28 PM, Sean Farley  wrote:

>
> Martin von Zweigbergk via Mercurial-devel  scm.org> writes:
>
> > On Tue, Jul 11, 2017 at 12:44 PM, Durham Goode  wrote:
> >>
> >>
> >> On 7/11/17 11:01 AM, Gregory Szorc wrote:
> >>>
> >>> Durham, et al:
> >>>
> >>> 4.3 freeze is in a few days. While we have declared sparse as an
> >>> experimental feature and not subject to BC, shipping in 4.3 means
> >>> someone will inevitably use it and be impacted by future BC. (Probably
> >>> Mozilla for Firefox CI). I'd like to get in as much BC before 4.3 as
> >>> possible.
> >>>
> >>> Here are some changes I'm proposing. Are any of them too controversial?
> >>>
> >>> * Require a [section] in sparse config files. Right now, if there is no
> >>> section, entries go in [includes]. I think less magic is better. This
> >>> could break sparse profiles in old revisions in repos using sparse
> >>> today. Should be easy to monkeypatch over in an extension if you need
> >>> old behavior. Or we could add a function argument to control parsing
> >>> strictness.
> >>
> >>
> >> +1. I don't think we rely on the old behavior anywhere.
> >>
> >>> * Remove requirement that [excludes] come after [includes]. This seems
> >>> arbitrary. Requirement may stem from implicit default [includes]
> >>> section. I doubt this will cause real world bustage.
> >>
> >>
> >> Sure, but the docs should be clear that order doesn't matter.
> >
> > I think I'm against this. I think it's clearer that excludes trump
> > includes if they appear after. Of course, *I* can continue to write it
> > that way and we can let anyone choose their own order, so it won't
> > really make a difference to me. What's the reason you want to be able
> > to put excludes before includes?
>
> To me, at least as a non-sparse person, I would expect the behavior to
> match what Mercurial does for includes/excludes in other places (e.g.
> convert, filemaps, command-line order, etc.). No?
>

And I would argue these are all not optimal because they aren't filesets.
Attempting to shoehorn queries into CLI arguments, config sections, etc
inevitably leads to limitations because you can't express rich primitives
that a query language allows you to. In theory, filesets is that query
language.

That being said, filesets aren't as polished as revsets and I don't think
they are ready to be used in this capacity. For example, we likely wouldn't
want to expose all the wdir related filesets to sparse/ignore processing.

Also, attempting to use filesets when including other files containing
patterns is wonky. The existing [include] [exclude] model where entries are
unioned does get the job done. We could achieve something similar if we
invented the concept of named queries and added a fileset function to
evaluate a named query.

It's worth noting that sparse profiles today can contain "set:" entries to
yield the power of filesets (the matcher constructor automatically
recognizes the type of pattern). What we can't do is change how [include]
and [exclude] interact with each other.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Potential BC changes to sparse before freeze

[Sean Farley]

Martin von Zweigbergk via Mercurial-devel  
writes:

> On Tue, Jul 11, 2017 at 12:44 PM, Durham Goode  wrote:
>>
>>
>> On 7/11/17 11:01 AM, Gregory Szorc wrote:
>>>
>>> Durham, et al:
>>>
>>> 4.3 freeze is in a few days. While we have declared sparse as an
>>> experimental feature and not subject to BC, shipping in 4.3 means
>>> someone will inevitably use it and be impacted by future BC. (Probably
>>> Mozilla for Firefox CI). I'd like to get in as much BC before 4.3 as
>>> possible.
>>>
>>> Here are some changes I'm proposing. Are any of them too controversial?
>>>
>>> * Require a [section] in sparse config files. Right now, if there is no
>>> section, entries go in [includes]. I think less magic is better. This
>>> could break sparse profiles in old revisions in repos using sparse
>>> today. Should be easy to monkeypatch over in an extension if you need
>>> old behavior. Or we could add a function argument to control parsing
>>> strictness.
>>
>>
>> +1. I don't think we rely on the old behavior anywhere.
>>
>>> * Remove requirement that [excludes] come after [includes]. This seems
>>> arbitrary. Requirement may stem from implicit default [includes]
>>> section. I doubt this will cause real world bustage.
>>
>>
>> Sure, but the docs should be clear that order doesn't matter.
>
> I think I'm against this. I think it's clearer that excludes trump
> includes if they appear after. Of course, *I* can continue to write it
> that way and we can let anyone choose their own order, so it won't
> really make a difference to me. What's the reason you want to be able
> to put excludes before includes?

To me, at least as a non-sparse person, I would expect the behavior to
match what Mercurial does for includes/excludes in other places (e.g.
convert, filemaps, command-line order, etc.). No?


signature.asc
Description: PGP signature
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Potential BC changes to sparse before freeze

[Gregory Szorc]

On Tue, Jul 11, 2017 at 1:47 PM, Siddharth Agarwal 
wrote:

> On 7/11/17 1:17 PM, Martin von Zweigbergk via Mercurial-devel wrote:
>
>> I think I'm against this. I think it's clearer that excludes trump
>> includes if they appear after. Of course,*I*  can continue to write it
>> that way and we can let anyone choose their own order, so it won't
>> really make a difference to me. What's the reason you want to be able
>> to put excludes before includes?
>>
>
> Yeah, I agree. This seems like a great way to make reading sparse profiles
> clearer.
>

OK. I'll keep the ordering requirement. I'll send patches for some of the
other things.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Potential BC changes to sparse before freeze

[Martin von Zweigbergk via Mercurial-devel]

On Tue, Jul 11, 2017 at 12:44 PM, Durham Goode  wrote:
>
>
> On 7/11/17 11:01 AM, Gregory Szorc wrote:
>>
>> Durham, et al:
>>
>> 4.3 freeze is in a few days. While we have declared sparse as an
>> experimental feature and not subject to BC, shipping in 4.3 means
>> someone will inevitably use it and be impacted by future BC. (Probably
>> Mozilla for Firefox CI). I'd like to get in as much BC before 4.3 as
>> possible.
>>
>> Here are some changes I'm proposing. Are any of them too controversial?
>>
>> * Require a [section] in sparse config files. Right now, if there is no
>> section, entries go in [includes]. I think less magic is better. This
>> could break sparse profiles in old revisions in repos using sparse
>> today. Should be easy to monkeypatch over in an extension if you need
>> old behavior. Or we could add a function argument to control parsing
>> strictness.
>
>
> +1. I don't think we rely on the old behavior anywhere.
>
>> * Remove requirement that [excludes] come after [includes]. This seems
>> arbitrary. Requirement may stem from implicit default [includes]
>> section. I doubt this will cause real world bustage.
>
>
> Sure, but the docs should be clear that order doesn't matter.

I think I'm against this. I think it's clearer that excludes trump
includes if they appear after. Of course, *I* can continue to write it
that way and we can let anyone choose their own order, so it won't
really make a difference to me. What's the reason you want to be able
to put excludes before includes?

Speaking of that, one change I'd like to make is so that an empty
sparse config matches nothing. Maybe that's already the case, but I
wouldn't be surprised if it matched everything.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Potential BC changes to sparse before freeze

[Jun Wu]

Excerpts from Gregory Szorc's message of 2017-07-11 12:50:02 -0700:
> Without typing up a formal proposal, I'd like to reuse the low-level config
> parser - not the higher-level config layer baked into ui.py. I /think/ that
> prevents things like aliases from creeping in. I'd also like to think about

I think even the low-level config parser might be over complicated for the
sparse usecase. Like its support of "%unset", order preserving, multiline
handling, and including relative paths etc.

If we are going to unify the file formats, I wonder if it's possible to be
built on top of hgignore format. They seem to be more similar, and is
simpler than the config parser. And if we made optimizations to regular
expressions for tree manifest hgignore in the future, a same format will
ensure the optimizations could be trivially reused in sparse too.

> how to use e.g. a .hgsparseprofiles file for centrally defining some
> profiles. Right now you have to know the path to a profile in the repo.
> This means repos can't do nice things like automatically advertise common
> profiles for users to use. If you have a central file defining multiple
> profiles, then you need sections. And all of a sudden a custom config
> format and parser seems a bit redundant.
> 
> I'll try to come up with a more formal proposal...
> 
> Thanks for the feedback!
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Potential BC changes to sparse before freeze

[Gregory Szorc]

On Tue, Jul 11, 2017 at 12:44 PM, Durham Goode  wrote:

>
>
> On 7/11/17 11:01 AM, Gregory Szorc wrote:
>
>> Durham, et al:
>>
>> 4.3 freeze is in a few days. While we have declared sparse as an
>> experimental feature and not subject to BC, shipping in 4.3 means
>> someone will inevitably use it and be impacted by future BC. (Probably
>> Mozilla for Firefox CI). I'd like to get in as much BC before 4.3 as
>> possible.
>>
>> Here are some changes I'm proposing. Are any of them too controversial?
>>
>> * Require a [section] in sparse config files. Right now, if there is no
>> section, entries go in [includes]. I think less magic is better. This
>> could break sparse profiles in old revisions in repos using sparse
>> today. Should be easy to monkeypatch over in an extension if you need
>> old behavior. Or we could add a function argument to control parsing
>> strictness.
>>
>
> +1. I don't think we rely on the old behavior anywhere.
>
> * Remove requirement that [excludes] come after [includes]. This seems
>> arbitrary. Requirement may stem from implicit default [includes]
>> section. I doubt this will cause real world bustage.
>>
>
> Sure, but the docs should be clear that order doesn't matter.
>
> * Add a repo requirement to use sparse and add the requirement when
>> sparse is activated on a repo. The way it is today, if you enable sparse
>> then use an old hg or an hg without sparse enabled, the sparse-less hg
>> sees an incomplete working directory and reports a bunch of missing
>> files. Not cool. It feels like we need to lock out clients that aren't
>> sparse aware from interacting with a working directory that has sparse
>> enabled.
>>
>
> As long as `hg sparse --reset` also remove that requirement.
>
> It'd be nice if the there was some documentation online or such that told
> old-hg users how to recover (remove sparse from .hg/requires && hg
> debugrebuilddirstate && hg revert --all)
>
> I'm also thinking about potentially massive changes to how sparse
>> configs work. Namely, I'd like to unify the file format with hgrc so we
>> don't have N+1 config file formats. That's obviously a massive BC break.
>> And I'm not sure we could land a refactor before the freeze. So we may
>> have to deal with the BC hit in the future.
>>
>
> I'd have to see more details on this.  My concern with hgrc files is that
> config keys can alias very easily, and the composing, multi-owner nature of
> sparse profiles could mean we'd hit this more frequently than with hgrcs.
>
>
> Again, I'm trying to walk a tight rope between what is best for
>> Mercurial and that doesn't inconvenience existing sparse users (notably
>> Facebook). I'll try to formulate any patches that make BC changes in
>> such a way that old behavior can be restored easily via an extension.
>>
>
Without typing up a formal proposal, I'd like to reuse the low-level config
parser - not the higher-level config layer baked into ui.py. I /think/ that
prevents things like aliases from creeping in. I'd also like to think about
how to use e.g. a .hgsparseprofiles file for centrally defining some
profiles. Right now you have to know the path to a profile in the repo.
This means repos can't do nice things like automatically advertise common
profiles for users to use. If you have a central file defining multiple
profiles, then you need sections. And all of a sudden a custom config
format and parser seems a bit redundant.

I'll try to come up with a more formal proposal...

Thanks for the feedback!
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Potential BC changes to sparse before freeze

[Durham Goode]

On 7/11/17 11:01 AM, Gregory Szorc wrote:

Durham, et al:

4.3 freeze is in a few days. While we have declared sparse as an
experimental feature and not subject to BC, shipping in 4.3 means
someone will inevitably use it and be impacted by future BC. (Probably
Mozilla for Firefox CI). I'd like to get in as much BC before 4.3 as
possible.

Here are some changes I'm proposing. Are any of them too controversial?

* Require a [section] in sparse config files. Right now, if there is no
section, entries go in [includes]. I think less magic is better. This
could break sparse profiles in old revisions in repos using sparse
today. Should be easy to monkeypatch over in an extension if you need
old behavior. Or we could add a function argument to control parsing
strictness.


+1. I don't think we rely on the old behavior anywhere.


* Remove requirement that [excludes] come after [includes]. This seems
arbitrary. Requirement may stem from implicit default [includes]
section. I doubt this will cause real world bustage.


Sure, but the docs should be clear that order doesn't matter.


* Add a repo requirement to use sparse and add the requirement when
sparse is activated on a repo. The way it is today, if you enable sparse
then use an old hg or an hg without sparse enabled, the sparse-less hg
sees an incomplete working directory and reports a bunch of missing
files. Not cool. It feels like we need to lock out clients that aren't
sparse aware from interacting with a working directory that has sparse
enabled.


As long as `hg sparse --reset` also remove that requirement.

It'd be nice if the there was some documentation online or such that 
told old-hg users how to recover (remove sparse from .hg/requires && hg 
debugrebuilddirstate && hg revert --all)



I'm also thinking about potentially massive changes to how sparse
configs work. Namely, I'd like to unify the file format with hgrc so we
don't have N+1 config file formats. That's obviously a massive BC break.
And I'm not sure we could land a refactor before the freeze. So we may
have to deal with the BC hit in the future.


I'd have to see more details on this.  My concern with hgrc files is 
that config keys can alias very easily, and the composing, multi-owner 
nature of sparse profiles could mean we'd hit this more frequently than 
with hgrcs.



Again, I'm trying to walk a tight rope between what is best for
Mercurial and that doesn't inconvenience existing sparse users (notably
Facebook). I'll try to formulate any patches that make BC changes in
such a way that old behavior can be restored easily via an extension.

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Potential BC changes to sparse before freeze

[Gregory Szorc]

Durham, et al:

4.3 freeze is in a few days. While we have declared sparse as an
experimental feature and not subject to BC, shipping in 4.3 means someone
will inevitably use it and be impacted by future BC. (Probably Mozilla for
Firefox CI). I'd like to get in as much BC before 4.3 as possible.

Here are some changes I'm proposing. Are any of them too controversial?

* Require a [section] in sparse config files. Right now, if there is no
section, entries go in [includes]. I think less magic is better. This could
break sparse profiles in old revisions in repos using sparse today. Should
be easy to monkeypatch over in an extension if you need old behavior. Or we
could add a function argument to control parsing strictness.

* Remove requirement that [excludes] come after [includes]. This seems
arbitrary. Requirement may stem from implicit default [includes] section. I
doubt this will cause real world bustage.

* Add a repo requirement to use sparse and add the requirement when sparse
is activated on a repo. The way it is today, if you enable sparse then use
an old hg or an hg without sparse enabled, the sparse-less hg sees an
incomplete working directory and reports a bunch of missing files. Not
cool. It feels like we need to lock out clients that aren't sparse aware
from interacting with a working directory that has sparse enabled.

I'm also thinking about potentially massive changes to how sparse configs
work. Namely, I'd like to unify the file format with hgrc so we don't have
N+1 config file formats. That's obviously a massive BC break. And I'm not
sure we could land a refactor before the freeze. So we may have to deal
with the BC hit in the future.

Again, I'm trying to walk a tight rope between what is best for Mercurial
and that doesn't inconvenience existing sparse users (notably Facebook).
I'll try to formulate any patches that make BC changes in such a way that
old behavior can be restored easily via an extension.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel