date:20160922

Re: news from the topic experiment

2016-09-22 Thread Kevin Bullock

> On Sep 22, 2016, at 20:14, Jun Wu  wrote:
> 
> Excerpts from Sean Farley's message of 2016-09-22 17:30:34 -0700:
>> Completely false. Take a look at the hgtopics repo:
>> 
>> $ hg log --debug -vr 093a | grep "\(topic=\|phase\)"
>> phase:   public
>> extra:   topic=stackheader
> 
> I admit I didn't look into the details carefully. Sorry for that.
> 
> I think the major difference is the "local" vs "global" part. I do think
> having the ability to give changesets private labels locally can be useful -
> that's probably another extension other than "topic".

It's not even an extension -- if a local label is what you want, we've had 
localtags since before bookmarks existed.

pacem in terris / мир / शान्ति / ‎‫سَلاَم‬ / 平和
Kevin R. Bullock

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: news from the topic experiment

2016-09-22 Thread Sean Farley

Pierre-Yves David  writes:

> On 09/23/2016 02:30 AM, Jun Wu wrote:
>> Excerpts from Pierre-Yves David's message of 2016-09-23 02:01:11 +0200:
>>>
>>> On 09/22/2016 10:09 PM, Jun Wu wrote:
 Could we consider storing the topic of a changeset elsewhere so it's not
 part of the changeset metadata? This will make it more lightweight and
 help preserve hashes with remote peers.
>>>
>>> One could definitely consider it. I've never been thrilled with having
>>> the topic as part of the hash. I agree if makes it more heavy weight
>>> that I would like to create and rename them. Not having them part of the
>>> hash with part of my initial criteria for a lightweight solution.
>>>
>>> However, when Matt, Augie and I were discussing topic somewhere in
>>> Minneapolis last year, Augie made a good case for storing them in the
>>> changesets at least until someone come with something better. Having
>>> them part of extra is solving many of hard problems right away:
>>>
>>> * We already how to discover and exchange them (just reuse changeset and
>>> named branch discovery)
>>>
>>> * We already can track history of changes (just reuse evolution related
>>> data)
>>>
>>> * We can handle rename, cyclic rename and and divergent rename (just
>>> reuse evolution related feature set).
>>
>> I think the following is the major disagreement. (this also applies to other
>> emails)
>>
>> Is the exchange thing the desired behavior? Or, should topic be global or
>> local? I'd prefer the later, because 1. this is *D*VCS and people should
>> have freedom to name things whatever they want. 2. topic names do not affect
>> the actual content.
>
> The exchanged is not only a desired behavior, it is a -required- 
> behavior. The main motivation being solving the feature branch struggle 
> and topic as a solution is to offer a way for people to identify feature 
> branches they exchange. This is central and extremely important. 
> Solution that does not fill this needs fully are not solution :-)
>
>> Consider the following example:
>>
>> 1. Alice makes 3 commits under the "smartfixup" topic.
>> 2. Bob pulls from Alice, got those 3 commits.
>> 3. Bob dislikes the "smartfixup" name, and renamed the topic to "tidy".
>> 4. Bob adds a typo fix commit. Of course, it has the topic name "tidy".
>> 5. Alice pulls the typo fix from Bob. Suddenly all commits get rewritten
>>to use the "tidy" name.
>> 6. Alice dislike the "tidy" name, and thinks wtf you renamed *my* topics...
>> 7. Alice dislike the behavior.
>
> I would clarify this as a "human and workflow issue" This does not only 
> apply to topic. Bob could be pull Alice and rename all the variables. 
> (Or even worse, rewrite the whole project in Ruby!).

Yes, I think this is going to have to be an understood limitation.
Renaming your own branch (with evolve) though seems good enough for me.

> People should not mess around with other people Changesets without prior 
> consent (or explicit workflow) and it is likely that we implement some 
> (friendly) barrier to have people think twice about it. But this is 
> something human being need to sort out between themself.
>
> In the same way large organisation will probably want to define and 
> enforce naming scheme for topics. But they already need to do so today 
> for named-branch of git-branch. So nothing specific to topic here.

I want to point out (because it seems it is implied but not explicit):
while git's ref model is simple, the downside is that it relies on
social definitions.

For instance, there is absolutely nothing preventing a dev from rebasing
'master'. In Mercurial, this restriction is built into the workflow by
building topics upon phases. "Can I rebase 'default'?" <- No. This would
prevent so many support issues.

For the average dev, prepending the username (as most large teams using
git suggest) would solve the same situation here.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: news from the topic experiment

2016-09-22 Thread Jun Wu

Excerpts from Sean Farley's message of 2016-09-22 17:30:34 -0700:
> Completely false. Take a look at the hgtopics repo:
> 
> $ hg log --debug -vr 093a | grep "\(topic=\|phase\)"
> phase:   public
> extra:   topic=stackheader

I admit I didn't look into the details carefully. Sorry for that.

I think the major difference is the "local" vs "global" part. I do think
having the ability to give changesets private labels locally can be useful -
that's probably another extension other than "topic".
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: news from the topic experiment

2016-09-22 Thread Jun Wu

Excerpts from Jun Wu's message of 2016-09-23 01:30:28 +0100:
> Excerpts from Pierre-Yves David's message of 2016-09-23 02:01:11 +0200:
> > 
> > On 09/22/2016 10:09 PM, Jun Wu wrote:
> > > Could we consider storing the topic of a changeset elsewhere so it's not
> > > part of the changeset metadata? This will make it more lightweight and
> > > help preserve hashes with remote peers.
> > 
> > One could definitely consider it. I've never been thrilled with having 
> > the topic as part of the hash. I agree if makes it more heavy weight 
> > that I would like to create and rename them. Not having them part of the 
> > hash with part of my initial criteria for a lightweight solution.
> > 
> > However, when Matt, Augie and I were discussing topic somewhere in 
> > Minneapolis last year, Augie made a good case for storing them in the 
> > changesets at least until someone come with something better. Having 
> > them part of extra is solving many of hard problems right away:
> > 
> > * We already how to discover and exchange them (just reuse changeset and 
> > named branch discovery)
> > 
> > * We already can track history of changes (just reuse evolution related 
> > data)
> > 
> > * We can handle rename, cyclic rename and and divergent rename (just 
> > reuse evolution related feature set).
> 
> I think the following is the major disagreement. (this also applies to other
> emails)
> 
> Is the exchange thing the desired behavior? Or, should topic be global or
> local? I'd prefer the later, because 1. this is *D*VCS and people should
> have freedom to name things whatever they want. 2. topic names do not affect
> the actual content.

Also, the *D* part is important to deal with name collisions.

Say if both Alice and Bob decide to use "topic1" as the topic name
locally, independently. So Alice's "topic1" and Bob's "topic1" mean totally
different things. If Bob pulls from Alice, he probably doesn't want to get
Alice's "topic1" which will mess up his own stack.

> 
> Consider the following example:
> 
> 1. Alice makes 3 commits under the "smartfixup" topic.
> 2. Bob pulls from Alice, got those 3 commits.
> 3. Bob dislikes the "smartfixup" name, and renamed the topic to "tidy".
> 4. Bob adds a typo fix commit. Of course, it has the topic name "tidy".
> 5. Alice pulls the typo fix from Bob. Suddenly all commits get rewritten
>to use the "tidy" name.
> 6. Alice dislike the "tidy" name, and thinks wtf you renamed *my* topics...
> 7. Alice dislike the behavior.
> 
> If we don't exchange topic names, Bob can change them freely locally, and
> have no issues exchange the actual commits with Alice. Bob can even write
> a hook to automatically rename the topic of incoming commits from
> "smartfixup" to "tidy".
> 
> > The last one is especially important because this is one of the main 
> > issue with bookmark. The life cycle is messy, conflict are annoying to 
> > express/resolve and propagation of all that is a hell.
> > 
> > So, yes ideally I would be happy with an ever lighter-weight solution. 
> > However given the vast amount of critical advantage we gain from this 
> > -in-extra- solutions it is unlikely we can divert our scarce resources 
> > to build a whole better solution.
> > 
> > But sure if someone show up with a solid alternative and his willing to 
> > implement it. It would certainly get the attention it deserve.
> > 
> > Cheers,
> > 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: news from the topic experiment

2016-09-22 Thread Sean Farley

Jun Wu  writes:

> Excerpts from Sean Farley's message of 2016-09-22 14:21:12 -0700:
>> > Having topic in commit metadata makes it hard to rename them, and they are
>> > not fundamentally different from branches imo - i.e. we can probably also
>> > make branches work like the current topics implementation with a some 
>> > tweaks
>> > - does not seem to need a fresh new concept.
>> 
>> I'm not convinced by how hard it is to rename them. That's what
>> evolution is for.
>
> Today I can "vim .hg/bookmarks" and do a massive change to bookmarks. If
> topics is designed to be lightweight and local, it'd better support
> something similar for power users.

This is an extremely advanced method for something I have never seen
used on a dev team. I find it very dubious that "it'd better support
something similar." *Very* dubious.

> I dislike hash change without actual content change as always. I think most
> people will prefer / expect no hash change with no actual content change.

I actually haven't seen that when introducing this idea.

>> > Conceptually, I think (I may be wrong) this is a lightweight way to specify
>> > a set of changesets, like the common "tag" concept to changesets. And it
>> > should be easy to add / remove a "tag" to / from a changeset easily.
>> 
>> It's (to me) more than that. The data model as currently defined is
>> attractive for a few reasons:
>> 
>> * a commit can only be on one topic (for better or worse)
>
> True but we can also enforce this if topics live outside changelog.

https://xkcd.com/974/

>> * you can follow the evolution of a topic very easily
>> * it's very familiar with the average user of Mercurial (because of
>>   named branches)
>
> I think our understanding differs on the "it should 'dissolve' upon being
> finished" part (from README). To me, it feels the topic names exist locally
> and they shouldn't be exposed to other peers in changelog. So they are
> guaranteed "dissolved", clean and simple.

I completely disagree. A benefit of having them in the metadata is that
we can offer something git cannot: archaeology. I have heard this
request / complaint about git many times while I've never seen the
*need* for editing refs.

Git's ref model is simple. Just as you state. But the downside is that
you cannot track what happens between ref movement. There are
heuristics, sure, but they are just that: heuristics. Evolve + topics
solves that quite elegantly and simply.

> We can have ".hg/bookmarks" or
> ".hg/remotenames" style state to track them if we do want to exchange
> the information between peers - it will be like remotenames, but one name
> points to a set of changesets, instead of just the head.

Writing remotenames taught me a lot. Mainly, it taught me what not to do
for Mercurial. There are pieces of tracking names, sure, that belong in
core (and I will hopefully do that soon) but it is too easy to wind up
with a ref model. Things just don't make sense after enabling
remotenames: what exactly are these anonymous heads? Where do named
branches fit in? Why can't I delete these commits when I delete the
bookmark?

At the end of the day, I was fighting the DVCS I was working with. There
is a way to get a ref-like workflow: use git. Seriously. I'm not trying
to be rude or condescending but by re-implementing bookmarks / remote
bookmarks you end up with a python version of git.

Mercurial's basic philosophy is that everything[1] is part of the DAG. I
think instead of fighting that, we should embrace a branching design
that is built on top of it.

> IIUC, the current implementation rewrites changesets to strip the metadata
> when the phase of changesets changes from "non-public" to "public". It
> sounds unnecessary complex to me and brings new problems unnecessarily -
> what if I "hg phase --public -f a-changeset-with-topic" ?

Completely false. Take a look at the hgtopics repo:

$ hg log --debug -vr 093a | grep "\(topic=\|phase\)"
phase:   public
extra:   topic=stackheader

>> * it works with bundle1
>
> This is some implementation detail that should not affect the design.
> Eventually everyone can upgrade.

Again: https://xkcd.com/974/

One of the biggest benefits of having a simpler model is that it enables
easier tooling around the DVCS. This point should really, really be
emphasized more than one sentence.

>> >> One of the nice features of the topic name being in the hash is that a
>> >> server (Bitbucket in this case) can enforce settings with bundle1 since
>> >> we can't really control the client of our users (unlike Facebook and
>> >> Google).
>> >
>> > I don't really understand the feature. Is it some kind of ACL or so?
>> 
>> * it allows branch permissions / restrictions
>
> Since it will eventually "dissolve", what's the usecase for permissions on
> topics? For example, if I'm allowed to use topic A but not topic B, I just
> push everything using topic A. The metadata "dissolves" anyway so I don't
> see the difference on the final

Re: news from the topic experiment

2016-09-22 Thread Jun Wu

Excerpts from Pierre-Yves David's message of 2016-09-23 02:01:11 +0200:
> 
> On 09/22/2016 10:09 PM, Jun Wu wrote:
> > Could we consider storing the topic of a changeset elsewhere so it's not
> > part of the changeset metadata? This will make it more lightweight and
> > help preserve hashes with remote peers.
> 
> One could definitely consider it. I've never been thrilled with having 
> the topic as part of the hash. I agree if makes it more heavy weight 
> that I would like to create and rename them. Not having them part of the 
> hash with part of my initial criteria for a lightweight solution.
> 
> However, when Matt, Augie and I were discussing topic somewhere in 
> Minneapolis last year, Augie made a good case for storing them in the 
> changesets at least until someone come with something better. Having 
> them part of extra is solving many of hard problems right away:
> 
> * We already how to discover and exchange them (just reuse changeset and 
> named branch discovery)
> 
> * We already can track history of changes (just reuse evolution related 
> data)
> 
> * We can handle rename, cyclic rename and and divergent rename (just 
> reuse evolution related feature set).

I think the following is the major disagreement. (this also applies to other
emails)

Is the exchange thing the desired behavior? Or, should topic be global or
local? I'd prefer the later, because 1. this is *D*VCS and people should
have freedom to name things whatever they want. 2. topic names do not affect
the actual content.

Consider the following example:

1. Alice makes 3 commits under the "smartfixup" topic.
2. Bob pulls from Alice, got those 3 commits.
3. Bob dislikes the "smartfixup" name, and renamed the topic to "tidy".
4. Bob adds a typo fix commit. Of course, it has the topic name "tidy".
5. Alice pulls the typo fix from Bob. Suddenly all commits get rewritten
   to use the "tidy" name.
6. Alice dislike the "tidy" name, and thinks wtf you renamed *my* topics...
7. Alice dislike the behavior.

If we don't exchange topic names, Bob can change them freely locally, and
have no issues exchange the actual commits with Alice. Bob can even write
a hook to automatically rename the topic of incoming commits from
"smartfixup" to "tidy".

> The last one is especially important because this is one of the main 
> issue with bookmark. The life cycle is messy, conflict are annoying to 
> express/resolve and propagation of all that is a hell.
> 
> So, yes ideally I would be happy with an ever lighter-weight solution. 
> However given the vast amount of critical advantage we gain from this 
> -in-extra- solutions it is unlikely we can divert our scarce resources 
> to build a whole better solution.
> 
> But sure if someone show up with a solid alternative and his willing to 
> implement it. It would certainly get the attention it deserve.
> 
> Cheers,
> 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: news from the topic experiment

2016-09-22 Thread Augie Fackler


> On Sep 22, 2016, at 8:18 PM, Pierre-Yves David 
>  wrote:
> 
>>> I assume this is along the spirit of your 'hg undo' for evolve (that
>>> preserves the hash)?
>> 
>> No. We are thinking about using topic to replace bookmark as the recommended
>> workflow at fb. People can get confused if local bookmarks point to public
>> changesets.
> 
> I would be happy to discuss that ;-)

As would I. I’ve fielded several user complaints lately that boil down to 
wanting an ephemeral, user-selected name for each draft commit (a la what we 
had with mq patch names). I suspect topics might be able to fill that niche as 
well.


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: news from the topic experiment

2016-09-22 Thread Pierre-Yves David




On 09/22/2016 11:58 PM, Jun Wu wrote:

Excerpts from Sean Farley's message of 2016-09-22 14:21:12 -0700:

Having topic in commit metadata makes it hard to rename them, and they are
not fundamentally different from branches imo - i.e. we can probably also
make branches work like the current topics implementation with a some tweaks
- does not seem to need a fresh new concept.


I'm not convinced by how hard it is to rename them. That's what
evolution is for.


Today I can "vim .hg/bookmarks" and do a massive change to bookmarks. If
topics is designed to be lightweight and local, it'd better support
something similar for power users.


It would be easy to have a 'topicedit' command that pops up an editor. 
As no content changes happens, it is easy to rewrite all the changeset 
in memory in a very smooth manners.



I dislike hash change without actual content change as always. I think most
people will prefer / expect no hash change with no actual content change.


I much as I would like to avoid it, the current benefit of having it 
(see earlier email) in extra largely out weight the inconvenience. I've 
also found that renaming topic is not something that I end up doing so 
often.



[…]

* you can follow the evolution of a topic very easily
* it's very familiar with the average user of Mercurial (because of
  named branches)


I think our understanding differs on the "it should 'dissolve' upon being
finished" part (from README). To me, it feels the topic names exist locally
and they shouldn't be exposed to other peers in changelog. So they are
guaranteed "dissolved", clean and simple. We can have ".hg/bookmarks" or
".hg/remotenames" style state to track them if we do want to exchange
the information between peers - it will be like remotenames, but one name
points to a set of changesets, instead of just the head.


We definitely want to be able to exchange them. Evolution allow feature 
branch exchange and topic is giving use the ability to express these 
feature branches. So smooth experience when exchanging is a very 
important part of the design/concept/



IIUC, the current implementation rewrites changesets to strip the metadata
when the phase of changesets changes from "non-public" to "public". It
sounds unnecessary complex to me and brings new problems unnecessarily -
what if I "hg phase --public -f a-changeset-with-topic" ?


Urg, no. We do -not- "rewrite changeset when the phase of the changeset 
change". We simply stop reading the metadata on public changeset. Phase 
movement stay a simple operation that does not affect the hash.


In the Facebook case, if you want to have an history without (unread) 
topic metadata, you could easily have the pushrebase extension removing 
it when performing its duty.



* it works with bundle1


This is some implementation detail that should not affect the design.
Eventually everyone can upgrade.


One of the nice features of the topic name being in the hash is that a
server (Bitbucket in this case) can enforce settings with bundle1 since
we can't really control the client of our users (unlike Facebook and
Google).


I don't really understand the feature. Is it some kind of ACL or so?


* it allows branch permissions / restrictions


Since it will eventually "dissolve", what's the usecase for permissions on
topics? For example, if I'm allowed to use topic A but not topic B, I just
push everything using topic A. The metadata "dissolves" anyway so I don't
see the difference on the final outcome.


* only one head per branch


Sounds like remote bookmark is a better fit here.


I'm not sure what you mean here.

(also note that topic+branch allow to tend toward 1 head per 
branch:topic (and could be enforced for bitbucket) while still allowing 
for multiple heads (from pull or intentionally) temporarily which is a 
very useful intermediate state.


Cheers.

--
Pierre-Yves David
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: news from the topic experiment

2016-09-22 Thread Pierre-Yves David




On 09/22/2016 11:12 PM, Jun Wu wrote:

Excerpts from Sean Farley's message of 2016-09-22 13:54:22 -0700:

Jun Wu  writes:


Could we consider storing the topic of a changeset elsewhere so it's not
part of the changeset metadata? This will make it more lightweight and
help preserve hashes with remote peers.


I assume this is along the spirit of your 'hg undo' for evolve (that
preserves the hash)?


No. We are thinking about using topic to replace bookmark as the recommended
workflow at fb. People can get confused if local bookmarks point to public
changesets.


I would be happy to discuss that ;-)


Having topic in commit metadata makes it hard to rename them, and they are
not fundamentally different from branches imo - i.e. we can probably also
make branches work like the current topics implementation with a some tweaks
- does not seem to need a fresh new concept.


No, we want both named branch and topics. They are and important 
difference in life-cycle and therefor semantic. One of the main issue 
with bookmark is their life-cycle. Having life cycle expectation 
(mutable life only) built in the topic design is important here. And 
then we still need named-branches for more long-lived target.



Conceptually, I think (I may be wrong) this is a lightweight way to specify
a set of changesets, like the common "tag" concept to changesets. And it
should be easy to add / remove a "tag" to / from a changeset easily.


(I know I'm repeating myself from the previous email, but better safe 
than sorry).


Defining the set locally is not too hard. But all the exchange part is 
much harder. You have to handle the distributed aspect of DVCS with all 
the possible renames (possibly divergent renames) and other related 
cases. And that is the very hard problem that we can just make vanish by 
reusing all our solution regarding changesets.


Cheers,

--
Pierre-Yves David
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: news from the topic experiment

2016-09-22 Thread Pierre-Yves David




On 09/22/2016 10:09 PM, Jun Wu wrote:

Could we consider storing the topic of a changeset elsewhere so it's not
part of the changeset metadata? This will make it more lightweight and
help preserve hashes with remote peers.


One could definitely consider it. I've never been thrilled with having 
the topic as part of the hash. I agree if makes it more heavy weight 
that I would like to create and rename them. Not having them part of the 
hash with part of my initial criteria for a lightweight solution.


However, when Matt, Augie and I were discussing topic somewhere in 
Minneapolis last year, Augie made a good case for storing them in the 
changesets at least until someone come with something better. Having 
them part of extra is solving many of hard problems right away:


* We already how to discover and exchange them (just reuse changeset and 
named branch discovery)


* We already can track history of changes (just reuse evolution related 
data)


* We can handle rename, cyclic rename and and divergent rename (just 
reuse evolution related feature set).


The last one is especially important because this is one of the main 
issue with bookmark. The life cycle is messy, conflict are annoying to 
express/resolve and propagation of all that is a hell.


So, yes ideally I would be happy with an ever lighter-weight solution. 
However given the vast amount of critical advantage we gain from this 
-in-extra- solutions it is unlikely we can divert our scarce resources 
to build a whole better solution.


But sure if someone show up with a solid alternative and his willing to 
implement it. It would certainly get the attention it deserve.


Cheers,

--
Pierre-Yves David
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 5] extdata: add extdatasource reader

2016-09-22 Thread Kevin Bullock

> On Sep 22, 2016, at 13:21, Matt Mackall  wrote:
> 
> # HG changeset patch
> # User Matt Mackall 
> # Date 1473794045 18000
> #  Tue Sep 13 14:14:05 2016 -0500
> # Node ID 19bf2776dfe39befdc479253e1e7d030b41c08f9
> # Parent  5271ae66615207f39cc41d78f4541bc6f8ca6ff6
> extdata: add extdatasource reader
> 
> This adds basic support for extdata, a way to add external data
> sources for revsets and templates. An extdata data source is simply a
> list of lines of the form:
> 
> []\n
> 
> An extdata source is configured thusly:
> 
> [extdata]
> name = 
> 
> urls of the form shell: are launch shell commands to generate data.
> 
> diff -r 5271ae666152 -r 19bf2776dfe3 mercurial/scmutil.py
> --- a/mercurial/scmutil.pyWed Sep 21 17:05:27 2016 -0400
> +++ b/mercurial/scmutil.pyTue Sep 13 14:14:05 2016 -0500
> @@ -29,6 +29,7 @@
> phases,
> revset,
> similar,
> +url,
> util,
> )
> 
> @@ -1418,3 +1419,66 @@
> return
> 
> self._queue.put(fh, block=True, timeout=None)
> +
> +def extdatasources(repo):
> +sources = set()
> +for k, v in repo.ui.configitems("extdata"):
> +sources.add(k)
> +return sources
> +
> +def extdatasource(repo, source):
> +"""gather a map of rev -> value dict from the specified source
> +
> +A source spec is treated as a URL, with a special case shell: type
> +for parsing the output from a shell command.
> +
> +The data is parsed as a series of newline-separated records where
> +each record is a revision specifier optionally followed by a space
> +and a freeform string value. If the revision is known locally, it
> +is converted to a rev, otherwise the record is skipped.
> +
> +Note that both key and value are treated as UTF-8 and converted to
> +the local encoding. This allows uniformity between local and
> +remote data sources.

That's a bit unfortunate. If we're expecting them to be read as UTF-8, can't we 
just keep them in UTF-8 all the way thru?

> +"""
> +
> +spec = repo.ui.config("extdata", source)
> +if not spec:
> +raise util.Abourt(_("unknown extdata source '%s'") % source)

Typo here. I suppose there's no great way to test against this.

> +
> +try:
> +# prepare for future expansion
> +expand = spec % ()
> +except TypeError:
> +raise error.Abort(_("extdata doesn't support parameters yet"),
> +  hint=_("use double % for escaping"))
> +
> +data = {}
> +if spec.startswith("shell:"):
> +# external commands should be run relative to the repo root
> +cmd = spec[6:]
> +cwd = os.getcwd()
> +os.chdir(repo.root)
> +try:
> +src = util.popen(cmd)

Erm, don't we want to use util.popen2 or one of the other variants that use 
subprocess instead?

...and maybe handle ENOENT gracefully?

pacem in terris / мир / शान्ति / ‎‫سَلاَم‬ / 平和
Kevin R. Bullock

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: news from the topic experiment

2016-09-22 Thread Sean Farley

Jun Wu  writes:

> Excerpts from Sean Farley's message of 2016-09-22 13:54:22 -0700:
>> Jun Wu  writes:
>> 
>> > Could we consider storing the topic of a changeset elsewhere so it's not
>> > part of the changeset metadata? This will make it more lightweight and
>> > help preserve hashes with remote peers.
>> 
>> I assume this is along the spirit of your 'hg undo' for evolve (that
>> preserves the hash)?
>
> No. We are thinking about using topic to replace bookmark as the recommended
> workflow at fb. People can get confused if local bookmarks point to public
> changesets.

Ah, cool.

> Having topic in commit metadata makes it hard to rename them, and they are
> not fundamentally different from branches imo - i.e. we can probably also
> make branches work like the current topics implementation with a some tweaks
> - does not seem to need a fresh new concept.

I'm not convinced by how hard it is to rename them. That's what
evolution is for.

> Conceptually, I think (I may be wrong) this is a lightweight way to specify
> a set of changesets, like the common "tag" concept to changesets. And it
> should be easy to add / remove a "tag" to / from a changeset easily.

It's (to me) more than that. The data model as currently defined is
attractive for a few reasons:

* a commit can only be on one topic (for better or worse)
* you can follow the evolution of a topic very easily
* it's very familiar with the average user of Mercurial (because of
  named branches)
* it works with bundle1

>> One of the nice features of the topic name being in the hash is that a
>> server (Bitbucket in this case) can enforce settings with bundle1 since
>> we can't really control the client of our users (unlike Facebook and
>> Google).
>
> I don't really understand the feature. Is it some kind of ACL or so?

* it allows branch permissions / restrictions
* only one head per branch
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: news from the topic experiment

2016-09-22 Thread Jun Wu

Excerpts from Sean Farley's message of 2016-09-22 13:54:22 -0700:
> Jun Wu  writes:
> 
> > Could we consider storing the topic of a changeset elsewhere so it's not
> > part of the changeset metadata? This will make it more lightweight and
> > help preserve hashes with remote peers.
> 
> I assume this is along the spirit of your 'hg undo' for evolve (that
> preserves the hash)?

No. We are thinking about using topic to replace bookmark as the recommended
workflow at fb. People can get confused if local bookmarks point to public
changesets.

Having topic in commit metadata makes it hard to rename them, and they are
not fundamentally different from branches imo - i.e. we can probably also
make branches work like the current topics implementation with a some tweaks
- does not seem to need a fresh new concept.

Conceptually, I think (I may be wrong) this is a lightweight way to specify
a set of changesets, like the common "tag" concept to changesets. And it
should be easy to add / remove a "tag" to / from a changeset easily.

> One of the nice features of the topic name being in the hash is that a
> server (Bitbucket in this case) can enforce settings with bundle1 since
> we can't really control the client of our users (unlike Facebook and
> Google).

I don't really understand the feature. Is it some kind of ACL or so?
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: news from the topic experiment

2016-09-22 Thread Jun Wu

Could we consider storing the topic of a changeset elsewhere so it's not
part of the changeset metadata? This will make it more lightweight and
help preserve hashes with remote peers.

Excerpts from Pierre-Yves David's message of 2016-09-14 21:14:16 +0200:
> In the past couple of weeks I made a couple of extra changes to the 
> topic experiment,
> 
> 'hg topic --verbose' got a large update and now list various information 
> about the topics, this should help people getting a grasp on the current 
> state of they topic in a single command:
> 
> Example output
> 
> > $ hg topic -v
> >bisect(on branch: default, 11 changesets, 149 
> > behind)
> >diff.order.issue  (on branch: default, 4 changesets, 2 
> > heads, 3 behind)
> >exception-too-wide(on branch: default, 1 changesets, 2027 
> > behind)
> >vfs.cleanup   (on branch: default, 12 changesets, 3 
> > troubled, 2 heads, 253 behind)
> >  * vfs.ward  (on branch: default, 7 changesets, 2 
> > troubled, 2 heads, 189 behind)
> 
> 
> The most notable change is probably the creation of the 'hg stack' 
> command. The 'hg stack' command display comprehensive information about 
> all changesets in your current topic.
> 
> Here is some example output (without the Christmas color)
> 
>  > $ hg stack
> > ### topic: bisect
> > ### branch: default, 149 behind
> > t9: may update
> > t8: move checkstate
> > t7: checkstate #2
> > t6: checkstate #1
> > t5: move plain update code around
> > t4: printresult
> > t3: extract extendrange
> > t2: extract reset
> > t1: use vfs for reset
> >   ^ hgweb: document why we don't allow untrusted settings to control zlib
> 
> Example output for messy state:
> 
> > ### topic: vfs.ward (2 heads)
> > ### branch: default, 189 behind
> > t7$ reposvfs: add a ward to check if locks are properly... (unstable)
> > t6@ mq: release lock after transaction in qrefresh (current)
> > t5: perf: release lock after transaction in perffncachewrite
> > t3^ repovfs: add a ward to check if locks are properly taken (base)
> > t4$ ignore bisect (unstable)
> > t3: repovfs: add a ward to check if locks are properly taken
> > t2: vfs: add the possibility to have a "ward" to check vfs usage
> > t1: pull: grab wlock during pull
> >   ^ journal: take wlock for writting the 'shared' file
> 
> 
> On the performance side, timeless introduced some improved regarding 
> caching that should reduce some of the performance impact. There is 
> still low hanging fruit there, but situation is already much better.
> 
> Sean Farley initiated a flake8 crusade and enforced some more coding 
> style through the code.
> 
> The topic extension also gained some raw documentation about its various 
> features, its not great but is better than nothing. Feel free to send 
> patch to improve it.
> 
> reminder, the extension can be found there:
> 
>https://www.mercurial-scm.org/repo/topic-experiment/ 
> 
> Cheers,
> 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[Bug 5378] New: hgrc points to the wrong place when using pooled clones

2016-09-22 Thread bugzilla

https://bz.mercurial-scm.org/show_bug.cgi?id=5378

Bug ID: 5378
   Summary: hgrc points to the wrong place when using pooled
clones
   Product: Mercurial
   Version: 3.8.2
  Hardware: PC
OS: Linux
Status: UNCONFIRMED
  Severity: feature
  Priority: wish
 Component: share
  Assignee: bugzi...@selenic.com
  Reporter: vgatien-ba...@janestreet.com
CC: mercurial-de...@selenic.com

I expect that when calling 'hg clone abc', the default path of the resulting
clone is abc. Pooled clones break this expectation, because the hgrc points to
the path of the initial clone.

Demonstration:
#!/bin/bash
cd /tmp/
rm -rf a a-share b b-share b-clone pool
hg init a; (cd a; echo a > a; hg add -q a; hg commit -q -m a)
hg clone -q a b -U
hg --config share.pool=/tmp/pool clone a a-share -U -q

# now we have two clones a and b with same rev 0,
# and one clone in the pool points to a, here is the bug
hg --config share.pool=/tmp/pool clone b b-share -U -q
head -n 2 b-share/.hg/hgrc
# prints
# [paths]
# default = /tmp/a
hg clone b b-clone -U -q
head -n 3 b-clone/.hg/hgrc
# prints
# # example repository config (see "hg help config" for more info)
# [paths]
# default = /tmp/b

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 2 v4] dispatch: change indentation level in _dispatch()

2016-09-22 Thread Arun Kulshreshtha

# HG changeset patch
# User Arun Kulshreshtha 
# Date 1474571988 25200
#  Thu Sep 22 12:19:48 2016 -0700
# Node ID 75b224273359ed924528491efb88c612f3caa4ca
# Parent  5271ae66615207f39cc41d78f4541bc6f8ca6ff6
dispatch: change indentation level in _dispatch()

Add an if True: placeholder for a profiling context manager that
will be added in the next commit, for the purpose of reducing size
of the diff due to trivial indentation changes.

This change should be a no-op.

diff --git a/mercurial/dispatch.py b/mercurial/dispatch.py
--- a/mercurial/dispatch.py
+++ b/mercurial/dispatch.py
@@ -844,60 +844,63 @@
 elif not cmd:
 return commands.help_(ui, 'shortlist')
 
-repo = None
-cmdpats = args[:]
-if not _cmdattr(ui, cmd, func, 'norepo'):
-# use the repo from the request only if we don't have -R
-if not rpath and not cwd:
-repo = req.repo
+if True:
+repo = None
+cmdpats = args[:]
+if not _cmdattr(ui, cmd, func, 'norepo'):
+# use the repo from the request only if we don't have -R
+if not rpath and not cwd:
+repo = req.repo
 
-if repo:
-# set the descriptors of the repo ui to those of ui
-repo.ui.fin = ui.fin
-repo.ui.fout = ui.fout
-repo.ui.ferr = ui.ferr
-else:
-try:
-repo = hg.repository(ui, path=path)
-if not repo.local():
-raise error.Abort(_("repository '%s' is not local") % path)
-repo.ui.setconfig("bundle", "mainreporoot", repo.root, 'repo')
-except error.RequirementError:
-raise
-except error.RepoError:
-if rpath and rpath[-1]: # invalid -R path
+if repo:
+# set the descriptors of the repo ui to those of ui
+repo.ui.fin = ui.fin
+repo.ui.fout = ui.fout
+repo.ui.ferr = ui.ferr
+else:
+try:
+repo = hg.repository(ui, path=path)
+if not repo.local():
+raise error.Abort(_("repository '%s' is not local")
+% path)
+repo.ui.setconfig("bundle", "mainreporoot", repo.root,
+  'repo')
+except error.RequirementError:
 raise
-if not _cmdattr(ui, cmd, func, 'optionalrepo'):
-if (_cmdattr(ui, cmd, func, 'inferrepo') and
-args and not path):
-# try to infer -R from command args
-repos = map(cmdutil.findrepo, args)
-guess = repos[0]
-if guess and repos.count(guess) == len(repos):
-req.args = ['--repository', guess] + fullargs
-return _dispatch(req)
-if not path:
-raise error.RepoError(_("no repository found in '%s'"
-" (.hg not found)")
-  % os.getcwd())
-raise
-if repo:
-ui = repo.ui
-if options['hidden']:
-repo = repo.unfiltered()
-args.insert(0, repo)
-elif rpath:
-ui.warn(_("warning: --repository ignored\n"))
+except error.RepoError:
+if rpath and rpath[-1]: # invalid -R path
+raise
+if not _cmdattr(ui, cmd, func, 'optionalrepo'):
+if (_cmdattr(ui, cmd, func, 'inferrepo') and
+args and not path):
+# try to infer -R from command args
+repos = map(cmdutil.findrepo, args)
+guess = repos[0]
+if guess and repos.count(guess) == len(repos):
+req.args = ['--repository', guess] + fullargs
+return _dispatch(req)
+if not path:
+raise error.RepoError(_("no repository found in"
+" '%s' (.hg not found)")
+  % os.getcwd())
+raise
+if repo:
+ui = repo.ui
+if options['hidden']:
+repo = repo.unfiltered()
+args.insert(0, repo)
+elif rpath:
+ui.warn(_("warning: --repository ignored\n"))
 
-msg = ' '.join(' ' in a and repr(a) or a for a in fullargs)
-ui.log("command", '%s\n', msg)
-d = lambda: util.checksignature(func)(ui, *args, **cmdoptions)
-try:
-return runcommand(lui, repo, cmd,

mercurial@29989: 8 new changesets

2016-09-22 Thread Mercurial Commits

8 new changesets in mercurial:

http://selenic.com/repo/hg//rev/c89b55c8f936
changeset:   29982:c89b55c8f936
user:Anton Shestakov 
date:Wed Sep 21 19:52:48 2016 +0800
summary: gitweb: tweak branch table's last row's cell colspan on summary 
page

http://selenic.com/repo/hg//rev/c6e829347561
changeset:   29983:c6e829347561
user:Anton Shestakov 
date:Wed Sep 21 20:19:35 2016 +0800
summary: monoblue: tweak branch table's last row's cell colspan on summary 
page

http://selenic.com/repo/hg//rev/10d27ee97032
changeset:   29984:10d27ee97032
user:Anton Shestakov 
date:Wed Sep 21 20:35:46 2016 +0800
summary: gitweb: remove unused template "changelogchild"

http://selenic.com/repo/hg//rev/ee60b1927b10
changeset:   29985:ee60b1927b10
user:Anton Shestakov 
date:Wed Sep 21 20:43:10 2016 +0800
summary: monoblue: remove unused template "changelogchild"

http://selenic.com/repo/hg//rev/12eeb3b7da8e
changeset:   29986:12eeb3b7da8e
user:Anton Shestakov 
date:Wed Sep 21 20:43:41 2016 +0800
summary: paper: remove unused template "changelogchild"

http://selenic.com/repo/hg//rev/d532ef155b0e
changeset:   29987:d532ef155b0e
user:timeless 
date:Wed Sep 21 16:33:37 2016 +
summary: help: clarify quotes are needed for filesets.size expressions

http://selenic.com/repo/hg//rev/96358865edb3
changeset:   29988:96358865edb3
user:timeless 
date:Wed Sep 21 17:23:05 2016 +
summary: help: move revsets.## documentation into infix section

http://selenic.com/repo/hg//rev/5271ae666152
changeset:   29989:5271ae666152
bookmark:@
tag: tip
user:timeless 
date:Wed Sep 21 17:05:27 2016 -0400
summary: help: add sections for revsets

-- 
Repository URL: http://selenic.com/repo/hg/
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[Bug 5377] New: Enabling pager on interactive commands breaks them

2016-09-22 Thread bugzilla

https://bz.mercurial-scm.org/show_bug.cgi?id=5377

Bug ID: 5377
   Summary: Enabling pager on interactive commands breaks them
   Product: Mercurial
   Version: default branch
  Hardware: All
OS: All
Status: UNCONFIRMED
  Severity: bug
  Priority: wish
 Component: pager
  Assignee: bugzi...@selenic.com
  Reporter: duri...@gmail.com
CC: mercurial-de...@selenic.com

Eg if you have

[pager]
attend = [some list that includes shelve]

then this happens:

$ hg shelve -i
abort: running non-interactively

reporting a bug so that interested parties can have something to watch for the
inevitable day when someone refactors all of pager.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 3 v2] profiling: allow nested usage of maybeprofile

2016-09-22 Thread Arun Kulshreshtha




On 9/22/16, 4:21 AM, "Yuya Nishihara"  wrote:

On Thu, 22 Sep 2016 09:04:40 +, Arun Kulshreshtha wrote:
> On 9/21/16, 8:51 PM, "Yuya Nishihara"  wrote:
> Perhaps we can move both maybeprofile() calls to _dispatch() without 
thinking
> about how nested profilers behave.
> 
> --- a/mercurial/dispatch.py
> +++ b/mercurial/dispatch.py
> @@ -774,7 +774,8 @@ def _dispatch(req):
>  # Check abbreviation/ambiguity of shell alias.
>  shellaliasfn = _checkshellalias(lui, ui, args)
>  if shellaliasfn:
> -return shellaliasfn()
> +with profiling.maybeprofile(lui):
> +return shellaliasfn()
>  
>  # check for fallback encoding
>  fallback = lui.config('ui', 'fallbackencoding')
> @@ -844,6 +845,10 @@ def _dispatch(req):
>  elif not cmd:
>  return commands.help_(ui, 'shortlist')
>  
> +with profiling.maybeprofile(lui):
> +return _dispatchcommand(...)
> +
> +def _dispatchcommand(...):
>  repo = None
>  cmdpats = args[:]
>  if not _cmdattr(ui, cmd, func, 'norepo'):
> 
> 
> Yes, in fact this is originally what I had intended to do. However, this 
would mean that it
> would not be possible to enable profiling from the repo-specific config 
file.

That's why 'lui' is passed to maybeprofile().

Ah, I see. OK, I’ll change it then using lui and see if everything works.

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 5 of 5] extdata: pass contexts to hgweb templates

2016-09-22 Thread Matt Mackall

# HG changeset patch
# User Matt Mackall 
# Date 1474310889 18000
#  Mon Sep 19 13:48:09 2016 -0500
# Node ID e6e68432abcbeec0a74478f03762b0b3192c09b5
# Parent  6db1b2e7d19ed317404c1275db46780d40ececb8
extdata: pass contexts to hgweb templates

This makes hgweb more in line with the command line templater, and
gives access to the extdata sources.

diff -r 6db1b2e7d19e -r e6e68432abcb mercurial/hgweb/webutil.py
--- a/mercurial/hgweb/webutil.pyMon Sep 19 09:16:00 2016 -0500
+++ b/mercurial/hgweb/webutil.pyMon Sep 19 13:48:09 2016 -0500
@@ -325,6 +325,7 @@
 'bookmarks': nodebookmarksdict(repo, node),
 'parent': lambda **x: parents(ctx),
 'child': lambda **x: children(ctx),
+'ctx': ctx
 }
 
 def changelistentry(web, ctx, tmpl):
diff -r 6db1b2e7d19e -r e6e68432abcb tests/test-hgweb-raw.t
--- a/tests/test-hgweb-raw.tMon Sep 19 09:16:00 2016 -0500
+++ b/tests/test-hgweb-raw.tMon Sep 19 13:48:09 2016 -0500
@@ -55,4 +55,35 @@
   $ cat access.log error.log
   127.0.0.1 - - [*] "GET 
/?f=bf0ff59095c9;file=sub/some%20text%25.txt;style=raw HTTP/1.1" 200 - (glob)
 
-  $ cd ..
+Test extdata support in hgweb
+
+  $ mkdir mystyle
+  $ echo "__base__ = raw" > mystyle/map
+  $ echo 'changelogentry = changelog.tmpl' >> mystyle/map
+  $ echo '{node|short} {if(push, "(pushed by {push})")}' > 
mystyle/changelog.tmpl
+
+  $ echo "[extdata]" >> .hg/hgrc
+  $ echo "push = pushlog.txt" >> .hg/hgrc
+  $ echo "0 bob" > pushlog.txt
+
+  $ rm access.log error.log
+  $ hg serve -p $HGPORT -A access.log -E error.log -d --pid-file=hg.pid \
+  > --config web.guessmime=True -t mystyle
+
+  $ cat hg.pid >> $DAEMON_PIDS
+  $ (get-with-headers.py localhost:$HGPORT 'changelog' content-type 
content-length content-disposition) >getoutput.txt
+  $ killdaemons.py hg.pid
+
+  $ cat getoutput.txt
+  200 Script output follows
+  content-type: text/plain; charset=ascii
+  
+  
+  # HG changelog
+  # Node ID bf0ff59095c91b192667cfe903dcdba4aced4833
+  
+  bf0ff59095c9 (pushed by bob)
+  
+  $ cat access.log error.log
+  127.0.0.1 - - [*] "GET /changelog HTTP/1.1" 200 - (glob)
+
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 5] extdata: add extdatasource reader

2016-09-22 Thread Matt Mackall

# HG changeset patch
# User Matt Mackall 
# Date 1473794045 18000
#  Tue Sep 13 14:14:05 2016 -0500
# Node ID 19bf2776dfe39befdc479253e1e7d030b41c08f9
# Parent  5271ae66615207f39cc41d78f4541bc6f8ca6ff6
extdata: add extdatasource reader

This adds basic support for extdata, a way to add external data
sources for revsets and templates. An extdata data source is simply a
list of lines of the form:

[]\n

An extdata source is configured thusly:

[extdata]
name = 

urls of the form shell: are launch shell commands to generate data.

diff -r 5271ae666152 -r 19bf2776dfe3 mercurial/scmutil.py
--- a/mercurial/scmutil.py  Wed Sep 21 17:05:27 2016 -0400
+++ b/mercurial/scmutil.py  Tue Sep 13 14:14:05 2016 -0500
@@ -29,6 +29,7 @@
 phases,
 revset,
 similar,
+url,
 util,
 )
 
@@ -1418,3 +1419,66 @@
 return
 
 self._queue.put(fh, block=True, timeout=None)
+
+def extdatasources(repo):
+sources = set()
+for k, v in repo.ui.configitems("extdata"):
+sources.add(k)
+return sources
+
+def extdatasource(repo, source):
+"""gather a map of rev -> value dict from the specified source
+
+A source spec is treated as a URL, with a special case shell: type
+for parsing the output from a shell command.
+
+The data is parsed as a series of newline-separated records where
+each record is a revision specifier optionally followed by a space
+and a freeform string value. If the revision is known locally, it
+is converted to a rev, otherwise the record is skipped.
+
+Note that both key and value are treated as UTF-8 and converted to
+the local encoding. This allows uniformity between local and
+remote data sources.
+"""
+
+spec = repo.ui.config("extdata", source)
+if not spec:
+raise util.Abourt(_("unknown extdata source '%s'") % source)
+
+try:
+# prepare for future expansion
+expand = spec % ()
+except TypeError:
+raise error.Abort(_("extdata doesn't support parameters yet"),
+  hint=_("use double % for escaping"))
+
+data = {}
+if spec.startswith("shell:"):
+# external commands should be run relative to the repo root
+cmd = spec[6:]
+cwd = os.getcwd()
+os.chdir(repo.root)
+try:
+src = util.popen(cmd)
+finally:
+os.chdir(cwd)
+else:
+# treat as a URL or file
+src = url.open(repo.ui, spec)
+
+try:
+for l in src.readlines():
+if " " in l:
+k, v = l.strip().split(" ", 1)
+else:
+k, v = l.strip(), ""
+
+k = encoding.tolocal(k)
+if k in repo:
+# we ignore data for nodes that don't exist locally
+data[repo[k].rev()] = encoding.tolocal(v)
+finally:
+src.close()
+
+return data
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 5] extdata: add revset support for extdata

2016-09-22 Thread Matt Mackall

# HG changeset patch
# User Matt Mackall 
# Date 1474293900 18000
#  Mon Sep 19 09:05:00 2016 -0500
# Node ID 9c8847df32a0c5045e60aded2e03a9c97507f909
# Parent  19bf2776dfe39befdc479253e1e7d030b41c08f9
extdata: add revset support for extdata

This inserts extdata into the revset function support. Planned
extensions of extdata support arguments, so this is the most
appropriate place for it.

Unfortunately, the registrar framework is not a good fit here. First,
setting an appropriate load point is still an unsolved problem (we
want the code to live in revset.py, but that module may never be loaded).
Second, registered methods become global and the data sources are likely to be
repo-specific. This won't work well in a context like hgwebdir.

diff -r 19bf2776dfe3 -r 9c8847df32a0 mercurial/revset.py
--- a/mercurial/revset.py   Tue Sep 13 14:14:05 2016 -0500
+++ b/mercurial/revset.py   Mon Sep 19 09:05:00 2016 -0500
@@ -432,6 +432,14 @@
 return fn(repo, subset, b, order)
 return fn(repo, subset, b)
 
+# avoid import cycle
+from . import scmutil
+
+# check external data sources (can't override built-ins)
+if f in scmutil.extdatasources(repo):
+extdata = scmutil.extdatasource(repo, f)
+return baseset([r for r in subset if r in extdata])
+
 keep = lambda fn: getattr(fn, '__doc__', None) is not None
 
 syms = [s for (s, fn) in symbols.items() if keep(fn)]
diff -r 19bf2776dfe3 -r 9c8847df32a0 tests/test-revset.t
--- a/tests/test-revset.t   Tue Sep 13 14:14:05 2016 -0500
+++ b/tests/test-revset.t   Mon Sep 19 09:05:00 2016 -0500
@@ -3540,6 +3540,31 @@
   1
   3
 
+test extdata revset support
+
+  $ echo "[extdata]" >> .hg/hgrc
+  $ echo "filedata = file:extdata.txt" >> .hg/hgrc
+  $ echo "shelldata = shell:cat extdata.txt | grep 2" >> .hg/hgrc
+  $ echo "2" > extdata.txt
+  $ echo "3" >> extdata.txt
+
+  $ hg log -qr "filedata()"
+  2:842c8a27ccf2
+  3:b7e1eaa35f23
+  $ hg log -qr "shelldata()"
+  2:842c8a27ccf2
+
+we don't fix up relative file URLs, but we do run shell commands in repo root
+
+  $ mkdir sub
+  $ cd sub
+  $ hg log -qr "filedata()"
+  abort: error: No such file or directory
+  [255]
+  $ hg log -qr "shelldata()"
+  2:842c8a27ccf2
+  $ cd ..
+
 test error message of bad revset
   $ hg log -r 'foo\\'
   hg: parse error at 3: syntax error in revset 'foo\\'
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[Bug 5376] New: hg log -pf FILE does not print patches sometimes when it should

2016-09-22 Thread bugzilla

https://bz.mercurial-scm.org/show_bug.cgi?id=5376

Bug ID: 5376
   Summary: hg log -pf FILE does not print patches sometimes when
it should
   Product: Mercurial
   Version: default branch
  Hardware: All
OS: All
Status: UNCONFIRMED
  Severity: bug
  Priority: wish
 Component: Mercurial
  Assignee: bugzi...@selenic.com
  Reporter: arcppzju+hg...@gmail.com
CC: mercurial-de...@selenic.com

Test case:

  $ cat >> $HGRCPATH << EOF
  > [diff]
  > git=1
  > [experimental]
  > evolution = createmarkers
  > EOF

  $ hg init repo
  $ cd repo

  $ for i in 1 2; do
  >   echo $i >> a
  >   hg commit -A a -m $i
  > done

  $ hg update 0 -q
  $ echo 2 >> a
  $ touch b
  $ hg commit -A b -A a -m '2 with b' -q

the following works as expected

  $ hg log -G -p -T '{rev}:{node|short} {desc}'
  @  2:281cd4d94b56 2 with bdiff --git a/a b/a
  |  --- a/a
  |  +++ b/a
  |  @@ -1,1 +1,2 @@
  |   1
  |  +2
  |  diff --git a/b b/b
  |  new file mode 100644
  |
  | o  1:1ed24be7e7a0 2diff --git a/a b/a
  |/   --- a/a
  |+++ b/a
  |@@ -1,1 +1,2 @@
  | 1
  |+2
  |
  o  0:eff892de26ec 1diff --git a/a b/a
 new file mode 100644
 --- /dev/null
 +++ b/a
 @@ -0,0 +1,1 @@
 +1

with -p FILE, it produces wrong output:

  $ hg log -G -p -f a -T '{rev}:{node|short} {desc}'
  @  2:281cd4d94b56 2 with b
  |
  o  0:eff892de26ec 1diff --git a/a b/a
 new file mode 100644
 --- /dev/null
 +++ b/a
 @@ -0,0 +1,1 @@
 +1

expected output is:

  $ hg log -G -p -f a -T '{rev}:{node|short} {desc}'
  @  2:281cd4d94b56 2 with b
  |  --- a/a
  |  +++ b/a
  |  @@ -1,1 +1,2 @@
  |   1
  |  +2
  |
  o  0:eff892de26ec 1diff --git a/a b/a
 new file mode 100644
 --- /dev/null
 +++ b/a
 @@ -0,0 +1,1 @@
 +1

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH remotenames-ext] tests: fix failures related to double->single quote changes

2016-09-22 Thread Kostia Balytskyi

# HG changeset patch
# User Kostia Balytskyi 
# Date 1474562531 25200
#  Thu Sep 22 09:42:11 2016 -0700
# Node ID 3aa5c4dbf5086615dbbe42b0d2d9fd5ca0488cf1
# Parent  7a6c5ff76f225c8ebe9baef9d5ef753da915aa8b
tests: fix failures related to double->single quote changes

diff --git a/tests/test-pull-rebase.t b/tests/test-pull-rebase.t
--- a/tests/test-pull-rebase.t
+++ b/tests/test-pull-rebase.t
@@ -119,7 +119,7 @@ Tests the behavior of a pull followed by
   searching for changes
   no changes found
   abort: can't rebase public changeset 4557926d2166
-  (see "hg help phases" for details)
+  (see 'hg help phases' for details)
   [255]
 
 Tests that there are no race condition between pulling changesets and remote 
bookmarks
diff --git a/tests/test-tracking.t b/tests/test-tracking.t
--- a/tests/test-tracking.t
+++ b/tests/test-tracking.t
@@ -14,7 +14,7 @@ Make sure we don't fail when rebase does
   
   rebasecommand to move sets of revisions to a different ancestor
   
-  (use "hg help extensions" for information on enabling extensions)
+  (use 'hg help extensions' for information on enabling extensions)
   [255]
   $ echo "rebase=" >> $HGRCPATH
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH evolve-ext] evolve: fix the way evolve checks whether dirstate.write has arguments

2016-09-22 Thread Kostia Balytskyi

# HG changeset patch
# User Kostia Balytskyi 
# Date 1474560467 25200
#  Thu Sep 22 09:07:47 2016 -0700
# Branch stable
# Node ID ac125f907dfe8f85d0c2771b3f71eaec788d9ea0
# Parent  8f902ec9ed9a296d92c2c2df1536af8c44b5321c
evolve: fix the way evolve checks whether dirstate.write has arguments


Earlier dirstate.write had the following signature: def write(self, tr=_token)
Since it had a default argument, func_defaults would list it.
In 52ff07e1de9150a8cec8e6cbe02125fb67978b8d the signature was changed to be
def write(self, tr), thus making the func_defaults check incorrect. This breaks
some evolve tests.

diff --git a/hgext/evolve.py b/hgext/evolve.py
--- a/hgext/evolve.py
+++ b/hgext/evolve.py
@@ -1024,7 +1024,7 @@ def bmactive(repo):
 ### dirstate compatibility layer < hg 3.6
 
 def writedirstate(dirstate, tr):
-if dirstate.write.func_defaults is not None: # mercurial 3.6 and above
+if dirstate.write.func_code.co_argcount: # mercurial 3.6 and above
 return dirstate.write(tr)
 return dirstate.write()
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 2] graphlog: preserve topo sort even if additional filter options specified

2016-09-22 Thread Yuya Nishihara

# HG changeset patch
# User Yuya Nishihara 
# Date 1474545233 -32400
#  Thu Sep 22 20:53:53 2016 +0900
# Node ID 63cfb37603c39abe9e7ef58462c39dd1bb83c2df
# Parent  5271ae66615207f39cc41d78f4541bc6f8ca6ff6
graphlog: preserve topo sort even if additional filter options specified

Use ordered=revset.followorder instead. This change is logically the same
as fa5e4f58dfbc.

diff --git a/mercurial/cmdutil.py b/mercurial/cmdutil.py
--- a/mercurial/cmdutil.py
+++ b/mercurial/cmdutil.py
@@ -2155,12 +2155,9 @@ def getgraphlogrevs(repo, pats, opts):
 # Revset matchers often operate faster on revisions in changelog
 # order, because most filters deal with the changelog.
 revs.reverse()
-matcher = revset.match(repo.ui, expr)
-# Revset matches can reorder revisions. "A or B" typically returns
-# returns the revision matching A then the revision matching B. Sort
-# again to fix that.
+matcher = revset.match(repo.ui, expr, order=revset.followorder)
 revs = matcher(repo, revs)
-revs.sort(reverse=True)
+revs.reverse()
 if limit is not None:
 limitedrevs = []
 for idx, rev in enumerate(revs):
diff --git a/tests/test-glog-topological.t b/tests/test-glog-topological.t
--- a/tests/test-glog-topological.t
+++ b/tests/test-glog-topological.t
@@ -60,6 +60,20 @@ later.
   o  0
   
 
+(display nodes filtered by log options)
+
+  $ hg log -G -r 'sort(all(), topo)' -k '.3'
+  o  8
+  |
+  o  3
+  |
+  ~
+  o  7
+  |
+  o  6
+  |
+  ~
+
 (revset skipping nodes)
 
   $ hg log -G --rev 'sort(not (2+6), topo)'
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 2] log: drop outdated optimization to walk revisions in reverse order

2016-09-22 Thread Yuya Nishihara

# HG changeset patch
# User Yuya Nishihara 
# Date 1474545564 -32400
#  Thu Sep 22 20:59:24 2016 +0900
# Node ID ffd93984937821bf449e2596acdb15a2c4ee88af
# Parent  63cfb37603c39abe9e7ef58462c39dd1bb83c2df
log: drop outdated optimization to walk revisions in reverse order

Since revset is computed lazily, there would be no (or little) benefit to
reverse 'revs' temporarily.

diff --git a/mercurial/cmdutil.py b/mercurial/cmdutil.py
--- a/mercurial/cmdutil.py
+++ b/mercurial/cmdutil.py
@@ -2152,12 +2152,8 @@ def getgraphlogrevs(repo, pats, opts):
 if not (revs.isdescending() or revs.istopo()):
 revs.sort(reverse=True)
 if expr:
-# Revset matchers often operate faster on revisions in changelog
-# order, because most filters deal with the changelog.
-revs.reverse()
 matcher = revset.match(repo.ui, expr, order=revset.followorder)
 revs = matcher(repo, revs)
-revs.reverse()
 if limit is not None:
 limitedrevs = []
 for idx, rev in enumerate(revs):
@@ -2182,14 +2178,8 @@ def getlogrevs(repo, pats, opts):
 return revset.baseset([]), None, None
 expr, filematcher = _makelogrevset(repo, pats, opts, revs)
 if expr:
-# Revset matchers often operate faster on revisions in changelog
-# order, because most filters deal with the changelog.
-if not opts.get('rev'):
-revs.reverse()
 matcher = revset.match(repo.ui, expr, order=revset.followorder)
 revs = matcher(repo, revs)
-if not opts.get('rev'):
-revs.reverse()
 if limit is not None:
 limitedrevs = []
 for idx, r in enumerate(revs):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 5 of 9 V2] manifest: specify checkambig=True to revlog.init, to avoid ambiguity

2016-09-22 Thread FUJIWARA Katsunori

# HG changeset patch
# User FUJIWARA Katsunori 
# Date 1474548718 -32400
#  Thu Sep 22 21:51:58 2016 +0900
# Node ID f41572305dfab3686526fcc3a1339689542b6b53
# Parent  770044f70593a8f3932e176784ed09e33d734ddf
manifest: specify checkambig=True to revlog.__init__, to avoid ambiguity

If steps below occurs at "the same time in sec", all of mtime, ctime
and size are same between (1) and (3).

  1. append data to 00manifest.i (and close transaction)
  2. discard appended data by truncation (strip or rollback)
  3. append same size but different data to 00manifest.i again

Therefore, cache validation doesn't work after (3) as expected.

To avoid such file stat ambiguity around truncation, this patch
specifies checkambig=True to revlog.__init__(). This makes revlog
write changes out with checkambig=True.

Even after this patch, avoiding file stat ambiguity of 00manifest.i
around truncation isn't yet completed, because truncation side isn't
aware of this issue.

This is a part of ExactCacheValidationPlan.

https://www.mercurial-scm.org/wiki/ExactCacheValidationPlan

diff --git a/mercurial/manifest.py b/mercurial/manifest.py
--- a/mercurial/manifest.py
+++ b/mercurial/manifest.py
@@ -927,7 +927,8 @@ class manifestrevlog(revlog.revlog):
 else:
 self._dirlogcache = {'': self}
 
-super(manifestrevlog, self).__init__(opener, indexfile)
+super(manifestrevlog, self).__init__(opener, indexfile,
+ checkambig=bool(dir))
 
 @property
 def fulltextcache(self):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 9 V2] scmutil: factor out common logic of delayclosedfile to reuse it

2016-09-22 Thread FUJIWARA Katsunori

# HG changeset patch
# User FUJIWARA Katsunori 
# Date 1474548716 -32400
#  Thu Sep 22 21:51:56 2016 +0900
# Node ID f2f665437a4939b06d9d03ae70f41f51bc5338f4
# Parent  5271ae66615207f39cc41d78f4541bc6f8ca6ff6
scmutil: factor out common logic of delayclosedfile to reuse it

This is a preparation for the subsequent patch, which adds another
proxy class for a file object.

diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1300,15 +1300,13 @@ def gddeltaconfig(ui):
 # experimental config: format.generaldelta
 return ui.configbool('format', 'generaldelta', False)
 
-class delayclosedfile(object):
-"""Proxy for a file object whose close is delayed.
+class closewrapbase(object):
+"""Base class of wrapper, which hooks closing
 
 Do not instantiate outside of the vfs layer.
 """
-
-def __init__(self, fh, closer):
+def __init__(self, fh):
 object.__setattr__(self, '_origfh', fh)
-object.__setattr__(self, '_closer', closer)
 
 def __getattr__(self, attr):
 return getattr(self._origfh, attr)
@@ -1323,6 +1321,21 @@ class delayclosedfile(object):
 return self._origfh.__enter__()
 
 def __exit__(self, exc_type, exc_value, exc_tb):
+raise NotImplementedError('attempted instantiating ' + str(type(self)))
+
+def close(self):
+raise NotImplementedError('attempted instantiating ' + str(type(self)))
+
+class delayclosedfile(closewrapbase):
+"""Proxy for a file object whose close is delayed.
+
+Do not instantiate outside of the vfs layer.
+"""
+def __init__(self, fh, closer):
+super(delayclosedfile, self).__init__(fh)
+object.__setattr__(self, '_closer', closer)
+
+def __exit__(self, exc_type, exc_value, exc_tb):
 self._closer.close(self._origfh)
 
 def close(self):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 6 of 9 V2] changelog: specify checkambig=True to avoid ambiguity around truncation

2016-09-22 Thread FUJIWARA Katsunori

# HG changeset patch
# User FUJIWARA Katsunori 
# Date 1474548719 -32400
#  Thu Sep 22 21:51:59 2016 +0900
# Node ID 550a2388d189a079822ee13d901f3cb9854e5a3e
# Parent  f41572305dfab3686526fcc3a1339689542b6b53
changelog: specify checkambig=True to avoid ambiguity around truncation

If steps below occurs at "the same time in sec", all of mtime, ctime
and size are same between (1) and (3).

  1. append data to 00changelog.i (and close transaction)
  2. discard appended data by truncation (strip or rollback)
  3. append same size but different data to 00changelog.i again

Therefore, cache validation doesn't work after (3) as expected.

To avoid such file stat ambiguity around truncation, this patch
specifies checkambig=True for renaming or opening to write changes out
at finalization.

Even after this patch, avoiding file stat ambiguity of 00changelog.i
around truncation isn't yet completed, because truncation side isn't
aware of this issue.

This is a part of ExactCacheValidationPlan.

https://www.mercurial-scm.org/wiki/ExactCacheValidationPlan

diff --git a/mercurial/changelog.py b/mercurial/changelog.py
--- a/mercurial/changelog.py
+++ b/mercurial/changelog.py
@@ -382,9 +382,9 @@ class changelog(revlog.revlog):
 tmpname = self.indexfile + ".a"
 nfile = self.opener.open(tmpname)
 nfile.close()
-self.opener.rename(tmpname, self.indexfile)
+self.opener.rename(tmpname, self.indexfile, checkambig=True)
 elif self._delaybuf:
-fp = self.opener(self.indexfile, 'a')
+fp = self.opener(self.indexfile, 'a', checkambig=True)
 fp.write("".join(self._delaybuf))
 fp.close()
 self._delaybuf = None
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 8 of 9 V2] repair: open a file with checkambig=True to avoid file stat ambiguity

2016-09-22 Thread FUJIWARA Katsunori

# HG changeset patch
# User FUJIWARA Katsunori 
# Date 1474548720 -32400
#  Thu Sep 22 21:52:00 2016 +0900
# Node ID 338febde64d5fe6a5aac79aff54741e201b5d88f
# Parent  5ec24143ad2e85bf1567d9af83df9fc5187f0a4e
repair: open a file with checkambig=True to avoid file stat ambiguity

Before this patch, if steps below occurs at "the same time in sec",
all of mtime, ctime and size are same between (1) and (3).

  1. append data to revlog-style file (and close transaction)
  2. discard appended data by truncation of strip
  3. append same size but different data to revlog-style file again

Therefore, cache validation doesn't work after (3) as expected.

To avoid such file stat ambiguity around truncation, this patch opens
a file with checkambig=True.

This patch also introduces "with" statement style, to ensure immediate
invocation of close() after truncation, because closing file is the
only trigger to check (and get rid of) file stat ambiguity.

This is a part of ExactCacheValidationPlan.

https://www.mercurial-scm.org/wiki/ExactCacheValidationPlan

diff --git a/mercurial/repair.py b/mercurial/repair.py
--- a/mercurial/repair.py
+++ b/mercurial/repair.py
@@ -181,7 +181,8 @@ def strip(ui, repo, nodelist, backup=Tru
 
 for i in xrange(offset, len(tr.entries)):
 file, troffset, ignore = tr.entries[i]
-repo.svfs(file, 'a').truncate(troffset)
+with repo.svfs(file, 'a', checkambig=True) as fp:
+fp.truncate(troffset)
 if troffset == 0:
 repo.store.markremoved(file)
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 9 V2] vfs: use checkambigatclosing in checkambig=True but atomictemp=False case

2016-09-22 Thread FUJIWARA Katsunori

# HG changeset patch
# User FUJIWARA Katsunori 
# Date 1474548717 -32400
#  Thu Sep 22 21:51:57 2016 +0900
# Node ID 012c44bc655e19121818ab0c3e34bf610a6dc1cf
# Parent  6b0eb9c2264b6fe8d4d70dfe15c600058d9f0f41
vfs: use checkambigatclosing in checkambig=True but atomictemp=False case

In Mercurial source tree, opening a file in "a"/"a+" mode like below
doesn't specify atomictemp=True for vfs, and this avoids file stat
ambiguity check by atomictempfile.

  - writing changes out in revlog layer uses "a+" mode
  - truncation in repair.strip() uses "a" mode
  - truncation in transaction._playback() uses "a" mode

If steps below occurs at "the same time in sec", all of mtime, ctime
and size are same between (1) and (3).

  1. append data to revlog-style file (and close transaction)
  2. discard appended data by truncation (strip or rollback)
  3. append same size but different data to revlog-style file again

Therefore, cache validation doesn't work after (3) as expected.

This patch uses checkambigatclosing in checkambig=True but
atomictemp=False case, to check (and get rid of) file stat ambiguity
at closing.

This is a part of ExactCacheValidationPlan.

https://www.mercurial-scm.org/wiki/ExactCacheValidationPlan

diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -587,6 +587,12 @@ class vfs(abstractvfs):
 if nlink == 0:
 self._fixfilemode(f)
 
+if checkambig:
+if mode in ('r', 'rb'):
+raise error.Abort(_('implementation error: mode %s is not'
+' valid for checkambig=True') % mode)
+fp = checkambigatclosing(fp)
+
 if backgroundclose:
 if not self._backgroundfilecloser:
 raise error.Abort(_('backgroundclose can only be used when a '
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 7 of 9 V2] changelog: specify checkambig=True to revlog.init, to avoid ambiguity

2016-09-22 Thread FUJIWARA Katsunori

# HG changeset patch
# User FUJIWARA Katsunori 
# Date 1474548719 -32400
#  Thu Sep 22 21:51:59 2016 +0900
# Node ID 5ec24143ad2e85bf1567d9af83df9fc5187f0a4e
# Parent  550a2388d189a079822ee13d901f3cb9854e5a3e
changelog: specify checkambig=True to revlog.__init__, to avoid ambiguity

If steps below occurs at "the same time in sec", all of mtime, ctime
and size are same between (1) and (3).

  1. append data to 00changelog.i (and close transaction)
  2. discard appended data by truncation (strip or rollback)
  3. append same size but different data to 00changelog.i again

Therefore, cache validation doesn't work after (3) as expected.

To avoid such file stat ambiguity around truncation, this patch
specifies checkambig=True to revlog.__init__(). This makes revlog
write changes out with checkambig=True.

Even though changes of 00changelog.i themselves are written out at
changelog._finalize(), this checkambig=True is needed, because
revlog.checkinlinesize(), which is invoked at the end of
changelog._finalize(), might replace already changed 00changelog.i by
converted one.

Even after this patch, avoiding file stat ambiguity of 00changelog.i
around truncation isn't yet completed, because truncation side isn't
aware of this issue.

This is a part of ExactCacheValidationPlan.

https://www.mercurial-scm.org/wiki/ExactCacheValidationPlan

diff --git a/mercurial/changelog.py b/mercurial/changelog.py
--- a/mercurial/changelog.py
+++ b/mercurial/changelog.py
@@ -257,7 +257,8 @@ class changelogrevision(object):
 
 class changelog(revlog.revlog):
 def __init__(self, opener):
-revlog.revlog.__init__(self, opener, "00changelog.i")
+revlog.revlog.__init__(self, opener, "00changelog.i",
+   checkambig=True)
 if self._initempty:
 # changelogs don't benefit from generaldelta
 self.version &= ~revlog.REVLOGGENERALDELTA
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 9 of 9 V2] transaction: open a file with checkambig=True to avoid file stat ambiguity

2016-09-22 Thread FUJIWARA Katsunori

# HG changeset patch
# User FUJIWARA Katsunori 
# Date 1474548720 -32400
#  Thu Sep 22 21:52:00 2016 +0900
# Node ID c47d67e9412df24914ff4caaf1437a2c35a0af6b
# Parent  338febde64d5fe6a5aac79aff54741e201b5d88f
transaction: open a file with checkambig=True to avoid file stat ambiguity

Before this patch, if steps below occurs at "the same time in sec",
all of mtime, ctime and size are same between (1) and (3).

  1. append data to revlog-style file (and close transaction)
  2. discard appended data by truncation of rollback
  3. append same size but different data to revlog-style file again

Therefore, cache validation doesn't work after (3) as expected.

To avoid file stat ambiguity around truncation, this patch opens a
file with checkambig=True.

This is a part of ExactCacheValidationPlan.

https://www.mercurial-scm.org/wiki/ExactCacheValidationPlan

diff --git a/mercurial/transaction.py b/mercurial/transaction.py
--- a/mercurial/transaction.py
+++ b/mercurial/transaction.py
@@ -48,7 +48,7 @@ def _playback(journal, report, opener, v
 for f, o, _ignore in entries:
 if o or not unlink:
 try:
-fp = opener(f, 'a')
+fp = opener(f, 'a', checkambig=True)
 fp.truncate(o)
 fp.close()
 except IOError:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 4 of 9 V2] revlog: specify checkambig at writing to avoid file stat ambiguity

2016-09-22 Thread FUJIWARA Katsunori

# HG changeset patch
# User FUJIWARA Katsunori 
# Date 1474548718 -32400
#  Thu Sep 22 21:51:58 2016 +0900
# Node ID 770044f70593a8f3932e176784ed09e33d734ddf
# Parent  012c44bc655e19121818ab0c3e34bf610a6dc1cf
revlog: specify checkambig at writing to avoid file stat ambiguity

This allows revlog-style files to be written out with checkambig=True
easily.

Because avoiding file stat ambiguity is needed only for filecache-ed
manifest and changelog, this patch does:

  - use False for default value of checkambig
  - focus only on writing changes of index file out

This patch also adds optional argument checkambig to _divert/_delay
for changelog, to safely accept checkambig specified in revlog
layer. But this argument can be fully ignored, because:

  - changes are written into other than index file, if name != target
  - changes are never written into index file, otherwise
(into pending file by _divert, or into in-memory buffer by _delay)

This is a part of ExactCacheValidationPlan.

https://www.mercurial-scm.org/wiki/ExactCacheValidationPlan

diff --git a/mercurial/changelog.py b/mercurial/changelog.py
--- a/mercurial/changelog.py
+++ b/mercurial/changelog.py
@@ -124,7 +124,7 @@ class appender(object):
 
 def _divertopener(opener, target):
 """build an opener that writes in 'target.a' instead of 'target'"""
-def _divert(name, mode='r'):
+def _divert(name, mode='r', checkambig=False):
 if name != target:
 return opener(name, mode)
 return opener(name + ".a", mode)
@@ -132,7 +132,7 @@ def _divertopener(opener, target):
 
 def _delayopener(opener, target, buf):
 """build an opener that stores chunks in 'buf' instead of 'target'"""
-def _delay(name, mode='r'):
+def _delay(name, mode='r', checkambig=False):
 if name != target:
 return opener(name, mode)
 return appender(opener, name, mode, buf)
diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -212,8 +212,11 @@ class revlog(object):
 fashion, which means we never need to rewrite a file to insert or
 remove data, and can use some simple techniques to avoid the need
 for locking while reading.
+
+If checkambig, indexfile is opened with checkambig=True at
+writing, to avoid file stat ambiguity.
 """
-def __init__(self, opener, indexfile):
+def __init__(self, opener, indexfile, checkambig=False):
 """
 create a revlog object
 
@@ -223,6 +226,7 @@ class revlog(object):
 self.indexfile = indexfile
 self.datafile = indexfile[:-2] + ".d"
 self.opener = opener
+self._checkambig = checkambig
 # 3-tuple of (node, rev, text) for a raw revision.
 self._cache = None
 # Maps rev to chain base rev.
@@ -1276,7 +1280,8 @@ class revlog(object):
 finally:
 df.close()
 
-fp = self.opener(self.indexfile, 'w', atomictemp=True)
+fp = self.opener(self.indexfile, 'w', atomictemp=True,
+ checkambig=self._checkambig)
 self.version &= ~(REVLOGNGINLINEDATA)
 self._inline = False
 for i in self:
@@ -1319,7 +1324,7 @@ class revlog(object):
 dfh = None
 if not self._inline:
 dfh = self.opener(self.datafile, "a+")
-ifh = self.opener(self.indexfile, "a+")
+ifh = self.opener(self.indexfile, "a+", checkambig=self._checkambig)
 try:
 return self._addrevision(node, text, transaction, link, p1, p2,
  REVIDX_DEFAULT_FLAGS, cachedelta, ifh, 
dfh)
@@ -1567,7 +1572,7 @@ class revlog(object):
 end = 0
 if r:
 end = self.end(r - 1)
-ifh = self.opener(self.indexfile, "a+")
+ifh = self.opener(self.indexfile, "a+", checkambig=self._checkambig)
 isize = r * self._io.size
 if self._inline:
 transaction.add(self.indexfile, end + isize, r)
@@ -1641,7 +1646,8 @@ class revlog(object):
 # reopen the index
 ifh.close()
 dfh = self.opener(self.datafile, "a+")
-ifh = self.opener(self.indexfile, "a+")
+ifh = self.opener(self.indexfile, "a+",
+  checkambig=self._checkambig)
 finally:
 if dfh:
 dfh.close()
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 3 v2] profiling: allow nested usage of maybeprofile

2016-09-22 Thread Yuya Nishihara

On Thu, 22 Sep 2016 09:04:40 +, Arun Kulshreshtha wrote:
> On 9/21/16, 8:51 PM, "Yuya Nishihara"  y...@tcha.org> wrote:
> Perhaps we can move both maybeprofile() calls to _dispatch() without 
> thinking
> about how nested profilers behave.
> 
> --- a/mercurial/dispatch.py
> +++ b/mercurial/dispatch.py
> @@ -774,7 +774,8 @@ def _dispatch(req):
>  # Check abbreviation/ambiguity of shell alias.
>  shellaliasfn = _checkshellalias(lui, ui, args)
>  if shellaliasfn:
> -return shellaliasfn()
> +with profiling.maybeprofile(lui):
> +return shellaliasfn()
>  
>  # check for fallback encoding
>  fallback = lui.config('ui', 'fallbackencoding')
> @@ -844,6 +845,10 @@ def _dispatch(req):
>  elif not cmd:
>  return commands.help_(ui, 'shortlist')
>  
> +with profiling.maybeprofile(lui):
> +return _dispatchcommand(...)
> +
> +def _dispatchcommand(...):
>  repo = None
>  cmdpats = args[:]
>  if not _cmdattr(ui, cmd, func, 'norepo'):
> 
> 
> Yes, in fact this is originally what I had intended to do. However, this 
> would mean that it
> would not be possible to enable profiling from the repo-specific config file.

That's why 'lui' is passed to maybeprofile().
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 3 v2] profiling: allow nested usage of maybeprofile

2016-09-22 Thread Arun Kulshreshtha




On 9/21/16, 8:51 PM, "Yuya Nishihara"  wrote:

On Wed, 21 Sep 2016 18:07:57 +, Arun Kulshreshtha wrote:
> On 9/21/16, 7:41 AM, "Yuya Nishihara"  wrote:
> > maybeprofile() can be called in threads. If we need to prevent 
nesting, we'll
> > have to save the flag in TLS.
> 
> That said, I'm not sure if nested maybeprofile() can be noop. Last 
time, Greg
> tried to make statprof stackable, which would imply maybeprofile() 
designed
> to be nested.
> 
> Hm, right now it seems like if you stack profilers, you’ll get multiple 
profiling statistics printouts at the end of command execution, which is hard 
to read. But this this behavior is actually desirable, then we’ll need to deal 
with this situation differently.

Perhaps we can move both maybeprofile() calls to _dispatch() without 
thinking
about how nested profilers behave.

--- a/mercurial/dispatch.py
+++ b/mercurial/dispatch.py
@@ -774,7 +774,8 @@ def _dispatch(req):
 # Check abbreviation/ambiguity of shell alias.
 shellaliasfn = _checkshellalias(lui, ui, args)
 if shellaliasfn:
-return shellaliasfn()
+with profiling.maybeprofile(lui):
+return shellaliasfn()
 
 # check for fallback encoding
 fallback = lui.config('ui', 'fallbackencoding')
@@ -844,6 +845,10 @@ def _dispatch(req):
 elif not cmd:
 return commands.help_(ui, 'shortlist')
 
+with profiling.maybeprofile(lui):
+return _dispatchcommand(...)
+
+def _dispatchcommand(...):
 repo = None
 cmdpats = args[:]
 if not _cmdattr(ui, cmd, func, 'norepo'):


Yes, in fact this is originally what I had intended to do. However, this would 
mean that it
would not be possible to enable profiling from the repo-specific config file. 
Of course, we could
simply add a third call to maybeprofile after the call to hg.repository(), but 
prevent having duplicate
output with a boolean flag to check if profiling was already enabled before 
reposetup. This approach
seems a little inelegant, but it’s also simple enough.

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

37 matches

Mail list logo