mercurial@42410: 2 new changesets
2 new changesets in mercurial: https://www.mercurial-scm.org/repo/hg/rev/72522fe7fb95 changeset: 42409:72522fe7fb95 user:Martin von Zweigbergk date:Sat Mar 09 22:11:27 2019 -0800 summary: merge: reorder some initialization to make more sense https://www.mercurial-scm.org/repo/hg/rev/a5b5ecff5f37 changeset: 42410:a5b5ecff5f37 bookmark:@ tag: tip user:Martin von Zweigbergk date:Sat Mar 09 22:13:06 2019 -0800 summary: merge: simplify initialization of "pas" -- Repository URL: https://www.mercurial-scm.org/repo/hg ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D6436: narrow: use narrow_widen wireproto command to widen in case of ellipses
rdamazio added inline comments. INLINE COMMENTS > narrowcommands.py:306 > +'ellipses': True, > +}).result() > +trmanager = exchange.transactionmanager(repo, 'widen', Being nosy here: does this mean that it'll download the whole bundle into memory before it starts applying it, rather than streaming it into the store? Bundles, especially in repos using narrow, can be very large, so that would not be ideal. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D6436 To: pulkit, durin42, martinvonz, #hg-reviewers Cc: rdamazio, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D6436: narrow: use narrow_widen wireproto command to widen in case of ellipses
martinvonz added inline comments. INLINE COMMENTS > narrowcommands.py:149 > # we have all the nodes > if wireprototypes.ELLIPSESCAP in pullop.remote.capabilities(): > kwargs['known'] = [node.hex(ctx.node()) for ctx in Also need to check old capability here > narrowcommands.py:257-258 > +remotecap = remote.capabilities() > +ellipsesremote = any(True for cap in wireprototypes.SUPPORTED_ELLIPSESCAP > + if cap in remotecap) > + nit: `any(cap in remotecap for cap in wireprototypes.SUPPORTED_ELLIPSESCAP)` > narrowcommands.py:282 > with wrappedextraprepare: > -with repo.ui.configoverride(overrides, 'widen'): > exchange.pull(repo, remote, heads=common) Should keep this in the "old" case > narrowcommands.py:303-305 > +'commonheads': common, > +'known': known, > +'ellipses': True, Is the `known` set needed when not using ellipses? Conversely, `commonheads` shouldn't be needed when using ellipses, but perhaps it's still useful to have it there (it's usually way smaller, so it's much less of a concern). These things can be fixed in a separate patch, of course. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D6436 To: pulkit, durin42, martinvonz, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
mercurial@42408: 7 new changesets
7 new changesets in mercurial:
https://www.mercurial-scm.org/repo/hg/rev/5b34972a0094
changeset: 42402:5b34972a0094
user:Martin von Zweigbergk
date:Thu Mar 08 11:08:24 2018 -0800
summary: setdiscovery: make progress on most connected groups each roundtrip
https://www.mercurial-scm.org/repo/hg/rev/4ce7cdd78da3
changeset: 42403:4ce7cdd78da3
user:Martin von Zweigbergk
date:Fri May 31 13:25:28 2019 -0700
summary: help: remove a superfluous "the" in revlogs text
https://www.mercurial-scm.org/repo/hg/rev/0c0a22f5b0b5
changeset: 42404:0c0a22f5b0b5
user:Martin von Zweigbergk
date:Fri May 31 09:25:51 2019 -0700
summary: narrowspec: use vfs.tryread() instead of reimplementing
https://www.mercurial-scm.org/repo/hg/rev/0c72eddb4be5
changeset: 42405:0c72eddb4be5
user:Martin von Zweigbergk
date:Thu Apr 18 13:35:02 2019 -0700
summary: templatekw: make {file_*} compare to both merge parents (issue4292)
https://www.mercurial-scm.org/repo/hg/rev/f385ba70e4af
changeset: 42406:f385ba70e4af
user:Martin von Zweigbergk
date:Tue May 14 22:19:51 2019 -0700
summary: changelog: optionally store added and removed files in changeset
extras
https://www.mercurial-scm.org/repo/hg/rev/602469a91550
changeset: 42407:602469a91550
user:Martin von Zweigbergk
date:Tue May 14 22:20:10 2019 -0700
summary: context: get filesadded() and filesremoved() from changeset if
configured
https://www.mercurial-scm.org/repo/hg/rev/4e4fa3a95406
changeset: 42408:4e4fa3a95406
bookmark:@
tag: tip
user:Georges Racinet
date:Wed May 22 08:27:02 2019 +
summary: rust-dirstate: architecture independence fix
--
Repository URL: https://www.mercurial-scm.org/repo/hg
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D6430: rust-discovery: using from Python code
martinvonz added a comment. I think this series needs to be updated for https://phab.mercurial-scm.org/D2647. I'm curious to see the timing numbers after that patch (I can get those myself once the series is updated if you're tired of running the perf commands). REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D6430 To: gracinet, #hg-reviewers Cc: martinvonz, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D6393: rust-dirstate: add "dirs" Rust implementation
martinvonz added a comment. Does this need updating after https://phab.mercurial-scm.org/D6403? Based on a cursory look at the patch, it has not been updated yet. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D6393 To: Alphare, #hg-reviewers Cc: martinvonz, durin42, kevincox, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D6462: commitextras: try to remove localrepo hacking to add extras to commit
martinvonz added a comment. Can you elaborate a bit about what the current hack does and why it's bad? REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D6462 To: pulkit, #hg-reviewers Cc: martinvonz, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D6468: narrowspec: replace one recursion-avoidance hack with another
This revision was automatically updated to reflect the committed changes. Closed by commit rHG4738c292a520: narrowspec: replace one recursion-avoidance hack with another (authored by martinvonz, committed by ). REPOSITORY rHG Mercurial CHANGES SINCE LAST UPDATE https://phab.mercurial-scm.org/D6468?vs=15316=15331 REVISION DETAIL https://phab.mercurial-scm.org/D6468 AFFECTED FILES mercurial/narrowspec.py CHANGE DETAILS diff --git a/mercurial/narrowspec.py b/mercurial/narrowspec.py --- a/mercurial/narrowspec.py +++ b/mercurial/narrowspec.py @@ -262,6 +262,9 @@ mctx=repo['.'], overwrite=False) def checkworkingcopynarrowspec(repo): +# Avoid infinite recursion when updating the working copy +if getattr(repo, '_updatingnarrowspec', False): +return storespec = repo.svfs.tryread(FILENAME) wcspec = repo.vfs.tryread(DIRSTATE_FILENAME) if wcspec != storespec: @@ -276,6 +279,7 @@ """ oldspec = repo.vfs.tryread(DIRSTATE_FILENAME) newspec = repo.svfs.tryread(FILENAME) +repo._updatingnarrowspec = True oldincludes, oldexcludes = parseconfig(repo.ui, oldspec) newincludes, newexcludes = parseconfig(repo.ui, newspec) @@ -305,10 +309,9 @@ for f in clean + trackeddirty: ds.drop(f) -repo.narrowpats = newincludes, newexcludes -repo._narrowmatch = newmatch pctx = repo['.'] newfiles = [f for f in pctx.manifest().walk(addedmatch) if f not in ds] for f in newfiles: ds.normallookup(f) _writeaddedfiles(repo, pctx, newfiles) +repo._updatingnarrowspec = False To: martinvonz, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D6472: merge: simplify initialization of "pas"
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGa5b5ecff5f37: merge: simplify initialization of
pas (authored by martinvonz, committed by ).
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D6472?vs=15320=15330
REVISION DETAIL
https://phab.mercurial-scm.org/D6472
AFFECTED FILES
mercurial/merge.py
CHANGE DETAILS
diff --git a/mercurial/merge.py b/mercurial/merge.py
--- a/mercurial/merge.py
+++ b/mercurial/merge.py
@@ -1978,10 +1978,9 @@
pl = wc.parents()
p1 = pl[0]
p2 = repo[node]
-pas = [None]
if ancestor is not None:
pas = [repo[ancestor]]
-if pas[0] is None:
+else:
if repo.ui.configlist('merge', 'preferancestor') == ['*']:
cahs = repo.changelog.commonancestorsheads(p1.node(),
p2.node())
pas = [repo[anc] for anc in (sorted(cahs) or [nullid])]
To: martinvonz, #hg-reviewers, pulkit
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D6471: merge: reorder some initialization to make more sense
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG72522fe7fb95: merge: reorder some initialization to make
more sense (authored by martinvonz, committed by ).
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D6471?vs=15319=15329
REVISION DETAIL
https://phab.mercurial-scm.org/D6471
AFFECTED FILES
mercurial/merge.py
CHANGE DETAILS
diff --git a/mercurial/merge.py b/mercurial/merge.py
--- a/mercurial/merge.py
+++ b/mercurial/merge.py
@@ -1977,13 +1977,10 @@
wc = repo[None]
pl = wc.parents()
p1 = pl[0]
+p2 = repo[node]
pas = [None]
if ancestor is not None:
pas = [repo[ancestor]]
-
-overwrite = force and not branchmerge
-
-p2 = repo[node]
if pas[0] is None:
if repo.ui.configlist('merge', 'preferancestor') == ['*']:
cahs = repo.changelog.commonancestorsheads(p1.node(),
p2.node())
@@ -1993,6 +1990,7 @@
fp1, fp2, xp1, xp2 = p1.node(), p2.node(), bytes(p1), bytes(p2)
+overwrite = force and not branchmerge
### check phase
if not overwrite:
if len(pl) > 1:
To: martinvonz, #hg-reviewers, pulkit
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D6473: rust-dirstate: architecture independence fix
This revision was automatically updated to reflect the committed changes. Closed by commit rHG7f22b45ab36c: rust-dirstate: architecture independence fix (authored by Georges Racinet georges.raci...@octobus.net, committed by ). REPOSITORY rHG Mercurial CHANGES SINCE LAST UPDATE https://phab.mercurial-scm.org/D6473?vs=15325=15328 REVISION DETAIL https://phab.mercurial-scm.org/D6473 AFFECTED FILES rust/hg-cpython/src/dirstate.rs CHANGE DETAILS diff --git a/rust/hg-cpython/src/dirstate.rs b/rust/hg-cpython/src/dirstate.rs --- a/rust/hg-cpython/src/dirstate.rs +++ b/rust/hg-cpython/src/dirstate.rs @@ -70,7 +70,7 @@ py, PyBytes::new(py, [..]), decapsule_make_dirstate_tuple(py)?( -entry.state, +entry.state as c_char, entry.mode, entry.size, entry.mtime, @@ -170,7 +170,7 @@ py, PyBytes::new(py, [..]), decapsule_make_dirstate_tuple(py)?( -state, mode, size, mtime, +state as c_char, mode, size, mtime, ), )?; } To: gracinet, #hg-reviewers Cc: durin42, kevincox, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D6475: merge: fix race that could cause wrong size in dirstate
valentin.gatienbaron created this revision.
Herald added a reviewer: durin42.
Herald added a reviewer: martinvonz.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
The problem is that hg merge/update/etc work the following way:
1. figure out what files to update
2. apply the update to disk
3. apply the update to in-memory dirstate
4. write dirstate
where step3 looks at the filesystem and assumes it sees the result of
step2. If a file is changed between step2 and step3, step3 will record
incorrect information in the dirstate.
I avoid this by passing the size step3 needs directly from step2, for
the common path (not implemented for change/delete conflicts for
instance). The code is a bit awkward, as there doesn't seem to be a
standard way in python to return a value at the end of a generator.
I didn't fix the same race for the exec bit for now, because it's less
likely to be problematic and I had trouble due to the fact that the
dirstate stores the permissions differently from the manifest (st_mode
vs '' 'l' 'x'), in combination with tests that pretend that symlinks
are not supported.
However, I moved the lstat from step3 to step2, which should tighten
the race window markedly, both for the exec bit and for the mtime.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D6475
AFFECTED FILES
hgext/largefiles/overrides.py
hgext/narrow/narrowdirstate.py
hgext/remotefilelog/__init__.py
hgext/sparse.py
mercurial/context.py
mercurial/dirstate.py
mercurial/merge.py
mercurial/narrowspec.py
mercurial/sparse.py
mercurial/worker.py
tests/test-dirstate-race2.t
tests/test-dirstate.t
CHANGE DETAILS
diff --git a/tests/test-dirstate.t b/tests/test-dirstate.t
--- a/tests/test-dirstate.t
+++ b/tests/test-dirstate.t
@@ -73,7 +73,7 @@
> merge,
> )
>
- > def wraprecordupdates(orig, repo, actions, branchmerge):
+ > def wraprecordupdates(*args):
> raise error.Abort("simulated error while recording dirstateupdates")
>
> def reposetup(ui, repo):
diff --git a/tests/test-dirstate-race2.t b/tests/test-dirstate-race2.t
--- a/tests/test-dirstate-race2.t
+++ b/tests/test-dirstate-race2.t
@@ -27,13 +27,11 @@
> EOF
Do an update where file 'a' is changed between hg writing it to disk
-and hg writing the dirstate. It results in a corrupted dirstate, which
-stores the wrong size, and thus hg status shows spuriously modified
-files.
+and hg writing the dirstate. The dirstate is correct nonetheless, and
+so hg status correctly shows a as clean.
$ hg up -r 0 --config extensions.race=$TESTTMP/dirstaterace.py
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg debugdirstate --no-dates
- n 644 0 (set |unset) a (re)
+ n 644 2 (set |unset) a (re)
$ echo a > a; hg status; hg diff
- M a
diff --git a/mercurial/worker.py b/mercurial/worker.py
--- a/mercurial/worker.py
+++ b/mercurial/worker.py
@@ -83,31 +83,36 @@
benefit = linear - (_STARTUP_COST * workers + linear / workers)
return benefit >= 0.15
-def worker(ui, costperarg, func, staticargs, args, threadsafe=True):
+def worker(ui, costperarg, func, staticargs, args, hasretval=False,
+ threadsafe=True):
'''run a function, possibly in parallel in multiple worker
processes.
returns a progress iterator
costperarg - cost of a single task
-func - function to run
+func - function to run. It is expected to return a progress iterator.
staticargs - arguments to pass to every invocation of the function
args - arguments to split into chunks, to pass to individual
workers
+hasretval - when True, func and the current function return an progress
+iterator then a list (encoded as an iterator that yield many (False, ..)
+then a (True, list)). The resulting list is in the natural order.
+
threadsafe - whether work items are thread safe and can be executed using
a thread-based worker. Should be disabled for CPU heavy tasks that don't
release the GIL.
'''
enabled = ui.configbool('worker', 'enabled')
if enabled and worthwhile(ui, costperarg, len(args),
threadsafe=threadsafe):
-return _platformworker(ui, func, staticargs, args)
+return _platformworker(ui, func, staticargs, args, hasretval)
return func(*staticargs + (args,))
-def _posixworker(ui, func, staticargs, args):
+def _posixworker(ui, func, staticargs, args, hasretval):
workers = _numworkers(ui)
oldhandler = signal.getsignal(signal.SIGINT)
signal.signal(signal.SIGINT, signal.SIG_IGN)
@@ -157,14 +162,16 @@
ui.flush()
parentpid = os.getpid()
pipes = []
-for pargs in partition(args, workers):
+retvals = []
+for i, pargs in enumerate(partition(args, workers)):
# Every worker gets its own pipe to send results
D6474: tests: show how the dirstate can end up containing wrong information
valentin.gatienbaron created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
which can result in bad status output.
Concretely, this seems to be easily triggered by having a build system
watching the filesystem for changes, and rebuilding files that are
both tracked and generated while an update is happening.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D6474
AFFECTED FILES
tests/test-dirstate-race2.t
CHANGE DETAILS
diff --git a/tests/test-dirstate-race2.t b/tests/test-dirstate-race2.t
new file mode 100644
--- /dev/null
+++ b/tests/test-dirstate-race2.t
@@ -0,0 +1,39 @@
+Checking the size/permissions/file-type of files stored in the
+dirstate after an update where the files are changed concurrently
+outside of hg's control.
+
+ $ hg init repo
+ $ cd repo
+ $ echo a > a
+ $ hg commit -qAm _
+ $ echo aa > a
+ $ hg commit -m _
+
+ $ hg debugdirstate --no-dates
+ n 644 3 (set |unset) a (re)
+
+ $ cat >> $TESTTMP/dirstaterace.py << EOF
+ > from mercurial import (
+ > extensions,
+ > merge,
+ > )
+ > def extsetup(ui):
+ > extensions.wrapfunction(merge, 'applyupdates', wrap)
+ > def wrap(orig, *args, **kwargs):
+ > res = orig(*args, **kwargs)
+ > with open("a", "w"):
+ > pass # just truncate the file
+ > return res
+ > EOF
+
+Do an update where file 'a' is changed between hg writing it to disk
+and hg writing the dirstate. It results in a corrupted dirstate, which
+stores the wrong size, and thus hg status shows spuriously modified
+files.
+
+ $ hg up -r 0 --config extensions.race=$TESTTMP/dirstaterace.py
+ 1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+ $ hg debugdirstate --no-dates
+ n 644 0 (set |unset) a (re)
+ $ echo a > a; hg status; hg diff
+ M a
To: valentin.gatienbaron, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D6473: rust-dirstate: architecture independence fix
gracinet created this revision. Herald added subscribers: mercurial-devel, kevincox, durin42. Herald added a reviewer: hg-reviewers. REVISION SUMMARY Apparently, c_char is u8 on ppc64le and i8 on amd64 REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D6473 AFFECTED FILES rust/hg-cpython/src/dirstate.rs CHANGE DETAILS diff --git a/rust/hg-cpython/src/dirstate.rs b/rust/hg-cpython/src/dirstate.rs --- a/rust/hg-cpython/src/dirstate.rs +++ b/rust/hg-cpython/src/dirstate.rs @@ -70,7 +70,7 @@ py, PyBytes::new(py, [..]), decapsule_make_dirstate_tuple(py)?( -entry.state, +entry.state as c_char, entry.mode, entry.size, entry.mtime, @@ -170,7 +170,7 @@ py, PyBytes::new(py, [..]), decapsule_make_dirstate_tuple(py)?( -state, mode, size, mtime, +state as c_char, mode, size, mtime, ), )?; } To: gracinet, #hg-reviewers Cc: durin42, kevincox, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
