D11253: test: disable test-subrepo-git.t in python2 + chg

[marmoute (Pierre-Yves David)]

marmoute created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.

REVISION SUMMARY
  I am a couple of days in try to debug that at it seems minor enough with 
enough
  other priority to simply disable it for now.

REPOSITORY
  rHG Mercurial

BRANCH
  stable

REVISION DETAIL
  https://phab.mercurial-scm.org/D11253

AFFECTED FILES
  tests/test-subrepo-git.t

CHANGE DETAILS

diff --git a/tests/test-subrepo-git.t b/tests/test-subrepo-git.t
--- a/tests/test-subrepo-git.t
+++ b/tests/test-subrepo-git.t
@@ -1,5 +1,20 @@
 #require git
 
+# XXX-CHG When running with python2 + chg this test tend to get stuck and end 
up
+# as a time-out error. My effort to reproduce this outside of the CI failed. 
The
+# test itself seems to pass fine, but never "complete". Debugging it is slow 
and
+# tedious. This as a bad impact on the development process as most CI run end 
up
+# wasting abotu 1h until that one fails.
+#
+# Pierre-Yves David, Augie Fackler and Raphaël Gomès all agreed to disable this
+# case in that specific case until we figure this out (or we drop python2 o:-) 
)
+
+#if no-py3 chg
+  $ echo 'skipped: this test get stuck on the CI with python2 + chg. 
investigation needed'
+  $ exit 80
+#endif
+
+
 make git commits repeatable
 
   $ cat >> $HGRCPATH 

D11248: store: return just one filename in walk functions

[valentin.gatienbaron (Valentin Gatien-Baron)]

valentin.gatienbaron created this revision.
Herald added a reviewer: durin42.
Herald added a reviewer: martinvonz.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.

REVISION SUMMARY
  Various walk functions return `(revlog_type, decoded, encoded)` where
  decoded could be None. But no-one cares about `encoded` and expects
  `unencoded` to be present, except verify (because this can only happen
  with old repo formats).
  
  Simplify all this by either failing outright if a decoding a filename
  fails (instead of almost certainly failing with a type error due to
  treating None as a bytes), or skipping the filename but providing in
  an out argument for hg verify.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D11248

AFFECTED FILES
  hgext/narrow/narrowcommands.py
  hgext/remotefilelog/contentstore.py
  hgext/remotefilelog/remotefilelogserver.py
  mercurial/repair.py
  mercurial/store.py
  mercurial/streamclone.py
  mercurial/upgrade_utils/engine.py
  mercurial/verify.py
  mercurial/wireprotov2server.py
  tests/simplestorerepo.py

CHANGE DETAILS

diff --git a/tests/simplestorerepo.py b/tests/simplestorerepo.py
--- a/tests/simplestorerepo.py
+++ b/tests/simplestorerepo.py
@@ -665,20 +665,24 @@
 
 
 class simplestore(store.encodedstore):
-def datafiles(self):
+def datafiles(self, undecodable=None):
 for x in super(simplestore, self).datafiles():
 yield x
 
 # Supplement with non-revlog files.
 extrafiles = self._walk('data', True, filefilter=issimplestorefile)
 
-for unencoded, encoded, size in extrafiles:
+for f1, size in extrafiles:
 try:
-unencoded = store.decodefilename(unencoded)
+f2 = store.decodefilename(f1)
 except KeyError:
-unencoded = None
-
-yield unencoded, encoded, size
+if undecodable is None:
+raise error.StorageError(b'undecodable revlog name %s' % 
a1)
+else:
+undecodable.append(a1)
+continue
+
+yield f2, size
 
 
 def reposetup(ui, repo):
diff --git a/mercurial/wireprotov2server.py b/mercurial/wireprotov2server.py
--- a/mercurial/wireprotov2server.py
+++ b/mercurial/wireprotov2server.py
@@ -1579,7 +1579,7 @@
 
 # TODO this is a bunch of storage layer interface abstractions because
 # it assumes revlogs.
-for rl_type, name, encodedname, size in topfiles:
+for rl_type, name, size in topfiles:
 # XXX use the `rl_type` for that
 if b'changelog' in files and name.startswith(b'00changelog'):
 pass
diff --git a/mercurial/verify.py b/mercurial/verify.py
--- a/mercurial/verify.py
+++ b/mercurial/verify.py
@@ -395,12 +395,13 @@
 storefiles = set()
 subdirs = set()
 revlogv1 = self.revlogv1
-for t, f, f2, size in repo.store.datafiles():
-if not f:
-self._err(None, _(b"cannot decode filename '%s'") % f2)
-elif (size > 0 or not revlogv1) and f.startswith(b'meta/'):
+undecodable = []
+for t, f, size in repo.store.datafiles(undecodable=undecodable):
+if (size > 0 or not revlogv1) and f.startswith(b'meta/'):
 storefiles.add(_normpath(f))
 subdirs.add(os.path.dirname(f))
+for f in undecodable:
+self._err(None, _(b"cannot decode filename '%s'") % f)
 subdirprogress = ui.makeprogress(
 _(b'checking'), unit=_(b'manifests'), total=len(subdirs)
 )
@@ -459,11 +460,12 @@
 ui.status(_(b"checking files\n"))
 
 storefiles = set()
-for rl_type, f, f2, size in repo.store.datafiles():
-if not f:
-self._err(None, _(b"cannot decode filename '%s'") % f2)
-elif (size > 0 or not revlogv1) and f.startswith(b'data/'):
+undecodable = []
+for t, f, size in repo.store.datafiles(undecodable=undecodable):
+if (size > 0 or not revlogv1) and f.startswith(b'data/'):
 storefiles.add(_normpath(f))
+for f in undecodable:
+self._err(None, _(b"cannot decode filename '%s'") % f)
 
 state = {
 # TODO this assumes revlog storage for changelog.
diff --git a/mercurial/upgrade_utils/engine.py 
b/mercurial/upgrade_utils/engine.py
--- a/mercurial/upgrade_utils/engine.py
+++ b/mercurial/upgrade_utils/engine.py
@@ -201,7 +201,7 @@
 
 # Perform a pass to collect metadata. This validates we can open all
 # source files and allows a unified progress bar to be displayed.
-for rl_type, unencoded, encoded, size in alldatafiles:
+for rl_type, unencoded, size in alldatafiles:
 if not rl_type & store.FILEFLAGS_REVLOG_MAIN:
 continue
 
diff --git 

D11252: debugupgraderepo: add fix-metaencoding-flag pass for issue6528

[joerg.sonnenberger (Joerg Sonnenberger)]

joerg.sonnenberger created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.
marmoute added a comment.


  This is similar to D11239 , having 
this done during upgrade too seems useful, but having a simpler/more-focussed 
command for this seems important (and "on-fly-fix" during exchange too.
  
  Ideally we would take D11239  first 
and reuse its bulding block for that diff afterward.

REPOSITORY
  rHG Mercurial

BRANCH
  stable

REVISION DETAIL
  https://phab.mercurial-scm.org/D11252

AFFECTED FILES
  mercurial/revlog.py
  mercurial/upgrade_utils/actions.py
  mercurial/upgrade_utils/engine.py

CHANGE DETAILS

diff --git a/mercurial/upgrade_utils/engine.py 
b/mercurial/upgrade_utils/engine.py
--- a/mercurial/upgrade_utils/engine.py
+++ b/mercurial/upgrade_utils/engine.py
@@ -136,6 +136,10 @@
 ):
 """returns the new revlog object created"""
 newrl = None
+recheckmetaencoding = (
+upgrade_op.recheckmetaencoding
+and (rl_type & store.FILEFLAGS_FILELOG) != 0
+)
 if matchrevlog(upgrade_op.revlogs_to_process, rl_type):
 ui.note(
 _(b'cloning %d revisions from %s\n') % (len(old_revlog), unencoded)
@@ -148,6 +152,7 @@
 deltareuse=upgrade_op.delta_reuse_mode,
 forcedeltabothparents=upgrade_op.force_re_delta_both_parents,
 sidedatacompanion=sidedatacompanion,
+recheckmetaencoding=recheckmetaencoding,
 )
 else:
 msg = _(b'blindly copying %s containing %i revisions\n')
diff --git a/mercurial/upgrade_utils/actions.py 
b/mercurial/upgrade_utils/actions.py
--- a/mercurial/upgrade_utils/actions.py
+++ b/mercurial/upgrade_utils/actions.py
@@ -616,6 +616,22 @@
 )
 )
 
+register_optimization(
+improvement(
+name=b'fix-metaencoding-flag',
+type=OPTIMISATION,
+description=_(
+b'filelog entries with copy metadata may have been flagged '
+b'incorrectly in Mercurial 5.8. This option will look for such '
+b'revisions and fix them.'
+),
+upgrademessage=_(
+b'revision content will be recomputed; this will likely 
drastically '
+b'slow down execution time'
+),
+)
+)
+
 
 def findoptimizations(repo):
 """Determine optimisation that could be used during upgrade"""
@@ -710,6 +726,11 @@
 elif b're-delta-fulladd' in self._upgrade_actions_names:
 self.delta_reuse_mode = revlog.revlog.DELTAREUSEFULLADD
 
+# should this operation search for misordered parents of copy filelog 
entries
+self.recheckmetaencoding = (
+b'fix-metaencoding-flag' in self._upgrade_actions_names
+)
+
 # should this operation force re-delta of both parents
 self.force_re_delta_both_parents = (
 b're-delta-multibase' in self._upgrade_actions_names
diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -2779,6 +2779,7 @@
 deltareuse=DELTAREUSESAMEREVS,
 forcedeltabothparents=None,
 sidedatacompanion=None,
+recheckmetaencoding=False,
 ):
 """Copy this revlog to another, possibly with format changes.
 
@@ -2876,6 +2877,7 @@
 deltareuse,
 forcedeltabothparents,
 sidedatacompanion,
+recheckmetaencoding,
 )
 
 finally:
@@ -2891,10 +2893,18 @@
 deltareuse,
 forcedeltabothparents,
 sidedatacompanion,
+recheckmetaencoding,
 ):
 """perform the core duty of `revlog.clone` after parameter 
processing"""
+
+def ismetaencoded(text, flags):
+return text.startswith(b'\x01\n') and (
+flags & REVIDX_ISCENSORED == 0
+)
+
 deltacomputer = deltautil.deltacomputer(destrevlog)
 index = self.index
+
 for rev in self:
 entry = index[rev]
 
@@ -2914,7 +2924,11 @@
 # the revlog chunk is a delta.
 cachedelta = None
 rawtext = None
-if any(sidedataactions) or deltareuse == self.DELTAREUSEFULLADD:
+if (
+any(sidedataactions)
+or deltareuse == self.DELTAREUSEFULLADD
+or recheckmetaencoding
+):
 dropall = sidedataactions[0]
 filterout = sidedataactions[1]
 update = sidedataactions[2]
@@ -2928,6 +2942,13 @@
 sidedata.update(update)
 if not sidedata:
 sidedata = None
+if (
+recheckmetaencoding
+and ismetaencoded(text, flags)
+and p1 != self.nullid
+and p2 == self.nullid
+):
+p1, p2 = p2, p1
 

D11251: test-nointerrupt: make "sure" the handler "might" trigger (issue6558)

[marmoute (Pierre-Yves David)]

marmoute created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.

REVISION SUMMARY
  We are sure that the signal got sent in the right time frame, however, we 
still
  have race, so either the code is actually buggy or we need some security to 
make
  sure the signal get processed.
  
  We might be affected by https://bugs.python.org/issue43406 ?

REPOSITORY
  rHG Mercurial

BRANCH
  stable

REVISION DETAIL
  https://phab.mercurial-scm.org/D11251

AFFECTED FILES
  tests/test-nointerrupt.t

CHANGE DETAILS

diff --git a/tests/test-nointerrupt.t b/tests/test-nointerrupt.t
--- a/tests/test-nointerrupt.t
+++ b/tests/test-nointerrupt.t
@@ -27,6 +27,8 @@
   > with ui.uninterruptible():
   > testing.write_file(sync_file, b'%d' % os.getpid())
   > testing.wait_file(done_file)
+  > # make sure we get rescheduled and the signal get a chance to be 
handled
+  > time.sleep(0.1)
   > ui.warn(b"end of unsafe operation\n")
   > ui.warn(b"%d second(s) passed\n" % int(time.time() - start))
   > EOF
@@ -40,6 +42,7 @@
   > fi
   > "$RUNTESTDIR/testlib/wait-on-file" 10 "$SYNC_FILE" || exit 2
   > kill -s \$SIG \`cat "$SYNC_FILE"\`
+  > sleep 1
   > touch "$DONE_FILE"
   > EOF
 



To: marmoute, #hg-reviewers
Cc: mercurial-patches, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D11249: check-module-imports: ignore non-stdlib module installed by distribution

[marmoute (Pierre-Yves David)]

marmoute created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.
pulkit accepted this revision.
This revision is now accepted and ready to land.
Alphare accepted this revision.

REVISION SUMMARY
  Previously, the check script would detect breezy as part of the stdlib if
  installed using the debian package manager.
  
  This silence the following complains:
  
  tests/test-convert-bzr.t:117: imports not lexically sorted: breezy.bzr.bzrdir 
< sys
  tests/test-convert-bzr.t:117: stdlib import "breezy.bzr.bzrdir" follows local 
import: breezy
  tests/test-convert-bzr-ghosts.t:7: imports not lexically sorted: 
breezy.bzr.bzrdir < sys
  tests/test-convert-bzr-ghosts.t:7: stdlib import "breezy.bzr.bzrdir" follows 
local import: breezy
  tests/test-convert-bzr-treeroot.t:7: imports not lexically sorted: 
breezy.bzr.bzrdir < sys
  tests/test-convert-bzr-treeroot.t:7: stdlib import "breezy.bzr.bzrdir" 
follows local import: breezy

REPOSITORY
  rHG Mercurial

BRANCH
  stable

REVISION DETAIL
  https://phab.mercurial-scm.org/D11249

AFFECTED FILES
  contrib/import-checker.py

CHANGE DETAILS

diff --git a/contrib/import-checker.py b/contrib/import-checker.py
--- a/contrib/import-checker.py
+++ b/contrib/import-checker.py
@@ -278,6 +278,8 @@
 ):
 continue
 for top, dirs, files in os.walk(libpath):
+if 'dist-packages' in top.split(os.path.sep):
+continue
 for i, d in reversed(list(enumerate(dirs))):
 if (
 not os.path.exists(os.path.join(top, d, '__init__.py'))



To: marmoute, #hg-reviewers
Cc: mercurial-patches, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D11250: testing: make sure write_file is "atomic"

[marmoute (Pierre-Yves David)]

marmoute created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.
marmoute added a comment.
Alphare accepted this revision.
This revision is now accepted and ready to land.


  This does not seems to be all of the issue we have with nointerrupt.t … 
https://foss.heptapod.net/octobus/mercurial-devel/-/jobs/233269

REVISION SUMMARY
  This make sure viewer cannot see the new file with partial content.
  
  This was likely the cause of some flakiness in `test-nointerrupt.t`

REPOSITORY
  rHG Mercurial

BRANCH
  stable

REVISION DETAIL
  https://phab.mercurial-scm.org/D11250

AFFECTED FILES
  mercurial/testing/__init__.py

CHANGE DETAILS

diff --git a/mercurial/testing/__init__.py b/mercurial/testing/__init__.py
--- a/mercurial/testing/__init__.py
+++ b/mercurial/testing/__init__.py
@@ -33,5 +33,11 @@
 
 
 def write_file(path, content=b''):
-with open(path, 'wb') as f:
+if content:
+write_path = b'%s.tmp' % path
+else:
+write_path = path
+with open(write_path, 'wb') as f:
 f.write(content)
+if path != write_path:
+os.rename(write_path, path)



To: marmoute, #hg-reviewers
Cc: mercurial-patches, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

mercurial@47801: 3 new changesets (3 on stable)

[Mercurial Commits]

3 new changesets (3 on stable) in mercurial:

https://www.mercurial-scm.org/repo/hg/rev/3fffb48539ee
changeset:   47799:3fffb48539ee
branch:  stable
user:Georges Racinet 
date:Sun Aug 01 14:39:38 2021 +0200
summary: rust-nodemap: falling back to C impl as mitigation

https://www.mercurial-scm.org/repo/hg/rev/6802422a1ae0
changeset:   47800:6802422a1ae0
branch:  stable
user:Valentin Gatien-Baron 
date:Mon Aug 02 08:06:27 2021 -0400
summary: remotefilelog: fix what looks like a wrong refactoring

https://www.mercurial-scm.org/repo/hg/rev/42e2cdb50db0
changeset:   47801:42e2cdb50db0
branch:  stable
tag: tip
user:Pierre-Yves David 
date:Tue Aug 03 18:29:31 2021 +0200
summary: check-module-imports: ignore non-stdlib module installed by 
distribution

-- 
Repository URL: https://www.mercurial-scm.org/repo/hg
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Failed pipeline for branch/stable | mercurial-devel | 49f3ac25

[Heptapod]

Pipeline #25533 has failed!

Project: mercurial-devel ( https://foss.heptapod.net/octobus/mercurial-devel )
Branch: branch/stable ( 
https://foss.heptapod.net/octobus/mercurial-devel/-/commits/branch/stable )

Commit: 49f3ac25 ( 
https://foss.heptapod.net/octobus/mercurial-devel/-/commit/49f3ac2571fcb7b35ed5eda8e33c5492579481c0
 )
Commit Message: check-module-imports: ignore non-stdlib module ...
Commit Author: Pierre-Yves David ( https://foss.heptapod.net/marmoute )

Pipeline #25533 ( 
https://foss.heptapod.net/octobus/mercurial-devel/-/pipelines/25533 ) triggered 
by Pulkit Goyal ( https://foss.heptapod.net/pulkit.goyal )
had 3 failed builds.

Job #233565 ( 
https://foss.heptapod.net/octobus/mercurial-devel/-/jobs/233565/raw )

Stage: tests
Name: test-py2
Job #233572 ( 
https://foss.heptapod.net/octobus/mercurial-devel/-/jobs/233572/raw )

Stage: tests
Name: test-py2-chg
Job #233573 ( 
https://foss.heptapod.net/octobus/mercurial-devel/-/jobs/233573/raw )

Stage: tests
Name: test-py3-chg

-- 
You're receiving this email because of your account on foss.heptapod.net.



___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel