D5536: testrunner: avoid capturing a regex group we don't care about
martinvonz created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5536 AFFECTED FILES tests/run-tests.py CHANGE DETAILS diff --git a/tests/run-tests.py b/tests/run-tests.py --- a/tests/run-tests.py +++ b/tests/run-tests.py @@ -2786,7 +2786,8 @@ expanded_args.append(arg) args = expanded_args -testcasepattern = re.compile(br'([\w-]+\.t|py)(#([a-zA-Z0-9_\-\.#]+))') +testcasepattern = re.compile( +br'([\w-]+\.t|py)(?:#([a-zA-Z0-9_\-\.#]+))') tests = [] for t in args: case = [] @@ -2796,7 +2797,7 @@ m = testcasepattern.match(os.path.basename(t)) if m is not None: -t_basename, _, casestr = m.groups() +t_basename, casestr = m.groups() t = os.path.join(os.path.dirname(t), t_basename) if casestr: case = casestr.split(b'#') To: martinvonz, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5535: tests: support passing testcase after .t paths that have path separators
This revision was automatically updated to reflect the committed changes. Closed by commit rHG8ddc5d8bea25: tests: support passing testcase after .t paths that have path separators (authored by spectral, committed by ). REPOSITORY rHG Mercurial CHANGES SINCE LAST UPDATE https://phab.mercurial-scm.org/D5535?vs=13087=13088 REVISION DETAIL https://phab.mercurial-scm.org/D5535 AFFECTED FILES tests/run-tests.py CHANGE DETAILS diff --git a/tests/run-tests.py b/tests/run-tests.py --- a/tests/run-tests.py +++ b/tests/run-tests.py @@ -2794,9 +2794,10 @@ if not (os.path.basename(t).startswith(b'test-') and (t.endswith(b'.py') or t.endswith(b'.t'))): -m = testcasepattern.match(t) +m = testcasepattern.match(os.path.basename(t)) if m is not None: -t, _, casestr = m.groups() +t_basename, _, casestr = m.groups() +t = os.path.join(os.path.dirname(t), t_basename) if casestr: case = casestr.split(b'#') else: To: spectral, #hg-reviewers, martinvonz Cc: martinvonz, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5535: tests: support passing testcase after .t paths that have path separators
martinvonz accepted this revision. martinvonz added a comment. This revision is now accepted and ready to land. Thanks for fixing! I have run into this many times but never spent time trying to understand what the problem was. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5535 To: spectral, #hg-reviewers, martinvonz Cc: martinvonz, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5535: tests: support passing testcase after .t paths that have path separators
spectral created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY This probably could have been implemented by changing the regex above this bit of code, but I wasn't sure if it would end up handling various OSes correctly, so I decided to go with this version instead. Previously: $ tests/run-tests.py tests/test-ssh.t -l running 2 tests using 2 parallel processes .. # Ran 2 tests, 0 skipped, 0 failed. $ tests/run-tests.py tests/test-ssh.t#sshv1 -l running 0 tests using 0 parallel processes # Ran 0 tests, 0 skipped, 0 failed. Now: $ tests/run-tests.py tests/test-ssh.t -l running 2 tests using 2 parallel processes .. # Ran 2 tests, 0 skipped, 0 failed. $ tests/run-tests.py tests/test-ssh.t#sshv1 -l running 1 tests using 1 parallel processes . # Ran 1 tests, 0 skipped, 0 failed. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5535 AFFECTED FILES tests/run-tests.py CHANGE DETAILS diff --git a/tests/run-tests.py b/tests/run-tests.py --- a/tests/run-tests.py +++ b/tests/run-tests.py @@ -2794,9 +2794,10 @@ if not (os.path.basename(t).startswith(b'test-') and (t.endswith(b'.py') or t.endswith(b'.t'))): -m = testcasepattern.match(t) +m = testcasepattern.match(os.path.basename(t)) if m is not None: -t, _, casestr = m.groups() +t_basename, _, casestr = m.groups() +t = os.path.join(os.path.dirname(t), t_basename) if casestr: case = casestr.split(b'#') else: To: spectral, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5534: merge: make local file storage in the .hg/merge directory extensible
dploch created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
This is similar to remotefilelog's 'getlocalkey' method, which must be
overridden by systems which rely on full path names for access control purposes.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5534
AFFECTED FILES
mercurial/merge.py
CHANGE DETAILS
diff --git a/mercurial/merge.py b/mercurial/merge.py
--- a/mercurial/merge.py
+++ b/mercurial/merge.py
@@ -478,6 +478,13 @@
f.write(_pack(format, key, len(data), data))
f.close()
+@staticmethod
+def getlocalkey(path):
+"""hash the path of a local file context for storage in the .hg/merge
+directory."""
+
+return hex(hashlib.sha1(path).digest())
+
def add(self, fcl, fco, fca, fd):
"""add a new (potentially?) conflicting file the merge state
fcl: file context for local,
@@ -488,11 +495,11 @@
note: also write the local version to the `.hg/merge` directory.
"""
if fcl.isabsent():
-hash = nullhex
+localkey = nullhex
else:
-hash = hex(hashlib.sha1(fcl.path()).digest())
-self._repo.vfs.write('merge/' + hash, fcl.data())
-self._state[fd] = [MERGE_RECORD_UNRESOLVED, hash, fcl.path(),
+localkey = mergestate.getlocalkey(fcl.path())
+self._repo.vfs.write('merge/' + localkey, fcl.data())
+self._state[fd] = [MERGE_RECORD_UNRESOLVED, localkey, fcl.path(),
fca.path(), hex(fca.filenode()),
fco.path(), hex(fco.filenode()),
fcl.flags()]
@@ -551,15 +558,15 @@
MERGE_RECORD_DRIVER_RESOLVED):
return True, 0
stateentry = self._state[dfile]
-state, hash, lfile, afile, anode, ofile, onode, flags = stateentry
+state, localkey, lfile, afile, anode, ofile, onode, flags = stateentry
octx = self._repo[self._other]
extras = self.extras(dfile)
anccommitnode = extras.get('ancestorlinknode')
if anccommitnode:
actx = self._repo[anccommitnode]
else:
actx = None
-fcd = self._filectxorabsent(hash, wctx, dfile)
+fcd = self._filectxorabsent(localkey, wctx, dfile)
fco = self._filectxorabsent(onode, octx, ofile)
# TODO: move this to filectxorabsent
fca = self._repo.filectx(afile, fileid=anode, changectx=actx)
@@ -577,8 +584,8 @@
flags = flo
if preresolve:
# restore local
-if hash != nullhex:
-f = self._repo.vfs('merge/' + hash)
+if localkey != nullhex:
+f = self._repo.vfs('merge/' + localkey)
wctx[dfile].write(f.read(), flags)
f.close()
else:
To: dploch, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5533: remotefilelog: add newlines to ui.log() invocations
spectral created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
The comment at the top of ui.log() says that the message should be a
newline-terminated string. When using the blackbox logger, if the string does
not end in a newline, the appearance in the file gets quite messy.
This sometimes leaves a string with just the newline as the message, these are
logged by the blackbox logger just fine. I don't know what other loggers do
when
logging structured data and a message that is just the newline.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5533
AFFECTED FILES
hgext/remotefilelog/remotefilectx.py
hgext/remotefilelog/shallowutil.py
CHANGE DETAILS
diff --git a/hgext/remotefilelog/shallowutil.py
b/hgext/remotefilelog/shallowutil.py
--- a/hgext/remotefilelog/shallowutil.py
+++ b/hgext/remotefilelog/shallowutil.py
@@ -106,7 +106,7 @@
def reportpackmetrics(ui, prefix, *stores):
dicts = [s.getmetrics() for s in stores]
dict = prefixkeys(sumdicts(*dicts), prefix + '_')
-ui.log(prefix + "_packsizes", "", **pycompat.strkwargs(dict))
+ui.log(prefix + "_packsizes", "\n", **pycompat.strkwargs(dict))
def _parsepackmeta(metabuf):
"""parse datapack meta, bytes () -> dict
diff --git a/hgext/remotefilelog/remotefilectx.py
b/hgext/remotefilelog/remotefilectx.py
--- a/hgext/remotefilelog/remotefilectx.py
+++ b/hgext/remotefilelog/remotefilectx.py
@@ -226,7 +226,7 @@
r'reponame': shallowutil.getreponame(repo.ui),
}
-repo.ui.log('linkrevfixup', 'adjusting linknode', **commonlogkwargs)
+repo.ui.log('linkrevfixup', 'adjusting linknode\n', **commonlogkwargs)
pc = repo._phasecache
seenpublic = False
@@ -315,7 +315,7 @@
return None
finally:
elapsed = time.time() - start
-repo.ui.log('linkrevfixup', logmsg, elapsed=elapsed * 1000,
+repo.ui.log('linkrevfixup', logmsg + '\n', elapsed=elapsed * 1000,
**pycompat.strkwargs(commonlogkwargs))
def _verifylinknode(self, revs, linknode):
To: spectral, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5532: context: schedule file prefetch before comparing for cleanliness
spectral created this revision.
Herald added subscribers: mercurial-devel, mjpieters.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
When using a system like remotefilelog, we can occasionally run into scenarios
where the local content cache does not have the data we need to perform these
comparisons. These will be fetched on-demand, but individually; if the
remotefilelog server isn't extremely low latency, the runtime of the command
becomes dominated by the multiple getfile requests for individual files.
By performing the prefetch, we can download these files in bulk, and save
server
resources and many network roundtrips.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5532
AFFECTED FILES
mercurial/context.py
CHANGE DETAILS
diff --git a/mercurial/context.py b/mercurial/context.py
--- a/mercurial/context.py
+++ b/mercurial/context.py
@@ -2018,6 +2018,12 @@
to resolve a conflict.
"""
keys = []
+# This won't be perfect, but can help performance significantly when
+# using things like remotefilelog.
+scmutil.prefetchfiles(
+self.repo(), [self.p1().rev()],
+matchmod.match('', '', patterns=self._cache.keys(), exact=True))
+
for path in self._cache.keys():
cache = self._cache[path]
try:
To: spectral, #hg-reviewers
Cc: mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: Sprint 5.0 (Asia, Spring 2019)
I've started https://www.mercurial-scm.org/wiki/5.0sprint, please fill it in with your interest/availability, or if there's other places that can host! On Tue, Oct 16, 2018 at 9:59 AM Gregory Szorc wrote: > On Sun, Oct 14, 2018 at 12:30 PM Kyle Lippincott > wrote: > >> Per the recent discussion at the 4.8 Sprint, let's discuss the next >> sprint now. The next one, for what is currently assumed to be the 5.0 >> release in Spring 2019, is currently scheduled to happen in Asia and I'm >> proposing Japan specifically. >> >> The past two sprints have been planned by Googlers and hosted by Google. >> Ideally a separate entity offers hosting space and handle logistics for >> this one. If no other entity steps up, I will attempt to organize this >> through Google again at Google's Tokyo office, but it is unclear if this is >> going to be acceptable. For Google's Tokyo office in particular, >> scheduling of a conference room is difficult, so getting agreement on the >> host, location, and timing earlier is *extremely* beneficial; let's be >> proactive about this. :) >> >> Please let me know if there are any questions or concerns. >> > > From a visa and transportation standpoint, Japan is likely our best option > in Asia. I think Hong Kong and Taiwan would also be up there. But I believe > some have more difficulty with visas for those locations compared to Japan. > > I would volunteer Mozilla to host. But I'm pretty sure we don't have a > suitable space in Japan. We /may/ have a suitable space in Taiwan. I'd have > to look into it... > ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5526: progress: document progress.debug config option
This revision was automatically updated to reflect the committed changes. Closed by commit rHG785a75f0ddcb: progress: document progress.debug config option (authored by martinvonz, committed by ). REPOSITORY rHG Mercurial CHANGES SINCE LAST UPDATE https://phab.mercurial-scm.org/D5526?vs=13076=13083 REVISION DETAIL https://phab.mercurial-scm.org/D5526 AFFECTED FILES mercurial/help/config.txt CHANGE DETAILS diff --git a/mercurial/help/config.txt b/mercurial/help/config.txt --- a/mercurial/help/config.txt +++ b/mercurial/help/config.txt @@ -1756,6 +1756,9 @@ possible. Some progress bars only offer indeterminate information, while others have a definite end point. +``debug`` +Whether to print debug info when updating the progress bar. (default: False) + ``delay`` Number of seconds (float) before showing the progress bar. (default: 3) To: martinvonz, #hg-reviewers, pulkit Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
4.9-rc0 coming up - prepare accordingly
Hey everyone! As per our time based release plan, we have a major release i.e. hg-4.9 coming 1st Feb. 4.9rc0 will be released on upcoming 16th or 17th. So if you want something to be included in upcoming major release, now is a good time to send patches. If you need help getting something landed between now and the 17th, please coordinate with a reviewer /now/ or reply to this email and we can discuss further. Also, last cycle we tried something new by accepting new functionality on default branch which we will like to continue this cycle too. Bugfixes in the rc should go on stable as usual. Thanks! Regards Pulkit ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5531: progress: deprecate ui.progress()
martinvonz updated this revision to Diff 13082.
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D5531?vs=13081=13082
REVISION DETAIL
https://phab.mercurial-scm.org/D5531
AFFECTED FILES
mercurial/ui.py
CHANGE DETAILS
diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -1691,6 +1691,8 @@
All topics should be marked closed by setting pos to None at
termination.
'''
+self.deprecwarn("use ui.makeprogress() instead of ui.progress()",
+"5.0")
progress = self.makeprogress(topic, unit, total)
if pos is not None:
progress.update(pos, item=item)
To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5531: progress: deprecate ui.progress()
martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
It is now just a weird wrapper for ui.makeprogress().
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5531
AFFECTED FILES
mercurial/ui.py
CHANGE DETAILS
diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -1691,6 +1691,8 @@
All topics should be marked closed by setting pos to None at
termination.
'''
+self.deprecwarn("use ui.makeprogress() instead of ui.progress()",
+"4.10")
progress = self.makeprogress(topic, unit, total)
if pos is not None:
progress.update(pos, item=item)
To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5530: progress: check what type of progress bar to use only once per topic
martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
This seems to have sped up `hg perfprogress` from 1.78 s to 1.41 s.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5530
AFFECTED FILES
mercurial/scmutil.py
CHANGE DETAILS
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1422,6 +1422,24 @@
self.unit = unit
self.total = total
self.debug = ui.configbool('progress', 'debug')
+if getattr(ui._fmsgerr, 'structured', False):
+# channel for machine-readable output with metadata, just send
+# raw information
+# TODO: consider porting some useful information (e.g. estimated
+# time) from progbar. we might want to support update delay to
+# reduce the cost of transferring progress messages.
+def updatebar(item):
+ui._fmsgerr.write(None, type=b'progress', topic=self.topic,
+ pos=self.pos, item=item, unit=self.unit,
+ total=self.total)
+elif ui._progbar is not None:
+def updatebar(item):
+ui._progbar.progress(self.topic, self.pos, item=item,
+ unit=self.unit, total=self.total)
+else:
+def updatebar(item):
+pass
+self._updatebar = updatebar
def __enter__(self):
return self
@@ -1447,20 +1465,6 @@
self.total = None
self._updatebar("")
-def _updatebar(self, item):
-if getattr(self.ui._fmsgerr, 'structured', False):
-# channel for machine-readable output with metadata, just send
-# raw information
-# TODO: consider porting some useful information (e.g. estimated
-# time) from progbar. we might want to support update delay to
-# reduce the cost of transferring progress messages.
-self.ui._fmsgerr.write(None, type=b'progress', topic=self.topic,
- pos=self.pos, item=item, unit=self.unit,
- total=self.total)
-elif self.ui._progbar is not None:
-self.ui._progbar.progress(self.topic, self.pos, item=item,
- unit=self.unit, total=self.total)
-
def _printdebug(self, item):
if self.unit:
unit = ' ' + self.unit
To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5527: progress: write ui.progress() in terms of ui.makeprogress()
martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
I think ui.makeprogress() should be the preferred interface and we
should deprecate ui.progress(). All in-core callers already use
ui.makeprogress(). Moving the logic to the scmutil.progress() will let
us make further improvements.
This seems to have sped up `hg perfprogress` from 1.92 s to 1.85 s,
perhaps because we now skip the indirection of updating the progress
bar via ui.progress().
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5527
AFFECTED FILES
mercurial/scmutil.py
mercurial/ui.py
CHANGE DETAILS
diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -1691,39 +1691,11 @@
All topics should be marked closed by setting pos to None at
termination.
'''
-if getattr(self._fmsgerr, 'structured', False):
-# channel for machine-readable output with metadata, just send
-# raw information
-# TODO: consider porting some useful information (e.g. estimated
-# time) from progbar. we might want to support update delay to
-# reduce the cost of transferring progress messages.
-self._fmsgerr.write(None, type=b'progress', topic=topic, pos=pos,
-item=item, unit=unit, total=total)
-elif self._progbar is not None:
-self._progbar.progress(topic, pos, item=item, unit=unit,
- total=total)
-
-# Looking up progress.debug in tight loops is expensive. The value
-# is cached on the progbar object and we can avoid the lookup in
-# the common case where a progbar is active.
-if pos is None or not self._progbar.debug:
-return
-
-# Keep this logic in sync with above.
-if pos is None or not self.configbool('progress', 'debug'):
-return
-
-if unit:
-unit = ' ' + unit
-if item:
-item = ' ' + item
-
-if total:
-pct = 100.0 * pos / total
-self.debug('%s:%s %d/%d%s (%4.2f%%)\n'
- % (topic, item, pos, total, unit, pct))
+progress = self.makeprogress(topic, unit, total)
+if pos is not None:
+progress.update(pos, item=item)
else:
-self.debug('%s:%s %d%s\n' % (topic, item, pos, unit))
+progress.complete()
def makeprogress(self, topic, unit="", total=None):
'''exists only so low-level modules won't need to import scmutil'''
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1439,11 +1439,46 @@
self.update(self.pos + step, item, total)
def complete(self):
-self.ui.progress(self.topic, None)
+self.pos = None
+self.unit = ""
+self.total = None
+self._print("")
def _print(self, item):
-self.ui.progress(self.topic, self.pos, item, self.unit,
- self.total)
+if getattr(self.ui._fmsgerr, 'structured', False):
+# channel for machine-readable output with metadata, just send
+# raw information
+# TODO: consider porting some useful information (e.g. estimated
+# time) from progbar. we might want to support update delay to
+# reduce the cost of transferring progress messages.
+self.ui._fmsgerr.write(None, type=b'progress', topic=self.topic,
+ pos=self.pos, item=item, unit=self.unit,
+ total=self.total)
+elif self.ui._progbar is not None:
+self.ui._progbar.progress(self.topic, self.pos, item=item,
+ unit=self.unit, total=self.total)
+
+# Looking up progress.debug in tight loops is expensive. The value
+# is cached on the progbar object and we can avoid the lookup in
+# the common case where a progbar is active.
+if self.pos is None or not self.ui._progbar.debug:
+return
+
+# Keep this logic in sync with above.
+if self.pos is None or not self.ui.configbool('progress', 'debug'):
+return
+
+if self.unit:
+unit = ' ' + self.unit
+if item:
+item = ' ' + item
+
+if self.total:
+pct = 100.0 * self.pos / self.total
+self.ui.debug('%s:%s %d/%d%s (%4.2f%%)\n'
+ % (self.topic, item, self.pos, self.total, unit, pct))
+else:
+self.ui.debug('%s:%s %d%s\n' % (self.topic, item, self.pos, unit))
def gdinitconfig(ui):
"""helper function to know if a repo should be created as general delta
To: martinvonz, #hg-reviewers
Cc:
D5529: progress: split up _print() method in bar-updating and debug-printing
martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
I just thought this was clearer, but it turned out to also simplify
the next patch.
This seems to have sped up `hg perfprogress` from 1.85 s to 1.78 s.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5529
AFFECTED FILES
mercurial/scmutil.py
CHANGE DETAILS
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1434,18 +1434,20 @@
if total:
self.total = total
self.pos = pos
-self._print(item)
+self._updatebar(item)
+if self.debug:
+self._printdebug(item)
def increment(self, step=1, item="", total=None):
self.update(self.pos + step, item, total)
def complete(self):
self.pos = None
self.unit = ""
self.total = None
-self._print("")
+self._updatebar("")
-def _print(self, item):
+def _updatebar(self, item):
if getattr(self.ui._fmsgerr, 'structured', False):
# channel for machine-readable output with metadata, just send
# raw information
@@ -1459,9 +1461,7 @@
self.ui._progbar.progress(self.topic, self.pos, item=item,
unit=self.unit, total=self.total)
-if self.pos is None or not self.debug:
-return
-
+def _printdebug(self, item):
if self.unit:
unit = ' ' + self.unit
if item:
To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5528: progress: move cached debug flag from progress.progbar to scmutil.progress
martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
It's simpler this way. One possible drawback (and a possisble
advantage) is that we now check the debug flag once per topic, so
processes that generate new topics all the time will still check the
flag frequently.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5528
AFFECTED FILES
mercurial/progress.py
mercurial/scmutil.py
CHANGE DETAILS
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1421,6 +1421,7 @@
self.topic = topic
self.unit = unit
self.total = total
+self.debug = ui.configbool('progress', 'debug')
def __enter__(self):
return self
@@ -1458,14 +1459,7 @@
self.ui._progbar.progress(self.topic, self.pos, item=item,
unit=self.unit, total=self.total)
-# Looking up progress.debug in tight loops is expensive. The value
-# is cached on the progbar object and we can avoid the lookup in
-# the common case where a progbar is active.
-if self.pos is None or not self.ui._progbar.debug:
-return
-
-# Keep this logic in sync with above.
-if self.pos is None or not self.ui.configbool('progress', 'debug'):
+if self.pos is None or not self.debug:
return
if self.unit:
diff --git a/mercurial/progress.py b/mercurial/progress.py
--- a/mercurial/progress.py
+++ b/mercurial/progress.py
@@ -104,8 +104,6 @@
self.order = self.ui.configlist('progress', 'format')
self.estimateinterval = self.ui.configwith(
float, 'progress', 'estimateinterval')
-# developer config: progress.debug
-self.debug = self.ui.configbool('progress', 'debug')
def show(self, now, topic, pos, item, unit, total):
if not shouldprint(self.ui):
To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5526: progress: document progress.debug config option
martinvonz created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY I think it was not spotted by test-check-config.t that we had not documented it because no caller refers to the ui object simply as "ui" (it was either "self.ui" or just "self"). REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5526 AFFECTED FILES mercurial/help/config.txt CHANGE DETAILS diff --git a/mercurial/help/config.txt b/mercurial/help/config.txt --- a/mercurial/help/config.txt +++ b/mercurial/help/config.txt @@ -1756,6 +1756,9 @@ possible. Some progress bars only offer indeterminate information, while others have a definite end point. +``debug`` +Whether to print debug info when updating the progress bar. (default: False) + ``delay`` Number of seconds (float) before showing the progress bar. (default: 3) To: martinvonz, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5514: test: change test's diff generation to use mdiff for nicer output
pulkit added a comment. > If you couldn't follow those steps, you can simply check the diffs here: > > test's diff: https://pastebin.com/Z4LRg4vx > diff used by hg diff: https://pastebin.com/qanQEsiA Can you make the diffs a part of commit message since pastebin's pastes can be removed in future? REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5514 To: sangeet259, #hg-reviewers, durin42 Cc: pulkit, durin42, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5514: test: change test's diff generation to use mdiff for nicer output
durin42 requested changes to this revision. durin42 added inline comments. This revision now requires changes to proceed. INLINE COMMENTS > run-tests.py:72 > import xml.dom.minidom as minidom > +from mercurial import mdiff, patch > This will fail if Mercurial isn't already globally installed on the machine, as would be the case in at least some packaging circumstances when building from a source tarball. Probably need a try/except, and _maybe_ a sanity check that mdiff's API matches what we expect? REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5514 To: sangeet259, #hg-reviewers, durin42 Cc: durin42, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5525: xdiff: don't attempt to use fuzzer inputs larger than 100k
durin42 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
This is the recommended approach from [0], and limiting the input was
suggested in https://github.com/google/oss-fuzz/issues/2076 when
discussing our broken coverage build.
0:
https://github.com/google/oss-fuzz/blob/master/docs/new_project_guide.md#custom-libfuzzer-options-for-clusterfuzz
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5525
AFFECTED FILES
contrib/fuzz/xdiff.cc
CHANGE DETAILS
diff --git a/contrib/fuzz/xdiff.cc b/contrib/fuzz/xdiff.cc
--- a/contrib/fuzz/xdiff.cc
+++ b/contrib/fuzz/xdiff.cc
@@ -22,6 +22,11 @@
int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size)
{
+ // Don't allow fuzzer inputs larger than 100k, since we'll just bog
+ // down and not accomplish much.
+ if (Size > 10) {
+ return 0;
+ }
auto maybe_inputs = SplitInputs(Data, Size);
if (!maybe_inputs) {
return 0;
To: durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 4 of 4] mmapindex: set default to 1MB
On Mon, 7 Jan 2019 09:45:32 +0100, Boris FELD wrote: > On 03/01/2019 09:58, Yuya Nishihara wrote: > > On Wed, 2 Jan 2019 23:40:11 +0100, Boris FELD wrote: > >> On 04/12/2018 12:09, Yuya Nishihara wrote: > >>> On Sun, 02 Dec 2018 17:17:43 +0100, Boris Feld wrote: > # HG changeset patch > # User Boris Feld > # Date 1542949784 -3600 > # Fri Nov 23 06:09:44 2018 +0100 > # Node ID 9708243274585f9544c70925eb0b0fa0ec7aba4f > # Parent 0fff68dfbe48d87dce8b8736c0347ed5aa79030e > # EXP-Topic mmap > # Available At https://bitbucket.org/octobus/mercurial-devel/ > # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r > 970824327458 > mmapindex: set default to 1MB > >>> Can you check if strip/rollback properly copy the revlog before > >>> truncating it? > >>> > >>> If a mmapped revlog is truncated by another process, the mapped memory > >>> could be > >>> invalid. The worst case, the process would be killed by SIGBUS. > >> Hum good catch, process reading a repository being stripped have always > >> been up for troubles. However, mmap makes it worse by raising a signal > >> instead of just having wonky seek. It also introduces new cases where > >> this can happen. > > mmap isn't worse because of SIGBUS, but because the index data can be > > updated > > after the index length is determined. Before, a single in-memory revlog > > index > > was at least consistent. > > > >> What shall we do here, I guess our best bet is to intercept these SIGBUS > >> when reading revlog index? > Yes, but it would be inconsistent with the data it was pointing to. > Access to this data would result in error too. Correct. > The new thing is that we > can get SIGBUS while accessing the index data themselves, as you are > pointing out. Another new thing is that truncated revisions can be seen as valid changesets of '000...' hash with 0-length text. If the whole (or maybe dozens of) revisions were truncated, SIGBUS would be raised. In other cases, the truncated part would probably be read as zeros. > > I don't think it'll be easy to handle SIGBUS properly. And SIGBUS won't > > always > > be raised. Perhaps, the easiest solution is to copy the revlog index before > > strip/rollback. > > I'm afraid at the performance impact, we are talking of potentially > hundreds of MB of index data to be rolled back. > > Maybe we can keep the current truncation in normal transaction rollback > and use the slower copies for the hg strip command itself (and rewrite)? > > However, I'm afraid we need to come up with a solution for mmap as it > would be useful to use it more and more. > > Maybe we can come up with something catching the SIGBUS? Or maybe we > need to never truncate files and keep an alternative way to track the > maximum offset? Any other ideas? I've no idea. My point is that catching SIGBUS wouldn't save us from many possible failures. > > IIRC, the mmap implementation was highly experimental. I don't know if it's > > widely tested other than in FB where strip is never used. > We have been using it internally, and one of our clients deployed it > too. It results in significant speed and memory improvement. Yup. I'm just afraid of enabling it by default. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5522: tests: migrate test-wireproto-serverreactor.py to our internal CBOR
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGa181a1c8af1d: tests: migrate
test-wireproto-serverreactor.py to our internal CBOR (authored by durin42,
committed by ).
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D5522?vs=13069=13074
REVISION DETAIL
https://phab.mercurial-scm.org/D5522
AFFECTED FILES
tests/test-wireproto-serverreactor.py
CHANGE DETAILS
diff --git a/tests/test-wireproto-serverreactor.py
b/tests/test-wireproto-serverreactor.py
--- a/tests/test-wireproto-serverreactor.py
+++ b/tests/test-wireproto-serverreactor.py
@@ -2,9 +2,6 @@
import unittest
-from mercurial.thirdparty import (
-cbor,
-)
from mercurial import (
ui as uimod,
util,
@@ -16,7 +13,7 @@
ffs = framing.makeframefromhumanstring
-OK = cbor.dumps({b'status': b'ok'})
+OK = b''.join(cborutil.streamencode({b'status': b'ok'}))
def makereactor(deferoutput=False):
ui = uimod.ui()
@@ -270,20 +267,20 @@
})
def testinterleavedcommands(self):
-cbor1 = cbor.dumps({
+cbor1 = b''.join(cborutil.streamencode({
b'name': b'command1',
b'args': {
b'foo': b'bar',
b'key1': b'val',
}
-}, canonical=True)
-cbor3 = cbor.dumps({
+}))
+cbor3 = b''.join(cborutil.streamencode({
b'name': b'command3',
b'args': {
b'biz': b'baz',
b'key': b'val',
},
-}, canonical=True)
+}))
results = list(sendframes(makereactor(), [
ffs(b'1 1 stream-begin command-request new|more %s' % cbor1[0:6]),
To: durin42, indygreg, #hg-reviewers, pulkit
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5518: state: update comment about use of CBOR
This revision was automatically updated to reflect the committed changes. Closed by commit rHG050ea8eb42a5: state: update comment about use of CBOR (authored by durin42, committed by ). REPOSITORY rHG Mercurial CHANGES SINCE LAST UPDATE https://phab.mercurial-scm.org/D5518?vs=13065=13073 REVISION DETAIL https://phab.mercurial-scm.org/D5518 AFFECTED FILES mercurial/state.py CHANGE DETAILS diff --git a/mercurial/state.py b/mercurial/state.py --- a/mercurial/state.py +++ b/mercurial/state.py @@ -13,8 +13,8 @@ The class has methods using which the data can be stored to disk in a file under .hg/ directory. -We store the data on disk in cbor, for which we use the third party cbor library -to serialize and deserialize data. +We store the data on disk in cbor, for which we use the CBOR format to encode +the data. """ from __future__ import absolute_import To: durin42, indygreg, #hg-reviewers, pulkit Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 4 of 4] mmapindex: set default to 1MB
On Thu, Jan 3, 2019 at 2:37 PM Yuya Nishihara wrote: > On Wed, 2 Jan 2019 23:40:11 +0100, Boris FELD wrote: > > On 04/12/2018 12:09, Yuya Nishihara wrote: > > > On Sun, 02 Dec 2018 17:17:43 +0100, Boris Feld wrote: > > >> # HG changeset patch > > >> # User Boris Feld > > >> # Date 1542949784 -3600 > > >> # Fri Nov 23 06:09:44 2018 +0100 > > >> # Node ID 9708243274585f9544c70925eb0b0fa0ec7aba4f > > >> # Parent 0fff68dfbe48d87dce8b8736c0347ed5aa79030e > > >> # EXP-Topic mmap > > >> # Available At https://bitbucket.org/octobus/mercurial-devel/ > > >> # hg pull https://bitbucket.org/octobus/mercurial-devel/ > -r 970824327458 > > >> mmapindex: set default to 1MB > > > Can you check if strip/rollback properly copy the revlog before > truncating it? > > > > > > If a mmapped revlog is truncated by another process, the mapped memory > could be > > > invalid. The worst case, the process would be killed by SIGBUS. > > > > Hum good catch, process reading a repository being stripped have always > > been up for troubles. However, mmap makes it worse by raising a signal > > instead of just having wonky seek. It also introduces new cases where > > this can happen. > > mmap isn't worse because of SIGBUS, but because the index data can be > updated > after the index length is determined. Before, a single in-memory revlog > index > was at least consistent. > > > What shall we do here, I guess our best bet is to intercept these SIGBUS > > when reading revlog index? > > I don't think it'll be easy to handle SIGBUS properly. And SIGBUS won't > always > be raised. Perhaps, the easiest solution is to copy the revlog index before > strip/rollback. > > IIRC, the mmap implementation was highly experimental. I don't know if it's > widely tested other than in FB where strip is never used. > We did use mmap implementation and strip both for some time and didn't faced any issues except strip itself being an issue for slowdowns. However our use was limited. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5495: revset: add "branch" positional arguments to the merge revset
pulkit added a comment.
In https://phab.mercurial-scm.org/D5495#81562, @yuja wrote:
> > +@predicate('merge(*withbranch)', safe=True)
> >
> > def merge(repo, subset, x):
> >
> > - """Changeset is a merge changeset. +"""Changeset is a merge
changeset + +All merge revisions are returned by default. If one or more
"withbranch" +names are provided only merges with those branches (i.e.
whose +second parent belongs to one of those branches) will be returned.
>
> I understand this will be useful in a certain branch strategy, but the
> proposed syntax is hardly extensible. Maybe it can be a non-wildcard
argument
> of `stringmatcher` type so we can at least add another option later.
>
> Any thoughts? Do anyone love this feature?
I like this feature as it will be quite useful for us. Also I like your
filter() suggestion more than the syntax this series introduces.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5495
To: angel.ezquerra, #hg-reviewers
Cc: pulkit, yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
