D1998: wireproto: define and use types for wire protocol commands

[indygreg (Gregory Szorc)]

indygreg added a comment.


  In https://phab.mercurial-scm.org/D1998#34740, @durin42 wrote:
  
  > I'm curious what registrations you need that don't fit in 2-tuples. Can I 
see a sample of where that's going?
  
  
  I'll be adding additional attributes to `@wireprotocommand` in future series. 
One thing I know I'll add is a mechanism to limit which wire protocol 
versions/transports to expose a command to. This will prevent legacy commands 
from being exposed to modern protocols and vice versa.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1998

To: indygreg, #hg-reviewers
Cc: durin42, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2065: wireprotoserver: rename abstractserverproto and improve docstring

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5299.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2065?vs=5262=5299

REVISION DETAIL
  https://phab.mercurial-scm.org/D2065

AFFECTED FILES
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -38,10 +38,13 @@
 # to reflect BC breakages.
 SSHV2 = 'exp-ssh-v2-0001'
 
-class abstractserverproto(object):
-"""abstract class that summarizes the protocol API
+class baseprotocolhandler(object):
+"""Abstract base class for wire protocol handlers.
 
-Used as reference and documentation.
+A wire protocol handler serves as an interface between protocol command
+handlers and the wire protocol transport layer. Protocol handlers provide
+methods to read command arguments, redirect stdio for the duration of
+the request, handle response types, etc.
 """
 
 __metaclass__ = abc.ABCMeta
@@ -104,7 +107,7 @@
 
 return ''.join(chunks)
 
-class webproto(abstractserverproto):
+class webproto(baseprotocolhandler):
 def __init__(self, req, ui):
 self._req = req
 self._ui = ui
@@ -333,7 +336,7 @@
 
 return ''
 
-class sshserver(abstractserverproto):
+class sshserver(baseprotocolhandler):
 def __init__(self, ui, repo):
 self._ui = ui
 self._repo = repo



To: indygreg, #hg-reviewers, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2068: revlog: do not use delta for lfs revisions

[indygreg (Gregory Szorc)]

indygreg requested changes to this revision.
indygreg added inline comments.
This revision now requires changes to proceed.

INLINE COMMENTS

> revlog.py:408
> +# do not use flags != 0 (ex. LFS) revision as delta base
> +if revlog.flags(candidaterev) != REVIDX_DEFAULT_FLAGS:
> +continue

Same comment as previous review: this leaves a foot gun if we ever introduce 
revision flags that aren't related to content presence. We need something to 
help prevent this footgun.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2068

To: quark, indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2069: revlog: resolve lfs rawtext to vanilla rawtext before applying delta

[indygreg (Gregory Szorc)]

indygreg requested changes to this revision.
indygreg added a comment.
This revision now requires changes to proceed.


  I'd like to see the next versions of hte previous 2 patches before looking at 
this because this patch will likely get some minor rework as well.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2069

To: quark, indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2067: changegroup: do not delta lfs revisions

[indygreg (Gregory Szorc)]

indygreg requested changes to this revision.
indygreg added a comment.
This revision now requires changes to proceed.


  This looks mostly good. I would like a change to address a future footgun 
though.
  
  I would also appreciate someone familiar with censor and narrow to weigh in 
on the implications of disabling delta generation for revisions that have the 
censor and ellipsis flags set. I'm pretty sure narrow will cope since it 
reimplements changegroup generation. Not sure how censor will react. (But I 
know we already have random code for detecting censored nodes during 
changegroup generation.)

INLINE COMMENTS

> revlog.py:719
> +# disable delta if either rev uses non-default flag (ex. LFS)
> +if self.flags(baserev) or self.flags(rev):
> +return False

This logic assumes that revision flags will only ever be used to influence the 
presence of content. That is true today because our flags are for 
`REVIDX_ISCENSORED`, `REVIDX_ELLIPSIS`, and `REVIDX_EXTSTORED`. But if we ever 
introduced a new revision flag for e.g. compression strategy, then testing for 
non-empty revision flags would be wrong.

I think we want to capture the bitmask of revision flags influencing delta 
generation explicitly. Or we want a big comment by the revision flags constants 
at the top of the file telling people to audit `candelta()` when adding new 
revision flags.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2067

To: quark, indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2083: wireprotoserver: remove redirect() and restore() (API)

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5335.
indygreg edited the summary of this revision.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2083?vs=5317=5335

REVISION DETAIL
  https://phab.mercurial-scm.org/D2083

AFFECTED FILES
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -88,23 +88,6 @@
 won't be captured.
 """
 
-@abc.abstractmethod
-def redirect(self):
-"""may setup interception for stdout and stderr
-
-See also the `restore` method."""
-
-# If the `redirect` function does install interception, the `restore`
-# function MUST be defined. If interception is not used, this function
-# MUST NOT be defined.
-#
-# left commented here on purpose
-#
-#def restore(self):
-#"""reinstall previous stdout and stderr and return intercepted stdout
-#"""
-#raise NotImplementedError()
-
 def decodevaluefromheaders(req, headerprefix):
 """Decode a long value from multiple HTTP request headers.
 
@@ -181,15 +164,6 @@
 self._ui.fout = oldout
 self._ui.ferr = olderr
 
-def redirect(self):
-self._oldio = self._ui.fout, self._ui.ferr
-self._ui.ferr = self._ui.fout = stringio()
-
-def restore(self):
-val = self._ui.fout.getvalue()
-self._ui.ferr, self._ui.fout = self._oldio
-return val
-
 def _client(self):
 return 'remote:%s:%s:%s' % (
 self._req.env.get('wsgi.url_scheme') or 'http',
@@ -425,9 +399,6 @@
 def mayberedirectstdio(self):
 yield None
 
-def redirect(self):
-pass
-
 def _client(self):
 client = encoding.environ.get('SSH_CLIENT', '').split(' ', 1)[0]
 return 'remote:ssh:' + client



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2081: wireprotoserver: add context manager mechanism for redirecting stdio

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5333.
indygreg edited the summary of this revision.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2081?vs=5315=5333

REVISION DETAIL
  https://phab.mercurial-scm.org/D2081

AFFECTED FILES
  mercurial/wireproto.py
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -8,6 +8,7 @@
 
 import abc
 import cgi
+import contextlib
 import struct
 import sys
 
@@ -74,6 +75,20 @@
 """
 
 @abc.abstractmethod
+def mayberedirectstdio(self):
+"""Context manager to possibly redirect stdio.
+
+The context manager yields a file-object like object that receives
+stdout and stderr output when the context manager is active. Or it
+yields ``None`` if no I/O redirection occurs.
+
+The intent of this context manager is to capture stdio output
+so it may be sent in the response. Some transports support streaming
+stdio to the client in real time. For these transports, stdio output
+won't be captured.
+"""
+
+@abc.abstractmethod
 def redirect(self):
 """may setup interception for stdout and stderr
 
@@ -151,6 +166,21 @@
 for s in util.filechunkiter(self._req, limit=length):
 fp.write(s)
 
+@contextlib.contextmanager
+def mayberedirectstdio(self):
+oldout = self._ui.fout
+olderr = self._ui.ferr
+
+out = util.stringio()
+
+try:
+self._ui.fout = out
+self._ui.ferr = out
+yield out
+finally:
+self._ui.fout = oldout
+self._ui.ferr = olderr
+
 def redirect(self):
 self._oldio = self._ui.fout, self._ui.ferr
 self._ui.ferr = self._ui.fout = stringio()
@@ -393,6 +423,10 @@
 fpout.write(self._fin.read(count))
 count = int(self._fin.readline())
 
+@contextlib.contextmanager
+def mayberedirectstdio(self):
+yield None
+
 def redirect(self):
 pass
 
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -978,20 +978,12 @@
 else:
 new = encoding.tolocal(new) # normal path
 
-if util.safehasattr(proto, 'restore'):
-
-proto.redirect()
-
+with proto.mayberedirectstdio() as output:
 r = repo.pushkey(encoding.tolocal(namespace), encoding.tolocal(key),
  encoding.tolocal(old), new) or False
 
-output = proto.restore()
-
-return '%s\n%s' % (int(r), output)
-
-r = repo.pushkey(encoding.tolocal(namespace), encoding.tolocal(key),
- encoding.tolocal(old), new)
-return '%s\n' % int(r)
+output = output.getvalue() if output else ''
+return '%s\n%s' % (int(r), output)
 
 @wireprotocommand('stream_out')
 def stream(repo, proto):



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2092: wireprotoserver: add version to SSH protocol names (API)

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  We recently introduced version 2 of the SSH protocol. Like we
  did for the peer, we will need to differentiate version 1 and 2
  of the server. So, we add version components to the advertised
  name of the protocol handler.
  
  Nothing in core looks for the old value. But an extension may.
  
  .. api::
  
SSH protocol handler now advertises its name internally as
"ssh-v1" instead of "ssh."

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2092

AFFECTED FILES
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -367,7 +367,7 @@
 
 @property
 def name(self):
-return 'ssh'
+return SSHV1
 
 def getargs(self, args):
 data = {}



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2094: wireprotoserver: define and use parse_qs from urllib

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The cgi module is deprecated since Python 2.6. Let's replace uses
  of it in wireprotoserver with a similar function from urllib.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2094

AFFECTED FILES
  mercurial/urllibcompat.py
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -7,7 +7,6 @@
 from __future__ import absolute_import
 
 import abc
-import cgi
 import contextlib
 import struct
 import sys
@@ -134,12 +133,12 @@
 args = util.rapply(pycompat.bytesurl, self._req.form.copy())
 postlen = int(self._req.env.get(r'HTTP_X_HGARGS_POST', 0))
 if postlen:
-args.update(cgi.parse_qs(
+args.update(urlreq.parseqs(
 self._req.read(postlen), keep_blank_values=True))
 return args
 
 argvalue = decodevaluefromheaders(self._req, r'X-HgArg')
-args.update(cgi.parse_qs(argvalue, keep_blank_values=True))
+args.update(urlreq.parseqs(argvalue, keep_blank_values=True))
 return args
 
 def forwardpayload(self, fp):
diff --git a/mercurial/urllibcompat.py b/mercurial/urllibcompat.py
--- a/mercurial/urllibcompat.py
+++ b/mercurial/urllibcompat.py
@@ -47,6 +47,7 @@
 "urlparse",
 "urlunparse",
 ))
+urlreq._registeralias(urllib.parse, "parse_qs", "parseqs")
 urlreq._registeralias(urllib.parse, "unquote_to_bytes", "unquote")
 import urllib.request
 urlreq._registeraliases(urllib.request, (
@@ -157,6 +158,7 @@
 "urlparse",
 "urlunparse",
 ))
+urlreq._registeralias(urlparse, "parse_qs", "parseqs")
 urlerr._registeraliases(urllib2, (
 "HTTPError",
 "URLError",



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2057: translate base85.c into rust code

[indygreg (Gregory Szorc)]

indygreg added a comment.


  To be honest, we're not yet sure what we'll decide for the Python -> Rust 
bridge. The problem is summarized in the `Rust <=> Python Interop` section on 
https://www.mercurial-scm.org/wiki/OxidationPlan.
  
  I suspect at some level we'll need a CPython extension for CPython for 
performance reasons (especially for high volume function calls). PyPy obviously 
uses CFFI. I think the ideal outcome is we can write Rust that exposes a C API 
and use CFFI natively on PyPy and something like `cbindgen` + `Milksnake` to 
auto-generate a CPython extension that acts as a wrapper around the C API 
exposed by Rust. I'm not sure if anyone has invented this exact wheel yet. If 
not, it's probably faster to use `rust-cpython`. Maybe several months from now 
we have enough Rust and maintaining `rust-cpython` is painful enough that we 
pursue the auto-generated CPython extension route.
  
  What I'm trying to say is you have a green field to explore! But at this 
juncture, perfect is the enemy of done. We'll be happy with any forward 
progress, even failed experiments.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2057

To: Ivzhh, #hg-reviewers
Cc: krbullock, indygreg, durin42, kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2089: wireproto: introduce type for raw byte responses (API)

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5341.
indygreg edited the summary of this revision.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2089?vs=5325=5341

REVISION DETAIL
  https://phab.mercurial-scm.org/D2089

AFFECTED FILES
  hgext/largefiles/proto.py
  mercurial/wireproto.py
  mercurial/wireprotoserver.py
  mercurial/wireprototypes.py
  tests/sshprotoext.py
  tests/test-wireproto.py

CHANGE DETAILS

diff --git a/tests/test-wireproto.py b/tests/test-wireproto.py
--- a/tests/test-wireproto.py
+++ b/tests/test-wireproto.py
@@ -1,8 +1,10 @@
 from __future__ import absolute_import, print_function
 
 from mercurial import (
+error,
 util,
 wireproto,
+wireprototypes,
 )
 stringio = util.stringio
 
@@ -42,7 +44,13 @@
 return ['batch']
 
 def _call(self, cmd, **args):
-return wireproto.dispatch(self.serverrepo, proto(args), cmd)
+res = wireproto.dispatch(self.serverrepo, proto(args), cmd)
+if isinstance(res, wireprototypes.bytesresponse):
+return res.data
+elif isinstance(res, bytes):
+return res
+else:
+raise error.Abort('dummy client does not support response type')
 
 def _callstream(self, cmd, **args):
 return stringio(self._call(cmd, **args))
diff --git a/tests/sshprotoext.py b/tests/sshprotoext.py
--- a/tests/sshprotoext.py
+++ b/tests/sshprotoext.py
@@ -49,7 +49,7 @@
 l = self._fin.readline()
 assert l == b'between\n'
 rsp = wireproto.dispatch(self._repo, self._proto, b'between')
-wireprotoserver._sshv1respondbytes(self._fout, rsp)
+wireprotoserver._sshv1respondbytes(self._fout, rsp.data)
 
 super(prehelloserver, self).serve_forever()
 
@@ -74,7 +74,7 @@
 # Send the upgrade response.
 self._fout.write(b'upgraded %s %s\n' % (token, name))
 servercaps = wireproto.capabilities(self._repo, self._proto)
-rsp = b'capabilities: %s' % servercaps
+rsp = b'capabilities: %s' % servercaps.data
 self._fout.write(b'%d\n' % len(rsp))
 self._fout.write(rsp)
 self._fout.write(b'\n')
diff --git a/mercurial/wireprototypes.py b/mercurial/wireprototypes.py
--- a/mercurial/wireprototypes.py
+++ b/mercurial/wireprototypes.py
@@ -5,6 +5,11 @@
 
 from __future__ import absolute_import
 
+class bytesresponse(object):
+"""A wire protocol response consisting of raw bytes."""
+def __init__(self, data):
+self.data = data
+
 class ooberror(object):
 """wireproto reply: failure of a batch of operation
 
diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -274,6 +274,9 @@
 if isinstance(rsp, bytes):
 req.respond(HTTP_OK, HGTYPE, body=rsp)
 return []
+elif isinstance(rsp, wireprototypes.bytesresponse):
+req.respond(HTTP_OK, HGTYPE, body=rsp.data)
+return []
 elif isinstance(rsp, wireprototypes.streamreslegacy):
 gen = rsp.gen
 req.respond(HTTP_OK, HGTYPE)
@@ -435,6 +438,8 @@
 
 if isinstance(rsp, bytes):
 _sshv1respondbytes(self._fout, rsp)
+elif isinstance(rsp, wireprototypes.bytesresponse):
+_sshv1respondbytes(self._fout, rsp.data)
 elif isinstance(rsp, wireprototypes.streamres):
 _sshv1respondstream(self._fout, rsp)
 elif isinstance(rsp, wireprototypes.streamreslegacy):
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -37,6 +37,7 @@
 urlerr = util.urlerr
 urlreq = util.urlreq
 
+bytesresponse = wireprototypes.bytesresponse
 ooberror = wireprototypes.ooberror
 pushres = wireprototypes.pushres
 pusherr = wireprototypes.pusherr
@@ -696,16 +697,24 @@
 result = func(repo, proto)
 if isinstance(result, ooberror):
 return result
+
+# For now, all batchable commands must return bytesresponse or
+# raw bytes (for backwards compatibility).
+assert isinstance(result, (bytesresponse, bytes))
+if isinstance(result, bytesresponse):
+result = result.data
 res.append(escapearg(result))
-return ';'.join(res)
+
+return bytesresponse(';'.join(res))
 
 @wireprotocommand('between', 'pairs')
 def between(repo, proto, pairs):
 pairs = [decodelist(p, '-') for p in pairs.split(" ")]
 r = []
 for b in repo.between(pairs):
 r.append(encodelist(b) + "\n")
-return "".join(r)
+
+return bytesresponse(''.join(r))
 
 @wireprotocommand('branchmap')
 def branchmap(repo, proto):
@@ -715,15 +724,17 @@
 branchname = urlreq.quote(encoding.fromlocal(branch))
 branchnodes = encodelist(nodes)
 heads.append('%s %s' % (branchname, branchnodes))
-return '\n'.join(heads)
+
+return bytesresponse('\n'.join(heads))
 
 

D2086: wireproto: remove unused proto argument from supportedcompengines (API)

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5338.
indygreg retitled this revision from "wireproto: remove unused proto argument 
from supportedcompengines" to "wireproto: remove unused proto argument from 
supportedcompengines (API)".

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2086?vs=5320=5338

REVISION DETAIL
  https://phab.mercurial-scm.org/D2086

AFFECTED FILES
  mercurial/wireproto.py
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -192,7 +192,7 @@
 break
 
 # Now find an agreed upon compression format.
-for engine in wireproto.supportedcompengines(self._ui, self,
+for engine in wireproto.supportedcompengines(self._ui,
  util.SERVERROLE):
 if engine.wireprotosupport().name in compformats:
 opts = {}
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -587,7 +587,7 @@
 
 return ui.configbool('server', 'bundle1')
 
-def supportedcompengines(ui, proto, role):
+def supportedcompengines(ui, role):
 """Obtain the list of supported compression engines for a request."""
 assert role in (util.CLIENTROLE, util.SERVERROLE)
 
@@ -824,7 +824,7 @@
 # FUTURE advertise minrx and mintx after consulting config option
 caps.append('httpmediatype=0.1rx,0.1tx,0.2tx')
 
-compengines = supportedcompengines(repo.ui, proto, util.SERVERROLE)
+compengines = supportedcompengines(repo.ui, util.SERVERROLE)
 if compengines:
 comptypes = ','.join(urlreq.quote(e.wireprotosupport().name)
  for e in compengines)



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2082: wireproto: use maybecapturestdio() for push responses (API)

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5334.
indygreg edited the summary of this revision.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2082?vs=5316=5334

REVISION DETAIL
  https://phab.mercurial-scm.org/D2082

AFFECTED FILES
  hgext/largefiles/proto.py
  mercurial/wireproto.py
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -320,15 +320,13 @@
 req.respond(HTTP_OK, mediatype)
 return gen
 elif isinstance(rsp, wireproto.pushres):
-val = proto.restore()
-rsp = '%d\n%s' % (rsp.res, val)
+rsp = '%d\n%s' % (rsp.res, rsp.output)
 req.respond(HTTP_OK, HGTYPE, body=rsp)
 return []
 elif isinstance(rsp, wireproto.pusherr):
 # This is the httplib workaround documented in _handlehttperror().
 req.drain()
 
-proto.restore()
 rsp = '0\n%s\n' % rsp.res
 req.respond(HTTP_OK, HGTYPE, body=rsp)
 return []
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -510,16 +510,18 @@
 
 The call was successful and returned an integer contained in `self.res`.
 """
-def __init__(self, res):
+def __init__(self, res, output):
 self.res = res
+self.output = output
 
 class pusherr(object):
 """wireproto reply: failure
 
 The call failed. The `self.res` attribute contains the error message.
 """
-def __init__(self, res):
+def __init__(self, res, output):
 self.res = res
+self.output = output
 
 class ooberror(object):
 """wireproto reply: failure of a batch of operation
@@ -997,97 +999,98 @@
 def unbundle(repo, proto, heads):
 their_heads = decodelist(heads)
 
-try:
-proto.redirect()
-
-exchange.check_heads(repo, their_heads, 'preparing changes')
-
-# write bundle data to temporary file because it can be big
-fd, tempname = tempfile.mkstemp(prefix='hg-unbundle-')
-fp = os.fdopen(fd, pycompat.sysstr('wb+'))
-r = 0
+with proto.mayberedirectstdio() as output:
 try:
-proto.getfile(fp)
-fp.seek(0)
-gen = exchange.readbundle(repo.ui, fp, None)
-if (isinstance(gen, changegroupmod.cg1unpacker)
-and not bundle1allowed(repo, 'push')):
-if proto.name == 'http':
-# need to special case http because stderr do not get to
-# the http client on failed push so we need to abuse some
-# other error type to make sure the message get to the
-# user.
-return ooberror(bundle2required)
-raise error.Abort(bundle2requiredmain,
-  hint=bundle2requiredhint)
+exchange.check_heads(repo, their_heads, 'preparing changes')
 
-r = exchange.unbundle(repo, gen, their_heads, 'serve',
-  proto._client())
-if util.safehasattr(r, 'addpart'):
-# The return looks streamable, we are in the bundle2 case and
-# should return a stream.
-return streamres_legacy(gen=r.getchunks())
-return pushres(r)
-
-finally:
-fp.close()
-os.unlink(tempname)
-
-except (error.BundleValueError, error.Abort, error.PushRaced) as exc:
-# handle non-bundle2 case first
-if not getattr(exc, 'duringunbundle2', False):
+# write bundle data to temporary file because it can be big
+fd, tempname = tempfile.mkstemp(prefix='hg-unbundle-')
+fp = os.fdopen(fd, pycompat.sysstr('wb+'))
+r = 0
 try:
-raise
-except error.Abort:
-# The old code we moved used util.stderr directly.
-# We did not change it to minimise code change.
-# This need to be moved to something proper.
-# Feel free to do it.
-util.stderr.write("abort: %s\n" % exc)
-if exc.hint is not None:
-util.stderr.write("(%s)\n" % exc.hint)
-return pushres(0)
-except error.PushRaced:
-return pusherr(str(exc))
+proto.getfile(fp)
+fp.seek(0)
+gen = exchange.readbundle(repo.ui, fp, None)
+if (isinstance(gen, changegroupmod.cg1unpacker)
+and not bundle1allowed(repo, 'push')):
+if proto.name == 'http':
+# need to special case http because stderr do not get 
to
+# the http client on failed push so we need to abuse
+# some other error type to make 

D2087: wireprotoserver: move responsetype() out of http handler

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5339.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2087?vs=5321=5339

REVISION DETAIL
  https://phab.mercurial-scm.org/D2087

AFFECTED FILES
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -170,48 +170,6 @@
 urlreq.quote(self._req.env.get('REMOTE_HOST', '')),
 urlreq.quote(self._req.env.get('REMOTE_USER', '')))
 
-def responsetype(self, prefer_uncompressed):
-"""Determine the appropriate response type and compression settings.
-
-Returns a tuple of (mediatype, compengine, engineopts).
-"""
-# Determine the response media type and compression engine based
-# on the request parameters.
-protocaps = decodevaluefromheaders(self._req, r'X-HgProto').split(' ')
-
-if '0.2' in protocaps:
-# All clients are expected to support uncompressed data.
-if prefer_uncompressed:
-return HGTYPE2, util._noopengine(), {}
-
-# Default as defined by wire protocol spec.
-compformats = ['zlib', 'none']
-for cap in protocaps:
-if cap.startswith('comp='):
-compformats = cap[5:].split(',')
-break
-
-# Now find an agreed upon compression format.
-for engine in wireproto.supportedcompengines(self._ui,
- util.SERVERROLE):
-if engine.wireprotosupport().name in compformats:
-opts = {}
-level = self._ui.configint('server',
-  '%slevel' % engine.name())
-if level is not None:
-opts['level'] = level
-
-return HGTYPE2, engine, opts
-
-# No mutually supported compression format. Fall back to the
-# legacy protocol.
-
-# Don't allow untrusted settings because disabling compression or
-# setting a very high compression level could lead to flooding
-# the server's network or CPU.
-opts = {'level': self._ui.configint('server', 'zliblevel')}
-return HGTYPE, util.compengines['zlib'], opts
-
 def iscmd(cmd):
 return cmd in wireproto.commands
 
@@ -252,6 +210,46 @@
 'handleerror': lambda ex: _handlehttperror(ex, req, cmd),
 }
 
+def _httpresponsetype(ui, req, prefer_uncompressed):
+"""Determine the appropriate response type and compression settings.
+
+Returns a tuple of (mediatype, compengine, engineopts).
+"""
+# Determine the response media type and compression engine based
+# on the request parameters.
+protocaps = decodevaluefromheaders(req, r'X-HgProto').split(' ')
+
+if '0.2' in protocaps:
+# All clients are expected to support uncompressed data.
+if prefer_uncompressed:
+return HGTYPE2, util._noopengine(), {}
+
+# Default as defined by wire protocol spec.
+compformats = ['zlib', 'none']
+for cap in protocaps:
+if cap.startswith('comp='):
+compformats = cap[5:].split(',')
+break
+
+# Now find an agreed upon compression format.
+for engine in wireproto.supportedcompengines(ui, util.SERVERROLE):
+if engine.wireprotosupport().name in compformats:
+opts = {}
+level = ui.configint('server', '%slevel' % engine.name())
+if level is not None:
+opts['level'] = level
+
+return HGTYPE2, engine, opts
+
+# No mutually supported compression format. Fall back to the
+# legacy protocol.
+
+# Don't allow untrusted settings because disabling compression or
+# setting a very high compression level could lead to flooding
+# the server's network or CPU.
+opts = {'level': ui.configint('server', 'zliblevel')}
+return HGTYPE, util.compengines['zlib'], opts
+
 def _callhttp(repo, req, proto, cmd):
 def genversion2(gen, engine, engineopts):
 # application/mercurial-0.2 always sends a payload header
@@ -284,8 +282,8 @@
 
 # This code for compression should not be streamres specific. It
 # is here because we only compress streamres at the moment.
-mediatype, engine, engineopts = proto.responsetype(
-rsp.prefer_uncompressed)
+mediatype, engine, engineopts = _httpresponsetype(
+repo.ui, req, rsp.prefer_uncompressed)
 gen = engine.compressstream(gen, engineopts)
 
 if mediatype == HGTYPE2:



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org

D2091: wireprotoserver: extract SSH response handling functions

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The lookup/dispatch table was cute. But it isn't needed. Future
  refactors will benefit from the handlers for individual response
  types living outside the class.
  
  As part of this, I snuck in a change that changes a type compare
  from str to bytes. This has no effect on Python 2. But it might
  make Python 3 a bit happier.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2091

AFFECTED FILES
  mercurial/wireprotoserver.py
  tests/sshprotoext.py

CHANGE DETAILS

diff --git a/tests/sshprotoext.py b/tests/sshprotoext.py
--- a/tests/sshprotoext.py
+++ b/tests/sshprotoext.py
@@ -45,11 +45,11 @@
 l = self._fin.readline()
 assert l == b'hello\n'
 # Respond to unknown commands with an empty reply.
-self._sendresponse(b'')
+wireprotoserver._sshv1respondbytes(self._fout, b'')
 l = self._fin.readline()
 assert l == b'between\n'
 rsp = wireproto.dispatch(self._repo, self, b'between')
-self._handlers[rsp.__class__](self, rsp)
+wireprotoserver._sshv1respondbytes(self._fout, rsp)
 
 super(prehelloserver, self).serve_forever()
 
diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -336,6 +336,24 @@
 
 return ''
 
+def _sshv1respondbytes(fout, value):
+"""Send a bytes response for protocol version 1."""
+fout.write('%d\n' % len(value))
+fout.write(value)
+fout.flush()
+
+def _sshv1respondstream(fout, source):
+write = fout.write
+for chunk in source.gen:
+write(chunk)
+fout.flush()
+
+def _sshv1sendooberror(fout, ferr, rsp):
+ferr.write(b'%s\n-\n' % rsp.message)
+ferr.flush()
+fout.write(b'\n')
+fout.flush()
+
 class sshserver(baseprotocolhandler):
 def __init__(self, ui, repo):
 self._ui = ui
@@ -376,60 +394,43 @@
 return [data[k] for k in keys]
 
 def getfile(self, fpout):
-self._sendresponse('')
+_sshv1respondbytes(self._fout, b'')
 count = int(self._fin.readline())
 while count:
 fpout.write(self._fin.read(count))
 count = int(self._fin.readline())
 
 def redirect(self):
 pass
 
-def _sendresponse(self, v):
-self._fout.write("%d\n" % len(v))
-self._fout.write(v)
-self._fout.flush()
-
-def _sendstream(self, source):
-write = self._fout.write
-for chunk in source.gen:
-write(chunk)
-self._fout.flush()
-
-def _sendpushresponse(self, rsp):
-self._sendresponse('')
-self._sendresponse(str(rsp.res))
-
-def _sendpusherror(self, rsp):
-self._sendresponse(rsp.res)
-
-def _sendooberror(self, rsp):
-self._ui.ferr.write('%s\n-\n' % rsp.message)
-self._ui.ferr.flush()
-self._fout.write('\n')
-self._fout.flush()
-
 def serve_forever(self):
 while self.serve_one():
 pass
 sys.exit(0)
 
-_handlers = {
-str: _sendresponse,
-wireproto.streamres: _sendstream,
-wireproto.streamres_legacy: _sendstream,
-wireproto.pushres: _sendpushresponse,
-wireproto.pusherr: _sendpusherror,
-wireproto.ooberror: _sendooberror,
-}
-
 def serve_one(self):
 cmd = self._fin.readline()[:-1]
 if cmd and wireproto.commands.commandavailable(cmd, self):
 rsp = wireproto.dispatch(self._repo, self, cmd)
-self._handlers[rsp.__class__](self, rsp)
+
+if isinstance(rsp, bytes):
+_sshv1respondbytes(self._fout, rsp)
+elif isinstance(rsp, wireproto.streamres):
+_sshv1respondstream(self._fout, rsp)
+elif isinstance(rsp, wireproto.streamres_legacy):
+_sshv1respondstream(self._fout, rsp)
+elif isinstance(rsp, wireproto.pushres):
+_sshv1respondbytes(self._fout, b'')
+_sshv1respondbytes(self._fout, bytes(rsp.res))
+elif isinstance(rsp, wireproto.pusherr):
+_sshv1respondbytes(self._fout, rsp.res)
+elif isinstance(rsp, wireproto.ooberror):
+_sshv1sendooberror(self._fout, self._ui.ferr, rsp)
+else:
+raise error.ProgrammingError('unhandled response type from '
+ 'wire protocol command: %s' % rsp)
 elif cmd:
-self._sendresponse("")
+_sshv1respondbytes(self._fout, b'')
 return cmd != ''
 
 def _client(self):



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2080: wireprotoserver: split ssh protocol handler and server

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5332.
indygreg edited the summary of this revision.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2080?vs=5314=5332

REVISION DETAIL
  https://phab.mercurial-scm.org/D2080

AFFECTED FILES
  mercurial/wireprotoserver.py
  tests/sshprotoext.py
  tests/test-sshserver.py

CHANGE DETAILS

diff --git a/tests/test-sshserver.py b/tests/test-sshserver.py
--- a/tests/test-sshserver.py
+++ b/tests/test-sshserver.py
@@ -24,7 +24,7 @@
 def assertparse(self, cmd, input, expected):
 server = mockserver(input)
 _func, spec = wireproto.commands[cmd]
-self.assertEqual(server.getargs(spec), expected)
+self.assertEqual(server._proto.getargs(spec), expected)
 
 def mockserver(inbytes):
 ui = mockui(inbytes)
diff --git a/tests/sshprotoext.py b/tests/sshprotoext.py
--- a/tests/sshprotoext.py
+++ b/tests/sshprotoext.py
@@ -48,7 +48,7 @@
 wireprotoserver._sshv1respondbytes(self._fout, b'')
 l = self._fin.readline()
 assert l == b'between\n'
-rsp = wireproto.dispatch(self._repo, self, b'between')
+rsp = wireproto.dispatch(self._repo, self._proto, b'between')
 wireprotoserver._sshv1respondbytes(self._fout, rsp)
 
 super(prehelloserver, self).serve_forever()
@@ -73,7 +73,7 @@
 
 # Send the upgrade response.
 self._fout.write(b'upgraded %s %s\n' % (token, name))
-servercaps = wireproto.capabilities(self._repo, self)
+servercaps = wireproto.capabilities(self._repo, self._proto)
 rsp = b'capabilities: %s' % servercaps
 self._fout.write(b'%d\n' % len(rsp))
 self._fout.write(rsp)
diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -354,19 +354,12 @@
 fout.write(b'\n')
 fout.flush()
 
-class sshserver(baseprotocolhandler):
-def __init__(self, ui, repo):
+class sshv1protocolhandler(baseprotocolhandler):
+"""Handler for requests services via version 1 of SSH protocol."""
+def __init__(self, ui, fin, fout):
 self._ui = ui
-self._repo = repo
-self._fin = ui.fin
-self._fout = ui.fout
-
-hook.redirect(True)
-ui.fout = repo.ui.fout = ui.ferr
-
-# Prevent insertion/deletion of CRs
-util.setbinary(self._fin)
-util.setbinary(self._fout)
+self._fin = fin
+self._fout = fout
 
 @property
 def name(self):
@@ -403,15 +396,35 @@
 def redirect(self):
 pass
 
+def _client(self):
+client = encoding.environ.get('SSH_CLIENT', '').split(' ', 1)[0]
+return 'remote:ssh:' + client
+
+class sshserver(object):
+def __init__(self, ui, repo):
+self._ui = ui
+self._repo = repo
+self._fin = ui.fin
+self._fout = ui.fout
+
+hook.redirect(True)
+ui.fout = repo.ui.fout = ui.ferr
+
+# Prevent insertion/deletion of CRs
+util.setbinary(self._fin)
+util.setbinary(self._fout)
+
+self._proto = sshv1protocolhandler(self._ui, self._fin, self._fout)
+
 def serve_forever(self):
 while self.serve_one():
 pass
 sys.exit(0)
 
 def serve_one(self):
 cmd = self._fin.readline()[:-1]
-if cmd and wireproto.commands.commandavailable(cmd, self):
-rsp = wireproto.dispatch(self._repo, self, cmd)
+if cmd and wireproto.commands.commandavailable(cmd, self._proto):
+rsp = wireproto.dispatch(self._repo, self._proto, cmd)
 
 if isinstance(rsp, bytes):
 _sshv1respondbytes(self._fout, rsp)
@@ -432,7 +445,3 @@
 elif cmd:
 _sshv1respondbytes(self._fout, b'')
 return cmd != ''
-
-def _client(self):
-client = encoding.environ.get('SSH_CLIENT', '').split(' ', 1)[0]
-return 'remote:ssh:' + client



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2084: wireprotoserver: rename _client to client (API)

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5336.
indygreg edited the summary of this revision.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2084?vs=5318=5336

REVISION DETAIL
  https://phab.mercurial-scm.org/D2084

AFFECTED FILES
  mercurial/wireproto.py
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -88,6 +88,10 @@
 won't be captured.
 """
 
+@abc.abstractmethod
+def client(self):
+"""Returns a string representation of this client (as bytes)."""
+
 def decodevaluefromheaders(req, headerprefix):
 """Decode a long value from multiple HTTP request headers.
 
@@ -164,7 +168,7 @@
 self._ui.fout = oldout
 self._ui.ferr = olderr
 
-def _client(self):
+def client(self):
 return 'remote:%s:%s:%s' % (
 self._req.env.get('wsgi.url_scheme') or 'http',
 urlreq.quote(self._req.env.get('REMOTE_HOST', '')),
@@ -399,7 +403,7 @@
 def mayberedirectstdio(self):
 yield None
 
-def _client(self):
+def client(self):
 client = encoding.environ.get('SSH_CLIENT', '').split(' ', 1)[0]
 return 'remote:ssh:' + client
 
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -1023,7 +1023,7 @@
   hint=bundle2requiredhint)
 
 r = exchange.unbundle(repo, gen, their_heads, 'serve',
-  proto._client())
+  proto.client())
 if util.safehasattr(r, 'addpart'):
 # The return looks streamable, we are in the bundle2 case
 # and should return a stream.



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2085: wireprotoserver: rename getfile() to forwardpayload() (API)

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5337.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2085?vs=5319=5337

REVISION DETAIL
  https://phab.mercurial-scm.org/D2085

AFFECTED FILES
  hgext/largefiles/proto.py
  mercurial/wireproto.py
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -64,14 +64,10 @@
 returns a list of values (same order as )"""
 
 @abc.abstractmethod
-def getfile(self, fp):
-"""write the whole content of a file into a file like object
+def forwardpayload(self, fp):
+"""Read the raw payload and forward to a file.
 
-The file is in the form::
-
-(\n)+0\n
-
-chunk size is the ascii version of the int.
+The payload is read in full before the function returns.
 """
 
 @abc.abstractmethod
@@ -145,7 +141,7 @@
 args.update(cgi.parse_qs(argvalue, keep_blank_values=True))
 return args
 
-def getfile(self, fp):
+def forwardpayload(self, fp):
 length = int(self._req.env[r'CONTENT_LENGTH'])
 # If httppostargs is used, we need to read Content-Length
 # minus the amount that was consumed by args.
@@ -392,7 +388,12 @@
 data[arg] = val
 return [data[k] for k in keys]
 
-def getfile(self, fpout):
+def forwardpayload(self, fpout):
+# The file is in the form:
+#
+# \n
+# ...
+# 0\n
 _sshv1respondbytes(self._fout, b'')
 count = int(self._fin.readline())
 while count:
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -1008,7 +1008,7 @@
 fp = os.fdopen(fd, pycompat.sysstr('wb+'))
 r = 0
 try:
-proto.getfile(fp)
+proto.forwardpayload(fp)
 fp.seek(0)
 gen = exchange.readbundle(repo.ui, fp, None)
 if (isinstance(gen, changegroupmod.cg1unpacker)
diff --git a/hgext/largefiles/proto.py b/hgext/largefiles/proto.py
--- a/hgext/largefiles/proto.py
+++ b/hgext/largefiles/proto.py
@@ -40,7 +40,7 @@
 tmpfp = util.atomictempfile(path, createmode=repo.store.createmode)
 
 try:
-proto.getfile(tmpfp)
+proto.forwardpayload(tmpfp)
 tmpfp._fp.seek(0)
 if sha != lfutil.hexsha1(tmpfp._fp):
 raise IOError(0, _('largefile contents do not match hash'))



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2088: wireprototypes: move wire protocol response types to new module

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5340.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2088?vs=5324=5340

REVISION DETAIL
  https://phab.mercurial-scm.org/D2088

AFFECTED FILES
  mercurial/wireproto.py
  mercurial/wireprotoserver.py
  mercurial/wireprototypes.py

CHANGE DETAILS

diff --git a/mercurial/wireprototypes.py b/mercurial/wireprototypes.py
new file mode 100644
--- /dev/null
+++ b/mercurial/wireprototypes.py
@@ -0,0 +1,61 @@
+# Copyright 2018 Gregory Szorc 
+#
+# This software may be used and distributed according to the terms of the
+# GNU General Public License version 2 or any later version.
+
+from __future__ import absolute_import
+
+class ooberror(object):
+"""wireproto reply: failure of a batch of operation
+
+Something failed during a batch call. The error message is stored in
+`self.message`.
+"""
+def __init__(self, message):
+self.message = message
+
+class pushres(object):
+"""wireproto reply: success with simple integer return
+
+The call was successful and returned an integer contained in `self.res`.
+"""
+def __init__(self, res, output):
+self.res = res
+self.output = output
+
+class pusherr(object):
+"""wireproto reply: failure
+
+The call failed. The `self.res` attribute contains the error message.
+"""
+def __init__(self, res, output):
+self.res = res
+self.output = output
+
+class streamres(object):
+"""wireproto reply: binary stream
+
+The call was successful and the result is a stream.
+
+Accepts a generator containing chunks of data to be sent to the client.
+
+``prefer_uncompressed`` indicates that the data is expected to be
+uncompressable and that the stream should therefore use the ``none``
+engine.
+"""
+def __init__(self, gen=None, prefer_uncompressed=False):
+self.gen = gen
+self.prefer_uncompressed = prefer_uncompressed
+
+class streamreslegacy(object):
+"""wireproto reply: uncompressed binary stream
+
+The call was successful and the result is a stream.
+
+Accepts a generator containing chunks of data to be sent to the client.
+
+Like ``streamres``, but sends an uncompressed data for "version 1" clients
+using the application/mercurial-0.1 media type.
+"""
+def __init__(self, gen=None):
+self.gen = gen
diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -20,6 +20,7 @@
 pycompat,
 util,
 wireproto,
+wireprototypes,
 )
 
 stringio = util.stringio
@@ -273,11 +274,11 @@
 if isinstance(rsp, bytes):
 req.respond(HTTP_OK, HGTYPE, body=rsp)
 return []
-elif isinstance(rsp, wireproto.streamres_legacy):
+elif isinstance(rsp, wireprototypes.streamreslegacy):
 gen = rsp.gen
 req.respond(HTTP_OK, HGTYPE)
 return gen
-elif isinstance(rsp, wireproto.streamres):
+elif isinstance(rsp, wireprototypes.streamres):
 gen = rsp.gen
 
 # This code for compression should not be streamres specific. It
@@ -291,18 +292,18 @@
 
 req.respond(HTTP_OK, mediatype)
 return gen
-elif isinstance(rsp, wireproto.pushres):
+elif isinstance(rsp, wireprototypes.pushres):
 rsp = '%d\n%s' % (rsp.res, rsp.output)
 req.respond(HTTP_OK, HGTYPE, body=rsp)
 return []
-elif isinstance(rsp, wireproto.pusherr):
+elif isinstance(rsp, wireprototypes.pusherr):
 # This is the httplib workaround documented in _handlehttperror().
 req.drain()
 
 rsp = '0\n%s\n' % rsp.res
 req.respond(HTTP_OK, HGTYPE, body=rsp)
 return []
-elif isinstance(rsp, wireproto.ooberror):
+elif isinstance(rsp, wireprototypes.ooberror):
 rsp = rsp.message
 req.respond(HTTP_OK, HGERRTYPE, body=rsp)
 return []
@@ -434,16 +435,16 @@
 
 if isinstance(rsp, bytes):
 _sshv1respondbytes(self._fout, rsp)
-elif isinstance(rsp, wireproto.streamres):
+elif isinstance(rsp, wireprototypes.streamres):
 _sshv1respondstream(self._fout, rsp)
-elif isinstance(rsp, wireproto.streamres_legacy):
+elif isinstance(rsp, wireprototypes.streamreslegacy):
 _sshv1respondstream(self._fout, rsp)
-elif isinstance(rsp, wireproto.pushres):
+elif isinstance(rsp, wireprototypes.pushres):
 _sshv1respondbytes(self._fout, b'')
 _sshv1respondbytes(self._fout, bytes(rsp.res))
-elif isinstance(rsp, wireproto.pusherr):
+elif isinstance(rsp, wireprototypes.pusherr):
 _sshv1respondbytes(self._fout, rsp.res)
-elif isinstance(rsp, wireproto.ooberror):
+elif isinstance(rsp, wireprototypes.ooberror):
 

D2082: wireproto: use maybecapturestdio() for push responses (API)

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5345.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2082?vs=5334=5345

REVISION DETAIL
  https://phab.mercurial-scm.org/D2082

AFFECTED FILES
  hgext/largefiles/proto.py
  mercurial/wireproto.py
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -320,15 +320,13 @@
 req.respond(HTTP_OK, mediatype)
 return gen
 elif isinstance(rsp, wireproto.pushres):
-val = proto.restore()
-rsp = '%d\n%s' % (rsp.res, val)
+rsp = '%d\n%s' % (rsp.res, rsp.output)
 req.respond(HTTP_OK, HGTYPE, body=rsp)
 return []
 elif isinstance(rsp, wireproto.pusherr):
 # This is the httplib workaround documented in _handlehttperror().
 req.drain()
 
-proto.restore()
 rsp = '0\n%s\n' % rsp.res
 req.respond(HTTP_OK, HGTYPE, body=rsp)
 return []
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -510,16 +510,18 @@
 
 The call was successful and returned an integer contained in `self.res`.
 """
-def __init__(self, res):
+def __init__(self, res, output):
 self.res = res
+self.output = output
 
 class pusherr(object):
 """wireproto reply: failure
 
 The call failed. The `self.res` attribute contains the error message.
 """
-def __init__(self, res):
+def __init__(self, res, output):
 self.res = res
+self.output = output
 
 class ooberror(object):
 """wireproto reply: failure of a batch of operation
@@ -997,97 +999,98 @@
 def unbundle(repo, proto, heads):
 their_heads = decodelist(heads)
 
-try:
-proto.redirect()
-
-exchange.check_heads(repo, their_heads, 'preparing changes')
-
-# write bundle data to temporary file because it can be big
-fd, tempname = tempfile.mkstemp(prefix='hg-unbundle-')
-fp = os.fdopen(fd, pycompat.sysstr('wb+'))
-r = 0
+with proto.mayberedirectstdio() as output:
 try:
-proto.getfile(fp)
-fp.seek(0)
-gen = exchange.readbundle(repo.ui, fp, None)
-if (isinstance(gen, changegroupmod.cg1unpacker)
-and not bundle1allowed(repo, 'push')):
-if proto.name == 'http':
-# need to special case http because stderr do not get to
-# the http client on failed push so we need to abuse some
-# other error type to make sure the message get to the
-# user.
-return ooberror(bundle2required)
-raise error.Abort(bundle2requiredmain,
-  hint=bundle2requiredhint)
+exchange.check_heads(repo, their_heads, 'preparing changes')
 
-r = exchange.unbundle(repo, gen, their_heads, 'serve',
-  proto._client())
-if util.safehasattr(r, 'addpart'):
-# The return looks streamable, we are in the bundle2 case and
-# should return a stream.
-return streamres_legacy(gen=r.getchunks())
-return pushres(r)
-
-finally:
-fp.close()
-os.unlink(tempname)
-
-except (error.BundleValueError, error.Abort, error.PushRaced) as exc:
-# handle non-bundle2 case first
-if not getattr(exc, 'duringunbundle2', False):
+# write bundle data to temporary file because it can be big
+fd, tempname = tempfile.mkstemp(prefix='hg-unbundle-')
+fp = os.fdopen(fd, pycompat.sysstr('wb+'))
+r = 0
 try:
-raise
-except error.Abort:
-# The old code we moved used util.stderr directly.
-# We did not change it to minimise code change.
-# This need to be moved to something proper.
-# Feel free to do it.
-util.stderr.write("abort: %s\n" % exc)
-if exc.hint is not None:
-util.stderr.write("(%s)\n" % exc.hint)
-return pushres(0)
-except error.PushRaced:
-return pusherr(str(exc))
+proto.getfile(fp)
+fp.seek(0)
+gen = exchange.readbundle(repo.ui, fp, None)
+if (isinstance(gen, changegroupmod.cg1unpacker)
+and not bundle1allowed(repo, 'push')):
+if proto.name == 'http':
+# need to special case http because stderr do not get 
to
+# the http client on failed push so we need to abuse
+# some other error type to make sure the message get to
+

D2086: wireproto: remove unused proto argument from supportedcompengines (API)

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5349.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2086?vs=5338=5349

REVISION DETAIL
  https://phab.mercurial-scm.org/D2086

AFFECTED FILES
  mercurial/wireproto.py
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -192,7 +192,7 @@
 break
 
 # Now find an agreed upon compression format.
-for engine in wireproto.supportedcompengines(self._ui, self,
+for engine in wireproto.supportedcompengines(self._ui,
  util.SERVERROLE):
 if engine.wireprotosupport().name in compformats:
 opts = {}
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -587,7 +587,7 @@
 
 return ui.configbool('server', 'bundle1')
 
-def supportedcompengines(ui, proto, role):
+def supportedcompengines(ui, role):
 """Obtain the list of supported compression engines for a request."""
 assert role in (util.CLIENTROLE, util.SERVERROLE)
 
@@ -824,7 +824,7 @@
 # FUTURE advertise minrx and mintx after consulting config option
 caps.append('httpmediatype=0.1rx,0.1tx,0.2tx')
 
-compengines = supportedcompengines(repo.ui, proto, util.SERVERROLE)
+compengines = supportedcompengines(repo.ui, util.SERVERROLE)
 if compengines:
 comptypes = ','.join(urlreq.quote(e.wireprotosupport().name)
  for e in compengines)



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2089: wireproto: introduce type for raw byte responses (API)

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5352.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2089?vs=5341=5352

REVISION DETAIL
  https://phab.mercurial-scm.org/D2089

AFFECTED FILES
  hgext/largefiles/proto.py
  mercurial/wireproto.py
  mercurial/wireprotoserver.py
  mercurial/wireprototypes.py
  tests/sshprotoext.py
  tests/test-wireproto.py

CHANGE DETAILS

diff --git a/tests/test-wireproto.py b/tests/test-wireproto.py
--- a/tests/test-wireproto.py
+++ b/tests/test-wireproto.py
@@ -1,8 +1,10 @@
 from __future__ import absolute_import, print_function
 
 from mercurial import (
+error,
 util,
 wireproto,
+wireprototypes,
 )
 stringio = util.stringio
 
@@ -42,7 +44,13 @@
 return ['batch']
 
 def _call(self, cmd, **args):
-return wireproto.dispatch(self.serverrepo, proto(args), cmd)
+res = wireproto.dispatch(self.serverrepo, proto(args), cmd)
+if isinstance(res, wireprototypes.bytesresponse):
+return res.data
+elif isinstance(res, bytes):
+return res
+else:
+raise error.Abort('dummy client does not support response type')
 
 def _callstream(self, cmd, **args):
 return stringio(self._call(cmd, **args))
diff --git a/tests/sshprotoext.py b/tests/sshprotoext.py
--- a/tests/sshprotoext.py
+++ b/tests/sshprotoext.py
@@ -49,7 +49,7 @@
 l = self._fin.readline()
 assert l == b'between\n'
 rsp = wireproto.dispatch(self._repo, self._proto, b'between')
-wireprotoserver._sshv1respondbytes(self._fout, rsp)
+wireprotoserver._sshv1respondbytes(self._fout, rsp.data)
 
 super(prehelloserver, self).serve_forever()
 
@@ -74,7 +74,7 @@
 # Send the upgrade response.
 self._fout.write(b'upgraded %s %s\n' % (token, name))
 servercaps = wireproto.capabilities(self._repo, self._proto)
-rsp = b'capabilities: %s' % servercaps
+rsp = b'capabilities: %s' % servercaps.data
 self._fout.write(b'%d\n' % len(rsp))
 self._fout.write(rsp)
 self._fout.write(b'\n')
diff --git a/mercurial/wireprototypes.py b/mercurial/wireprototypes.py
--- a/mercurial/wireprototypes.py
+++ b/mercurial/wireprototypes.py
@@ -5,6 +5,11 @@
 
 from __future__ import absolute_import
 
+class bytesresponse(object):
+"""A wire protocol response consisting of raw bytes."""
+def __init__(self, data):
+self.data = data
+
 class ooberror(object):
 """wireproto reply: failure of a batch of operation
 
diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -274,6 +274,9 @@
 if isinstance(rsp, bytes):
 req.respond(HTTP_OK, HGTYPE, body=rsp)
 return []
+elif isinstance(rsp, wireprototypes.bytesresponse):
+req.respond(HTTP_OK, HGTYPE, body=rsp.data)
+return []
 elif isinstance(rsp, wireprototypes.streamreslegacy):
 gen = rsp.gen
 req.respond(HTTP_OK, HGTYPE)
@@ -435,6 +438,8 @@
 
 if isinstance(rsp, bytes):
 _sshv1respondbytes(self._fout, rsp)
+elif isinstance(rsp, wireprototypes.bytesresponse):
+_sshv1respondbytes(self._fout, rsp.data)
 elif isinstance(rsp, wireprototypes.streamres):
 _sshv1respondstream(self._fout, rsp)
 elif isinstance(rsp, wireprototypes.streamreslegacy):
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -37,6 +37,7 @@
 urlerr = util.urlerr
 urlreq = util.urlreq
 
+bytesresponse = wireprototypes.bytesresponse
 ooberror = wireprototypes.ooberror
 pushres = wireprototypes.pushres
 pusherr = wireprototypes.pusherr
@@ -696,16 +697,24 @@
 result = func(repo, proto)
 if isinstance(result, ooberror):
 return result
+
+# For now, all batchable commands must return bytesresponse or
+# raw bytes (for backwards compatibility).
+assert isinstance(result, (bytesresponse, bytes))
+if isinstance(result, bytesresponse):
+result = result.data
 res.append(escapearg(result))
-return ';'.join(res)
+
+return bytesresponse(';'.join(res))
 
 @wireprotocommand('between', 'pairs')
 def between(repo, proto, pairs):
 pairs = [decodelist(p, '-') for p in pairs.split(" ")]
 r = []
 for b in repo.between(pairs):
 r.append(encodelist(b) + "\n")
-return "".join(r)
+
+return bytesresponse(''.join(r))
 
 @wireprotocommand('branchmap')
 def branchmap(repo, proto):
@@ -715,15 +724,17 @@
 branchname = urlreq.quote(encoding.fromlocal(branch))
 branchnodes = encodelist(nodes)
 heads.append('%s %s' % (branchname, branchnodes))
-return '\n'.join(heads)
+
+return bytesresponse('\n'.join(heads))
 
 @wireprotocommand('branches', 'nodes')
 def 

D2084: wireprotoserver: rename _client to client (API)

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5347.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2084?vs=5336=5347

REVISION DETAIL
  https://phab.mercurial-scm.org/D2084

AFFECTED FILES
  mercurial/wireproto.py
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -88,6 +88,10 @@
 won't be captured.
 """
 
+@abc.abstractmethod
+def client(self):
+"""Returns a string representation of this client (as bytes)."""
+
 def decodevaluefromheaders(req, headerprefix):
 """Decode a long value from multiple HTTP request headers.
 
@@ -164,7 +168,7 @@
 self._ui.fout = oldout
 self._ui.ferr = olderr
 
-def _client(self):
+def client(self):
 return 'remote:%s:%s:%s' % (
 self._req.env.get('wsgi.url_scheme') or 'http',
 urlreq.quote(self._req.env.get('REMOTE_HOST', '')),
@@ -399,7 +403,7 @@
 def mayberedirectstdio(self):
 yield None
 
-def _client(self):
+def client(self):
 client = encoding.environ.get('SSH_CLIENT', '').split(' ', 1)[0]
 return 'remote:ssh:' + client
 
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -1023,7 +1023,7 @@
   hint=bundle2requiredhint)
 
 r = exchange.unbundle(repo, gen, their_heads, 'serve',
-  proto._client())
+  proto.client())
 if util.safehasattr(r, 'addpart'):
 # The return looks streamable, we are in the bundle2 case
 # and should return a stream.



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2093: tests: add tests for sending recognized command before handshake

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Rounding out our test coverage for the SSH server.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2093

AFFECTED FILES
  tests/test-ssh-proto.t

CHANGE DETAILS

diff --git a/tests/test-ssh-proto.t b/tests/test-ssh-proto.t
--- a/tests/test-ssh-proto.t
+++ b/tests/test-ssh-proto.t
@@ -389,6 +389,33 @@
   0
   0
 
+Send a valid command before the handshake
+
+  $ hg -R server serve --stdio << EOF
+  > heads
+  > hello
+  > between
+  > pairs 81
+  > 
-
+  > EOF
+  41
+  68986213bd4485ea51533535e3fc9e78007a711f
+  384
+  capabilities: lookup changegroupsubset branchmap pushkey known getbundle 
unbundlehash batch streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ 
unbundle=HG10GZ,HG10BZ,HG10UN
+  1
+  
+
+And a variation that doesn't send the between command
+
+  $ hg -R server serve --stdio << EOF
+  > heads
+  > hello
+  > EOF
+  41
+  68986213bd4485ea51533535e3fc9e78007a711f
+  384
+  capabilities: lookup changegroupsubset branchmap pushkey known getbundle 
unbundlehash batch streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ 
unbundle=HG10GZ,HG10BZ,HG10UN
+
 Send an upgrade request to a server that doesn't support that command
 
   $ hg -R server serve --stdio << EOF



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2087: wireprotoserver: move responsetype() out of http handler

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5350.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2087?vs=5339=5350

REVISION DETAIL
  https://phab.mercurial-scm.org/D2087

AFFECTED FILES
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -170,48 +170,6 @@
 urlreq.quote(self._req.env.get('REMOTE_HOST', '')),
 urlreq.quote(self._req.env.get('REMOTE_USER', '')))
 
-def responsetype(self, prefer_uncompressed):
-"""Determine the appropriate response type and compression settings.
-
-Returns a tuple of (mediatype, compengine, engineopts).
-"""
-# Determine the response media type and compression engine based
-# on the request parameters.
-protocaps = decodevaluefromheaders(self._req, r'X-HgProto').split(' ')
-
-if '0.2' in protocaps:
-# All clients are expected to support uncompressed data.
-if prefer_uncompressed:
-return HGTYPE2, util._noopengine(), {}
-
-# Default as defined by wire protocol spec.
-compformats = ['zlib', 'none']
-for cap in protocaps:
-if cap.startswith('comp='):
-compformats = cap[5:].split(',')
-break
-
-# Now find an agreed upon compression format.
-for engine in wireproto.supportedcompengines(self._ui,
- util.SERVERROLE):
-if engine.wireprotosupport().name in compformats:
-opts = {}
-level = self._ui.configint('server',
-  '%slevel' % engine.name())
-if level is not None:
-opts['level'] = level
-
-return HGTYPE2, engine, opts
-
-# No mutually supported compression format. Fall back to the
-# legacy protocol.
-
-# Don't allow untrusted settings because disabling compression or
-# setting a very high compression level could lead to flooding
-# the server's network or CPU.
-opts = {'level': self._ui.configint('server', 'zliblevel')}
-return HGTYPE, util.compengines['zlib'], opts
-
 def iscmd(cmd):
 return cmd in wireproto.commands
 
@@ -252,6 +210,46 @@
 'handleerror': lambda ex: _handlehttperror(ex, req, cmd),
 }
 
+def _httpresponsetype(ui, req, prefer_uncompressed):
+"""Determine the appropriate response type and compression settings.
+
+Returns a tuple of (mediatype, compengine, engineopts).
+"""
+# Determine the response media type and compression engine based
+# on the request parameters.
+protocaps = decodevaluefromheaders(req, r'X-HgProto').split(' ')
+
+if '0.2' in protocaps:
+# All clients are expected to support uncompressed data.
+if prefer_uncompressed:
+return HGTYPE2, util._noopengine(), {}
+
+# Default as defined by wire protocol spec.
+compformats = ['zlib', 'none']
+for cap in protocaps:
+if cap.startswith('comp='):
+compformats = cap[5:].split(',')
+break
+
+# Now find an agreed upon compression format.
+for engine in wireproto.supportedcompengines(ui, util.SERVERROLE):
+if engine.wireprotosupport().name in compformats:
+opts = {}
+level = ui.configint('server', '%slevel' % engine.name())
+if level is not None:
+opts['level'] = level
+
+return HGTYPE2, engine, opts
+
+# No mutually supported compression format. Fall back to the
+# legacy protocol.
+
+# Don't allow untrusted settings because disabling compression or
+# setting a very high compression level could lead to flooding
+# the server's network or CPU.
+opts = {'level': ui.configint('server', 'zliblevel')}
+return HGTYPE, util.compengines['zlib'], opts
+
 def _callhttp(repo, req, proto, cmd):
 def genversion2(gen, engine, engineopts):
 # application/mercurial-0.2 always sends a payload header
@@ -284,8 +282,8 @@
 
 # This code for compression should not be streamres specific. It
 # is here because we only compress streamres at the moment.
-mediatype, engine, engineopts = proto.responsetype(
-rsp.prefer_uncompressed)
+mediatype, engine, engineopts = _httpresponsetype(
+repo.ui, req, rsp.prefer_uncompressed)
 gen = engine.compressstream(gen, engineopts)
 
 if mediatype == HGTYPE2:



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org

D2088: wireprototypes: move wire protocol response types to new module

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5351.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2088?vs=5340=5351

REVISION DETAIL
  https://phab.mercurial-scm.org/D2088

AFFECTED FILES
  mercurial/wireproto.py
  mercurial/wireprotoserver.py
  mercurial/wireprototypes.py

CHANGE DETAILS

diff --git a/mercurial/wireprototypes.py b/mercurial/wireprototypes.py
new file mode 100644
--- /dev/null
+++ b/mercurial/wireprototypes.py
@@ -0,0 +1,61 @@
+# Copyright 2018 Gregory Szorc 
+#
+# This software may be used and distributed according to the terms of the
+# GNU General Public License version 2 or any later version.
+
+from __future__ import absolute_import
+
+class ooberror(object):
+"""wireproto reply: failure of a batch of operation
+
+Something failed during a batch call. The error message is stored in
+`self.message`.
+"""
+def __init__(self, message):
+self.message = message
+
+class pushres(object):
+"""wireproto reply: success with simple integer return
+
+The call was successful and returned an integer contained in `self.res`.
+"""
+def __init__(self, res, output):
+self.res = res
+self.output = output
+
+class pusherr(object):
+"""wireproto reply: failure
+
+The call failed. The `self.res` attribute contains the error message.
+"""
+def __init__(self, res, output):
+self.res = res
+self.output = output
+
+class streamres(object):
+"""wireproto reply: binary stream
+
+The call was successful and the result is a stream.
+
+Accepts a generator containing chunks of data to be sent to the client.
+
+``prefer_uncompressed`` indicates that the data is expected to be
+uncompressable and that the stream should therefore use the ``none``
+engine.
+"""
+def __init__(self, gen=None, prefer_uncompressed=False):
+self.gen = gen
+self.prefer_uncompressed = prefer_uncompressed
+
+class streamreslegacy(object):
+"""wireproto reply: uncompressed binary stream
+
+The call was successful and the result is a stream.
+
+Accepts a generator containing chunks of data to be sent to the client.
+
+Like ``streamres``, but sends an uncompressed data for "version 1" clients
+using the application/mercurial-0.1 media type.
+"""
+def __init__(self, gen=None):
+self.gen = gen
diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -20,6 +20,7 @@
 pycompat,
 util,
 wireproto,
+wireprototypes,
 )
 
 stringio = util.stringio
@@ -273,11 +274,11 @@
 if isinstance(rsp, bytes):
 req.respond(HTTP_OK, HGTYPE, body=rsp)
 return []
-elif isinstance(rsp, wireproto.streamres_legacy):
+elif isinstance(rsp, wireprototypes.streamreslegacy):
 gen = rsp.gen
 req.respond(HTTP_OK, HGTYPE)
 return gen
-elif isinstance(rsp, wireproto.streamres):
+elif isinstance(rsp, wireprototypes.streamres):
 gen = rsp.gen
 
 # This code for compression should not be streamres specific. It
@@ -291,18 +292,18 @@
 
 req.respond(HTTP_OK, mediatype)
 return gen
-elif isinstance(rsp, wireproto.pushres):
+elif isinstance(rsp, wireprototypes.pushres):
 rsp = '%d\n%s' % (rsp.res, rsp.output)
 req.respond(HTTP_OK, HGTYPE, body=rsp)
 return []
-elif isinstance(rsp, wireproto.pusherr):
+elif isinstance(rsp, wireprototypes.pusherr):
 # This is the httplib workaround documented in _handlehttperror().
 req.drain()
 
 rsp = '0\n%s\n' % rsp.res
 req.respond(HTTP_OK, HGTYPE, body=rsp)
 return []
-elif isinstance(rsp, wireproto.ooberror):
+elif isinstance(rsp, wireprototypes.ooberror):
 rsp = rsp.message
 req.respond(HTTP_OK, HGERRTYPE, body=rsp)
 return []
@@ -434,16 +435,16 @@
 
 if isinstance(rsp, bytes):
 _sshv1respondbytes(self._fout, rsp)
-elif isinstance(rsp, wireproto.streamres):
+elif isinstance(rsp, wireprototypes.streamres):
 _sshv1respondstream(self._fout, rsp)
-elif isinstance(rsp, wireproto.streamres_legacy):
+elif isinstance(rsp, wireprototypes.streamreslegacy):
 _sshv1respondstream(self._fout, rsp)
-elif isinstance(rsp, wireproto.pushres):
+elif isinstance(rsp, wireprototypes.pushres):
 _sshv1respondbytes(self._fout, b'')
 _sshv1respondbytes(self._fout, bytes(rsp.res))
-elif isinstance(rsp, wireproto.pusherr):
+elif isinstance(rsp, wireprototypes.pusherr):
 _sshv1respondbytes(self._fout, rsp.res)
-elif isinstance(rsp, wireproto.ooberror):
+elif isinstance(rsp, wireprototypes.ooberror):
 

D2083: wireprotoserver: remove redirect() and restore() (API)

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5346.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2083?vs=5335=5346

REVISION DETAIL
  https://phab.mercurial-scm.org/D2083

AFFECTED FILES
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -88,23 +88,6 @@
 won't be captured.
 """
 
-@abc.abstractmethod
-def redirect(self):
-"""may setup interception for stdout and stderr
-
-See also the `restore` method."""
-
-# If the `redirect` function does install interception, the `restore`
-# function MUST be defined. If interception is not used, this function
-# MUST NOT be defined.
-#
-# left commented here on purpose
-#
-#def restore(self):
-#"""reinstall previous stdout and stderr and return intercepted stdout
-#"""
-#raise NotImplementedError()
-
 def decodevaluefromheaders(req, headerprefix):
 """Decode a long value from multiple HTTP request headers.
 
@@ -181,15 +164,6 @@
 self._ui.fout = oldout
 self._ui.ferr = olderr
 
-def redirect(self):
-self._oldio = self._ui.fout, self._ui.ferr
-self._ui.ferr = self._ui.fout = stringio()
-
-def restore(self):
-val = self._ui.fout.getvalue()
-self._ui.ferr, self._ui.fout = self._oldio
-return val
-
 def _client(self):
 return 'remote:%s:%s:%s' % (
 self._req.env.get('wsgi.url_scheme') or 'http',
@@ -425,9 +399,6 @@
 def mayberedirectstdio(self):
 yield None
 
-def redirect(self):
-pass
-
 def _client(self):
 client = encoding.environ.get('SSH_CLIENT', '').split(' ', 1)[0]
 return 'remote:ssh:' + client



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2085: wireprotoserver: rename getfile() to forwardpayload() (API)

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5348.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2085?vs=5337=5348

REVISION DETAIL
  https://phab.mercurial-scm.org/D2085

AFFECTED FILES
  hgext/largefiles/proto.py
  mercurial/wireproto.py
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -64,14 +64,10 @@
 returns a list of values (same order as )"""
 
 @abc.abstractmethod
-def getfile(self, fp):
-"""write the whole content of a file into a file like object
+def forwardpayload(self, fp):
+"""Read the raw payload and forward to a file.
 
-The file is in the form::
-
-(\n)+0\n
-
-chunk size is the ascii version of the int.
+The payload is read in full before the function returns.
 """
 
 @abc.abstractmethod
@@ -145,7 +141,7 @@
 args.update(cgi.parse_qs(argvalue, keep_blank_values=True))
 return args
 
-def getfile(self, fp):
+def forwardpayload(self, fp):
 length = int(self._req.env[r'CONTENT_LENGTH'])
 # If httppostargs is used, we need to read Content-Length
 # minus the amount that was consumed by args.
@@ -392,7 +388,12 @@
 data[arg] = val
 return [data[k] for k in keys]
 
-def getfile(self, fpout):
+def forwardpayload(self, fpout):
+# The file is in the form:
+#
+# \n
+# ...
+# 0\n
 _sshv1respondbytes(self._fout, b'')
 count = int(self._fin.readline())
 while count:
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -1008,7 +1008,7 @@
 fp = os.fdopen(fd, pycompat.sysstr('wb+'))
 r = 0
 try:
-proto.getfile(fp)
+proto.forwardpayload(fp)
 fp.seek(0)
 gen = exchange.readbundle(repo.ui, fp, None)
 if (isinstance(gen, changegroupmod.cg1unpacker)
diff --git a/hgext/largefiles/proto.py b/hgext/largefiles/proto.py
--- a/hgext/largefiles/proto.py
+++ b/hgext/largefiles/proto.py
@@ -40,7 +40,7 @@
 tmpfp = util.atomictempfile(path, createmode=repo.store.createmode)
 
 try:
-proto.getfile(tmpfp)
+proto.forwardpayload(tmpfp)
 tmpfp._fp.seek(0)
 if sha != lfutil.hexsha1(tmpfp._fp):
 raise IOError(0, _('largefile contents do not match hash'))



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2081: wireprotoserver: add context manager mechanism for redirecting stdio

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5344.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2081?vs=5333=5344

REVISION DETAIL
  https://phab.mercurial-scm.org/D2081

AFFECTED FILES
  mercurial/wireproto.py
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -8,6 +8,7 @@
 
 import abc
 import cgi
+import contextlib
 import struct
 import sys
 
@@ -74,6 +75,20 @@
 """
 
 @abc.abstractmethod
+def mayberedirectstdio(self):
+"""Context manager to possibly redirect stdio.
+
+The context manager yields a file-object like object that receives
+stdout and stderr output when the context manager is active. Or it
+yields ``None`` if no I/O redirection occurs.
+
+The intent of this context manager is to capture stdio output
+so it may be sent in the response. Some transports support streaming
+stdio to the client in real time. For these transports, stdio output
+won't be captured.
+"""
+
+@abc.abstractmethod
 def redirect(self):
 """may setup interception for stdout and stderr
 
@@ -151,6 +166,21 @@
 for s in util.filechunkiter(self._req, limit=length):
 fp.write(s)
 
+@contextlib.contextmanager
+def mayberedirectstdio(self):
+oldout = self._ui.fout
+olderr = self._ui.ferr
+
+out = util.stringio()
+
+try:
+self._ui.fout = out
+self._ui.ferr = out
+yield out
+finally:
+self._ui.fout = oldout
+self._ui.ferr = olderr
+
 def redirect(self):
 self._oldio = self._ui.fout, self._ui.ferr
 self._ui.ferr = self._ui.fout = stringio()
@@ -393,6 +423,10 @@
 fpout.write(self._fin.read(count))
 count = int(self._fin.readline())
 
+@contextlib.contextmanager
+def mayberedirectstdio(self):
+yield None
+
 def redirect(self):
 pass
 
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -978,20 +978,12 @@
 else:
 new = encoding.tolocal(new) # normal path
 
-if util.safehasattr(proto, 'restore'):
-
-proto.redirect()
-
+with proto.mayberedirectstdio() as output:
 r = repo.pushkey(encoding.tolocal(namespace), encoding.tolocal(key),
  encoding.tolocal(old), new) or False
 
-output = proto.restore()
-
-return '%s\n%s' % (int(r), output)
-
-r = repo.pushkey(encoding.tolocal(namespace), encoding.tolocal(key),
- encoding.tolocal(old), new)
-return '%s\n' % int(r)
+output = output.getvalue() if output else ''
+return '%s\n%s' % (int(r), output)
 
 @wireprotocommand('stream_out')
 def stream(repo, proto):



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2091: wireprotoserver: extract SSH response handling functions

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5342.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2091?vs=5331=5342

REVISION DETAIL
  https://phab.mercurial-scm.org/D2091

AFFECTED FILES
  mercurial/wireprotoserver.py
  tests/sshprotoext.py

CHANGE DETAILS

diff --git a/tests/sshprotoext.py b/tests/sshprotoext.py
--- a/tests/sshprotoext.py
+++ b/tests/sshprotoext.py
@@ -45,11 +45,11 @@
 l = self._fin.readline()
 assert l == b'hello\n'
 # Respond to unknown commands with an empty reply.
-self._sendresponse(b'')
+wireprotoserver._sshv1respondbytes(self._fout, b'')
 l = self._fin.readline()
 assert l == b'between\n'
 rsp = wireproto.dispatch(self._repo, self, b'between')
-self._handlers[rsp.__class__](self, rsp)
+wireprotoserver._sshv1respondbytes(self._fout, rsp)
 
 super(prehelloserver, self).serve_forever()
 
diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -336,6 +336,24 @@
 
 return ''
 
+def _sshv1respondbytes(fout, value):
+"""Send a bytes response for protocol version 1."""
+fout.write('%d\n' % len(value))
+fout.write(value)
+fout.flush()
+
+def _sshv1respondstream(fout, source):
+write = fout.write
+for chunk in source.gen:
+write(chunk)
+fout.flush()
+
+def _sshv1respondooberror(fout, ferr, rsp):
+ferr.write(b'%s\n-\n' % rsp)
+ferr.flush()
+fout.write(b'\n')
+fout.flush()
+
 class sshserver(baseprotocolhandler):
 def __init__(self, ui, repo):
 self._ui = ui
@@ -376,60 +394,43 @@
 return [data[k] for k in keys]
 
 def getfile(self, fpout):
-self._sendresponse('')
+_sshv1respondbytes(self._fout, b'')
 count = int(self._fin.readline())
 while count:
 fpout.write(self._fin.read(count))
 count = int(self._fin.readline())
 
 def redirect(self):
 pass
 
-def _sendresponse(self, v):
-self._fout.write("%d\n" % len(v))
-self._fout.write(v)
-self._fout.flush()
-
-def _sendstream(self, source):
-write = self._fout.write
-for chunk in source.gen:
-write(chunk)
-self._fout.flush()
-
-def _sendpushresponse(self, rsp):
-self._sendresponse('')
-self._sendresponse(str(rsp.res))
-
-def _sendpusherror(self, rsp):
-self._sendresponse(rsp.res)
-
-def _sendooberror(self, rsp):
-self._ui.ferr.write('%s\n-\n' % rsp.message)
-self._ui.ferr.flush()
-self._fout.write('\n')
-self._fout.flush()
-
 def serve_forever(self):
 while self.serve_one():
 pass
 sys.exit(0)
 
-_handlers = {
-str: _sendresponse,
-wireproto.streamres: _sendstream,
-wireproto.streamres_legacy: _sendstream,
-wireproto.pushres: _sendpushresponse,
-wireproto.pusherr: _sendpusherror,
-wireproto.ooberror: _sendooberror,
-}
-
 def serve_one(self):
 cmd = self._fin.readline()[:-1]
 if cmd and wireproto.commands.commandavailable(cmd, self):
 rsp = wireproto.dispatch(self._repo, self, cmd)
-self._handlers[rsp.__class__](self, rsp)
+
+if isinstance(rsp, bytes):
+_sshv1respondbytes(self._fout, rsp)
+elif isinstance(rsp, wireproto.streamres):
+_sshv1respondstream(self._fout, rsp)
+elif isinstance(rsp, wireproto.streamres_legacy):
+_sshv1respondstream(self._fout, rsp)
+elif isinstance(rsp, wireproto.pushres):
+_sshv1respondbytes(self._fout, b'')
+_sshv1respondbytes(self._fout, bytes(rsp.res))
+elif isinstance(rsp, wireproto.pusherr):
+_sshv1respondbytes(self._fout, rsp.res)
+elif isinstance(rsp, wireproto.ooberror):
+_sshv1respondooberror(self._fout, self._ui.ferr, rsp.message)
+else:
+raise error.ProgrammingError('unhandled response type from '
+ 'wire protocol command: %s' % rsp)
 elif cmd:
-self._sendresponse("")
+_sshv1respondbytes(self._fout, b'')
 return cmd != ''
 
 def _client(self):



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2057: translate base85.c into rust code

[indygreg (Gregory Szorc)]

indygreg added a comment.


  Yes, we should definitely split things into multiple crates. Small, 
narrowly-focused crates does seem to be the Rust way, after all.
  
  `hgcli` should be for things specific to the Rust implementation of `hg`. I 
think this can also include the feature set of `chg` (once we've ported `chg` 
to Rust).
  
  I definitely support separating the "pure Rust" from the "Python Rust" via a 
crate boundary. It is generally useful to have Rust that isn't bound to Python 
because it will facilitate reuse outside of Python contexts. For example, 
someone could implement a Mercurial wire protocol server in pure Rust without 
needing to worry about Python. Of course, we're likely to encounter areas where 
we really want tight coupling in order to achieve optimal performance in 
Python. So we may have to design APIs on the pure Rust side to facilitate 
CPython use. I'm OK with that.
  
  As for how many crates to have, I don't have super strong opinions. I could 
see us putting every little component/subsystem in its own crate. I could also 
see us putting everything in one large crate. I don't think it is worth 
deciding at this early juncture. API design and ability to be reused outside 
its originally intended purpose is the important property to strive for. I 
think that has more to do with how the code is authored rather than which 
crates things are in.
  
  A missing piece of this patch is the build system and module loader 
integration. We have a //module policy// that dictates which implementation of 
a Python module we use. We probably want to introduce a `rust` policy that uses 
Rust-based modules where available and falls back to the `cext` modules/policy 
if a Rust module isn't available. We also need to figure out how to integrate 
Rust into `setup.py`. But I think the build system bit can be deferred until 
we're actually ready to ship Rust, which is still a bit of ways off. I'm happy 
for the workflow to be //run cargo in order to load Rust modules// for the time 
being. But if you can implement `Makefile` and/or `setup.py` integration to 
build these Rust extensions, that would be awesome.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2057

To: Ivzhh, #hg-reviewers
Cc: indygreg, durin42, kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2019: wireprotoserver: move protocol parsing and dispatch out of hgweb

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5259.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2019?vs=5173=5259

REVISION DETAIL
  https://phab.mercurial-scm.org/D2019

AFFECTED FILES
  mercurial/hgweb/hgweb_mod.py
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -202,9 +202,43 @@
 def iscmd(cmd):
 return cmd in wireproto.commands
 
-def callhttp(repo, req, cmd):
+def parsehttprequest(repo, req, query):
+"""Parse the HTTP request for a wire protocol request.
+
+If the current request appears to be a wire protocol request, this
+function returns a dict with details about that request, including
+an ``abstractprotocolserver`` instance suitable for handling the
+request. Otherwise, ``None`` is returned.
+
+``req`` is a ``wsgirequest`` instance.
+"""
+# HTTP version 1 wire protocol requests are denoted by a "cmd" query
+# string parameter. If it isn't present, this isn't a wire protocol
+# request.
+if r'cmd' not in req.form:
+return None
+
+cmd = pycompat.sysbytes(req.form[r'cmd'][0])
+
+# The "cmd" request parameter is used by both the wire protocol and hgweb.
+# While not all wire protocol commands are available for all transports,
+# if we see a "cmd" value that resembles a known wire protocol command, we
+# route it to a protocol handler. This is better than routing possible
+# wire protocol requests to hgweb because it prevents hgweb from using
+# known wire protocol commands and it is less confusing for machine
+# clients.
+if cmd not in wireproto.commands:
+return None
+
 proto = webproto(req, repo.ui)
 
+return {
+'cmd': cmd,
+'proto': proto,
+'dispatch': lambda: _callhttp(repo, req, proto, cmd),
+}
+
+def _callhttp(repo, req, proto, cmd):
 def genversion2(gen, engine, engineopts):
 # application/mercurial-0.2 always sends a payload header
 # identifying the compression engine.
diff --git a/mercurial/hgweb/hgweb_mod.py b/mercurial/hgweb/hgweb_mod.py
--- a/mercurial/hgweb/hgweb_mod.py
+++ b/mercurial/hgweb/hgweb_mod.py
@@ -357,26 +357,18 @@
 query = req.env[r'QUERY_STRING'].partition(r'&')[0]
 query = query.partition(r';')[0]
 
-# The ``cmd`` request parameter is used by both the wire protocol
-# and hgweb. We route all known wire protocol commands to the
-# wire protocol handler, even if the command isn't available for
-# this transport. That's better for machine clients in the case
-# of an errant request to an unavailable protocol command. And it
-# prevents hgweb from accidentally using ``cmd`` values used by
-# the wire protocol.
+# Route it to a wire protocol handler if it looks like a wire protocol
+# request.
+protohandler = wireprotoserver.parsehttprequest(rctx.repo, req, query)
 
-# process this if it's a protocol request
-# protocol bits don't need to create any URLs
-# and the clients always use the old URL structure
-
-cmd = pycompat.sysbytes(req.form.get(r'cmd', [r''])[0])
-if wireprotoserver.iscmd(cmd):
+if protohandler:
+cmd = protohandler['cmd']
 try:
 if query:
 raise ErrorResponse(HTTP_NOT_FOUND)
 if cmd in perms:
 self.check_perm(rctx, req, perms[cmd])
-return wireprotoserver.callhttp(rctx.repo, req, cmd)
+return protohandler['dispatch']()
 except ErrorResponse as inst:
 # A client that sends unbundle without 100-continue will
 # break if we respond early.
@@ -425,6 +417,8 @@
 if fn.endswith(ext):
 req.form['node'] = [fn[:-len(ext)]]
 req.form['type'] = [type_]
+else:
+cmd = pycompat.sysbytes(req.form.get(r'cmd', [r''])[0])
 
 # process the web interface request
 



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1999: wireproto: function for testing if wire protocol command is available

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5258.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D1999?vs=5138=5258

REVISION DETAIL
  https://phab.mercurial-scm.org/D1999

AFFECTED FILES
  mercurial/hgweb/hgweb_mod.py
  mercurial/wireproto.py
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -217,6 +217,13 @@
 yield chunk
 
 rsp = wireproto.dispatch(repo, proto, cmd)
+
+if not wireproto.commands.commandavailable(cmd, proto):
+req.respond(HTTP_OK, HGERRTYPE,
+body=_('requested wire protocol command is not available '
+   'over HTTP'))
+return []
+
 if isinstance(rsp, bytes):
 req.respond(HTTP_OK, HGTYPE, body=rsp)
 return []
@@ -345,7 +352,7 @@
 
 def serve_one(self):
 cmd = self._fin.readline()[:-1]
-if cmd and cmd in wireproto.commands:
+if cmd and wireproto.commands.commandavailable(cmd, self):
 rsp = wireproto.dispatch(self._repo, self, cmd)
 self._handlers[rsp.__class__](self, rsp)
 elif cmd:
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -691,6 +691,12 @@
 
 return super(commanddict, self).__setitem__(k, v)
 
+def commandavailable(self, command, proto):
+"""Determine if a command is available for the requested protocol."""
+# For now, commands are available for all protocols. So do a simple
+# membership test.
+return command in self
+
 commands = commanddict()
 
 def wireprotocommand(name, args=''):
diff --git a/mercurial/hgweb/hgweb_mod.py b/mercurial/hgweb/hgweb_mod.py
--- a/mercurial/hgweb/hgweb_mod.py
+++ b/mercurial/hgweb/hgweb_mod.py
@@ -357,6 +357,14 @@
 query = req.env[r'QUERY_STRING'].partition(r'&')[0]
 query = query.partition(r';')[0]
 
+# The ``cmd`` request parameter is used by both the wire protocol
+# and hgweb. We route all known wire protocol commands to the
+# wire protocol handler, even if the command isn't available for
+# this transport. That's better for machine clients in the case
+# of an errant request to an unavailable protocol command. And it
+# prevents hgweb from accidentally using ``cmd`` values used by
+# the wire protocol.
+
 # process this if it's a protocol request
 # protocol bits don't need to create any URLs
 # and the clients always use the old URL structure



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2021: wireprotoserver: move error response handling out of hgweb

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5260.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2021?vs=5175=5260

REVISION DETAIL
  https://phab.mercurial-scm.org/D2021

AFFECTED FILES
  mercurial/hgweb/hgweb_mod.py
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -236,6 +236,7 @@
 'cmd': cmd,
 'proto': proto,
 'dispatch': lambda: _callhttp(repo, req, proto, cmd),
+'handleerror': lambda ex: _handlehttperror(ex, req, cmd),
 }
 
 def _callhttp(repo, req, proto, cmd):
@@ -297,6 +298,22 @@
 return []
 raise error.ProgrammingError('hgweb.protocol internal failure', rsp)
 
+def _handlehttperror(e, req, cmd):
+"""Called when an ErrorResponse is raised during HTTP request 
processing."""
+# A client that sends unbundle without 100-continue will
+# break if we respond early.
+if (cmd == 'unbundle' and
+(req.env.get('HTTP_EXPECT',
+ '').lower() != '100-continue') or
+req.env.get('X-HgHttp2', '')):
+req.drain()
+else:
+req.headers.append((r'Connection', r'Close'))
+
+req.respond(e, HGTYPE, body='0\n%s\n' % e)
+
+return ''
+
 class sshserver(abstractserverproto):
 def __init__(self, ui, repo):
 self._ui = ui
diff --git a/mercurial/hgweb/hgweb_mod.py b/mercurial/hgweb/hgweb_mod.py
--- a/mercurial/hgweb/hgweb_mod.py
+++ b/mercurial/hgweb/hgweb_mod.py
@@ -369,18 +369,7 @@
 if cmd in perms:
 self.check_perm(rctx, req, perms[cmd])
 except ErrorResponse as inst:
-# A client that sends unbundle without 100-continue will
-# break if we respond early.
-if (cmd == 'unbundle' and
-(req.env.get('HTTP_EXPECT',
- '').lower() != '100-continue') or
-req.env.get('X-HgHttp2', '')):
-req.drain()
-else:
-req.headers.append((r'Connection', r'Close'))
-req.respond(inst, wireprotoserver.HGTYPE,
-body='0\n%s\n' % inst)
-return ''
+return protohandler['handleerror'](inst)
 
 return protohandler['dispatch']()
 



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2061: sshpeer: initial definition and implementation of new SSH protocol

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The existing SSH protocol has several design flaws. Future commits
  will elaborate on these flaws as new features are introduced
  to combat these flaws. For now, hopefully you can take me for my
  word that a ground up rewrite of the SSH protocol is needed.
  
  This commit lays the foundation for a new SSH protocol by defining
  a mechanism to upgrade the SSH transport channel away from the
  default (version 1) protocol to something modern (which we'll call
  "version 2" for now).
  
  This upgrade process is detailed in the internals documentation
  for the wire protocol. The gist of it is the client sends a
  request line preceding the "hello" command/line which basically
  says "I'm requesting an upgrade: here's what I support." If the
  server recognizes that line, it processes the upgrade request and
  the transport channel is switched to use the new version of the
  protocol. If not, it sends an empty response, which is how all
  Mercurial SSH servers from the beginning of time reacted to unknown
  commands. The upgrade request is effectively ignored and the client
  continues to use the existing version of the protocol as if nothing
  happened.
  
  The new version of the SSH protocol is completely identical to
  version 1 aside from the upgrade dance and the bytes that follow.
  The immediate bytes that follow the protocol switch are defined to
  be a length framed "capabilities: " line containing the remote's
  advertised capabilities. In reality, this looks very similar to
  what the "hello" response would look like. But it will evolve
  quickly.
  
  The methodology by which the protocol will evolve is important.
  
  I'm not going to introduce the new protocol all at once. That would
  likely lead to endless bike shedding and forward progress would
  stall. Instead, I intend to tricle out new features and diversions
  from the existing protocol in small, incremental changes.
  
  To support the gradual evolution of the protocol, the on-the-wire
  advertised protocol name contains an "exp" to denote "experimental"
  and a 4 digit field to capture the sub-version of the protocol.
  Whenever we make a BC change to the wire protocol, we can increment
  this version and lock out all older clients because it will appear
  as a completely different protocol version. This means we can incur
  as many breaking changes as we want. We don't have to commit to
  supporting any one feature or idea for a long period of time. We
  can even evolve the handshake mechanism, because that is defined
  as being an implementation detail of the negotiated protocol version!
  Hopefully this lowers the barrier to accepting changes to the
  protocol and for experimenting with "radical" ideas during its
  development.
  
  In core, sshpeer received most of the attention. We haven't even
  implemented the server bits for the new protocol in core yet.
  Instead, we add very primitive support to our test server, mainly
  just to exercise the added code paths in sshpeer.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2061

AFFECTED FILES
  mercurial/configitems.py
  mercurial/help/internals/wireprotocol.txt
  mercurial/sshpeer.py
  mercurial/wireprotoserver.py
  tests/sshprotoext.py
  tests/test-ssh-proto.t

CHANGE DETAILS

diff --git a/tests/test-ssh-proto.t b/tests/test-ssh-proto.t
--- a/tests/test-ssh-proto.t
+++ b/tests/test-ssh-proto.t
@@ -388,3 +388,107 @@
   0
   0
   0
+
+Send an upgrade request to a server that doesn't support that command
+
+  $ hg -R server serve --stdio << EOF
+  > upgrade 2e82ab3f-9ce3-4b4e-8f8c-6fd1c0e9e23a 
proto=irrelevant1%2Cirrelevant2
+  > hello
+  > between
+  > pairs 81
+  > 
-
+  > EOF
+  0
+  384
+  capabilities: lookup changegroupsubset branchmap pushkey known getbundle 
unbundlehash batch streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ 
unbundle=HG10GZ,HG10BZ,HG10UN
+  1
+  
+
+  $ hg --config experimental.sshpeer.advertise-v2=true --debug debugpeer 
ssh://user@dummy/server
+  running * "*/tests/dummyssh" 'user@dummy' 'hg -R server serve --stdio' (glob)
+  sending upgrade request: * proto=exp-ssh-v2-0001 (glob)
+  devel-peer-request: hello
+  sending hello command
+  devel-peer-request: between
+  devel-peer-request:   pairs: 81 bytes
+  sending between command
+  remote: 0
+  remote: 384
+  remote: capabilities: lookup changegroupsubset branchmap pushkey known 
getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 
$USUAL_BUNDLE2_CAPS_SERVER$ unbundle=HG10GZ,HG10BZ,HG10UN
+  remote: 1
+  url: ssh://user@dummy/server
+  local: no
+  pushable: yes
+
+Send an upgrade request to a server that supports upgrade
+
+  $ SSHSERVERMODE=upgradev2 hg -R server serve --stdio << EOF
+  > upgrade this-is-some-token 

D2062: sshpeer: rename sshpeer class to sshv1peer (API)

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  With the introduction of version 2 of the SSH wire protocol,
  we will need a new peer class to speak that protocol because
  it will be too difficult to shoehorn a single class to speak
  two protocols. We rename sshpeer.sshpeer to sshpeer.sshv1peer
  to reflect the fact that there will be multiple versions of
  the peer depending on the negotiated protocol.
  
  .. api::
  
sshpeer.sshpeer renamed to sshpeer.sshv1peer.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2062

AFFECTED FILES
  hgext/largefiles/uisetup.py
  mercurial/sshpeer.py
  tests/test-check-interfaces.py

CHANGE DETAILS

diff --git a/tests/test-check-interfaces.py b/tests/test-check-interfaces.py
--- a/tests/test-check-interfaces.py
+++ b/tests/test-check-interfaces.py
@@ -65,8 +65,8 @@
 checkobject(badpeer())
 checkobject(httppeer.httppeer(ui, 'http://localhost'))
 checkobject(localrepo.localpeer(dummyrepo()))
-checkobject(sshpeer.sshpeer(ui, 'ssh://localhost/foo', None, None, None,
-   None, None))
+checkobject(sshpeer.sshv1peer(ui, 'ssh://localhost/foo', None, None, None,
+  None, None))
 checkobject(bundlerepo.bundlepeer(dummyrepo()))
 checkobject(statichttprepo.statichttppeer(dummyrepo()))
 checkobject(unionrepo.unionpeer(dummyrepo()))
diff --git a/mercurial/sshpeer.py b/mercurial/sshpeer.py
--- a/mercurial/sshpeer.py
+++ b/mercurial/sshpeer.py
@@ -328,7 +328,7 @@
 
 return caps
 
-class sshpeer(wireproto.wirepeer):
+class sshv1peer(wireproto.wirepeer):
 def __init__(self, ui, url, proc, stdin, stdout, stderr, caps):
 """Create a peer from an existing SSH connection.
 
@@ -537,4 +537,4 @@
 _cleanuppipes(ui, stdout, stdin, stderr)
 raise
 
-return sshpeer(ui, path, proc, stdin, stdout, stderr, caps)
+return sshv1peer(ui, path, proc, stdin, stdout, stderr, caps)
diff --git a/hgext/largefiles/uisetup.py b/hgext/largefiles/uisetup.py
--- a/hgext/largefiles/uisetup.py
+++ b/hgext/largefiles/uisetup.py
@@ -185,9 +185,9 @@
 
 # can't do this in reposetup because it needs to have happened before
 # wirerepo.__init__ is called
-proto.ssholdcallstream = sshpeer.sshpeer._callstream
+proto.ssholdcallstream = sshpeer.sshv1peer._callstream
 proto.httpoldcallstream = httppeer.httppeer._callstream
-sshpeer.sshpeer._callstream = proto.sshrepocallstream
+sshpeer.sshv1peer._callstream = proto.sshrepocallstream
 httppeer.httppeer._callstream = proto.httprepocallstream
 
 # override some extensions' stuff as well



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2060: internals: refactor wire protocol documentation

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Upcoming work will introduce a new version of the HTTP and SSH
  transports. The differences will be significant enough to consider
  them new transports. So, we now attach a version number to each
  transport.
  
  In addition, having the handshake documented after the transport
  and in a single shared section made it harder to follow the flow
  of the connection. The handshake documentation is now moved to the
  protocol section it describes. We now have a generic section about
  the purpose of the handshake, which was rewritten significantly.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2060

AFFECTED FILES
  mercurial/help/internals/wireprotocol.txt

CHANGE DETAILS

diff --git a/mercurial/help/internals/wireprotocol.txt 
b/mercurial/help/internals/wireprotocol.txt
--- a/mercurial/help/internals/wireprotocol.txt
+++ b/mercurial/help/internals/wireprotocol.txt
@@ -10,11 +10,43 @@
 The protocol is synchronous and does not support multiplexing (concurrent
 commands).
 
-Transport Protocols
-===
+Handshake
+=
+
+It is required or common for clients to perform a *handshake* when connecting
+to a server. The handshake serves the following purposes:
+
+* Negotiating protocol/transport level options
+* Allows the client to learn about server capabilities to influence
+  future requests
+* Ensures the underlying transport channel is in a *clean* state
 
-HTTP Transport
---
+An important goal of the handshake is to allow clients to use more modern
+wire protocol features. By default, clients must assume they are talking
+to an old version of Mercurial server (possibly even the very first
+implementation). So, clients should not attempt to call or utilize modern
+wire protocol features until they have confirmation that the server
+supports them. The handshake implementation is designed to allow both
+ends to utilize the latest set of features and capabilities with as
+few round trips as possible.
+
+The handshake mechanism varies by transport and protocol and is documented
+in the sections below.
+
+HTTP Protocol
+=
+
+Handshake
+-
+
+The client sends a ``capabilities`` command request (``?cmd=capabilities``)
+as soon as HTTP requests may be issued.
+
+The server responds with a capabilities string, which the client parses to
+learn about the server's abilities.
+
+HTTP Version 1 Transport
+
 
 Commands are issued as HTTP/1.0 or HTTP/1.1 requests. Commands are
 sent to the base URL of the repository with the command name sent in
@@ -112,11 +144,86 @@
 ``application/mercurial-0.*`` media type and the HTTP response is typically
 using *chunked transfer* (``Transfer-Encoding: chunked``).
 
-SSH Transport
-=
+SSH Protocol
+
+
+Handshake
+-
+
+For all clients, the handshake consists of the client sending 1 or more
+commands to the server using version 1 of the transport. Servers respond
+to commands they know how to respond to and send an empty response (``0\n``)
+for unknown commands (per standard behavior of version 1 of the transport).
+Clients then typically look for a response to the newest sent command to
+determine which transport version to use and what the available features for
+the connection and server are.
+
+Preceding any response from client-issued commands, the server may print
+non-protocol output. It is common for SSH servers to print banners, message
+of the day announcements, etc when clients connect. It is assumed that any
+such *banner* output will precede any Mercurial server output. So clients
+must be prepared to handle server output on initial connect that isn't
+in response to any client-issued command and doesn't conform to Mercurial's
+wire protocol. This *banner* output should only be on stdout. However,
+some servers may send output on stderr.
+
+Pre 0.9.1 clients issue a ``between`` command with the ``pairs`` argument
+having the value
+``-``.
+
+The ``between`` command has been supported since the original Mercurial
+SSH server. Requesting the empty range will return a ``\n`` string response,
+which will be encoded as ``1\n\n`` (value length of ``1`` followed by a newline
+followed by the value, which happens to be a newline).
+
+For pre 0.9.1 clients and all servers, the exchange looks like::
+
+   c: between\n
+   c: pairs 81\n
+   c: 
-
+   s: 1\n
+   s: \n
 
-The SSH transport is a custom text-based protocol suitable for use over any
-bi-directional stream transport. It is most commonly used with SSH.
+0.9.1+ clients send a ``hello`` command (with no arguments) before the
+``between`` command. The response to this command allows clients to

D2063: sshpeer: implement peer for version 2 of wire protocol

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Since the protocol is now negotiated before we construct a
  peer instance, we can return the negotiated protocol from the
  handshake function and instantiate an appropriate peer class
  for the protocol.
  
  Version 2 of the SSH protocol is currently identical to version
  1 post handshake. So our sshv2peer class just inherits from
  sshv1peer for the time being. This will obviously change
  over time.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2063

AFFECTED FILES
  mercurial/sshpeer.py
  tests/test-check-interfaces.py

CHANGE DETAILS

diff --git a/tests/test-check-interfaces.py b/tests/test-check-interfaces.py
--- a/tests/test-check-interfaces.py
+++ b/tests/test-check-interfaces.py
@@ -67,6 +67,8 @@
 checkobject(localrepo.localpeer(dummyrepo()))
 checkobject(sshpeer.sshv1peer(ui, 'ssh://localhost/foo', None, None, None,
   None, None))
+checkobject(sshpeer.sshv2peer(ui, 'ssh://localhost/foo', None, None, None,
+  None, None))
 checkobject(bundlerepo.bundlepeer(dummyrepo()))
 checkobject(statichttprepo.statichttppeer(dummyrepo()))
 checkobject(unionrepo.unionpeer(dummyrepo()))
diff --git a/mercurial/sshpeer.py b/mercurial/sshpeer.py
--- a/mercurial/sshpeer.py
+++ b/mercurial/sshpeer.py
@@ -326,7 +326,7 @@
 if not caps:
 badresponse()
 
-return caps
+return protoname, caps
 
 class sshv1peer(wireproto.wirepeer):
 def __init__(self, ui, url, proc, stdin, stdout, stderr, caps):
@@ -497,6 +497,12 @@
 self._pipeo.flush()
 self._readerr()
 
+class sshv2peer(sshv1peer):
+"""A peer that speakers version 2 of the transport protocol."""
+# Currently version 2 is identical to version 1 post handshake.
+# And handshake is performed before the peer is instantiated. So
+# we need no custom code.
+
 def instance(ui, path, create):
 """Create an SSH peer.
 
@@ -532,9 +538,16 @@
   remotepath, sshenv)
 
 try:
-caps = _performhandshake(ui, stdin, stdout, stderr)
+protoname, caps = _performhandshake(ui, stdin, stdout, stderr)
 except Exception:
 _cleanuppipes(ui, stdout, stdin, stderr)
 raise
 
-return sshv1peer(ui, path, proc, stdin, stdout, stderr, caps)
+if protoname == wireprotoserver.SSHV1:
+return sshv1peer(ui, path, proc, stdin, stdout, stderr, caps)
+elif protoname == wireprotoserver.SSHV2:
+return sshv2peer(ui, path, proc, stdin, stdout, stderr, caps)
+else:
+_cleanuppipes(ui, stdout, stdin, stderr)
+raise error.RepoError(_('unknown version of SSH protocol: %s') %
+  protoname)



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2065: wireprotoserver: rename abstractserverproto and improve docstring

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The docstring isn't completely accurate for the current state
  of the world. But it does describe the direction future patches
  will be taking things.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2065

AFFECTED FILES
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -32,10 +32,13 @@
 HGTYPE2 = 'application/mercurial-0.2'
 HGERRTYPE = 'application/hg-error'
 
-class abstractserverproto(object):
-"""abstract class that summarizes the protocol API
+class baseprotocolhandler(object):
+"""Abstract base class for wire protocol handlers.
 
-Used as reference and documentation.
+A wire protocol handler serves as an interface between protocol command
+handlers and the wire protocol transport layer. Protocol handlers provide
+methods to read command arguments, redirect stdio for the duration of
+the request, handle response types, etc.
 """
 
 __metaclass__ = abc.ABCMeta
@@ -98,7 +101,7 @@
 
 return ''.join(chunks)
 
-class webproto(abstractserverproto):
+class webproto(baseprotocolhandler):
 def __init__(self, req, ui):
 self._req = req
 self._ui = ui
@@ -327,7 +330,7 @@
 
 return ''
 
-class sshserver(abstractserverproto):
+class sshserver(baseprotocolhandler):
 def __init__(self, ui, repo):
 self._ui = ui
 self._repo = repo



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2064: wireprotoserver: document and improve the httplib workaround

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This workaround dates all the way back to 
https://phab.mercurial-scm.org/rHGa42d27bc809dda4939c5b3807b397bcc811ebbe0 in 
2008.
  The code is esoteric enough to warrant an inline explanation.
  So I've added one.
  
  At the time the code was written, the only wire protocol command
  that accepted an HTTP request body was "unbundle." In the years
  since, we've grown the ability to accept command arguments via
  HTTP POST requests. So, the code has been changed to apply the
  httplib workaround to all HTTP POST requests.
  
  While staring at this code, I realized that the HTTP response
  body in case of error is always the same. And, it appears to
  mimic the behavior of a failed call to the "unbundle" command.
  Since we can hit this code path on theoretically any protocol
  request (since self.check_perm accepts custom auth checking
  functions which may raise), I'm having a hard time believing
  that clients react well to an "unbundle" response payload on
  any wire protocol command. I wouldn't be surprised if our test
  coverage for this feature only covers HTTP POST calls to
  "unbundle." In other words, the experimental support for sending
  arguments via HTTP POST request bodies may result in badness on
  the client. Something to investigate another time perhaps...

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2064

AFFECTED FILES
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -286,8 +286,9 @@
 req.respond(HTTP_OK, HGTYPE, body=rsp)
 return []
 elif isinstance(rsp, wireproto.pusherr):
-# drain the incoming bundle
+# This is the httplib workaround documented in _handlehttperror().
 req.drain()
+
 proto.restore()
 rsp = '0\n%s\n' % rsp.res
 req.respond(HTTP_OK, HGTYPE, body=rsp)
@@ -300,16 +301,28 @@
 
 def _handlehttperror(e, req, cmd):
 """Called when an ErrorResponse is raised during HTTP request 
processing."""
-# A client that sends unbundle without 100-continue will
-# break if we respond early.
-if (cmd == 'unbundle' and
+
+# Clients using Python's httplib are stateful: the HTTP client
+# won't process an HTTP response until all request data is
+# sent to the server. The intent of this code is to ensure
+# we always read HTTP request data from the client, thus
+# ensuring httplib transitions to a state that allows it to read
+# the HTTP response. In other words, it helps prevent deadlocks
+# on clients using httplib.
+
+if (req.env[r'REQUEST_METHOD'] == r'POST' and
+# But not if Expect: 100-continue is being used.
 (req.env.get('HTTP_EXPECT',
  '').lower() != '100-continue') or
+# Or the non-httplib HTTP library is being advertised by
+# the client.
 req.env.get('X-HgHttp2', '')):
 req.drain()
 else:
 req.headers.append((r'Connection', r'Close'))
 
+# TODO This response body assumes the failed command was
+# "unbundle." That assumption is not always valid.
 req.respond(e, HGTYPE, body='0\n%s\n' % e)
 
 return ''



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2057: translate base85.c into rust code

[indygreg (Gregory Szorc)]

indygreg added a comment.


  We generally prefer that patches to Mercurial be small and do a single thing. 
This makes it easier to review and understand changes, since each change can be 
evaluated in isolation. If you submit changesets together using `hg phabsend`, 
they automatically show up as a //stack// in Phabricator. And if changesets at 
the bottom of the stack are ready to land, we generally land those without 
waiting for the entire stack to land. This enables forward progress to be made 
and this is generally better for everyone than waiting until a series of 
commits is perfect before adding any of them.
  
  What that means is you should ideally split this work into smaller parts. For 
example:
  
  1. Add the pure Rust code/crate
  2. Add the Python Rust code/crate
  3. Build system / module policy changes
  
  I'm not sure of the order of things though. Since this is the first Rust 
extension, it's not clear what needs to be implemented in what order. I'm fine 
looking at a large commit if things are too tightly coupled to separate. But 
you should strive to make smaller commits.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2057

To: Ivzhh, #hg-reviewers
Cc: indygreg, durin42, kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2061: sshpeer: initial definition and implementation of new SSH protocol

[indygreg (Gregory Szorc)]

indygreg added a comment.


  I want to emphasize that I'm not committed to any implementation detail at 
this point in time. I've very opened minded about alternatives and making 
backwards incompatible changes throughout the 4.6 release cycle.
  
  That being said, I am trying to make forward progress on a ton of wire 
protocol changes. These are blocking my planned work for shallow clone this 
release cycle. (I don't want to deploy shallow clone on the existing wire 
protocol for various reasons.) So, I would prefer we //fall forward// and take 
commits even if there are open bikesheds. I'm more than happy to rework the 
protocol later. I just don't want my local work to be dozens of changesets 
ahead of what's reviewed and have to spend hours reworking my code because of a 
bikeshed. I'd rather commit the flawed work, fix it at the head of my local 
queue, and move forward. If nothing else, this approach will lead to a more 
feature complete protocol landing sooner. And only once it is feature complete 
will we all have the full perspective to bikeshed the protocol.

INLINE COMMENTS

> wireprotocol.txt:241-243
> +The transport capabilities string is a URL/percent encoded string
> +containing key-value pairs defining the client's transport-level
> +capabilities. The following capabilities are defined:

I chose the //query string// format here because I don't like reinventing 
wheels. However, if we wanted to make it a list of space delimited atoms (like 
the existing capabilities string), I'd be OK with that.

We can always change this later, since we're not locked into any BC guarantees 
at this juncture.

> wireprotocol.txt:245-247
> +proto
> +   A comma-delimited list of transport protocol versions the client
> +   supports. e.g. ``ssh-v2``.

In the future, I want to advertise:

- compression engine support
- compression engine settings (e.g. max window size for zstandard so a server 
won't choose a compression level that will result in excessive memory usage for 
client)
- max concurrent responses limit (in the future, the protocol will gain the 
ability to stream multiple responses for a single request concurrently)

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2061

To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2031: sshpeer: establish SSH connection before class instantiation

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5232.
indygreg edited the summary of this revision.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2031?vs=5189=5232

REVISION DETAIL
  https://phab.mercurial-scm.org/D2031

AFFECTED FILES
  mercurial/sshpeer.py
  tests/test-check-interfaces.py

CHANGE DETAILS

diff --git a/tests/test-check-interfaces.py b/tests/test-check-interfaces.py
--- a/tests/test-check-interfaces.py
+++ b/tests/test-check-interfaces.py
@@ -69,7 +69,8 @@
 checkobject(badpeer())
 checkobject(httppeer.httppeer(ui, 'http://localhost'))
 checkobject(localrepo.localpeer(dummyrepo()))
-checkobject(testingsshpeer(ui, 'ssh://localhost/foo', False, ()))
+checkobject(testingsshpeer(ui, 'ssh://localhost/foo', False,
+   (None, None, None, None)))
 checkobject(bundlerepo.bundlepeer(dummyrepo()))
 checkobject(statichttprepo.statichttppeer(dummyrepo()))
 checkobject(unionrepo.unionpeer(dummyrepo()))
diff --git a/mercurial/sshpeer.py b/mercurial/sshpeer.py
--- a/mercurial/sshpeer.py
+++ b/mercurial/sshpeer.py
@@ -131,16 +131,40 @@
 
 pipee.close()
 
+def _makeconnection(ui, sshcmd, args, remotecmd, path, sshenv=None):
+"""Create an SSH connection to a server.
+
+Returns a tuple of (process, stdin, stdout, stderr) for the
+spawned process.
+"""
+cmd = '%s %s %s' % (
+sshcmd,
+args,
+util.shellquote('%s -R %s serve --stdio' % (
+_serverquote(remotecmd), _serverquote(path
+
+ui.debug('running %s\n' % cmd)
+cmd = util.quotecommand(cmd)
+
+# no buffer allow the use of 'select'
+# feel free to remove buffering and select usage when we ultimately
+# move to threading.
+stdin, stdout, stderr, proc = util.popen4(cmd, bufsize=0, env=sshenv)
+
+stdout = doublepipe(ui, util.bufferedinputpipe(stdout), stderr)
+stdin = doublepipe(ui, stdin, stderr)
+
+return proc, stdin, stdout, stderr
+
 class sshpeer(wireproto.wirepeer):
 def __init__(self, ui, path, create=False, sshstate=None):
 self._url = path
 self._ui = ui
-self._pipeo = self._pipei = self._pipee = None
+# self._subprocess is unused. Keeping a handle on the process
+# holds a reference and prevents it from being garbage collected.
+self._subprocess, self._pipei, self._pipeo, self._pipee = sshstate
 
-u = util.url(path, parsequery=False, parsefragment=False)
-self._path = u.path or '.'
-
-self._validaterepo(*sshstate)
+self._validaterepo()
 
 # Begin of _basepeer interface.
 
@@ -172,28 +196,7 @@
 
 # End of _basewirecommands interface.
 
-def _validaterepo(self, sshcmd, args, remotecmd, sshenv=None):
-assert self._pipei is None
-
-cmd = '%s %s %s' % (sshcmd, args,
-util.shellquote("%s -R %s serve --stdio" %
-(_serverquote(remotecmd), _serverquote(self._path
-self.ui.debug('running %s\n' % cmd)
-cmd = util.quotecommand(cmd)
-
-# while self._subprocess isn't used, having it allows the subprocess to
-# to clean up correctly later
-#
-# no buffer allow the use of 'select'
-# feel free to remove buffering and select usage when we ultimately
-# move to threading.
-sub = util.popen4(cmd, bufsize=0, env=sshenv)
-self._pipeo, self._pipei, self._pipee, self._subprocess = sub
-
-self._pipei = util.bufferedinputpipe(self._pipei)
-self._pipei = doublepipe(self.ui, self._pipei, self._pipee)
-self._pipeo = doublepipe(self.ui, self._pipeo, self._pipee)
-
+def _validaterepo(self):
 def badresponse():
 msg = _("no suitable response from remote hg")
 hint = self.ui.config("ui", "ssherrorhint")
@@ -380,6 +383,9 @@
 if res != 0:
 raise error.RepoError(_('could not create remote repo'))
 
-sshstate = (sshcmd, args, remotecmd, sshenv)
+proc, stdin, stdout, stderr = _makeconnection(ui, sshcmd, args, remotecmd,
+  remotepath, sshenv)
+
+sshstate = (proc, stdout, stdin, stderr)
 
 return sshpeer(ui, path, create=create, sshstate=sshstate)



To: indygreg, #hg-reviewers, lothiraldan
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2032: sshpeer: clean up API for sshpeer.__init__ (API)

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5233.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2032?vs=5190=5233

REVISION DETAIL
  https://phab.mercurial-scm.org/D2032

AFFECTED FILES
  mercurial/sshpeer.py
  tests/test-check-interfaces.py

CHANGE DETAILS

diff --git a/tests/test-check-interfaces.py b/tests/test-check-interfaces.py
--- a/tests/test-check-interfaces.py
+++ b/tests/test-check-interfaces.py
@@ -69,8 +69,8 @@
 checkobject(badpeer())
 checkobject(httppeer.httppeer(ui, 'http://localhost'))
 checkobject(localrepo.localpeer(dummyrepo()))
-checkobject(testingsshpeer(ui, 'ssh://localhost/foo', False,
-   (None, None, None, None)))
+checkobject(testingsshpeer(ui, 'ssh://localhost/foo', None, None, None,
+   None))
 checkobject(bundlerepo.bundlepeer(dummyrepo()))
 checkobject(statichttprepo.statichttppeer(dummyrepo()))
 checkobject(unionrepo.unionpeer(dummyrepo()))
diff --git a/mercurial/sshpeer.py b/mercurial/sshpeer.py
--- a/mercurial/sshpeer.py
+++ b/mercurial/sshpeer.py
@@ -157,12 +157,21 @@
 return proc, stdin, stdout, stderr
 
 class sshpeer(wireproto.wirepeer):
-def __init__(self, ui, path, create=False, sshstate=None):
-self._url = path
+def __init__(self, ui, url, proc, stdin, stdout, stderr):
+"""Create a peer from an existing SSH connection.
+
+``proc`` is a handle on the underlying SSH process.
+``stdin``, ``stdout``, and ``stderr`` are handles on the stdio
+pipes for that process.
+"""
+self._url = url
 self._ui = ui
 # self._subprocess is unused. Keeping a handle on the process
 # holds a reference and prevents it from being garbage collected.
-self._subprocess, self._pipei, self._pipeo, self._pipee = sshstate
+self._subprocess = proc
+self._pipeo = stdin
+self._pipei = stdout
+self._pipee = stderr
 
 self._validaterepo()
 
@@ -386,6 +395,4 @@
 proc, stdin, stdout, stderr = _makeconnection(ui, sshcmd, args, remotecmd,
   remotepath, sshenv)
 
-sshstate = (proc, stdout, stdin, stderr)
-
-return sshpeer(ui, path, create=create, sshstate=sshstate)
+return sshpeer(ui, path, proc, stdin, stdout, stderr)



To: indygreg, #hg-reviewers, lothiraldan
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2035: sshpeer: document the handshake mechanism

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5236.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2035?vs=5230=5236

REVISION DETAIL
  https://phab.mercurial-scm.org/D2035

AFFECTED FILES
  mercurial/sshpeer.py

CHANGE DETAILS

diff --git a/mercurial/sshpeer.py b/mercurial/sshpeer.py
--- a/mercurial/sshpeer.py
+++ b/mercurial/sshpeer.py
@@ -162,6 +162,37 @@
 hint = ui.config('ui', 'ssherrorhint')
 raise error.RepoError(msg, hint=hint)
 
+# The handshake consists of sending 2 wire protocol commands:
+# ``hello`` and ``between``.
+#
+# The ``hello`` command (which was introduced in Mercurial 0.9.1)
+# instructs the server to advertise its capabilities.
+#
+# The ``between`` command (which has existed in all Mercurial servers
+# for as long as SSH support has existed), asks for the set of revisions
+# between a pair of revisions.
+#
+# The ``between`` command is issued with a request for the null
+# range. If the remote is a Mercurial server, this request will
+# generate a specific response: ``1\n\n``. This represents the
+# wire protocol encoded value for ``\n``. We look for ``1\n\n``
+# in the output stream and know this is the response to ``between``
+# and we're at the end of our handshake reply.
+#
+# The response to the ``hello`` command will be a line with the
+# length of the value returned by that command followed by that
+# value. If the server doesn't support ``hello`` (which should be
+# rare), that line will be ``0\n``. Otherwise, the value will contain
+# RFC 822 like lines. Of these, the ``capabilities:`` line contains
+# the capabilities of the server.
+#
+# In addition to the responses to our command requests, the server
+# may emit "banner" output on stdout. SSH servers are allowed to
+# print messages to stdout on login. Issuing commands on connection
+# allows us to flush this banner output from the server by scanning
+# for output to our well-known ``between`` command. Of course, if
+# the banner contains ``1\n\n``, this will throw off our detection.
+
 requestlog = ui.configbool('devel', 'debug.peer-request')
 
 try:
@@ -205,6 +236,8 @@
 
 caps = set()
 for l in reversed(lines):
+# Look for response to ``hello`` command. Scan from the back so
+# we don't misinterpret banner output as the command reply.
 if l.startswith('capabilities:'):
 caps.update(l[:-1].split(':')[1].split())
 break



To: indygreg, #hg-reviewers, lothiraldan
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2034: sshpeer: move handshake outside of sshpeer

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5235.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2034?vs=5229=5235

REVISION DETAIL
  https://phab.mercurial-scm.org/D2034

AFFECTED FILES
  mercurial/sshpeer.py
  tests/sshprotoext.py
  tests/test-check-interfaces.py
  tests/test-ssh-proto.t

CHANGE DETAILS

diff --git a/tests/test-ssh-proto.t b/tests/test-ssh-proto.t
--- a/tests/test-ssh-proto.t
+++ b/tests/test-ssh-proto.t
@@ -146,7 +146,6 @@
 
   $ hg --config sshpeer.mode=extra-handshake-commands --config 
sshpeer.handshake-mode=pre-no-args --debug debugpeer ssh://user@dummy/server
   running * "*/tests/dummyssh" 'user@dummy' 'hg -R server serve --stdio' (glob)
-  devel-peer-request: no-args
   sending no-args command
   devel-peer-request: hello
   sending hello command
@@ -182,11 +181,8 @@
 
   $ hg --config sshpeer.mode=extra-handshake-commands --config 
sshpeer.handshake-mode=pre-multiple-no-args --debug debugpeer 
ssh://user@dummy/server
   running * "*/tests/dummyssh" 'user@dummy' 'hg -R server serve --stdio' (glob)
-  devel-peer-request: unknown1
   sending unknown1 command
-  devel-peer-request: unknown2
   sending unknown2 command
-  devel-peer-request: unknown3
   sending unknown3 command
   devel-peer-request: hello
   sending hello command
diff --git a/tests/test-check-interfaces.py b/tests/test-check-interfaces.py
--- a/tests/test-check-interfaces.py
+++ b/tests/test-check-interfaces.py
@@ -51,10 +51,6 @@
 pass
 
 # Facilitates testing sshpeer without requiring an SSH server.
-class testingsshpeer(sshpeer.sshpeer):
-def _validaterepo(self, *args, **kwargs):
-pass
-
 class badpeer(httppeer.httppeer):
 def __init__(self):
 super(badpeer, self).__init__(uimod.ui(), 'http://localhost')
@@ -69,8 +65,8 @@
 checkobject(badpeer())
 checkobject(httppeer.httppeer(ui, 'http://localhost'))
 checkobject(localrepo.localpeer(dummyrepo()))
-checkobject(testingsshpeer(ui, 'ssh://localhost/foo', None, None, None,
-   None))
+checkobject(sshpeer.sshpeer(ui, 'ssh://localhost/foo', None, None, None,
+   None, None))
 checkobject(bundlerepo.bundlepeer(dummyrepo()))
 checkobject(statichttprepo.statichttppeer(dummyrepo()))
 checkobject(unionrepo.unionpeer(dummyrepo()))
diff --git a/tests/sshprotoext.py b/tests/sshprotoext.py
--- a/tests/sshprotoext.py
+++ b/tests/sshprotoext.py
@@ -12,6 +12,7 @@
 
 from mercurial import (
 error,
+extensions,
 registrar,
 sshpeer,
 wireproto,
@@ -52,30 +53,26 @@
 
 super(prehelloserver, self).serve_forever()
 
-class extrahandshakecommandspeer(sshpeer.sshpeer):
-"""An ssh peer that sends extra commands as part of initial handshake."""
-def _validaterepo(self):
-mode = self._ui.config(b'sshpeer', b'handshake-mode')
-if mode == b'pre-no-args':
-self._callstream(b'no-args')
-return super(extrahandshakecommandspeer, self)._validaterepo()
-elif mode == b'pre-multiple-no-args':
-self._callstream(b'unknown1')
-self._callstream(b'unknown2')
-self._callstream(b'unknown3')
-return super(extrahandshakecommandspeer, self)._validaterepo()
-else:
-raise error.ProgrammingError(b'unknown HANDSHAKECOMMANDMODE: %s' %
- mode)
-
-def registercommands():
-def dummycommand(repo, proto):
-raise error.ProgrammingError('this should never be called')
-
-wireproto.wireprotocommand(b'no-args', b'')(dummycommand)
-wireproto.wireprotocommand(b'unknown1', b'')(dummycommand)
-wireproto.wireprotocommand(b'unknown2', b'')(dummycommand)
-wireproto.wireprotocommand(b'unknown3', b'')(dummycommand)
+def performhandshake(orig, ui, stdin, stdout, stderr):
+"""Wrapped version of sshpeer._performhandshake to send extra commands."""
+mode = ui.config(b'sshpeer', b'handshake-mode')
+if mode == b'pre-no-args':
+ui.debug(b'sending no-args command\n')
+stdin.write(b'no-args\n')
+stdin.flush()
+return orig(ui, stdin, stdout, stderr)
+elif mode == b'pre-multiple-no-args':
+ui.debug(b'sending unknown1 command\n')
+stdin.write(b'unknown1\n')
+ui.debug(b'sending unknown2 command\n')
+stdin.write(b'unknown2\n')
+ui.debug(b'sending unknown3 command\n')
+stdin.write(b'unknown3\n')
+stdin.flush()
+return orig(ui, stdin, stdout, stderr)
+else:
+raise error.ProgrammingError(b'unknown HANDSHAKECOMMANDMODE: %s' %
+ mode)
 
 def extsetup(ui):
 # It's easier for tests to define the server behavior via environment
@@ -94,7 +91,6 @@
 peermode = ui.config(b'sshpeer', b'mode')
 
 if peermode == b'extra-handshake-commands':
-sshpeer.sshpeer = extrahandshakecommandspeer
-registercommands()
+   

D2036: sshpeer: remove support for connecting to <0.9.1 servers (BC)

[indygreg (Gregory Szorc)]

indygreg updated this revision to Diff 5237.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2036?vs=5231=5237

REVISION DETAIL
  https://phab.mercurial-scm.org/D2036

AFFECTED FILES
  mercurial/sshpeer.py
  tests/test-ssh-proto.t

CHANGE DETAILS

diff --git a/tests/test-ssh-proto.t b/tests/test-ssh-proto.t
--- a/tests/test-ssh-proto.t
+++ b/tests/test-ssh-proto.t
@@ -109,7 +109,9 @@
   1
   
 
-Connecting to a <0.9.1 server that doesn't support the hello command
+Connecting to a <0.9.1 server that doesn't support the hello command.
+The client should refuse, as we dropped support for connecting to such
+servers.
 
   $ SSHSERVERMODE=no-hello hg --debug debugpeer ssh://user@dummy/server
   running * "*/tests/dummyssh" 'user@dummy' 'hg -R server serve --stdio' (glob)
@@ -120,14 +122,8 @@
   sending between command
   remote: 0
   remote: 1
-  url: ssh://user@dummy/server
-  local: no
-  pushable: yes
-
-The client should interpret this as no capabilities
-
-  $ SSHSERVERMODE=no-hello hg debugcapabilities ssh://user@dummy/server
-  Main capabilities:
+  abort: no suitable response from remote hg!
+  [255]
 
 Sending an unknown command to the server results in an empty response to that 
command
 
diff --git a/mercurial/sshpeer.py b/mercurial/sshpeer.py
--- a/mercurial/sshpeer.py
+++ b/mercurial/sshpeer.py
@@ -242,6 +242,16 @@
 caps.update(l[:-1].split(':')[1].split())
 break
 
+# Error if we couldn't find a response to ``hello``. This could
+# mean:
+#
+# 1. Remote isn't a Mercurial server
+# 2. Remote is a <0.9.1 Mercurial server
+# 3. Remote is a future Mercurial server that dropped ``hello``
+#support.
+if not caps:
+badresponse()
+
 return caps
 
 class sshpeer(wireproto.wirepeer):



To: indygreg, #hg-reviewers, lothiraldan
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1973: bdiff: write a native version of splitnewlines

[indygreg (Gregory Szorc)]

indygreg accepted this revision.
indygreg added a comment.


  I'm happy with this as a first revision.
  
  While I'm accepting as hg-reviewers, I think C code should have an extra set 
of eyes. So I'll defer to @yuja to queue it.
  
  For the record, I'm no fan of not having braces for all bodies of 
conditionals. Can't wait to globally reformat our code to fix that.

INLINE COMMENTS

> yuja wrote in bdiff.c:185
> Nit: `static bool sliceintolist(`

This doesn't need to be static. I'd declare it as `inline bool sliceintolist(`.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1973

To: durin42, #hg-reviewers, indygreg, yuja
Cc: indygreg, yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2178: pathencode: allow clang-format oversight

[indygreg (Gregory Szorc)]

indygreg accepted this revision.
indygreg added inline comments.
This revision is now accepted and ready to land.

INLINE COMMENTS

> pathencode.c:129-130
>   charcopy(dest, , destsize, src[i++]);
> - }
> - else state = DDEFAULT;
> + } else
> + state = DDEFAULT;
>   break;

I actually prefer the old style. But the good thing about using clang-format is 
we can bikeshed on the style later and mass rewrite things after that debate 
has concluded with minimal effort. So getting things to clang format is the 
important goal here, not bikeshedding about the style it is using today.

> pathencode.c:283-284
>   switch (src[i]) {
> - case '1': case '2': case '3': case '4': case '5':
> - case '6': case '7': case '8': case '9':
>   state = COMLPTn;

/me screams

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2178

To: durin42, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2181: charencode: allow clang-format oversight

[indygreg (Gregory Szorc)]

indygreg requested changes to this revision.
indygreg added a comment.
This revision now requires changes to proceed.


  I want a second opinion about the `#include` order.

INLINE COMMENTS

> charencode.h:11-12
>  
> +#include "compat.h"
>  #include 
>  

Huh? Why is it putting a system include after a local include? This feels wrong 
to me.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2181

To: durin42, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1974: narrow: import experimental extension from narrowhg revision cb51d673e9c5

[indygreg (Gregory Szorc)]

indygreg accepted this revision.
indygreg added a comment.
This revision is now accepted and ready to land.


  I think I've seen enough follow-ups to feel comfortable taking this in core. 
There's still a ton of work that needs to get done. But it will be easier to 
iterate and for others to get involved when the code is committed than when it 
is sitting around in review.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1974

To: durin42, #hg-reviewers, indygreg
Cc: idlsoft, martinvonz, indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2093: tests: add tests for sending recognized command before handshake

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG465858451347: tests: add tests for sending recognized 
command before handshake (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2093?vs=5354=5537

REVISION DETAIL
  https://phab.mercurial-scm.org/D2093

AFFECTED FILES
  tests/test-ssh-proto.t

CHANGE DETAILS

diff --git a/tests/test-ssh-proto.t b/tests/test-ssh-proto.t
--- a/tests/test-ssh-proto.t
+++ b/tests/test-ssh-proto.t
@@ -394,6 +394,33 @@
   0
   0
 
+Send a valid command before the handshake
+
+  $ hg -R server serve --stdio << EOF
+  > heads
+  > hello
+  > between
+  > pairs 81
+  > 
-
+  > EOF
+  41
+  68986213bd4485ea51533535e3fc9e78007a711f
+  384
+  capabilities: lookup changegroupsubset branchmap pushkey known getbundle 
unbundlehash batch streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ 
unbundle=HG10GZ,HG10BZ,HG10UN
+  1
+  
+
+And a variation that doesn't send the between command
+
+  $ hg -R server serve --stdio << EOF
+  > heads
+  > hello
+  > EOF
+  41
+  68986213bd4485ea51533535e3fc9e78007a711f
+  384
+  capabilities: lookup changegroupsubset branchmap pushkey known getbundle 
unbundlehash batch streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ 
unbundle=HG10GZ,HG10BZ,HG10UN
+
 Send an upgrade request to a server that doesn't support that command
 
   $ hg -R server serve --stdio << EOF



To: indygreg, #hg-reviewers, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2094: wireprotoserver: define and use parse_qs from urllib

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGa3d42d1865f1: wireprotoserver: define and use parse_qs from 
urllib (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2094?vs=5355=5538

REVISION DETAIL
  https://phab.mercurial-scm.org/D2094

AFFECTED FILES
  mercurial/urllibcompat.py
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -7,7 +7,6 @@
 from __future__ import absolute_import
 
 import abc
-import cgi
 import contextlib
 import struct
 import sys
@@ -134,12 +133,12 @@
 args = util.rapply(pycompat.bytesurl, self._req.form.copy())
 postlen = int(self._req.env.get(r'HTTP_X_HGARGS_POST', 0))
 if postlen:
-args.update(cgi.parse_qs(
+args.update(urlreq.parseqs(
 self._req.read(postlen), keep_blank_values=True))
 return args
 
 argvalue = decodevaluefromheaders(self._req, r'X-HgArg')
-args.update(cgi.parse_qs(argvalue, keep_blank_values=True))
+args.update(urlreq.parseqs(argvalue, keep_blank_values=True))
 return args
 
 def forwardpayload(self, fp):
diff --git a/mercurial/urllibcompat.py b/mercurial/urllibcompat.py
--- a/mercurial/urllibcompat.py
+++ b/mercurial/urllibcompat.py
@@ -47,6 +47,7 @@
 "urlparse",
 "urlunparse",
 ))
+urlreq._registeralias(urllib.parse, "parse_qs", "parseqs")
 urlreq._registeralias(urllib.parse, "unquote_to_bytes", "unquote")
 import urllib.request
 urlreq._registeraliases(urllib.request, (
@@ -157,6 +158,7 @@
 "urlparse",
 "urlunparse",
 ))
+urlreq._registeralias(urlparse, "parse_qs", "parseqs")
 urlerr._registeraliases(urllib2, (
 "HTTPError",
 "URLError",



To: indygreg, #hg-reviewers, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2108: infinitepush: drop the `--to` flag to push and use `-B` instead

[indygreg (Gregory Szorc)]

indygreg added a comment.


  In https://phab.mercurial-scm.org/D2108#36335, @durham wrote:
  
  > > There are things which I am not sure whether to keep or not:
  > > 
  > > - the --bundle-store flag to push command
  >
  > This is useful for scripts or tools that want to upload a commit to the 
cloud without having to give it a name. For instance, you can use it to push a 
commit then send that commit hash to some build service which can checkout the 
commit without having to worry about a bookmark name.  But this could always be 
added back later, so it's probably fine to drop it if there's not an immediate 
need in Mozilla's use case.
  
  
  To be clear, Mozilla has 2 use cases where infinitepush could be useful:
  
  1. For our Try repository. Upload a nameless bundle somewhere and CI consumes 
it.
  2. For user repositories (basically forks of the main repos).

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2108

To: pulkit, #hg-reviewers, indygreg
Cc: indygreg, durham, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2088: wireprototypes: move wire protocol response types to new module

[indygreg (Gregory Szorc)]

indygreg added a comment.


  In https://phab.mercurial-scm.org/D2088#36287, @durin42 wrote:
  
  > I wonder if these should move to being attrs-generated at some point.
  
  
  Probably. I'm half considering blowing up all these types because they are... 
not well-defined and behavior is overloaded. I'll definitely be creating new 
types for version 2 of the wire protocol. I'm waiting on that code to come into 
existence before touching these types because I'm not yet sure how everything 
will pan out.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2088

To: indygreg, #hg-reviewers, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2082: wireproto: use maybecapturestdio() for push responses (API)

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGcaca3ac2ac04: wireproto: use maybecapturestdio() for push 
responses (API) (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2082?vs=5345=5527

REVISION DETAIL
  https://phab.mercurial-scm.org/D2082

AFFECTED FILES
  hgext/largefiles/proto.py
  mercurial/wireproto.py
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -320,15 +320,13 @@
 req.respond(HTTP_OK, mediatype)
 return gen
 elif isinstance(rsp, wireproto.pushres):
-val = proto.restore()
-rsp = '%d\n%s' % (rsp.res, val)
+rsp = '%d\n%s' % (rsp.res, rsp.output)
 req.respond(HTTP_OK, HGTYPE, body=rsp)
 return []
 elif isinstance(rsp, wireproto.pusherr):
 # This is the httplib workaround documented in _handlehttperror().
 req.drain()
 
-proto.restore()
 rsp = '0\n%s\n' % rsp.res
 req.respond(HTTP_OK, HGTYPE, body=rsp)
 return []
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -510,16 +510,18 @@
 
 The call was successful and returned an integer contained in `self.res`.
 """
-def __init__(self, res):
+def __init__(self, res, output):
 self.res = res
+self.output = output
 
 class pusherr(object):
 """wireproto reply: failure
 
 The call failed. The `self.res` attribute contains the error message.
 """
-def __init__(self, res):
+def __init__(self, res, output):
 self.res = res
+self.output = output
 
 class ooberror(object):
 """wireproto reply: failure of a batch of operation
@@ -997,97 +999,98 @@
 def unbundle(repo, proto, heads):
 their_heads = decodelist(heads)
 
-try:
-proto.redirect()
-
-exchange.check_heads(repo, their_heads, 'preparing changes')
-
-# write bundle data to temporary file because it can be big
-fd, tempname = tempfile.mkstemp(prefix='hg-unbundle-')
-fp = os.fdopen(fd, pycompat.sysstr('wb+'))
-r = 0
+with proto.mayberedirectstdio() as output:
 try:
-proto.getfile(fp)
-fp.seek(0)
-gen = exchange.readbundle(repo.ui, fp, None)
-if (isinstance(gen, changegroupmod.cg1unpacker)
-and not bundle1allowed(repo, 'push')):
-if proto.name == 'http':
-# need to special case http because stderr do not get to
-# the http client on failed push so we need to abuse some
-# other error type to make sure the message get to the
-# user.
-return ooberror(bundle2required)
-raise error.Abort(bundle2requiredmain,
-  hint=bundle2requiredhint)
+exchange.check_heads(repo, their_heads, 'preparing changes')
 
-r = exchange.unbundle(repo, gen, their_heads, 'serve',
-  proto._client())
-if util.safehasattr(r, 'addpart'):
-# The return looks streamable, we are in the bundle2 case and
-# should return a stream.
-return streamres_legacy(gen=r.getchunks())
-return pushres(r)
-
-finally:
-fp.close()
-os.unlink(tempname)
-
-except (error.BundleValueError, error.Abort, error.PushRaced) as exc:
-# handle non-bundle2 case first
-if not getattr(exc, 'duringunbundle2', False):
+# write bundle data to temporary file because it can be big
+fd, tempname = tempfile.mkstemp(prefix='hg-unbundle-')
+fp = os.fdopen(fd, pycompat.sysstr('wb+'))
+r = 0
 try:
-raise
-except error.Abort:
-# The old code we moved used util.stderr directly.
-# We did not change it to minimise code change.
-# This need to be moved to something proper.
-# Feel free to do it.
-util.stderr.write("abort: %s\n" % exc)
-if exc.hint is not None:
-util.stderr.write("(%s)\n" % exc.hint)
-return pushres(0)
-except error.PushRaced:
-return pusherr(str(exc))
+proto.getfile(fp)
+fp.seek(0)
+gen = exchange.readbundle(repo.ui, fp, None)
+if (isinstance(gen, changegroupmod.cg1unpacker)
+and not bundle1allowed(repo, 'push')):
+if proto.name == 'http':
+# need to special case http because stderr do not get 
to
+

D2089: wireproto: introduce type for raw byte responses (API)

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG2f7290555c96: wireproto: introduce type for raw byte 
responses (API) (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2089?vs=5352=5534

REVISION DETAIL
  https://phab.mercurial-scm.org/D2089

AFFECTED FILES
  hgext/largefiles/proto.py
  mercurial/wireproto.py
  mercurial/wireprotoserver.py
  mercurial/wireprototypes.py
  tests/sshprotoext.py
  tests/test-wireproto.py

CHANGE DETAILS

diff --git a/tests/test-wireproto.py b/tests/test-wireproto.py
--- a/tests/test-wireproto.py
+++ b/tests/test-wireproto.py
@@ -1,8 +1,10 @@
 from __future__ import absolute_import, print_function
 
 from mercurial import (
+error,
 util,
 wireproto,
+wireprototypes,
 )
 stringio = util.stringio
 
@@ -42,7 +44,13 @@
 return ['batch']
 
 def _call(self, cmd, **args):
-return wireproto.dispatch(self.serverrepo, proto(args), cmd)
+res = wireproto.dispatch(self.serverrepo, proto(args), cmd)
+if isinstance(res, wireprototypes.bytesresponse):
+return res.data
+elif isinstance(res, bytes):
+return res
+else:
+raise error.Abort('dummy client does not support response type')
 
 def _callstream(self, cmd, **args):
 return stringio(self._call(cmd, **args))
diff --git a/tests/sshprotoext.py b/tests/sshprotoext.py
--- a/tests/sshprotoext.py
+++ b/tests/sshprotoext.py
@@ -49,7 +49,7 @@
 l = self._fin.readline()
 assert l == b'between\n'
 rsp = wireproto.dispatch(self._repo, self._proto, b'between')
-wireprotoserver._sshv1respondbytes(self._fout, rsp)
+wireprotoserver._sshv1respondbytes(self._fout, rsp.data)
 
 super(prehelloserver, self).serve_forever()
 
@@ -74,7 +74,7 @@
 # Send the upgrade response.
 self._fout.write(b'upgraded %s %s\n' % (token, name))
 servercaps = wireproto.capabilities(self._repo, self._proto)
-rsp = b'capabilities: %s' % servercaps
+rsp = b'capabilities: %s' % servercaps.data
 self._fout.write(b'%d\n' % len(rsp))
 self._fout.write(rsp)
 self._fout.write(b'\n')
diff --git a/mercurial/wireprototypes.py b/mercurial/wireprototypes.py
--- a/mercurial/wireprototypes.py
+++ b/mercurial/wireprototypes.py
@@ -5,6 +5,11 @@
 
 from __future__ import absolute_import
 
+class bytesresponse(object):
+"""A wire protocol response consisting of raw bytes."""
+def __init__(self, data):
+self.data = data
+
 class ooberror(object):
 """wireproto reply: failure of a batch of operation
 
diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -274,6 +274,9 @@
 if isinstance(rsp, bytes):
 req.respond(HTTP_OK, HGTYPE, body=rsp)
 return []
+elif isinstance(rsp, wireprototypes.bytesresponse):
+req.respond(HTTP_OK, HGTYPE, body=rsp.data)
+return []
 elif isinstance(rsp, wireprototypes.streamreslegacy):
 gen = rsp.gen
 req.respond(HTTP_OK, HGTYPE)
@@ -435,6 +438,8 @@
 
 if isinstance(rsp, bytes):
 _sshv1respondbytes(self._fout, rsp)
+elif isinstance(rsp, wireprototypes.bytesresponse):
+_sshv1respondbytes(self._fout, rsp.data)
 elif isinstance(rsp, wireprototypes.streamres):
 _sshv1respondstream(self._fout, rsp)
 elif isinstance(rsp, wireprototypes.streamreslegacy):
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -37,6 +37,7 @@
 urlerr = util.urlerr
 urlreq = util.urlreq
 
+bytesresponse = wireprototypes.bytesresponse
 ooberror = wireprototypes.ooberror
 pushres = wireprototypes.pushres
 pusherr = wireprototypes.pusherr
@@ -696,16 +697,24 @@
 result = func(repo, proto)
 if isinstance(result, ooberror):
 return result
+
+# For now, all batchable commands must return bytesresponse or
+# raw bytes (for backwards compatibility).
+assert isinstance(result, (bytesresponse, bytes))
+if isinstance(result, bytesresponse):
+result = result.data
 res.append(escapearg(result))
-return ';'.join(res)
+
+return bytesresponse(';'.join(res))
 
 @wireprotocommand('between', 'pairs')
 def between(repo, proto, pairs):
 pairs = [decodelist(p, '-') for p in pairs.split(" ")]
 r = []
 for b in repo.between(pairs):
 r.append(encodelist(b) + "\n")
-return "".join(r)
+
+return bytesresponse(''.join(r))
 
 @wireprotocommand('branchmap')
 def branchmap(repo, proto):
@@ -715,15 +724,17 @@
 branchname = urlreq.quote(encoding.fromlocal(branch))
 branchnodes = encodelist(nodes)
 heads.append('%s %s' % 

D2088: wireprototypes: move wire protocol response types to new module

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGcd6ab329c5c7: wireprototypes: move wire protocol response 
types to new module (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2088?vs=5351=5535

REVISION DETAIL
  https://phab.mercurial-scm.org/D2088

AFFECTED FILES
  mercurial/wireproto.py
  mercurial/wireprotoserver.py
  mercurial/wireprototypes.py

CHANGE DETAILS

diff --git a/mercurial/wireprototypes.py b/mercurial/wireprototypes.py
new file mode 100644
--- /dev/null
+++ b/mercurial/wireprototypes.py
@@ -0,0 +1,61 @@
+# Copyright 2018 Gregory Szorc 
+#
+# This software may be used and distributed according to the terms of the
+# GNU General Public License version 2 or any later version.
+
+from __future__ import absolute_import
+
+class ooberror(object):
+"""wireproto reply: failure of a batch of operation
+
+Something failed during a batch call. The error message is stored in
+`self.message`.
+"""
+def __init__(self, message):
+self.message = message
+
+class pushres(object):
+"""wireproto reply: success with simple integer return
+
+The call was successful and returned an integer contained in `self.res`.
+"""
+def __init__(self, res, output):
+self.res = res
+self.output = output
+
+class pusherr(object):
+"""wireproto reply: failure
+
+The call failed. The `self.res` attribute contains the error message.
+"""
+def __init__(self, res, output):
+self.res = res
+self.output = output
+
+class streamres(object):
+"""wireproto reply: binary stream
+
+The call was successful and the result is a stream.
+
+Accepts a generator containing chunks of data to be sent to the client.
+
+``prefer_uncompressed`` indicates that the data is expected to be
+uncompressable and that the stream should therefore use the ``none``
+engine.
+"""
+def __init__(self, gen=None, prefer_uncompressed=False):
+self.gen = gen
+self.prefer_uncompressed = prefer_uncompressed
+
+class streamreslegacy(object):
+"""wireproto reply: uncompressed binary stream
+
+The call was successful and the result is a stream.
+
+Accepts a generator containing chunks of data to be sent to the client.
+
+Like ``streamres``, but sends an uncompressed data for "version 1" clients
+using the application/mercurial-0.1 media type.
+"""
+def __init__(self, gen=None):
+self.gen = gen
diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -20,6 +20,7 @@
 pycompat,
 util,
 wireproto,
+wireprototypes,
 )
 
 stringio = util.stringio
@@ -273,11 +274,11 @@
 if isinstance(rsp, bytes):
 req.respond(HTTP_OK, HGTYPE, body=rsp)
 return []
-elif isinstance(rsp, wireproto.streamres_legacy):
+elif isinstance(rsp, wireprototypes.streamreslegacy):
 gen = rsp.gen
 req.respond(HTTP_OK, HGTYPE)
 return gen
-elif isinstance(rsp, wireproto.streamres):
+elif isinstance(rsp, wireprototypes.streamres):
 gen = rsp.gen
 
 # This code for compression should not be streamres specific. It
@@ -291,18 +292,18 @@
 
 req.respond(HTTP_OK, mediatype)
 return gen
-elif isinstance(rsp, wireproto.pushres):
+elif isinstance(rsp, wireprototypes.pushres):
 rsp = '%d\n%s' % (rsp.res, rsp.output)
 req.respond(HTTP_OK, HGTYPE, body=rsp)
 return []
-elif isinstance(rsp, wireproto.pusherr):
+elif isinstance(rsp, wireprototypes.pusherr):
 # This is the httplib workaround documented in _handlehttperror().
 req.drain()
 
 rsp = '0\n%s\n' % rsp.res
 req.respond(HTTP_OK, HGTYPE, body=rsp)
 return []
-elif isinstance(rsp, wireproto.ooberror):
+elif isinstance(rsp, wireprototypes.ooberror):
 rsp = rsp.message
 req.respond(HTTP_OK, HGERRTYPE, body=rsp)
 return []
@@ -434,16 +435,16 @@
 
 if isinstance(rsp, bytes):
 _sshv1respondbytes(self._fout, rsp)
-elif isinstance(rsp, wireproto.streamres):
+elif isinstance(rsp, wireprototypes.streamres):
 _sshv1respondstream(self._fout, rsp)
-elif isinstance(rsp, wireproto.streamres_legacy):
+elif isinstance(rsp, wireprototypes.streamreslegacy):
 _sshv1respondstream(self._fout, rsp)
-elif isinstance(rsp, wireproto.pushres):
+elif isinstance(rsp, wireprototypes.pushres):
 _sshv1respondbytes(self._fout, b'')
 _sshv1respondbytes(self._fout, bytes(rsp.res))
-elif isinstance(rsp, wireproto.pusherr):
+elif isinstance(rsp, wireprototypes.pusherr):
 

D2092: wireprotoserver: add version to SSH protocol names (API)

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGac33dc94e1d5: wireprotoserver: add version to SSH protocol 
names (API) (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2092?vs=5353=5536

REVISION DETAIL
  https://phab.mercurial-scm.org/D2092

AFFECTED FILES
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -367,7 +367,7 @@
 
 @property
 def name(self):
-return 'ssh'
+return SSHV1
 
 def getargs(self, args):
 data = {}



To: indygreg, #hg-reviewers, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2081: wireprotoserver: add context manager mechanism for redirecting stdio

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG2ad145fbde54: wireprotoserver: add context manager 
mechanism for redirecting stdio (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2081?vs=5344=5528

REVISION DETAIL
  https://phab.mercurial-scm.org/D2081

AFFECTED FILES
  mercurial/wireproto.py
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -8,6 +8,7 @@
 
 import abc
 import cgi
+import contextlib
 import struct
 import sys
 
@@ -74,6 +75,20 @@
 """
 
 @abc.abstractmethod
+def mayberedirectstdio(self):
+"""Context manager to possibly redirect stdio.
+
+The context manager yields a file-object like object that receives
+stdout and stderr output when the context manager is active. Or it
+yields ``None`` if no I/O redirection occurs.
+
+The intent of this context manager is to capture stdio output
+so it may be sent in the response. Some transports support streaming
+stdio to the client in real time. For these transports, stdio output
+won't be captured.
+"""
+
+@abc.abstractmethod
 def redirect(self):
 """may setup interception for stdout and stderr
 
@@ -151,6 +166,21 @@
 for s in util.filechunkiter(self._req, limit=length):
 fp.write(s)
 
+@contextlib.contextmanager
+def mayberedirectstdio(self):
+oldout = self._ui.fout
+olderr = self._ui.ferr
+
+out = util.stringio()
+
+try:
+self._ui.fout = out
+self._ui.ferr = out
+yield out
+finally:
+self._ui.fout = oldout
+self._ui.ferr = olderr
+
 def redirect(self):
 self._oldio = self._ui.fout, self._ui.ferr
 self._ui.ferr = self._ui.fout = stringio()
@@ -393,6 +423,10 @@
 fpout.write(self._fin.read(count))
 count = int(self._fin.readline())
 
+@contextlib.contextmanager
+def mayberedirectstdio(self):
+yield None
+
 def redirect(self):
 pass
 
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -978,20 +978,12 @@
 else:
 new = encoding.tolocal(new) # normal path
 
-if util.safehasattr(proto, 'restore'):
-
-proto.redirect()
-
+with proto.mayberedirectstdio() as output:
 r = repo.pushkey(encoding.tolocal(namespace), encoding.tolocal(key),
  encoding.tolocal(old), new) or False
 
-output = proto.restore()
-
-return '%s\n%s' % (int(r), output)
-
-r = repo.pushkey(encoding.tolocal(namespace), encoding.tolocal(key),
- encoding.tolocal(old), new)
-return '%s\n' % int(r)
+output = output.getvalue() if output else ''
+return '%s\n%s' % (int(r), output)
 
 @wireprotocommand('stream_out')
 def stream(repo, proto):



To: indygreg, #hg-reviewers, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2084: wireprotoserver: rename _client to client (API)

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG957e773614d0: wireprotoserver: rename _client to client 
(API) (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2084?vs=5347=5530

REVISION DETAIL
  https://phab.mercurial-scm.org/D2084

AFFECTED FILES
  mercurial/wireproto.py
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -88,6 +88,10 @@
 won't be captured.
 """
 
+@abc.abstractmethod
+def client(self):
+"""Returns a string representation of this client (as bytes)."""
+
 def decodevaluefromheaders(req, headerprefix):
 """Decode a long value from multiple HTTP request headers.
 
@@ -164,7 +168,7 @@
 self._ui.fout = oldout
 self._ui.ferr = olderr
 
-def _client(self):
+def client(self):
 return 'remote:%s:%s:%s' % (
 self._req.env.get('wsgi.url_scheme') or 'http',
 urlreq.quote(self._req.env.get('REMOTE_HOST', '')),
@@ -399,7 +403,7 @@
 def mayberedirectstdio(self):
 yield None
 
-def _client(self):
+def client(self):
 client = encoding.environ.get('SSH_CLIENT', '').split(' ', 1)[0]
 return 'remote:ssh:' + client
 
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -1023,7 +1023,7 @@
   hint=bundle2requiredhint)
 
 r = exchange.unbundle(repo, gen, their_heads, 'serve',
-  proto._client())
+  proto.client())
 if util.safehasattr(r, 'addpart'):
 # The return looks streamable, we are in the bundle2 case
 # and should return a stream.



To: indygreg, #hg-reviewers, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2085: wireprotoserver: rename getfile() to forwardpayload() (API)

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG90ca4986616c: wireprotoserver: rename getfile() to 
forwardpayload() (API) (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2085?vs=5348=5531

REVISION DETAIL
  https://phab.mercurial-scm.org/D2085

AFFECTED FILES
  hgext/largefiles/proto.py
  mercurial/wireproto.py
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -64,14 +64,10 @@
 returns a list of values (same order as )"""
 
 @abc.abstractmethod
-def getfile(self, fp):
-"""write the whole content of a file into a file like object
+def forwardpayload(self, fp):
+"""Read the raw payload and forward to a file.
 
-The file is in the form::
-
-(\n)+0\n
-
-chunk size is the ascii version of the int.
+The payload is read in full before the function returns.
 """
 
 @abc.abstractmethod
@@ -145,7 +141,7 @@
 args.update(cgi.parse_qs(argvalue, keep_blank_values=True))
 return args
 
-def getfile(self, fp):
+def forwardpayload(self, fp):
 length = int(self._req.env[r'CONTENT_LENGTH'])
 # If httppostargs is used, we need to read Content-Length
 # minus the amount that was consumed by args.
@@ -392,7 +388,12 @@
 data[arg] = val
 return [data[k] for k in keys]
 
-def getfile(self, fpout):
+def forwardpayload(self, fpout):
+# The file is in the form:
+#
+# \n
+# ...
+# 0\n
 _sshv1respondbytes(self._fout, b'')
 count = int(self._fin.readline())
 while count:
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -1008,7 +1008,7 @@
 fp = os.fdopen(fd, pycompat.sysstr('wb+'))
 r = 0
 try:
-proto.getfile(fp)
+proto.forwardpayload(fp)
 fp.seek(0)
 gen = exchange.readbundle(repo.ui, fp, None)
 if (isinstance(gen, changegroupmod.cg1unpacker)
diff --git a/hgext/largefiles/proto.py b/hgext/largefiles/proto.py
--- a/hgext/largefiles/proto.py
+++ b/hgext/largefiles/proto.py
@@ -40,7 +40,7 @@
 tmpfp = util.atomictempfile(path, createmode=repo.store.createmode)
 
 try:
-proto.getfile(tmpfp)
+proto.forwardpayload(tmpfp)
 tmpfp._fp.seek(0)
 if sha != lfutil.hexsha1(tmpfp._fp):
 raise IOError(0, _('largefile contents do not match hash'))



To: indygreg, #hg-reviewers, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2086: wireproto: remove unused proto argument from supportedcompengines (API)

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG038bcb759b75: wireproto: remove unused proto argument from 
supportedcompengines (API) (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2086?vs=5349=5532

REVISION DETAIL
  https://phab.mercurial-scm.org/D2086

AFFECTED FILES
  mercurial/wireproto.py
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -192,7 +192,7 @@
 break
 
 # Now find an agreed upon compression format.
-for engine in wireproto.supportedcompengines(self._ui, self,
+for engine in wireproto.supportedcompengines(self._ui,
  util.SERVERROLE):
 if engine.wireprotosupport().name in compformats:
 opts = {}
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -587,7 +587,7 @@
 
 return ui.configbool('server', 'bundle1')
 
-def supportedcompengines(ui, proto, role):
+def supportedcompengines(ui, role):
 """Obtain the list of supported compression engines for a request."""
 assert role in (util.CLIENTROLE, util.SERVERROLE)
 
@@ -824,7 +824,7 @@
 # FUTURE advertise minrx and mintx after consulting config option
 caps.append('httpmediatype=0.1rx,0.1tx,0.2tx')
 
-compengines = supportedcompengines(repo.ui, proto, util.SERVERROLE)
+compengines = supportedcompengines(repo.ui, util.SERVERROLE)
 if compengines:
 comptypes = ','.join(urlreq.quote(e.wireprotosupport().name)
  for e in compengines)



To: indygreg, #hg-reviewers, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2083: wireprotoserver: remove redirect() and restore() (API)

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG56fe8a3b2d52: wireprotoserver: remove redirect() and 
restore() (API) (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2083?vs=5346=5529

REVISION DETAIL
  https://phab.mercurial-scm.org/D2083

AFFECTED FILES
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -88,23 +88,6 @@
 won't be captured.
 """
 
-@abc.abstractmethod
-def redirect(self):
-"""may setup interception for stdout and stderr
-
-See also the `restore` method."""
-
-# If the `redirect` function does install interception, the `restore`
-# function MUST be defined. If interception is not used, this function
-# MUST NOT be defined.
-#
-# left commented here on purpose
-#
-#def restore(self):
-#"""reinstall previous stdout and stderr and return intercepted stdout
-#"""
-#raise NotImplementedError()
-
 def decodevaluefromheaders(req, headerprefix):
 """Decode a long value from multiple HTTP request headers.
 
@@ -181,15 +164,6 @@
 self._ui.fout = oldout
 self._ui.ferr = olderr
 
-def redirect(self):
-self._oldio = self._ui.fout, self._ui.ferr
-self._ui.ferr = self._ui.fout = stringio()
-
-def restore(self):
-val = self._ui.fout.getvalue()
-self._ui.ferr, self._ui.fout = self._oldio
-return val
-
 def _client(self):
 return 'remote:%s:%s:%s' % (
 self._req.env.get('wsgi.url_scheme') or 'http',
@@ -425,9 +399,6 @@
 def mayberedirectstdio(self):
 yield None
 
-def redirect(self):
-pass
-
 def _client(self):
 client = encoding.environ.get('SSH_CLIENT', '').split(' ', 1)[0]
 return 'remote:ssh:' + client



To: indygreg, #hg-reviewers, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2091: wireprotoserver: extract SSH response handling functions

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG5767664d39a5: wireprotoserver: extract SSH response 
handling functions (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2091?vs=5342=5525

REVISION DETAIL
  https://phab.mercurial-scm.org/D2091

AFFECTED FILES
  mercurial/wireprotoserver.py
  tests/sshprotoext.py

CHANGE DETAILS

diff --git a/tests/sshprotoext.py b/tests/sshprotoext.py
--- a/tests/sshprotoext.py
+++ b/tests/sshprotoext.py
@@ -45,11 +45,11 @@
 l = self._fin.readline()
 assert l == b'hello\n'
 # Respond to unknown commands with an empty reply.
-self._sendresponse(b'')
+wireprotoserver._sshv1respondbytes(self._fout, b'')
 l = self._fin.readline()
 assert l == b'between\n'
 rsp = wireproto.dispatch(self._repo, self, b'between')
-self._handlers[rsp.__class__](self, rsp)
+wireprotoserver._sshv1respondbytes(self._fout, rsp)
 
 super(prehelloserver, self).serve_forever()
 
diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -336,6 +336,24 @@
 
 return ''
 
+def _sshv1respondbytes(fout, value):
+"""Send a bytes response for protocol version 1."""
+fout.write('%d\n' % len(value))
+fout.write(value)
+fout.flush()
+
+def _sshv1respondstream(fout, source):
+write = fout.write
+for chunk in source.gen:
+write(chunk)
+fout.flush()
+
+def _sshv1respondooberror(fout, ferr, rsp):
+ferr.write(b'%s\n-\n' % rsp)
+ferr.flush()
+fout.write(b'\n')
+fout.flush()
+
 class sshserver(baseprotocolhandler):
 def __init__(self, ui, repo):
 self._ui = ui
@@ -376,60 +394,43 @@
 return [data[k] for k in keys]
 
 def getfile(self, fpout):
-self._sendresponse('')
+_sshv1respondbytes(self._fout, b'')
 count = int(self._fin.readline())
 while count:
 fpout.write(self._fin.read(count))
 count = int(self._fin.readline())
 
 def redirect(self):
 pass
 
-def _sendresponse(self, v):
-self._fout.write("%d\n" % len(v))
-self._fout.write(v)
-self._fout.flush()
-
-def _sendstream(self, source):
-write = self._fout.write
-for chunk in source.gen:
-write(chunk)
-self._fout.flush()
-
-def _sendpushresponse(self, rsp):
-self._sendresponse('')
-self._sendresponse(str(rsp.res))
-
-def _sendpusherror(self, rsp):
-self._sendresponse(rsp.res)
-
-def _sendooberror(self, rsp):
-self._ui.ferr.write('%s\n-\n' % rsp.message)
-self._ui.ferr.flush()
-self._fout.write('\n')
-self._fout.flush()
-
 def serve_forever(self):
 while self.serve_one():
 pass
 sys.exit(0)
 
-_handlers = {
-str: _sendresponse,
-wireproto.streamres: _sendstream,
-wireproto.streamres_legacy: _sendstream,
-wireproto.pushres: _sendpushresponse,
-wireproto.pusherr: _sendpusherror,
-wireproto.ooberror: _sendooberror,
-}
-
 def serve_one(self):
 cmd = self._fin.readline()[:-1]
 if cmd and wireproto.commands.commandavailable(cmd, self):
 rsp = wireproto.dispatch(self._repo, self, cmd)
-self._handlers[rsp.__class__](self, rsp)
+
+if isinstance(rsp, bytes):
+_sshv1respondbytes(self._fout, rsp)
+elif isinstance(rsp, wireproto.streamres):
+_sshv1respondstream(self._fout, rsp)
+elif isinstance(rsp, wireproto.streamres_legacy):
+_sshv1respondstream(self._fout, rsp)
+elif isinstance(rsp, wireproto.pushres):
+_sshv1respondbytes(self._fout, b'')
+_sshv1respondbytes(self._fout, bytes(rsp.res))
+elif isinstance(rsp, wireproto.pusherr):
+_sshv1respondbytes(self._fout, rsp.res)
+elif isinstance(rsp, wireproto.ooberror):
+_sshv1respondooberror(self._fout, self._ui.ferr, rsp.message)
+else:
+raise error.ProgrammingError('unhandled response type from '
+ 'wire protocol command: %s' % rsp)
 elif cmd:
-self._sendresponse("")
+_sshv1respondbytes(self._fout, b'')
 return cmd != ''
 
 def _client(self):



To: indygreg, #hg-reviewers, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2080: wireprotoserver: split ssh protocol handler and server

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGbf676267f64f: wireprotoserver: split ssh protocol handler 
and server (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2080?vs=5343=5526

REVISION DETAIL
  https://phab.mercurial-scm.org/D2080

AFFECTED FILES
  mercurial/wireprotoserver.py
  tests/sshprotoext.py
  tests/test-sshserver.py

CHANGE DETAILS

diff --git a/tests/test-sshserver.py b/tests/test-sshserver.py
--- a/tests/test-sshserver.py
+++ b/tests/test-sshserver.py
@@ -24,7 +24,7 @@
 def assertparse(self, cmd, input, expected):
 server = mockserver(input)
 _func, spec = wireproto.commands[cmd]
-self.assertEqual(server.getargs(spec), expected)
+self.assertEqual(server._proto.getargs(spec), expected)
 
 def mockserver(inbytes):
 ui = mockui(inbytes)
diff --git a/tests/sshprotoext.py b/tests/sshprotoext.py
--- a/tests/sshprotoext.py
+++ b/tests/sshprotoext.py
@@ -48,7 +48,7 @@
 wireprotoserver._sshv1respondbytes(self._fout, b'')
 l = self._fin.readline()
 assert l == b'between\n'
-rsp = wireproto.dispatch(self._repo, self, b'between')
+rsp = wireproto.dispatch(self._repo, self._proto, b'between')
 wireprotoserver._sshv1respondbytes(self._fout, rsp)
 
 super(prehelloserver, self).serve_forever()
@@ -73,7 +73,7 @@
 
 # Send the upgrade response.
 self._fout.write(b'upgraded %s %s\n' % (token, name))
-servercaps = wireproto.capabilities(self._repo, self)
+servercaps = wireproto.capabilities(self._repo, self._proto)
 rsp = b'capabilities: %s' % servercaps
 self._fout.write(b'%d\n' % len(rsp))
 self._fout.write(rsp)
diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -354,19 +354,12 @@
 fout.write(b'\n')
 fout.flush()
 
-class sshserver(baseprotocolhandler):
-def __init__(self, ui, repo):
+class sshv1protocolhandler(baseprotocolhandler):
+"""Handler for requests services via version 1 of SSH protocol."""
+def __init__(self, ui, fin, fout):
 self._ui = ui
-self._repo = repo
-self._fin = ui.fin
-self._fout = ui.fout
-
-hook.redirect(True)
-ui.fout = repo.ui.fout = ui.ferr
-
-# Prevent insertion/deletion of CRs
-util.setbinary(self._fin)
-util.setbinary(self._fout)
+self._fin = fin
+self._fout = fout
 
 @property
 def name(self):
@@ -403,15 +396,35 @@
 def redirect(self):
 pass
 
+def _client(self):
+client = encoding.environ.get('SSH_CLIENT', '').split(' ', 1)[0]
+return 'remote:ssh:' + client
+
+class sshserver(object):
+def __init__(self, ui, repo):
+self._ui = ui
+self._repo = repo
+self._fin = ui.fin
+self._fout = ui.fout
+
+hook.redirect(True)
+ui.fout = repo.ui.fout = ui.ferr
+
+# Prevent insertion/deletion of CRs
+util.setbinary(self._fin)
+util.setbinary(self._fout)
+
+self._proto = sshv1protocolhandler(self._ui, self._fin, self._fout)
+
 def serve_forever(self):
 while self.serve_one():
 pass
 sys.exit(0)
 
 def serve_one(self):
 cmd = self._fin.readline()[:-1]
-if cmd and wireproto.commands.commandavailable(cmd, self):
-rsp = wireproto.dispatch(self._repo, self, cmd)
+if cmd and wireproto.commands.commandavailable(cmd, self._proto):
+rsp = wireproto.dispatch(self._repo, self._proto, cmd)
 
 if isinstance(rsp, bytes):
 _sshv1respondbytes(self._fout, rsp)
@@ -432,7 +445,3 @@
 elif cmd:
 _sshv1respondbytes(self._fout, b'')
 return cmd != ''
-
-def _client(self):
-client = encoding.environ.get('SSH_CLIENT', '').split(' ', 1)[0]
-return 'remote:ssh:' + client



To: indygreg, #hg-reviewers, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2087: wireprotoserver: move responsetype() out of http handler

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG341c886e411e: wireprotoserver: move responsetype() out of 
http handler (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2087?vs=5350=5533

REVISION DETAIL
  https://phab.mercurial-scm.org/D2087

AFFECTED FILES
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -170,48 +170,6 @@
 urlreq.quote(self._req.env.get('REMOTE_HOST', '')),
 urlreq.quote(self._req.env.get('REMOTE_USER', '')))
 
-def responsetype(self, prefer_uncompressed):
-"""Determine the appropriate response type and compression settings.
-
-Returns a tuple of (mediatype, compengine, engineopts).
-"""
-# Determine the response media type and compression engine based
-# on the request parameters.
-protocaps = decodevaluefromheaders(self._req, r'X-HgProto').split(' ')
-
-if '0.2' in protocaps:
-# All clients are expected to support uncompressed data.
-if prefer_uncompressed:
-return HGTYPE2, util._noopengine(), {}
-
-# Default as defined by wire protocol spec.
-compformats = ['zlib', 'none']
-for cap in protocaps:
-if cap.startswith('comp='):
-compformats = cap[5:].split(',')
-break
-
-# Now find an agreed upon compression format.
-for engine in wireproto.supportedcompengines(self._ui,
- util.SERVERROLE):
-if engine.wireprotosupport().name in compformats:
-opts = {}
-level = self._ui.configint('server',
-  '%slevel' % engine.name())
-if level is not None:
-opts['level'] = level
-
-return HGTYPE2, engine, opts
-
-# No mutually supported compression format. Fall back to the
-# legacy protocol.
-
-# Don't allow untrusted settings because disabling compression or
-# setting a very high compression level could lead to flooding
-# the server's network or CPU.
-opts = {'level': self._ui.configint('server', 'zliblevel')}
-return HGTYPE, util.compengines['zlib'], opts
-
 def iscmd(cmd):
 return cmd in wireproto.commands
 
@@ -252,6 +210,46 @@
 'handleerror': lambda ex: _handlehttperror(ex, req, cmd),
 }
 
+def _httpresponsetype(ui, req, prefer_uncompressed):
+"""Determine the appropriate response type and compression settings.
+
+Returns a tuple of (mediatype, compengine, engineopts).
+"""
+# Determine the response media type and compression engine based
+# on the request parameters.
+protocaps = decodevaluefromheaders(req, r'X-HgProto').split(' ')
+
+if '0.2' in protocaps:
+# All clients are expected to support uncompressed data.
+if prefer_uncompressed:
+return HGTYPE2, util._noopengine(), {}
+
+# Default as defined by wire protocol spec.
+compformats = ['zlib', 'none']
+for cap in protocaps:
+if cap.startswith('comp='):
+compformats = cap[5:].split(',')
+break
+
+# Now find an agreed upon compression format.
+for engine in wireproto.supportedcompengines(ui, util.SERVERROLE):
+if engine.wireprotosupport().name in compformats:
+opts = {}
+level = ui.configint('server', '%slevel' % engine.name())
+if level is not None:
+opts['level'] = level
+
+return HGTYPE2, engine, opts
+
+# No mutually supported compression format. Fall back to the
+# legacy protocol.
+
+# Don't allow untrusted settings because disabling compression or
+# setting a very high compression level could lead to flooding
+# the server's network or CPU.
+opts = {'level': ui.configint('server', 'zliblevel')}
+return HGTYPE, util.compengines['zlib'], opts
+
 def _callhttp(repo, req, proto, cmd):
 def genversion2(gen, engine, engineopts):
 # application/mercurial-0.2 always sends a payload header
@@ -284,8 +282,8 @@
 
 # This code for compression should not be streamres specific. It
 # is here because we only compress streamres at the moment.
-mediatype, engine, engineopts = proto.responsetype(
-rsp.prefer_uncompressed)
+mediatype, engine, engineopts = _httpresponsetype(
+repo.ui, req, rsp.prefer_uncompressed)
 gen = engine.compressstream(gen, engineopts)
 
 if mediatype == HGTYPE2:



To: indygreg, 

D2219: wireprotoserver: add version to HTTP protocol name (API)

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This matches what we did for the SSH protocol handler in
  https://phab.mercurial-scm.org/rHGac33dc94e1d53cf3fae22fd7e7c07805300ab42a.
  
  .. api::
  
HTTP protocol handlers now advertises its internal name as
``http-v1`` instead of ``http``.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2219

AFFECTED FILES
  mercurial/wireproto.py
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -112,7 +112,7 @@
 
 @property
 def name(self):
-return 'http'
+return 'http-v1'
 
 def getargs(self, args):
 knownargs = self._args()
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -778,7 +778,7 @@
 caps.append('bundle2=' + urlreq.quote(capsblob))
 caps.append('unbundle=%s' % ','.join(bundle2.bundlepriority))
 
-if proto.name == 'http':
+if proto.name == 'http-v1':
 caps.append('httpheader=%d' %
 repo.ui.configint('server', 'maxhttpheaderlen'))
 if repo.ui.configbool('experimental', 'httppostargs'):
@@ -852,7 +852,7 @@
 
 if not bundle1allowed(repo, 'pull'):
 if not exchange.bundle2requested(opts.get('bundlecaps')):
-if proto.name == 'http':
+if proto.name == 'http-v1':
 return ooberror(bundle2required)
 raise error.Abort(bundle2requiredmain,
   hint=bundle2requiredhint)
@@ -878,7 +878,7 @@
 except error.Abort as exc:
 # cleanly forward Abort error to the client
 if not exchange.bundle2requested(opts.get('bundlecaps')):
-if proto.name == 'http':
+if proto.name == 'http-v1':
 return ooberror(str(exc) + '\n')
 raise # cannot do better for bundle1 + ssh
 # bundle2 request expect a bundle2 reply
@@ -983,7 +983,7 @@
 gen = exchange.readbundle(repo.ui, fp, None)
 if (isinstance(gen, changegroupmod.cg1unpacker)
 and not bundle1allowed(repo, 'push')):
-if proto.name == 'http':
+if proto.name == 'http-v1':
 # need to special case http because stderr do not get 
to
 # the http client on failed push so we need to abuse
 # some other error type to make sure the message get to



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2217: wireproto: improve docstring for "hello"

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2217

AFFECTED FILES
  mercurial/wireproto.py

CHANGE DETAILS

diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -901,13 +901,16 @@
 
 @wireprotocommand('hello')
 def hello(repo, proto):
-'''the hello command returns a set of lines describing various
-interesting things about the server, in an RFC822-like format.
-Currently the only one defined is "capabilities", which
-consists of a line in the form:
+"""Called as part of SSH handshake to obtain server info.
+
+Returns a list of lines describing interesting things about the
+server, in an RFC822-like format.
 
-capabilities: space separated list of tokens
-'''
+Currently, the only one defined is ``capabilities``, which consists of a
+line of space separated tokens describing server abilities:
+
+capabilities:   
+"""
 caps = capabilities(repo, proto).data
 return bytesresponse('capabilities: %s\n' % caps)
 



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2215: httppeer: remove redundant code to fetch capabilities

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  _fetchcaps() is called by httppeer.instance(), which is the only
  instantiator of httppeer. Since _fetchcaps() always sets self._caps
  and since 
https://phab.mercurial-scm.org/rHG197d10e157ce848129ff5e7a53cf81d4ca63a932 
removed the fallback for cases where the
  remote doesn't support capabilities, we can remove some dead
  code from httppeer.capabilities().

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2215

AFFECTED FILES
  mercurial/httppeer.py

CHANGE DETAILS

diff --git a/mercurial/httppeer.py b/mercurial/httppeer.py
--- a/mercurial/httppeer.py
+++ b/mercurial/httppeer.py
@@ -221,13 +221,9 @@
 # Begin of _basewirepeer interface.
 
 def capabilities(self):
-if self._caps is None:
-try:
-self._fetchcaps()
-except error.RepoError:
-self._caps = set()
-self.ui.debug('capabilities: %s\n' %
-  (' '.join(self._caps or ['none'])))
+# self._fetchcaps() should have been called as part of peer
+# handshake. So self._caps should always be set.
+assert self._caps is not None
 return self._caps
 
 # End of _basewirepeer interface.



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2218: wireprotoserver: rename webproto to httpv1protocolhandler

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This matches our naming convention for the SSH server's protocol
  handler.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2218

AFFECTED FILES
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -105,7 +105,7 @@
 
 return ''.join(chunks)
 
-class webproto(baseprotocolhandler):
+class httpv1protocolhandler(baseprotocolhandler):
 def __init__(self, req, ui):
 self._req = req
 self._ui = ui
@@ -201,7 +201,7 @@
 if cmd not in wireproto.commands:
 return None
 
-proto = webproto(req, repo.ui)
+proto = httpv1protocolhandler(req, repo.ui)
 
 return {
 'cmd': cmd,



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2216: httppeer: remove httpspeer

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  All it did was verify at construction time that Mercurial supports
  TLS. instance() is what's used to construct peer instances. So
  we can just inline this check into that function and do away with
  the type variant.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2216

AFFECTED FILES
  mercurial/httppeer.py

CHANGE DETAILS

diff --git a/mercurial/httppeer.py b/mercurial/httppeer.py
--- a/mercurial/httppeer.py
+++ b/mercurial/httppeer.py
@@ -480,22 +480,15 @@
 def _abort(self, exception):
 raise exception
 
-class httpspeer(httppeer):
-def __init__(self, ui, path):
-if not url.has_https:
-raise error.Abort(_('Python support for SSL and HTTPS '
-   'is not installed'))
-httppeer.__init__(self, ui, path)
-
 def instance(ui, path, create):
 if create:
 raise error.Abort(_('cannot create new http repository'))
 try:
-if path.startswith('https:'):
-inst = httpspeer(ui, path)
-else:
-inst = httppeer(ui, path)
+if path.startswith('https:') and not url.has_https:
+raise error.Abort(_('Python support for SSL and HTTPS '
+'is not installed'))
 
+inst = httppeer(ui, path)
 inst._fetchcaps()
 
 return inst



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2175: py3: use b'' in mockblackbox.py

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGf49c3ee5b02f: py3: use b in mockblackbox.py 
(authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2175?vs=5476=5576

REVISION DETAIL
  https://phab.mercurial-scm.org/D2175

AFFECTED FILES
  tests/mockblackbox.py

CHANGE DETAILS

diff --git a/tests/mockblackbox.py b/tests/mockblackbox.py
--- a/tests/mockblackbox.py
+++ b/tests/mockblackbox.py
@@ -5,7 +5,7 @@
 
 # XXX: we should probably offer a devel option to do this in blackbox directly
 def getuser():
-return 'bob'
+return b'bob'
 def getpid():
 return 5000
 



To: indygreg, #hg-reviewers, pulkit, durin42
Cc: pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2154: py3: use system strings when calling __import__

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGc4146cf4dd20: py3: use system strings when calling 
__import__ (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2154?vs=5455=5579

REVISION DETAIL
  https://phab.mercurial-scm.org/D2154

AFFECTED FILES
  mercurial/hook.py

CHANGE DETAILS

diff --git a/mercurial/hook.py b/mercurial/hook.py
--- a/mercurial/hook.py
+++ b/mercurial/hook.py
@@ -49,12 +49,12 @@
 modname = modfile
 with demandimport.deactivated():
 try:
-obj = __import__(modname)
+obj = __import__(pycompat.sysstr(modname))
 except (ImportError, SyntaxError):
 e1 = sys.exc_info()
 try:
 # extensions are loaded with hgext_ prefix
-obj = __import__("hgext_%s" % modname)
+obj = __import__(r"hgext_%s" % pycompat.sysstr(modname))
 except (ImportError, SyntaxError):
 e2 = sys.exc_info()
 if ui.tracebackflag:



To: indygreg, #hg-reviewers, pulkit, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2152: py3: compare against bytes instead of str

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGc33a99506e13: py3: compare against bytes instead of str 
(authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2152?vs=5453=5577

REVISION DETAIL
  https://phab.mercurial-scm.org/D2152

AFFECTED FILES
  hgext/mq.py

CHANGE DETAILS

diff --git a/hgext/mq.py b/hgext/mq.py
--- a/hgext/mq.py
+++ b/hgext/mq.py
@@ -650,7 +650,7 @@
 self.seriesdirty = True
 
 def pushable(self, idx):
-if isinstance(idx, str):
+if isinstance(idx, bytes):
 idx = self.series.index(idx)
 patchguards = self.seriesguards[idx]
 if not patchguards:



To: indygreg, #hg-reviewers, pulkit, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2159: py3: use hex(hasher.digest())

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG6426878f7f0f: py3: use hex(hasher.digest()) (authored by 
indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2159?vs=5460=5582

REVISION DETAIL
  https://phab.mercurial-scm.org/D2159

AFFECTED FILES
  hgext/largefiles/lfutil.py

CHANGE DETAILS

diff --git a/hgext/largefiles/lfutil.py b/hgext/largefiles/lfutil.py
--- a/hgext/largefiles/lfutil.py
+++ b/hgext/largefiles/lfutil.py
@@ -15,6 +15,7 @@
 import stat
 
 from mercurial.i18n import _
+from mercurial.node import hex
 
 from mercurial import (
 dirstate,
@@ -371,7 +372,7 @@
 for data in instream:
 hasher.update(data)
 outfile.write(data)
-return hasher.hexdigest()
+return hex(hasher.digest())
 
 def hashfile(file):
 if not os.path.exists(file):
@@ -404,7 +405,7 @@
 h = hashlib.sha1()
 for chunk in util.filechunkiter(fileobj):
 h.update(chunk)
-return h.hexdigest()
+return hex(h.digest())
 
 def httpsendfile(ui, filename):
 return httpconnection.httpsendfile(ui, filename, 'rb')



To: indygreg, #hg-reviewers, pulkit, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2161: py3: use b'' for changegroup version literals

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG83246d6920f2: py3: use b for changegroup 
version literals (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2161?vs=5462=5585

REVISION DETAIL
  https://phab.mercurial-scm.org/D2161

AFFECTED FILES
  tests/flagprocessorext.py

CHANGE DETAILS

diff --git a/tests/flagprocessorext.py b/tests/flagprocessorext.py
--- a/tests/flagprocessorext.py
+++ b/tests/flagprocessorext.py
@@ -45,14 +45,14 @@
 
 def supportedoutgoingversions(orig, repo):
 versions = orig(repo)
-versions.discard('01')
-versions.discard('02')
-versions.add('03')
+versions.discard(b'01')
+versions.discard(b'02')
+versions.add(b'03')
 return versions
 
 def allsupportedversions(orig, ui):
 versions = orig(ui)
-versions.add('03')
+versions.add(b'03')
 return versions
 
 def noopaddrevision(orig, self, text, transaction, link, p1, p2,
@@ -106,7 +106,7 @@
 
 # Teach exchange to use changegroup 3
 for k in exchange._bundlespeccgversions.keys():
-exchange._bundlespeccgversions[k] = '03'
+exchange._bundlespeccgversions[k] = b'03'
 
 # Add wrappers for addrevision, responsible to set flags depending on the
 # revision data contents.



To: indygreg, #hg-reviewers, pulkit
Cc: pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2160: py3: use b'' in inline extension

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGa42817fede27: py3: use b in inline extension 
(authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2160?vs=5461=5584

REVISION DETAIL
  https://phab.mercurial-scm.org/D2160

AFFECTED FILES
  tests/test-largefiles-small-disk.t

CHANGE DETAILS

diff --git a/tests/test-largefiles-small-disk.t 
b/tests/test-largefiles-small-disk.t
--- a/tests/test-largefiles-small-disk.t
+++ b/tests/test-largefiles-small-disk.t
@@ -11,7 +11,7 @@
   > _origcopyfileobj = shutil.copyfileobj
   > def copyfileobj(fsrc, fdst, length=16*1024):
   > # allow journal files (used by transaction) to be written
-  > if 'journal.' in fdst.name:
+  > if b'journal.' in fdst.name:
   > return _origcopyfileobj(fsrc, fdst, length)
   > fdst.write(fsrc.read(4))
   > raise IOError(errno.ENOSPC, os.strerror(errno.ENOSPC))



To: indygreg, #hg-reviewers, pulkit
Cc: pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2156: py3: catch TypeError during template operations

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG230489fc0b41: py3: catch TypeError during template 
operations (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2156?vs=5457=5580

REVISION DETAIL
  https://phab.mercurial-scm.org/D2156

AFFECTED FILES
  mercurial/templatekw.py

CHANGE DETAILS

diff --git a/mercurial/templatekw.py b/mercurial/templatekw.py
--- a/mercurial/templatekw.py
+++ b/mercurial/templatekw.py
@@ -192,11 +192,15 @@
 def one(v, tag=name):
 try:
 vmapping.update(v)
-except (AttributeError, ValueError):
+# Python 2 raises ValueError if the type of v is wrong. Python
+# 3 raises TypeError.
+except (AttributeError, TypeError, ValueError):
 try:
+# Python 2 raises ValueError trying to destructure an e.g.
+# bytes. Python 3 raises TypeError.
 for a, b in v:
 vmapping[a] = b
-except ValueError:
+except (TypeError, ValueError):
 vmapping[name] = v
 return templ(tag, **pycompat.strkwargs(vmapping))
 lastname = 'last_' + name



To: indygreg, #hg-reviewers, pulkit, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2167: py3: cast character set to bytes

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG6ea7f1c10c81: py3: cast character set to bytes (authored by 
indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2167?vs=5468=5590

REVISION DETAIL
  https://phab.mercurial-scm.org/D2167

AFFECTED FILES
  mercurial/mail.py

CHANGE DETAILS

diff --git a/mercurial/mail.py b/mercurial/mail.py
--- a/mercurial/mail.py
+++ b/mercurial/mail.py
@@ -187,7 +187,7 @@
 
 def codec2iana(cs):
 ''
-cs = email.charset.Charset(cs).input_charset.lower()
+cs = pycompat.sysbytes(email.charset.Charset(cs).input_charset.lower())
 
 # "latin1" normalizes to "iso8859-1", standard calls for "iso-8859-1"
 if cs.startswith("iso") and not cs.startswith("iso-"):



To: indygreg, #hg-reviewers, pulkit, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2163: py3: use raw string for key in **kwargs

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGb587a889b97e: py3: use raw string for key in **kwargs 
(authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2163?vs=5464=5587

REVISION DETAIL
  https://phab.mercurial-scm.org/D2163

AFFECTED FILES
  mercurial/bundle2.py

CHANGE DETAILS

diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -1729,7 +1729,7 @@
 extrakwargs = {}
 targetphase = inpart.params.get('targetphase')
 if targetphase is not None:
-extrakwargs['targetphase'] = int(targetphase)
+extrakwargs[r'targetphase'] = int(targetphase)
 ret = _processchangegroup(op, cg, tr, 'bundle2', 'bundle2',
   expectedtotal=nbchangesets, **extrakwargs)
 if op.reply is not None:



To: indygreg, #hg-reviewers, pulkit, durin42
Cc: pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2155: py3: use bytes literals for test extension

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG361276a36d49: py3: use bytes literals for test extension 
(authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2155?vs=5456=5581

REVISION DETAIL
  https://phab.mercurial-scm.org/D2155

AFFECTED FILES
  tests/test-hook.t

CHANGE DETAILS

diff --git a/tests/test-hook.t b/tests/test-hook.t
--- a/tests/test-hook.t
+++ b/tests/test-hook.t
@@ -417,9 +417,9 @@
   > def printargs(ui, args):
   > a = list(args.items())
   > a.sort()
-  > ui.write('hook args:\n')
+  > ui.write(b'hook args:\n')
   > for k, v in a:
-  >ui.write('  %s %s\n' % (k, v))
+  >ui.write(b'  %s %s\n' % (k, v))
   > 
   > def passhook(ui, repo, **args):
   > printargs(ui, args)
@@ -432,19 +432,19 @@
   > pass
   > 
   > def raisehook(**args):
-  > raise LocalException('exception from hook')
+  > raise LocalException(b'exception from hook')
   > 
   > def aborthook(**args):
-  > raise error.Abort('raise abort from hook')
+  > raise error.Abort(b'raise abort from hook')
   > 
   > def brokenhook(**args):
   > return 1 + {}
   > 
   > def verbosehook(ui, **args):
-  > ui.note('verbose output from hook\n')
+  > ui.note(b'verbose output from hook\n')
   > 
   > def printtags(ui, repo, **args):
-  > ui.write('%s\n' % sorted(repo.tags()))
+  > ui.write(b'%s\n' % sorted(repo.tags()))
   > 
   > class container:
   > unreachable = 1
@@ -667,7 +667,7 @@
   $ cd hooks
   $ cat > testhooks.py < def testhook(ui, **args):
-  > ui.write('hook works\n')
+  > ui.write(b'hook works\n')
   > EOF
   $ echo '[hooks]' > ../repo/.hg/hgrc
   $ echo "pre-commit.test = python:`pwd`/testhooks.py:testhook" >> 
../repo/.hg/hgrc
@@ -886,7 +886,7 @@
   > def uisetup(ui):
   > class untrustedui(ui.__class__):
   > def _trusted(self, fp, f):
-  > if util.normpath(fp.name).endswith('untrusted/.hg/hgrc'):
+  > if util.normpath(fp.name).endswith(b'untrusted/.hg/hgrc'):
   > return False
   > return super(untrustedui, self)._trusted(fp, f)
   > ui.__class__ = untrustedui



To: indygreg, #hg-reviewers, pulkit
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2153: py3: open patches.queue in binary mode

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGbff95b002e33: py3: open patches.queue in binary mode 
(authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2153?vs=5454=5578

REVISION DETAIL
  https://phab.mercurial-scm.org/D2153

AFFECTED FILES
  hgext/mq.py

CHANGE DETAILS

diff --git a/hgext/mq.py b/hgext/mq.py
--- a/hgext/mq.py
+++ b/hgext/mq.py
@@ -445,9 +445,9 @@
 def __init__(self, ui, baseui, path, patchdir=None):
 self.basepath = path
 try:
-fh = open(os.path.join(path, 'patches.queue'))
-cur = fh.read().rstrip()
-fh.close()
+with open(os.path.join(path, 'patches.queue'), r'rb') as fh:
+cur = fh.read().rstrip()
+
 if not cur:
 curpath = os.path.join(path, 'patches')
 else:



To: indygreg, #hg-reviewers, pulkit, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2162: py3: preserve chunks as an iterable of bytes

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGc1104fe76e69: py3: preserve chunks as an iterable of bytes 
(authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2162?vs=5463=5586

REVISION DETAIL
  https://phab.mercurial-scm.org/D2162

AFFECTED FILES
  mercurial/logcmdutil.py

CHANGE DETAILS

diff --git a/mercurial/logcmdutil.py b/mercurial/logcmdutil.py
--- a/mercurial/logcmdutil.py
+++ b/mercurial/logcmdutil.py
@@ -81,7 +81,7 @@
 if fp is not None or ui.canwritewithoutlabels():
 out = fp or ui
 if stat:
-chunks = patch.diffstat(util.iterlines(chunks), width=width)
+chunks = [patch.diffstat(util.iterlines(chunks), width=width)]
 for chunk in util.filechunkiter(util.chunkbuffer(chunks)):
 out.write(chunk)
 else:



To: indygreg, #hg-reviewers, pulkit, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2164: py3: avoid changing dictionary during iteration

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGc02771617a70: py3: avoid changing dictionary during 
iteration (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2164?vs=5465=5588

REVISION DETAIL
  https://phab.mercurial-scm.org/D2164

AFFECTED FILES
  mercurial/copies.py

CHANGE DETAILS

diff --git a/mercurial/copies.py b/mercurial/copies.py
--- a/mercurial/copies.py
+++ b/mercurial/copies.py
@@ -123,7 +123,7 @@
 t[k] = v
 
 # remove criss-crossed copies
-for k, v in t.items():
+for k, v in list(t.items()):
 if k in src and v in dst:
 del t[k]
 



To: indygreg, #hg-reviewers, pulkit, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2169: py3: explicitly cast bool to bytes

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGd78a51982262: py3: explicitly cast bool to bytes (authored 
by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2169?vs=5470=5592

REVISION DETAIL
  https://phab.mercurial-scm.org/D2169

AFFECTED FILES
  hgext/histedit.py

CHANGE DETAILS

diff --git a/hgext/histedit.py b/hgext/histedit.py
--- a/hgext/histedit.py
+++ b/hgext/histedit.py
@@ -344,7 +344,7 @@
 fp.write('v1\n')
 fp.write('%s\n' % node.hex(self.parentctxnode))
 fp.write('%s\n' % node.hex(self.topmost))
-fp.write('%s\n' % self.keep)
+fp.write('%s\n' % 'True' if self.keep else 'False')
 fp.write('%d\n' % len(self.actions))
 for action in self.actions:
 fp.write('%s\n' % action.tostate())



To: indygreg, durin42, #hg-reviewers, pulkit
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2168: mail: import email.utils not email.Utils

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG54dfb65e2f82: mail: import email.utils not email.Utils 
(authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2168?vs=5469=5591

REVISION DETAIL
  https://phab.mercurial-scm.org/D2168

AFFECTED FILES
  mercurial/mail.py

CHANGE DETAILS

diff --git a/mercurial/mail.py b/mercurial/mail.py
--- a/mercurial/mail.py
+++ b/mercurial/mail.py
@@ -288,13 +288,13 @@
 addr = addr.encode('ascii')
 except UnicodeDecodeError:
 raise error.Abort(_('invalid local address: %s') % addr)
-return email.Utils.formataddr((name, addr))
+return email.utils.formataddr((name, addr))
 
 def addressencode(ui, address, charsets=None, display=False):
 '''Turns address into RFC-2047 compliant header.'''
 if display or not address:
 return address or ''
-name, addr = email.Utils.parseaddr(address)
+name, addr = email.utils.parseaddr(address)
 return _addressencode(ui, name, addr, charsets)
 
 def addrlistencode(ui, addrs, charsets=None, display=False):
@@ -305,7 +305,7 @@
 return [a.strip() for a in addrs if a.strip()]
 
 result = []
-for name, addr in email.Utils.getaddresses(addrs):
+for name, addr in email.utils.getaddresses(addrs):
 if name or addr:
 result.append(_addressencode(ui, name, addr, charsets))
 return result



To: indygreg, #hg-reviewers, pulkit, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2166: py3: cast decode() argument to system string

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG9e47bfbeb723: py3: cast decode() argument to system string 
(authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2166?vs=5467=5589

REVISION DETAIL
  https://phab.mercurial-scm.org/D2166

AFFECTED FILES
  mercurial/mail.py

CHANGE DETAILS

diff --git a/mercurial/mail.py b/mercurial/mail.py
--- a/mercurial/mail.py
+++ b/mercurial/mail.py
@@ -206,7 +206,7 @@
 return mimetextqp(s, subtype, 'us-ascii')
 for charset in cs:
 try:
-s.decode(charset)
+s.decode(pycompat.sysstr(charset))
 return mimetextqp(s, subtype, codec2iana(charset))
 except UnicodeDecodeError:
 pass



To: indygreg, #hg-reviewers, pulkit, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2170: py3: make dummyssh compatible with Python 3

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG9996ec844c1e: py3: make dummyssh compatible with Python 3 
(authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2170?vs=5471=5593

REVISION DETAIL
  https://phab.mercurial-scm.org/D2170

AFFECTED FILES
  tests/dummyssh

CHANGE DETAILS

diff --git a/tests/dummyssh b/tests/dummyssh
--- a/tests/dummyssh
+++ b/tests/dummyssh
@@ -15,8 +15,8 @@
 log = open("dummylog", "ab")
 log.write(b"Got arguments")
 for i, arg in enumerate(sys.argv[1:]):
-log.write(b" %d:%s" % (i + 1, arg))
-log.write("\n")
+log.write(b" %d:%s" % (i + 1, arg.encode('latin1')))
+log.write(b"\n")
 log.close()
 hgcmd = sys.argv[2]
 if os.name == 'nt':



To: indygreg, #hg-reviewers, pulkit, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2176: py3: convert traceback representation to bytes when logging

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG9446246e4d1a: py3: convert traceback representation to 
bytes when logging (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2176?vs=5477=5598

REVISION DETAIL
  https://phab.mercurial-scm.org/D2176

AFFECTED FILES
  mercurial/dispatch.py

CHANGE DETAILS

diff --git a/mercurial/dispatch.py b/mercurial/dispatch.py
--- a/mercurial/dispatch.py
+++ b/mercurial/dispatch.py
@@ -992,6 +992,7 @@
 this function returns False, ignored otherwise.
 """
 warning = _exceptionwarning(ui)
-ui.log("commandexception", "%s\n%s\n", warning, traceback.format_exc())
+ui.log("commandexception", "%s\n%s\n", warning,
+   pycompat.sysbytes(traceback.format_exc()))
 ui.warn(warning)
 return False  # re-raise the exception



To: indygreg, #hg-reviewers, pulkit, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2171: ui: use named attributes on FrameInfo instance

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGffcc3b977e43: ui: use named attributes on FrameInfo 
instance (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2171?vs=5472=5594

REVISION DETAIL
  https://phab.mercurial-scm.org/D2171

AFFECTED FILES
  mercurial/ui.py

CHANGE DETAILS

diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -1620,11 +1620,15 @@
 else:
 curframe = inspect.currentframe()
 calframe = inspect.getouterframes(curframe, 2)
-self.write_err('%s at: %s:%s (%s)\n'
-   % ((msg,) + calframe[stacklevel][1:4]))
+frameinfo = calframe[stacklevel]
+
+self.write_err('%s at: %s:%s (%s)\n' % (
+msg, frameinfo.filename, frameinfo.lineno,
+frameinfo.function))
 self.log('develwarn', '%s at: %s:%s (%s)\n',
- msg, *calframe[stacklevel][1:4])
-curframe = calframe = None  # avoid cycles
+ msg, frameinfo.filename, frameinfo.lineno,
+ frameinfo.function)
+curframe = calframe = frameinfo = None  # avoid cycles
 
 def deprecwarn(self, msg, version, stacklevel=2):
 """issue a deprecation warning



To: indygreg, #hg-reviewers, pulkit, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2172: py3: convert FrameInfo members to bytes

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG20dbe0eee139: py3: convert FrameInfo members to bytes 
(authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2172?vs=5473=5595

REVISION DETAIL
  https://phab.mercurial-scm.org/D2172

AFFECTED FILES
  mercurial/ui.py

CHANGE DETAILS

diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -1622,12 +1622,12 @@
 calframe = inspect.getouterframes(curframe, 2)
 frameinfo = calframe[stacklevel]
 
-self.write_err('%s at: %s:%s (%s)\n' % (
-msg, frameinfo.filename, frameinfo.lineno,
-frameinfo.function))
-self.log('develwarn', '%s at: %s:%s (%s)\n',
- msg, frameinfo.filename, frameinfo.lineno,
- frameinfo.function)
+self.write_err('%s at: %s:%d (%s)\n' % (
+msg, pycompat.sysbytes(frameinfo.filename),
+frameinfo.lineno, pycompat.sysbytes(frameinfo.function)))
+self.log('develwarn', '%s at: %s:%d (%s)\n',
+ msg, pycompat.sysbytes(frameinfo.filename),
+ frameinfo.lineno, pycompat.sysbytes(frameinfo.function))
 curframe = calframe = frameinfo = None  # avoid cycles
 
 def deprecwarn(self, msg, version, stacklevel=2):



To: indygreg, #hg-reviewers, pulkit, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2173: py3: add missing b'' literal to sshprotoext.py

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG2e1d3924fa5b: py3: add missing b literal to 
sshprotoext.py (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2173?vs=5474=5596

REVISION DETAIL
  https://phab.mercurial-scm.org/D2173

AFFECTED FILES
  tests/sshprotoext.py

CHANGE DETAILS

diff --git a/tests/sshprotoext.py b/tests/sshprotoext.py
--- a/tests/sshprotoext.py
+++ b/tests/sshprotoext.py
@@ -68,7 +68,7 @@
 l = self._fin.readline()
 assert l == b'between\n'
 l = self._fin.readline()
-assert l == 'pairs 81\n'
+assert l == b'pairs 81\n'
 self._fin.read(81)
 
 # Send the upgrade response.



To: indygreg, #hg-reviewers, pulkit
Cc: pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2157: py3: use string for "close" value in commit extras

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGb44a47214122: py3: use string for close value 
in commit extras (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2157?vs=5458=5583

REVISION DETAIL
  https://phab.mercurial-scm.org/D2157

AFFECTED FILES
  mercurial/commands.py

CHANGE DETAILS

diff --git a/mercurial/commands.py b/mercurial/commands.py
--- a/mercurial/commands.py
+++ b/mercurial/commands.py
@@ -1551,7 +1551,7 @@
 
 extra = {}
 if opts.get('close_branch'):
-extra['close'] = 1
+extra['close'] = '1'
 
 if not bheads:
 raise error.Abort(_('can only close branch heads'))



To: indygreg, #hg-reviewers, pulkit, durin42
Cc: pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2174: py3: convert context to bytes instead of str

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGdd905ea1ea60: py3: convert context to bytes instead of str 
(authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2174?vs=5475=5597

REVISION DETAIL
  https://phab.mercurial-scm.org/D2174

AFFECTED FILES
  mercurial/debugcommands.py

CHANGE DETAILS

diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -2356,7 +2356,7 @@
 """
 # passed to successorssets caching computation from one call to another
 cache = {}
-ctx2str = str
+ctx2str = bytes
 node2str = short
 for rev in scmutil.revrange(repo, revs):
 ctx = repo[rev]



To: indygreg, #hg-reviewers, pulkit, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2198: tests: remove references to bundle2-exp config option

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This option was deleted a while ago. We don't even alias it in
  core.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2198

AFFECTED FILES
  tests/narrow-library.sh
  tests/test-pushvars.t

CHANGE DETAILS

diff --git a/tests/test-pushvars.t b/tests/test-pushvars.t
--- a/tests/test-pushvars.t
+++ b/tests/test-pushvars.t
@@ -11,8 +11,6 @@
   $ cat >> $HGRCPATH << EOF
   > [hooks]
   > pretxnchangegroup = sh $TESTTMP/pretxnchangegroup.sh
-  > [experimental]
-  > bundle2-exp = true
   > EOF
 
   $ hg init repo
diff --git a/tests/narrow-library.sh b/tests/narrow-library.sh
--- a/tests/narrow-library.sh
+++ b/tests/narrow-library.sh
@@ -4,6 +4,5 @@
 [ui]
 ssh=python "$TESTDIR/dummyssh"
 [experimental]
-bundle2-exp = True
 changegroup3 = True
 EOF



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2199: tests: glob over line number

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2199

AFFECTED FILES
  tests/test-narrow-pull.t

CHANGE DETAILS

diff --git a/tests/test-narrow-pull.t b/tests/test-narrow-pull.t
--- a/tests/test-narrow-pull.t
+++ b/tests/test-narrow-pull.t
@@ -166,7 +166,7 @@
 
 We should also be able to unshare without breaking everything:
   $ hg unshare
-  devel-warn: write with no wlock: "narrowspec" at: 
*/hgext/narrow/narrowrepo.py:43 (unsharenarrowspec) (glob)
+  devel-warn: write with no wlock: "narrowspec" at: 
*/hgext/narrow/narrowrepo.py:* (unsharenarrowspec) (glob)
   $ hg verify
   checking changesets
   checking manifests



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2200: hg: move share._getsrcrepo into core

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The fact we were calling this from extensions was a sign that it
  should live in core.
  
  We were also able to remove some extra attribute aliases from the
  share extension.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2200

AFFECTED FILES
  hgext/journal.py
  hgext/narrow/narrowrepo.py
  hgext/narrow/narrowspec.py
  hgext/share.py
  mercurial/hg.py

CHANGE DETAILS

diff --git a/mercurial/hg.py b/mercurial/hg.py
--- a/mercurial/hg.py
+++ b/mercurial/hg.py
@@ -202,6 +202,24 @@
 return ''
 return os.path.basename(os.path.normpath(path))
 
+def sharedreposource(repo):
+"""Returns repository object for source repository of a shared repo.
+
+If repo is not a shared repository, returns None.
+"""
+if repo.sharedpath == repo.path:
+return None
+
+if util.safehasattr(repo, 'srcrepo') and repo.srcrepo:
+return repo.srcrepo
+
+# the sharedpath always ends in the .hg; we want the path to the repo
+source = repo.vfs.split(repo.sharedpath)[0]
+srcurl, branches = parseurl(source)
+srcrepo = repository(repo.ui, srcurl)
+repo.srcrepo = srcrepo
+return srcrepo
+
 def share(ui, source, dest=None, update=True, bookmarks=True, defaultpath=None,
   relative=False):
 '''create a shared repository'''
diff --git a/hgext/share.py b/hgext/share.py
--- a/hgext/share.py
+++ b/hgext/share.py
@@ -52,9 +52,6 @@
 util,
 )
 
-repository = hg.repository
-parseurl = hg.parseurl
-
 cmdtable = {}
 command = registrar.command(cmdtable)
 # Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' 
for
@@ -135,27 +132,9 @@
 return False
 return hg.sharedbookmarks in shared
 
-def _getsrcrepo(repo):
-"""
-Returns the source repository object for a given shared repository.
-If repo is not a shared repository, return None.
-"""
-if repo.sharedpath == repo.path:
-return None
-
-if util.safehasattr(repo, 'srcrepo') and repo.srcrepo:
-return repo.srcrepo
-
-# the sharedpath always ends in the .hg; we want the path to the repo
-source = repo.vfs.split(repo.sharedpath)[0]
-srcurl, branches = parseurl(source)
-srcrepo = repository(repo.ui, srcurl)
-repo.srcrepo = srcrepo
-return srcrepo
-
 def getbkfile(orig, repo):
 if _hassharedbookmarks(repo):
-srcrepo = _getsrcrepo(repo)
+srcrepo = hg.sharedreposource(repo)
 if srcrepo is not None:
 # just orig(srcrepo) doesn't work as expected, because
 # HG_PENDING refers repo.root.
@@ -186,7 +165,7 @@
 orig(self, tr)
 
 if _hassharedbookmarks(self._repo):
-srcrepo = _getsrcrepo(self._repo)
+srcrepo = hg.sharedreposource(self._repo)
 if srcrepo is not None:
 category = 'share-bookmarks'
 tr.addpostclose(category, lambda tr: self._writerepo(srcrepo))
@@ -196,6 +175,6 @@
 orig(self, repo)
 
 if _hassharedbookmarks(self._repo):
-srcrepo = _getsrcrepo(self._repo)
+srcrepo = hg.sharedreposource(self._repo)
 if srcrepo is not None:
 orig(self, srcrepo)
diff --git a/hgext/narrow/narrowspec.py b/hgext/narrow/narrowspec.py
--- a/hgext/narrow/narrowspec.py
+++ b/hgext/narrow/narrowspec.py
@@ -12,14 +12,11 @@
 from mercurial.i18n import _
 from mercurial import (
 error,
+hg,
 match as matchmod,
 util,
 )
 
-from .. import (
-share,
-)
-
 FILENAME = 'narrowspec'
 
 def _parsestoredpatterns(text):
@@ -133,7 +130,7 @@
 
 def load(repo):
 if repo.shared():
-repo = share._getsrcrepo(repo)
+repo = hg.sharedreposource(repo)
 try:
 spec = repo.vfs.read(FILENAME)
 except IOError as e:
@@ -150,7 +147,7 @@
 def save(repo, includepats, excludepats):
 spec = format(includepats, excludepats)
 if repo.shared():
-repo = share._getsrcrepo(repo)
+repo = hg.sharedreposource(repo)
 repo.vfs.write(FILENAME, spec)
 
 def restrictpatterns(req_includes, req_excludes, repo_includes, repo_excludes):
diff --git a/hgext/narrow/narrowrepo.py b/hgext/narrow/narrowrepo.py
--- a/hgext/narrow/narrowrepo.py
+++ b/hgext/narrow/narrowrepo.py
@@ -9,15 +9,12 @@
 
 from mercurial import (
 bundlerepo,
+hg,
 localrepo,
 match as matchmod,
 scmutil,
 )
 
-from .. import (
-share,
-)
-
 from . import (
 narrowrevlog,
 narrowspec,
@@ -37,7 +34,7 @@
 def unsharenarrowspec(orig, ui, repo, repopath):
 if (REQUIREMENT in repo.requirements
 and repo.path == repopath and repo.shared()):
-srcrepo = share._getsrcrepo(repo)
+srcrepo = hg.sharedreposource(repo)
 with srcrepo.vfs(narrowspec.FILENAME) as f:
 spec = f.read()
 with repo.vfs(narrowspec.FILENAME, 'w') as f:
diff --git a/hgext/journal.py 

D2202: tests: remove code to support Mercurial 4.3

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Now that narrow lives in core, we don't need the legacy support.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2202

AFFECTED FILES
  tests/test-narrow-expanddirstate.t

CHANGE DETAILS

diff --git a/tests/test-narrow-expanddirstate.t 
b/tests/test-narrow-expanddirstate.t
--- a/tests/test-narrow-expanddirstate.t
+++ b/tests/test-narrow-expanddirstate.t
@@ -75,23 +75,13 @@
   >   def wrapds(orig, self):
   > ds = orig(self)
   > class expandingdirstate(ds.__class__):
-  >   # Mercurial 4.4 uses this version.
   >   @hgutil.propertycache
   >   def _map(self):
   > ret = super(expandingdirstate, self)._map
   > with repo.wlock(), repo.lock(), repo.transaction(
   > 'expandnarrowspec'):
   >   expandnarrowspec(ui, repo, os.environ.get('DIRSTATEINCLUDES'))
   > return ret
-  >   # Mercurial 4.3.3 and earlier uses this version. It seems that
-  >   # narrowhg does not currently support this version, but we include
-  >   # it just in case backwards compatibility is restored.
-  >   def _read(self):
-  > ret = super(expandingdirstate, self)._read()
-  > with repo.wlock(), repo.lock(), repo.transaction(
-  > 'expandnarrowspec'):
-  >   expandnarrowspec(ui, repo, os.environ.get('DIRSTATEINCLUDES'))
-  > return ret
   > ds.__class__ = expandingdirstate
   > return ds
   >   return wrapds



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2201: narrowspec: move module into core

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Having support for parsing the narrow specification in core is
  necessary for moving many other parts of narrow to core.
  
  We do still want to harmonize the narrow spec with the sparse
  spec. And the format needs to be documented. But this shouldn't
  hold up the code moving to core.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2201

AFFECTED FILES
  hgext/narrow/narrowbundle2.py
  hgext/narrow/narrowcommands.py
  hgext/narrow/narrowdirstate.py
  hgext/narrow/narrowrepo.py
  hgext/narrow/narrowspec.py
  hgext/narrow/narrowwirepeer.py
  mercurial/narrowspec.py
  tests/test-narrow-expanddirstate.t

CHANGE DETAILS

diff --git a/tests/test-narrow-expanddirstate.t 
b/tests/test-narrow-expanddirstate.t
--- a/tests/test-narrow-expanddirstate.t
+++ b/tests/test-narrow-expanddirstate.t
@@ -51,22 +51,22 @@
   > from mercurial import extensions
   > from mercurial import localrepo
   > from mercurial import match as matchmod
+  > from mercurial import narrowspec
   > from mercurial import patch
   > from mercurial import util as hgutil
   > 
   > def expandnarrowspec(ui, repo, newincludes=None):
   >   if not newincludes:
   > return
   >   import sys
   >   newincludes = set([newincludes])
-  >   narrowhg = extensions.find('narrow')
   >   includes, excludes = repo.narrowpats
-  >   currentmatcher = narrowhg.narrowspec.match(repo.root, includes, excludes)
+  >   currentmatcher = narrowspec.match(repo.root, includes, excludes)
   >   includes = includes | newincludes
   >   if not repo.currenttransaction():
   > ui.develwarn('expandnarrowspec called outside of transaction!')
   >   repo.setnarrowpats(includes, excludes)
-  >   newmatcher = narrowhg.narrowspec.match(repo.root, includes, excludes)
+  >   newmatcher = narrowspec.match(repo.root, includes, excludes)
   >   added = matchmod.differencematcher(newmatcher, currentmatcher)
   >   for f in repo['.'].manifest().walk(added):
   > repo.dirstate.normallookup(f)
diff --git a/hgext/narrow/narrowspec.py b/mercurial/narrowspec.py
rename from hgext/narrow/narrowspec.py
rename to mercurial/narrowspec.py
--- a/hgext/narrow/narrowspec.py
+++ b/mercurial/narrowspec.py
@@ -9,8 +9,8 @@
 
 import errno
 
-from mercurial.i18n import _
-from mercurial import (
+from .i18n import _
+from . import (
 error,
 hg,
 match as matchmod,
@@ -89,7 +89,7 @@
 # We use newlines as separators in the narrowspec file, so don't allow them
 # in patterns.
 if _numlines(pat) > 1:
-raise error.Abort('newlines are not allowed in narrowspec paths')
+raise error.Abort(_('newlines are not allowed in narrowspec paths'))
 
 components = pat.split('/')
 if '.' in components or '..' in components:
diff --git a/hgext/narrow/narrowwirepeer.py b/hgext/narrow/narrowwirepeer.py
--- a/hgext/narrow/narrowwirepeer.py
+++ b/hgext/narrow/narrowwirepeer.py
@@ -12,11 +12,10 @@
 error,
 extensions,
 hg,
+narrowspec,
 node,
 )
 
-from . import narrowspec
-
 def uisetup():
 def peersetup(ui, peer):
 # We must set up the expansion before reposetup below, since it's used
diff --git a/hgext/narrow/narrowrepo.py b/hgext/narrow/narrowrepo.py
--- a/hgext/narrow/narrowrepo.py
+++ b/hgext/narrow/narrowrepo.py
@@ -12,12 +12,12 @@
 hg,
 localrepo,
 match as matchmod,
+narrowspec,
 scmutil,
 )
 
 from . import (
 narrowrevlog,
-narrowspec,
 )
 
 # When narrowing is finalized and no longer subject to format changes,
diff --git a/hgext/narrow/narrowdirstate.py b/hgext/narrow/narrowdirstate.py
--- a/hgext/narrow/narrowdirstate.py
+++ b/hgext/narrow/narrowdirstate.py
@@ -13,11 +13,10 @@
 error,
 extensions,
 match as matchmod,
+narrowspec,
 util as hgutil,
 )
 
-from . import narrowspec
-
 def setup(repo):
 """Add narrow spec dirstate ignore, block changes outside narrow spec."""
 
diff --git a/hgext/narrow/narrowcommands.py b/hgext/narrow/narrowcommands.py
--- a/hgext/narrow/narrowcommands.py
+++ b/hgext/narrow/narrowcommands.py
@@ -18,6 +18,7 @@
 extensions,
 hg,
 merge,
+narrowspec,
 node,
 registrar,
 repair,
@@ -28,7 +29,6 @@
 from . import (
 narrowbundle2,
 narrowrepo,
-narrowspec,
 )
 
 table = {}
diff --git a/hgext/narrow/narrowbundle2.py b/hgext/narrow/narrowbundle2.py
--- a/hgext/narrow/narrowbundle2.py
+++ b/hgext/narrow/narrowbundle2.py
@@ -24,14 +24,14 @@
 error,
 exchange,
 extensions,
+narrowspec,
 repair,
 util,
 wireproto,
 )
 
 from . import (
 narrowrepo,
-narrowspec,
 )
 
 NARROWCAP = 'narrow'



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel