D2057: rust implementation of hg status

[glandium (Mike Hommey)]

glandium added a comment.


  Doesn't mononoke have code to read revlogs already?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2057

To: Ivzhh, #hg-reviewers
Cc: glandium, krbullock, indygreg, durin42, kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2057: rust implementation of hg status

[indygreg (Gregory Szorc)]

indygreg added a comment.


  First of all - wow! Thanks for writing all this code. There's definitely a 
lot to work with. And work with it we will!
  
  This is definitely too big to review as one commit. If you could do *any* 
work to split it up, it would be greatly appreciated. I'd focus on the pure 
Rust pieces first. Everything needed to open revlogs would be great!
  
  You may find our custom Phabricator extensions (linked from 
https://www.mercurial-scm.org/wiki/Phabricator) useful for submitting series of 
commits to Phabricator.
  
  Regarding the performance, that's pretty good! The dirstate code is some of 
the most optimized code in Mercurial. There are some gnarly Python C hacks to 
make it fast. Some of those tricks involve using special system calls to walk 
directories to minimize the number of system calls. I'm not sure if the crate 
you imported has those optimizations. (I wouldn't be surprised either way.) I 
wouldn't worry too much about performance at this juncture. But I suspect we 
could make the Rust code another 50% faster with some tweaking. It would also 
be interesting to test on a larger repo, say 
https://hg.mozilla.org/mozilla-unified. Also, I believe there are hooks in the 
dirstate code to use Watchman (fsmonitor). Those hooks are critical in order to 
achieve peak performance on large repositories.
  
  Since you seem to be proficient at writing lots of Rust code, if you are 
looking for another project, may I suggest porting `chg` to Rust? That code is 
in `contrib/chg`. That might be the easiest component to actually ship in Rust 
since it is a standalone binary that doesn't link against Python. But we 
shouldn't get ahead of ourselves :)
  
  Anyway, it is late for me and I need to detach from my computer. I'm sure 
others will have things to say as well...

INLINE COMMENTS

> build.rs:1
> +// build.rs -- Configure build environment for `hgcli` Rust package.
> +//

I see this file was copied. There's nothing wrong with that. But does this mean 
we will need a custom build.rs for each Rust package doing Python? If that's 
the case, then I would prefer to isolate all our rust-cpython code to a single 
package, if possible. I'm guessing that could be challenging due to crossing 
create boundaries. I'm sure there are placed where we don't want to expose 
symbols outside the crate.

I'm curious how others feel about this.

> main.rs:233-261
> +let matches = clap::App::new("hg rust oxidation")
> +.arg(
> +clap::Arg::with_name("repository")
> +.short("c")
> +.long("repository")
> +.value_name("dash_r"),
> +)

This is definitely nifty and an impressive achievement \o/

The `r-` commands for testing pure Rust code paths are an interesting idea!

I think I'm OK with including support for this in `hgcli`. But I think the code 
should live in a separate file so it doesn't pollute `main()`. And it should be 
behind a Cargo feature flag so we maintain compatibility with `hg` as much as 
possible by default.

Also, Mercurial's command line parser is extremely wonky and has some 
questionable behavior. If the intent is to make `rhg` compatible with `hg`, we 
would need to preserve this horrible behavior. We'll likely have to write a 
custom argument parser because of how quirky Mercurial's argument parser is :(

> config.rs:95
> +} else {
> +RevlogFormat::V0
> +}

I would not worry about supporting v0 or v2 at this time. v0 is only important 
for backwards compatibility with ancient repos. And v2 never got off the ground.

> revlog_v1.rs:279
> +let mut fhandle = BufReader::new(match self.dflag {
> +Inline => fs::File::open(self.index_file.as_path()).unwrap(),
> +Separated(ref dfile) => fs::File::open(dfile).unwrap(),

IIRC, core Mercurial keeps an open file handle on revlogs and ensures we don't 
run out of file handles by not keeping too many revlogs open at the same time. 
For scanning operations, not having to open and close the file handles all the 
time will make a difference for performance. Also, core Mercurial loads the 
entirety of the `.i` file into memory. That's a scaling problem for large 
revlogs. But it does make performance of index lookups really fast.

> revlog_v1.rs:290-293
> +while let Some(ref chld_r) = it.next() {
> +if let Some(bin) = self.get_content( fhandle, chld_r) {
> +bins.push(bin);
> +} else {

A thread pool to help with zlib decompression should go a long way here.

Probably too early to think about this, but we'll likely eventually want a 
global thread pool for doing I/O and CPU expensive tasks, such as reading 
chunks from a revlog and decompressing them.

FWIW, we're going to radically alter the storage format in order to better 
support shallow clones. But that work hasn't started yet. I still think there 
is a benefit to implementing the revlog code in Rust 

D2057: rust implementation of hg status

[Ivzhh (Sheng Mao)]

Ivzhh added a comment.


  Hi all,
  
  Based on the discussion a few weeks ago, I come up with a solution for review 
and discussion. After reading the Oxidation plan, the first thought is to 
bypass python engine and current plugin system IFF on request. If user (maybe 
background checker of IDE) request r-* subcommands, then hg gives rust 
implementations instead of python's. So I try to make hg r-status as fast as 
possible. The submitted version has comparable performance (as an example of 
the performance, not evidence, on my MacBook, in hg's own repo, hg r-status 
150ms, and hg status 220ms). I am using CodeXL to profile the performance, and 
plan to use Future.rs to make the loading parallel and maybe 30ms faster.
  
  The implementation originates from hg python implementation, because the 
python version is really fast. I tried to split into small changes, however, I 
eventually to combine all hgstorage module as one commit.
  
  Thank you for your comments!

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2057

To: Ivzhh, #hg-reviewers
Cc: krbullock, indygreg, durin42, kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2675: debugcommands: fix some %r output with bytestr() wrappers

[durin42 (Augie Fackler)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG7aae39d03139: debugcommands: fix some %r output with 
bytestr() wrappers (authored by durin42, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2675?vs=6627=6725

REVISION DETAIL
  https://phab.mercurial-scm.org/D2675

AFFECTED FILES
  mercurial/debugcommands.py

CHANGE DETAILS

diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -1770,15 +1770,15 @@
 overrides = {}
 if opts['tool']:
 overrides[('ui', 'forcemerge')] = opts['tool']
-ui.note(('with --tool %r\n') % (opts['tool']))
+ui.note(('with --tool %r\n') % (pycompat.bytestr(opts['tool'])))
 
 with ui.configoverride(overrides, 'debugmergepatterns'):
 hgmerge = encoding.environ.get("HGMERGE")
 if hgmerge is not None:
-ui.note(('with HGMERGE=%r\n') % (hgmerge))
+ui.note(('with HGMERGE=%r\n') % (pycompat.bytestr(hgmerge)))
 uimerge = ui.config("ui", "merge")
 if uimerge:
-ui.note(('with ui.merge=%r\n') % (uimerge))
+ui.note(('with ui.merge=%r\n') % (pycompat.bytestr(uimerge)))
 
 ctx = scmutil.revsingle(repo, opts.get('rev'))
 m = scmutil.match(ctx, pats, opts)



To: durin42, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2057: translate base85.c into rust code

[Ivzhh (Sheng Mao)]

Ivzhh updated this revision to Diff 6724.
Ivzhh added a comment.


  - merge with stable
  - translate base85.c into rust code
  - move hgbase85 into independent module
  - add hgstorage crate
  - hg status implementation in rust

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2057?vs=5238=6724

BRANCH
  phab-submit-D2057-2018-02-05 (bookmark) on default (branch)

REVISION DETAIL
  https://phab.mercurial-scm.org/D2057

AFFECTED FILES
  rust/Cargo.lock
  rust/Cargo.toml
  rust/hgbase85/Cargo.toml
  rust/hgbase85/build.rs
  rust/hgbase85/src/base85.rs
  rust/hgbase85/src/cpython_ext.rs
  rust/hgbase85/src/lib.rs
  rust/hgcli/Cargo.toml
  rust/hgcli/build.rs
  rust/hgcli/src/main.rs
  rust/hgstorage/Cargo.toml
  rust/hgstorage/src/changelog.rs
  rust/hgstorage/src/config.rs
  rust/hgstorage/src/dirstate.rs
  rust/hgstorage/src/lib.rs
  rust/hgstorage/src/local_repo.rs
  rust/hgstorage/src/manifest.rs
  rust/hgstorage/src/matcher.rs
  rust/hgstorage/src/mpatch.rs
  rust/hgstorage/src/path_encoding.rs
  rust/hgstorage/src/repository.rs
  rust/hgstorage/src/revlog.rs
  rust/hgstorage/src/revlog_v1.rs
  rust/hgstorage/src/working_context.rs

CHANGE DETAILS

diff --git a/rust/hgstorage/src/working_context.rs 
b/rust/hgstorage/src/working_context.rs
new file mode 100644
--- /dev/null
+++ b/rust/hgstorage/src/working_context.rs
@@ -0,0 +1,108 @@
+use std::path::PathBuf;
+use std::io::prelude::*;
+use std::fs;
+use std::collections::HashMap;
+use std::collections::HashSet as Set;
+use std::sync::{Arc, Mutex, RwLock};
+
+use threadpool::ThreadPool;
+use num_cpus;
+
+use dirstate::{CurrentState, DirState};
+use local_repo::LocalRepo;
+use manifest::{FlatManifest, ManifestEntry};
+use changelog::ChangeLog;
+
+pub struct WorkCtx {
+pub dirstate: Arc,
+pub file_revs: HashMap,
+}
+
+impl WorkCtx {
+pub fn new(
+dot_hg_path: Arc,
+manifest: Arc,
+changelog: Arc,
+) -> Self {
+let dirstate = DirState::new(dot_hg_path.join("dirstate"));
+
+let manifest_id = changelog.get_commit_info();
+
+let rev = manifest
+.inner
+.read()
+.unwrap()
+.node_id_to_rev(_id.manifest_id)
+.unwrap();
+
+let file_revs = manifest.build_file_rev_mapping();
+
+let dirstate = Arc::new(RwLock::new(dirstate));
+
+Self {
+dirstate,
+file_revs,
+}
+}
+
+pub fn status(, repo: ) -> CurrentState {
+let mut state = self.dirstate
+.write()
+.unwrap()
+.walk_dir(repo.repo_root.as_path(), );
+
+if !state.lookup.is_empty() {
+let ncpus = num_cpus::get();
+
+let nworkers = if state.lookup.len() < ncpus {
+state.lookup.len()
+} else {
+ncpus
+};
+
+let pool = ThreadPool::new(nworkers);
+
+let clean = Arc::new(Mutex::new(Set::new()));
+let modified = Arc::new(Mutex::new(Set::new()));
+
+for f in state.lookup.drain() {
+let rl = repo.get_filelog(f.as_path());
+let fl = Arc::new(repo.repo_root.join(f.as_path()));
+
+let (id, p1, p2) = {
+let id = _revs[f.as_path()].id;
+let gd = rl.read().unwrap();
+let rev = gd.node_id_to_rev(id).unwrap();
+
+let p1 = gd.p1_nodeid();
+let p2 = gd.p2_nodeid();
+(id.clone(), p1, p2)
+};
+
+let clean = clean.clone();
+let modified = modified.clone();
+
+pool.execute(move || {
+let mut wfile = fs::File::open(fl.as_path()).unwrap();
+let mut content = Vecnew();
+wfile.read_to_end( content).unwrap();
+if rl.read().unwrap().check_hash(, , ) == id 
{
+clean.lock().unwrap().insert(f);
+} else {
+modified.lock().unwrap().insert(f);
+}
+});
+}
+
+pool.join();
+assert_eq!(pool.panic_count(), 0);
+
+let mut gd = modified.lock().unwrap();
+state.modified.extend(gd.drain());
+let mut gd = clean.lock().unwrap();
+state.clean.extend(gd.drain());
+}
+
+return state;
+}
+}
diff --git a/rust/hgstorage/src/revlog_v1.rs b/rust/hgstorage/src/revlog_v1.rs
new file mode 100644
--- /dev/null
+++ b/rust/hgstorage/src/revlog_v1.rs
@@ -0,0 +1,422 @@
+use std::path::{Path, PathBuf};
+use std::io;
+use std::io::{BufReader, Read, Seek, SeekFrom};
+use std::fs;
+use std::cell::RefCell;
+use std::sync::{Arc, RwLock};
+use std::collections::HashMap as Map;
+
+use byteorder::{BigEndian, ReadBytesExt};
+use 

D2676: tests: stop over-specifying tempfile name

[indygreg (Gregory Szorc)]

indygreg added a comment.


  I'm -0 on this. I'd prefer to change the temp file code to be consistent 
between Python versions. Preferably standardizing on the Python 3 version.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2676

To: durin42, pulkit, #hg-reviewers
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2395: stack: add a new module for stack-related commands

[indygreg (Gregory Szorc)]

indygreg requested changes to this revision.
indygreg added a comment.
This revision now requires changes to proceed.


  I assume this stack will get a post-sprint refresh. So I'm going to wait on 
that before reviewing.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2395

To: lothiraldan, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2714: tests: add test for issue 5494 but with --collapse

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG66c569e57c70: tests: add test for issue 5494 but with 
--collapse (authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2714?vs=6702=6723

REVISION DETAIL
  https://phab.mercurial-scm.org/D2714

AFFECTED FILES
  tests/test-rebase-interruptions.t

CHANGE DETAILS

diff --git a/tests/test-rebase-interruptions.t 
b/tests/test-rebase-interruptions.t
--- a/tests/test-rebase-interruptions.t
+++ b/tests/test-rebase-interruptions.t
@@ -463,3 +463,23 @@
   $ hg resolve --list
   $ test -d .hg/merge
   [1]
+Now try again with --collapse
+  $ hg unbundle -q .hg/strip-backup/fdaca8533b86-7fd70513-rebase.hg
+  $ hg rebase -s 2 -d 1 --noninteractive --collapse
+  rebasing 2:fdaca8533b86 "b" (tip)
+  merging a
+  warning: conflicts while merging a! (edit, then use 'hg resolve --mark')
+  unresolved conflicts (see hg resolve, then hg rebase --continue)
+  [1]
+  $ echo a > a
+  $ echo c >> a
+  $ hg resolve --mark a
+  (no more unresolved files)
+  continue: hg rebase --continue
+  $ hg rebase --continue
+  rebasing 2:fdaca8533b86 "b" (tip)
+  saved backup bundle to 
$TESTTMP/repo/.hg/strip-backup/fdaca8533b86-7fd70513-rebase.hg
+BROKEN: the merge state was not cleared
+  $ hg resolve --list
+  R a
+  $ test -d .hg/merge



To: martinvonz, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2713: tests: .hg/merge is a directory, so use `test -d`

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGc164a3a282c1: tests: .hg/merge is a directory, so use `test 
-d` (authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2713?vs=6701=6722

REVISION DETAIL
  https://phab.mercurial-scm.org/D2713

AFFECTED FILES
  tests/test-rebase-interruptions.t

CHANGE DETAILS

diff --git a/tests/test-rebase-interruptions.t 
b/tests/test-rebase-interruptions.t
--- a/tests/test-rebase-interruptions.t
+++ b/tests/test-rebase-interruptions.t
@@ -461,5 +461,5 @@
   note: rebase of 1:fdaca8533b86 created no changes to commit
   saved backup bundle to 
$TESTTMP/repo/.hg/strip-backup/fdaca8533b86-7fd70513-rebase.hg
   $ hg resolve --list
-  $ test -f .hg/merge
+  $ test -d .hg/merge
   [1]



To: martinvonz, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 2 of 2 V2] hgweb: add a hook for processing LFS file transfer requests

[Gregory Szorc]

On Wed, Mar 7, 2018 at 8:49 PM, Matt Harbison  wrote:

> # HG changeset patch
> # User Matt Harbison 
> # Date 1519275362 18000
> #  Wed Feb 21 23:56:02 2018 -0500
> # Node ID 5544688dec388a7c8a988c074aab659f059f549f
> # Parent  89382cb20bb19e089513b2ce69ef8acfa1f523fd
> hgweb: add a hook for processing LFS file transfer requests
>

> As part of this, the PUT request needs to be handled to upload files.
> Unlike
> the requests to the Batch API, this URI is internally controlled, and
> provided
> to the client in the Batch API.  So without any interoperability concerns,
> the
> URI starts with '/.hg', and reflects where the files are actually stored.
>
> The permission check is deferred to the processing function, because this
> request is used for both uploads and downloads.
>
> diff --git a/mercurial/hgweb/hgweb_mod.py b/mercurial/hgweb/hgweb_mod.py
> --- a/mercurial/hgweb/hgweb_mod.py
> +++ b/mercurial/hgweb/hgweb_mod.py
> @@ -96,6 +96,15 @@
>  """
>  raise ErrorResponse(HTTP_NOT_FOUND)
>
> +def _processlfstransfer(repo, req):
> +"""A hook for the LFS extension to wrap that handles file transfer
> requests.
> +
> +The caller MUST call ``req.checkperm()`` with 'push' or 'pull' after
> it
> +determines whether this is an upload or a download, prior to
> accessing any
> +repository data.
> +"""
> +raise ErrorResponse(HTTP_NOT_FOUND)
> +
>  class requestcontext(object):
>  """Holds state/context for an individual request.
>
> @@ -382,14 +391,20 @@
>  except ErrorResponse as inst:
>  return protohandler['handleerror'](inst)
>
> -# Route LFS Batch API requests to the appropriate handler
> +# Route LFS Batch API and transfer requests to the appropriate
> handler
>
>  if req.env.get(r'HTTP_USER_AGENT', r'').startswith(r'git-lfs/'):
>  try:
> -path = req.env.get(r'PATH_INFO')
> +path = req.env.get(r'PATH_INFO', '')
>  if path == '/.git/info/lfs/objects/batch':
>  self.check_perm(rctx, req, None)
>  return _processlfsbatchreq(rctx.repo, req)
> +elif path.startswith('/.hg/store/lfs/objects'):
>

This scares me a bit because we're leaking internal storage paths into the
URI space. Must we do this? What's relying on this behavior?


> +# NB: This function is responsible for doing the
> appropriate
> +# permission checks after determining if this is an
> upload
> +# or a download.
> +req.checkperm = lambda op: self.check_perm(rctx, req,
> op)
> +return _processlfstransfer(rctx.repo, req)
>  else:
>  raise ErrorResponse(HTTP_NOT_FOUND)
>  except ErrorResponse as inst:
> diff --git a/mercurial/hgweb/server.py b/mercurial/hgweb/server.py
> --- a/mercurial/hgweb/server.py
> +++ b/mercurial/hgweb/server.py
> @@ -111,6 +111,9 @@
>  self.log_error(r"Exception happened during processing "
> r"request '%s':%s%s", self.path, newline, tb)
>
> +def do_PUT(self):
> +self.do_POST()
> +
>  def do_GET(self):
>  self.do_POST()
>
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 2 V2] hgweb: add a hook for processing LFS Batch API requests

[Gregory Szorc]

On Wed, Mar 7, 2018 at 8:49 PM, Matt Harbison  wrote:

> # HG changeset patch
> # User Matt Harbison 
> # Date 1519274700 18000
> #  Wed Feb 21 23:45:00 2018 -0500
> # Node ID 89382cb20bb19e089513b2ce69ef8acfa1f523fd
> # Parent  6dab3bdb1f00932a80ffa07f80ba240c3a4b48df
> hgweb: add a hook for processing LFS Batch API requests
>
> There really isn't a clean way to give LFS a crack at intercepting the
> requests
> without hardcoding some LFS knowledge in the core.  The rationale for this
> URI
> is that the spec for the Batch API[1] defines the URL as the LFS server
> url +
> '/objects/batch'.  The default git URLs are:
>
> Git remote: https://git-server.com/foo/bar
> LFS server: https://git-server.com/foo/bar.git/info/lfs
> Batch API: https://git-server.com/foo/bar.git/info/lfs/objects/batch
>
> '.git/' seems like it's not something a user would normally track.  If we
> adhere
> to how git defines the URLs, then the hg-git extension should be able to
> talk to
> a git based server without any additional work.
>
> I'm not sure if checking the User-Agent is a good idea, but this needs a
> specialized client, and it seems like everyone else is doing it
> (3d48ae1aaa5e,
> e7bb5fc4570c).  We can always back this off if it becomes a nuisance.  A
> web
> browser will see "400: bad method", the same as it would before this
> change.
>
> I'm not sure if None or 'pull' is the proper permission check, but the only
> difference is whether or not `hgweb.allowpull` is checked.  Since nothing
> of
> particular interest is transferred here, and the next phase handles the
> read or
> write, treating this like web interface request seems fine.
>
> [1] https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md
>
> diff --git a/mercurial/hgweb/hgweb_mod.py b/mercurial/hgweb/hgweb_mod.py
> --- a/mercurial/hgweb/hgweb_mod.py
> +++ b/mercurial/hgweb/hgweb_mod.py
> @@ -90,6 +90,12 @@
>  urlel = os.path.dirname(urlel)
>  return reversed(breadcrumb)
>
> +def _processlfsbatchreq(repo, req):
> +"""A hook for the LFS extension to wrap that handles requests to the
> Batch
> +API, and returns the appropriate JSON response.
> +"""
> +raise ErrorResponse(HTTP_NOT_FOUND)
> +
>  class requestcontext(object):
>  """Holds state/context for an individual request.
>
> @@ -376,6 +382,21 @@
>  except ErrorResponse as inst:
>  return protohandler['handleerror'](inst)
>
> +# Route LFS Batch API requests to the appropriate handler
> +
> +if req.env.get(r'HTTP_USER_AGENT', r'').startswith(r'git-lfs/'):
>

I don't like filtering by the user-agent. It is recommended to not do this.
Unless it will cause problems or tons of code complexity to properly lock
out actual Git clients who may get confused by this, my vote is to remove
it and just rely on path filtering.


> +try:
> +path = req.env.get(r'PATH_INFO')
> +if path == '/.git/info/lfs/objects/batch':
>

Relying on PATH_INFO is super annoying. But our code kind of sucks :/

My only suggestion would be to define `parts` in the `else` block of the
`if r'PATH_INFO' in req.env:` above and check `parts == ['.git', 'info',
'lfs', 'objects', 'batch']`. That's still a big ugly though.

Whatever happens, my guess is I will eventually write some patches to clean
the URL parsing code up.


> +self.check_perm(rctx, req, None)
>

Shouldn't this be `pull` instead of `None`? Otherwise, LFS won't honor
web.* to restrict data access.


> +return _processlfsbatchreq(rctx.repo, req)
> +else:
> +raise ErrorResponse(HTTP_NOT_FOUND)
> +except ErrorResponse as inst:
> +req.respond(inst, 'text/plain; charset=utf-8')
> +# No body, since only lfs clients are allowed here
> +return ['']
> +
>  # translate user-visible url structure to internal structure
>
>  args = query.split('/', 2)
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2686: xdiff: add a preprocessing step that trims files

[indygreg (Gregory Szorc)]

indygreg requested changes to this revision.
indygreg added a comment.
This revision now requires changes to proceed.


  I'm overall pretty happy with this. I'm requesting just a few minor fixups.
  
  Also, if you (or anyone else for that matter) wanted to spend time to use 
better variable names and add comments throughout this code, it would be 
greatly appreciated. I find this code challenging to read because of its almost 
non-existent documentation.

INLINE COMMENTS

> xprepare.c:169
> + long plines = 0, pbytes = 0, slines = 0, sbytes = 0, i;
> + const char *pp1, *pp2, *ps1, *ps2;
> +

Bonus points if you resubmit this with more expressive variable names. Just 
because xdiff's code is almost impossible to read doesn't mean we should follow 
suit :)

> xprepare.c:183-193
> + pp1 = msmall.ptr, pp2 = mlarge.ptr;
> + for (i = 0; i < msmall.size && *pp1 == *pp2; ++i) {
> + plines += (*pp1 == '\n');
> + pp1++, pp2++;
> + }
> +
> + ps1 = msmall.ptr + msmall.size - 1, ps2 = mlarge.ptr + mlarge.size - 1;

I'm still showing this as a hot point in the code when compiling with default 
settings used by Python packaging tools. I suspect we can get better results on 
typical compiler flags by tweaking things a bit. But we can do that after this 
lands.

> xprepare.c:199-202
> + for (i = 0; i <= reserved;) {
> + pp1--;
> + i += (*pp1 == '\n');
> + }

This is clever. But `memrchr()` will be easier to read. Plus I suspect it will 
be faster.

If you disagree, let's compromise at:

  i = 0;
  while (i <= reserved) {
 pp1--;
 i += (*pp1 == '\n');
  }

There's no sense using a `for` without the 3rd parameter IMO.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2686

To: quark, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2725: httppeer: refactor how httppeer is created (API)

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Previously, we passed a bunch of arguments to httppeer.__init__,
  validated them, then possibly constructed a valid instance.
  
  A short while ago, we refactored sshpeer so all the validation and
  setup work occurs before the constructor. We introduced a makepeer()
  to hold most of this logic.
  
  This commit gives httppeer the same treatment.
  
  As a sign that the previous design was poor, __del__ no longer
  conditionally checks for the presence of an attribute that may
  not be defined (it is always defined in the new code).
  
  .. api::
  
httppeer.httppeer.__init__ now takes additional arguments.
Instances should be obtained by calling httppeer.instance()
or httppeer.makepeer() instead.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2725

AFFECTED FILES
  mercurial/httppeer.py
  tests/test-check-interfaces.py

CHANGE DETAILS

diff --git a/tests/test-check-interfaces.py b/tests/test-check-interfaces.py
--- a/tests/test-check-interfaces.py
+++ b/tests/test-check-interfaces.py
@@ -50,10 +50,13 @@
 def _restrictcapabilities(self, caps):
 pass
 
+class dummyopener(object):
+handlers = []
+
 # Facilitates testing sshpeer without requiring an SSH server.
 class badpeer(httppeer.httppeer):
 def __init__(self):
-super(badpeer, self).__init__(uimod.ui(), 'http://localhost')
+super(badpeer, self).__init__(None, None, None, dummyopener())
 self.badattribute = True
 
 def badmethod(self):
@@ -67,7 +70,7 @@
 ui = uimod.ui()
 
 checkobject(badpeer())
-checkobject(httppeer.httppeer(ui, 'http://localhost'))
+checkobject(httppeer.httppeer(None, None, None, dummyopener()))
 checkobject(localrepo.localpeer(dummyrepo()))
 checkobject(sshpeer.sshv1peer(ui, 'ssh://localhost/foo', None, dummypipe(),
   dummypipe(), None, None))
diff --git a/mercurial/httppeer.py b/mercurial/httppeer.py
--- a/mercurial/httppeer.py
+++ b/mercurial/httppeer.py
@@ -134,29 +134,17 @@
 self._index = 0
 
 class httppeer(wireproto.wirepeer):
-def __init__(self, ui, path):
+def __init__(self, ui, path, url, opener):
+self._ui = ui
 self._path = path
+self._url = url
 self._caps = None
-self._urlopener = None
-u = util.url(path)
-if u.query or u.fragment:
-raise error.Abort(_('unsupported URL component: "%s"') %
- (u.query or u.fragment))
-
-# urllib cannot handle URLs with embedded user or passwd
-self._url, authinfo = u.authinfo()
-
-self._ui = ui
-ui.debug('using %s\n' % self._url)
-
-self._urlopener = urlmod.opener(ui, authinfo)
+self._urlopener = opener
 
 def __del__(self):
-urlopener = getattr(self, '_urlopener', None)
-if urlopener:
-for h in urlopener.handlers:
-h.close()
-getattr(h, "close_all", lambda: None)()
+for h in self._urlopener.handlers:
+h.close()
+getattr(h, "close_all", lambda: None)()
 
 def _openurl(self, req):
 if (self._ui.debugflag
@@ -480,15 +468,29 @@
 def _abort(self, exception):
 raise exception
 
+def makepeer(ui, path):
+u = util.url(path)
+if u.query or u.fragment:
+raise error.Abort(_('unsupported URL component: "%s"') %
+  (u.query or u.fragment))
+
+# urllib cannot handle URLs with embedded user or passwd.
+url, authinfo = u.authinfo()
+ui.debug('using %s\n' % url)
+
+opener = urlmod.opener(ui, authinfo)
+
+return httppeer(ui, path, url, opener)
+
 def instance(ui, path, create):
 if create:
 raise error.Abort(_('cannot create new http repository'))
 try:
 if path.startswith('https:') and not urlmod.has_https:
 raise error.Abort(_('Python support for SSL and HTTPS '
 'is not installed'))
 
-inst = httppeer(ui, path)
+inst = makepeer(ui, path)
 inst._fetchcaps()
 
 return inst



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2721: util: observable proxy objects for sockets

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  We previously introduced proxy objects and observers for file objects
  to help implement low-level tests for the SSH wire protocol.
  
  In this commit, we do the same for sockets in order to help test the
  HTTP server.
  
  We only proxy/observe some socket methods. I didn't feel like
  implementing all the methods because there are so many of them and
  implementing them will provide no short term value. We can always
  implement them later.
  
  1. no-check-commit because we implement foo_bar methods on stdlib types

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2721

AFFECTED FILES
  mercurial/util.py

CHANGE DETAILS

diff --git a/mercurial/util.py b/mercurial/util.py
--- a/mercurial/util.py
+++ b/mercurial/util.py
@@ -695,6 +695,120 @@
 
 return res
 
+PROXIED_SOCKET_METHODS = {
+r'makefile',
+r'recv',
+r'recvfrom',
+r'recvfrom_into',
+r'recv_into',
+r'send',
+r'sendall',
+r'sendto',
+r'setblocking',
+r'settimeout',
+r'gettimeout',
+r'setsockopt',
+}
+
+class socketproxy(object):
+"""A proxy around a socket that tells a watcher when events occur.
+
+This is like ``fileobjectproxy`` except for sockets.
+
+This type is intended to only be used for testing purposes. Think hard
+before using it in important code.
+"""
+__slots__ = (
+r'_orig',
+r'_observer',
+)
+
+def __init__(self, sock, observer):
+object.__setattr__(self, r'_orig', sock)
+object.__setattr__(self, r'_observer', observer)
+
+def __getattribute__(self, name):
+if name in PROXIED_SOCKET_METHODS:
+return object.__getattribute__(self, name)
+
+return getattr(object.__getattribute__(self, r'_orig'), name)
+
+def __delattr__(self, name):
+return delattr(object.__getattribute__(self, r'_orig'), name)
+
+def __setattr__(self, name, value):
+return setattr(object.__getattribute__(self, r'_orig'), name, value)
+
+def _observedcall(self, name, *args, **kwargs):
+# Call the original object.
+orig = object.__getattribute__(self, r'_orig')
+res = getattr(orig, name)(*args, **kwargs)
+
+# Call a method on the observer of the same name with arguments
+# so it can react, log, etc.
+observer = object.__getattribute__(self, r'_observer')
+fn = getattr(observer, name, None)
+if fn:
+fn(res, *args, **kwargs)
+
+return res
+
+def makefile(self, *args, **kwargs):
+res = object.__getattribute__(self, r'_observedcall')(
+r'makefile', *args, **kwargs)
+
+# The file object may be used for I/O. So we turn it into a
+# proxy using our observer.
+observer = object.__getattribute__(self, r'_observer')
+return makeloggingfileobject(observer.fh, res, observer.name,
+ reads=observer.reads,
+ writes=observer.writes,
+ logdata=observer.logdata)
+
+def recv(self, *args, **kwargs):
+return object.__getattribute__(self, r'_observedcall')(
+r'recv', *args, **kwargs)
+
+def recvfrom(self, *args, **kwargs):
+return object.__getattribute__(self, r'_observedcall')(
+r'recvfrom', *args, **kwargs)
+
+def recvfrom_into(self, *args, **kwargs):
+return object.__getattribute__(self, r'_observedcall')(
+r'recvfrom_into', *args, **kwargs)
+
+def recv_into(self, *args, **kwargs):
+return object.__getattribute__(self, r'_observedcall')(
+r'recv_info', *args, **kwargs)
+
+def send(self, *args, **kwargs):
+return object.__getattribute__(self, r'_observedcall')(
+r'send', *args, **kwargs)
+
+def sendall(self, *args, **kwargs):
+return object.__getattribute__(self, r'_observedcall')(
+r'sendall', *args, **kwargs)
+
+def sendto(self, *args, **kwargs):
+return object.__getattribute__(self, r'_observedcall')(
+r'sendto', *args, **kwargs)
+
+def setblocking(self, *args, **kwargs):
+return object.__getattribute__(self, r'_observedcall')(
+r'setblocking', *args, **kwargs)
+
+def settimeout(self, *args, **kwargs):
+return object.__getattribute__(self, r'_observedcall')(
+r'settimeout', *args, **kwargs)
+
+def gettimeout(self, *args, **kwargs):
+return object.__getattribute__(self, r'_observedcall')(
+r'gettimeout', *args, **kwargs)
+
+def setsockopt(self, *args, **kwargs):
+return object.__getattribute__(self, r'_observedcall')(
+r'setsockopt', *args, **kwargs)
+
 DATA_ESCAPE_MAP = {pycompat.bytechr(i): br'\x%02x' % i for i in range(256)}
 

D2723: httppeer: remove _requestbuilder attribute

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This was a glorified alias to a function in another module. I have no
  clue why it existed.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2723

AFFECTED FILES
  mercurial/httppeer.py

CHANGE DETAILS

diff --git a/mercurial/httppeer.py b/mercurial/httppeer.py
--- a/mercurial/httppeer.py
+++ b/mercurial/httppeer.py
@@ -138,7 +138,6 @@
 self._path = path
 self._caps = None
 self._urlopener = None
-self._requestbuilder = None
 u = util.url(path)
 if u.query or u.fragment:
 raise error.Abort(_('unsupported URL component: "%s"') %
@@ -151,7 +150,6 @@
 ui.debug('using %s\n' % self._url)
 
 self._urlopener = url.opener(ui, authinfo)
-self._requestbuilder = urlreq.request
 
 def __del__(self):
 urlopener = getattr(self, '_urlopener', None)
@@ -328,7 +326,7 @@
 if varyheaders:
 headers[r'Vary'] = r','.join(varyheaders)
 
-req = self._requestbuilder(pycompat.strurl(cu), data, headers)
+req = urlreq.request(pycompat.strurl(cu), data, headers)
 
 if data is not None:
 self.ui.debug("sending %d bytes\n" % size)



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2724: httppeer: alias url as urlmod

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  "url" is a common variable name. We do this aliasing elsewhere to
  avoid shadowing.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2724

AFFECTED FILES
  mercurial/httppeer.py

CHANGE DETAILS

diff --git a/mercurial/httppeer.py b/mercurial/httppeer.py
--- a/mercurial/httppeer.py
+++ b/mercurial/httppeer.py
@@ -22,7 +22,7 @@
 httpconnection,
 pycompat,
 statichttprepo,
-url,
+url as urlmod,
 util,
 wireproto,
 )
@@ -149,7 +149,7 @@
 self._ui = ui
 ui.debug('using %s\n' % self._url)
 
-self._urlopener = url.opener(ui, authinfo)
+self._urlopener = urlmod.opener(ui, authinfo)
 
 def __del__(self):
 urlopener = getattr(self, '_urlopener', None)
@@ -484,7 +484,7 @@
 if create:
 raise error.Abort(_('cannot create new http repository'))
 try:
-if path.startswith('https:') and not url.has_https:
+if path.startswith('https:') and not urlmod.has_https:
 raise error.Abort(_('Python support for SSL and HTTPS '
 'is not installed'))
 



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2722: url: add HTTP handler that uses a proxied socket

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Now that we have a socket proxy that can log I/O, we need to teach
  mechanisms that open URLs how to use.
  
  We invent a custom HTTP handler class that knows how to proxy
  sockets as soon as they are opened. We teach the high-level
  opener() to accept logging arguments so a logging HTTP handler
  can be constructed.
  
  1. no-check-commit because we must name http_open that way

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2722

AFFECTED FILES
  mercurial/url.py

CHANGE DETAILS

diff --git a/mercurial/url.py b/mercurial/url.py
--- a/mercurial/url.py
+++ b/mercurial/url.py
@@ -296,6 +296,34 @@
 _generic_start_transaction(self, h, req)
 return keepalive.HTTPHandler._start_transaction(self, h, req)
 
+class logginghttpconnection(keepalive.HTTPConnection):
+def __init__(self, createconn, *args, **kwargs):
+keepalive.HTTPConnection.__init__(self, *args, **kwargs)
+self._create_connection = createconn
+
+class logginghttphandler(httphandler):
+"""HTTP handler that logs socket I/O."""
+def __init__(self, logfh, name, observeropts):
+super(logginghttphandler, self).__init__()
+
+self._logfh = logfh
+self._logname = name
+self._observeropts = observeropts
+
+# do_open() calls the passed class to instantiate an HTTPConnection. We
+# pass in a callable method that creates a custom HTTPConnection instance
+# whose callback to create the socket knows how to proxy the socket.
+def http_open(self, req):
+return self.do_open(self._makeconnection, req)
+
+def _makeconnection(self, *args, **kwargs):
+def createconnection(*args, **kwargs):
+sock = socket.create_connection(*args, **kwargs)
+return util.makeloggingsocket(self._logfh, sock, self._logname,
+  **self._observeropts)
+
+return logginghttpconnection(createconnection, *args, **kwargs)
+
 if has_https:
 class httpsconnection(httplib.HTTPConnection):
 response_class = keepalive.HTTPResponse
@@ -465,14 +493,32 @@
 
 handlerfuncs = []
 
-def opener(ui, authinfo=None, useragent=None):
+def opener(ui, authinfo=None, useragent=None, loggingfh=None,
+   loggingname=b's', loggingopts=None):
 '''
 construct an opener suitable for urllib2
 authinfo will be added to the password manager
+
+The opener can be configured to log socket events if the various
+``logging*`` arguments are specified.
+
+``loggingfh`` denotes a file object to log events to.
+``loggingname`` denotes the name of the to print when logging.
+``loggingopts`` is a dict of keyword arguments to pass to the constructed
+``util.socketobserver`` instance.
 '''
-handlers = [httphandler()]
-if has_https:
-handlers.append(httpshandler(ui))
+handlers = []
+
+if loggingfh:
+handlers.append(logginghttphandler(loggingfh, loggingname,
+   loggingopts or {}))
+# We don't yet support HTTPS when logging I/O. If we attempt to open
+# an HTTPS URL, we'll likely fail due to unknown protocol.
+
+else:
+handlers.append(httphandler())
+if has_https:
+handlers.append(httpshandler(ui))
 
 handlers.append(proxyhandler(ui))
 



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2726: debugcommands: support connecting to HTTP peers

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Now that we have the plumbing for logging socket activity, let's hook
  it up to `hg debugwireproto` so we can collect low-level activity on
  sockets.
  
  The new code is a bit incomplete. But it is better than nothing:
  `hg debugwireproto` is still heavily evolving.
  
  The added test demonstrates some interesting behavior. For example,
  we're calling socket.makefile() and doing I/O on that. TIL. We're also
  sending an "Accept-Encoding: identity" request header. (This is
  probably not necessary.)
  
  I don't plan on implementing too many HTTP protocol tests at this
  time because they are a bit verbose and the liberal use of ignoring
  variable length fields is annoying. Although, I may evolve
  `hg debugwireproto` to make the output less verbose to facilitate
  testing. I'm still working things out...

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2726

AFFECTED FILES
  mercurial/debugcommands.py
  tests/test-http-protocol.t

CHANGE DETAILS

diff --git a/tests/test-http-protocol.t b/tests/test-http-protocol.t
--- a/tests/test-http-protocol.t
+++ b/tests/test-http-protocol.t
@@ -161,3 +161,68 @@
   : 32 30 30 20 53 63 72 69 70 74 20 6f 75 74 70 75 |200 Script outpu|
   0010: 74 20 66 6f 6c 6c 6f 77 73 0a 0a 04 7a 6c 69 62 |t follows...zlib|
   0020: 78  |x|
+
+  $ killdaemons.py
+  $ cd ..
+
+Test listkeys for listing namespaces
+
+  $ hg init empty
+  $ hg -R empty serve -p $HGPORT -d --pid-file hg.pid
+  $ cat hg.pid > $DAEMON_PIDS
+  $ hg --verbose debugwireproto http://$LOCALIP:$HGPORT << EOF
+  > command listkeys
+  > namespace namespaces
+  > EOF
+  s> sendall(*, 0): (glob)
+  s> GET /?cmd=capabilities HTTP/1.1\r\n
+  s> Accept-Encoding: identity\r\n
+  s> accept: application/mercurial-0.1\r\n
+  s> host: $LOCALIP:$HGPORT\r\n (glob)
+  s> user-agent: mercurial/proto-1.0 (Mercurial *)\r\n (glob)
+  s> \r\n
+  s> makefile('rb', None)
+  s> readline() -> 36:
+  s> HTTP/1.1 200 Script output follows\r\n
+  s> readline() -> *: (glob)
+  s> Server: *\r\n (glob)
+  s> readline() -> *: (glob)
+  s> Date: *\r\n (glob)
+  s> readline() -> 41:
+  s> Content-Type: application/mercurial-0.1\r\n
+  s> readline() -> 21:
+  s> Content-Length: *\r\n (glob)
+  s> readline() -> 2:
+  s> \r\n
+  s> read(*) -> *: lookup branchmap pushkey known getbundle unbundlehash batch 
changegroupsubset streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ 
unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx 
compression=$BUNDLE2_COMPRESSIONS$ (glob)
+  sending listkeys command
+  s> sendall(*, 0): (glob)
+  s> GET /?cmd=listkeys HTTP/1.1\r\n
+  s> Accept-Encoding: identity\r\n
+  s> vary: X-HgArg-1,X-HgProto-1\r\n
+  s> x-hgarg-1: namespace=namespaces\r\n
+  s> x-hgproto-1: 0.1 0.2 comp=$USUAL_COMPRESSIONS$\r\n
+  s> accept: application/mercurial-0.1\r\n
+  s> host: $LOCALIP:$HGPORT\r\n (glob)
+  s> user-agent: mercurial/proto-1.0 (Mercurial *)\r\n (glob)
+  s> \r\n
+  s> makefile('rb', None)
+  s> readline() -> 36:
+  s> HTTP/1.1 200 Script output follows\r\n
+  s> readline() -> 36:
+  s> Server: *\r\n (glob)
+  s> readline() -> *: (glob)
+  s> Date: *\r\n (glob)
+  s> readline() -> 41:
+  s> Content-Type: application/mercurial-0.1\r\n
+  s> readline() -> 20:
+  s> Content-Length: 30\r\n
+  s> readline() -> 2:
+  s> \r\n
+  s> read(30) -> 30:
+  s> bookmarks \n
+  s> namespaces\n
+  s> phases
+  response: bookmarks  \nnamespaces\nphases
+
+  $ killdaemons.py
diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -47,6 +47,7 @@
 fileset,
 formatter,
 hg,
+httppeer,
 localrepo,
 lock as lockmod,
 logcmdutil,
@@ -2571,9 +2572,9 @@
 ('', 'peer', '', _('construct a specific version of the peer')),
 ('', 'noreadstderr', False, _('do not read from stderr of the 
remote')),
 ] + cmdutil.remoteopts,
-_('[REPO]'),
+_('[PATH]'),
 optionalrepo=True)
-def debugwireproto(ui, repo, **opts):
+def debugwireproto(ui, repo, path=None, **opts):
 """send wire protocol commands to a server
 
 This command can be used to issue wire protocol commands to remote
@@ -2708,12 +2709,19 @@
 raise error.Abort(_('invalid value for --peer'),
   hint=_('valid values are "raw", "ssh1", and "ssh2"'))
 
+if path and opts['localssh']:
+raise error.Abort(_('cannot specify --localssh with an explicit '
+'path'))
+
 if ui.interactive():
 ui.write(_('(waiting for commands on stdin)\n'))
 
 blocks = list(_parsewirelangblocks(ui.fin))
 
  

[PATCH 1 of 2 V2] hgweb: add a hook for processing LFS Batch API requests

[Matt Harbison]

# HG changeset patch
# User Matt Harbison 
# Date 1519274700 18000
#  Wed Feb 21 23:45:00 2018 -0500
# Node ID 89382cb20bb19e089513b2ce69ef8acfa1f523fd
# Parent  6dab3bdb1f00932a80ffa07f80ba240c3a4b48df
hgweb: add a hook for processing LFS Batch API requests

There really isn't a clean way to give LFS a crack at intercepting the requests
without hardcoding some LFS knowledge in the core.  The rationale for this URI
is that the spec for the Batch API[1] defines the URL as the LFS server url +
'/objects/batch'.  The default git URLs are:

Git remote: https://git-server.com/foo/bar
LFS server: https://git-server.com/foo/bar.git/info/lfs
Batch API: https://git-server.com/foo/bar.git/info/lfs/objects/batch

'.git/' seems like it's not something a user would normally track.  If we adhere
to how git defines the URLs, then the hg-git extension should be able to talk to
a git based server without any additional work.

I'm not sure if checking the User-Agent is a good idea, but this needs a
specialized client, and it seems like everyone else is doing it (3d48ae1aaa5e,
e7bb5fc4570c).  We can always back this off if it becomes a nuisance.  A web
browser will see "400: bad method", the same as it would before this change.

I'm not sure if None or 'pull' is the proper permission check, but the only
difference is whether or not `hgweb.allowpull` is checked.  Since nothing of
particular interest is transferred here, and the next phase handles the read or
write, treating this like web interface request seems fine.

[1] https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md

diff --git a/mercurial/hgweb/hgweb_mod.py b/mercurial/hgweb/hgweb_mod.py
--- a/mercurial/hgweb/hgweb_mod.py
+++ b/mercurial/hgweb/hgweb_mod.py
@@ -90,6 +90,12 @@
 urlel = os.path.dirname(urlel)
 return reversed(breadcrumb)
 
+def _processlfsbatchreq(repo, req):
+"""A hook for the LFS extension to wrap that handles requests to the Batch
+API, and returns the appropriate JSON response.
+"""
+raise ErrorResponse(HTTP_NOT_FOUND)
+
 class requestcontext(object):
 """Holds state/context for an individual request.
 
@@ -376,6 +382,21 @@
 except ErrorResponse as inst:
 return protohandler['handleerror'](inst)
 
+# Route LFS Batch API requests to the appropriate handler
+
+if req.env.get(r'HTTP_USER_AGENT', r'').startswith(r'git-lfs/'):
+try:
+path = req.env.get(r'PATH_INFO')
+if path == '/.git/info/lfs/objects/batch':
+self.check_perm(rctx, req, None)
+return _processlfsbatchreq(rctx.repo, req)
+else:
+raise ErrorResponse(HTTP_NOT_FOUND)
+except ErrorResponse as inst:
+req.respond(inst, 'text/plain; charset=utf-8')
+# No body, since only lfs clients are allowed here
+return ['']
+
 # translate user-visible url structure to internal structure
 
 args = query.split('/', 2)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 2 V2] hgweb: add a hook for processing LFS file transfer requests

[Matt Harbison]

# HG changeset patch
# User Matt Harbison 
# Date 1519275362 18000
#  Wed Feb 21 23:56:02 2018 -0500
# Node ID 5544688dec388a7c8a988c074aab659f059f549f
# Parent  89382cb20bb19e089513b2ce69ef8acfa1f523fd
hgweb: add a hook for processing LFS file transfer requests

As part of this, the PUT request needs to be handled to upload files.  Unlike
the requests to the Batch API, this URI is internally controlled, and provided
to the client in the Batch API.  So without any interoperability concerns, the
URI starts with '/.hg', and reflects where the files are actually stored.

The permission check is deferred to the processing function, because this
request is used for both uploads and downloads.

diff --git a/mercurial/hgweb/hgweb_mod.py b/mercurial/hgweb/hgweb_mod.py
--- a/mercurial/hgweb/hgweb_mod.py
+++ b/mercurial/hgweb/hgweb_mod.py
@@ -96,6 +96,15 @@
 """
 raise ErrorResponse(HTTP_NOT_FOUND)
 
+def _processlfstransfer(repo, req):
+"""A hook for the LFS extension to wrap that handles file transfer 
requests.
+
+The caller MUST call ``req.checkperm()`` with 'push' or 'pull' after it
+determines whether this is an upload or a download, prior to accessing any
+repository data.
+"""
+raise ErrorResponse(HTTP_NOT_FOUND)
+
 class requestcontext(object):
 """Holds state/context for an individual request.
 
@@ -382,14 +391,20 @@
 except ErrorResponse as inst:
 return protohandler['handleerror'](inst)
 
-# Route LFS Batch API requests to the appropriate handler
+# Route LFS Batch API and transfer requests to the appropriate handler
 
 if req.env.get(r'HTTP_USER_AGENT', r'').startswith(r'git-lfs/'):
 try:
-path = req.env.get(r'PATH_INFO')
+path = req.env.get(r'PATH_INFO', '')
 if path == '/.git/info/lfs/objects/batch':
 self.check_perm(rctx, req, None)
 return _processlfsbatchreq(rctx.repo, req)
+elif path.startswith('/.hg/store/lfs/objects'):
+# NB: This function is responsible for doing the 
appropriate
+# permission checks after determining if this is an upload
+# or a download.
+req.checkperm = lambda op: self.check_perm(rctx, req, op)
+return _processlfstransfer(rctx.repo, req)
 else:
 raise ErrorResponse(HTTP_NOT_FOUND)
 except ErrorResponse as inst:
diff --git a/mercurial/hgweb/server.py b/mercurial/hgweb/server.py
--- a/mercurial/hgweb/server.py
+++ b/mercurial/hgweb/server.py
@@ -111,6 +111,9 @@
 self.log_error(r"Exception happened during processing "
r"request '%s':%s%s", self.path, newline, tb)
 
+def do_PUT(self):
+self.do_POST()
+
 def do_GET(self):
 self.do_POST()
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2720: debugcommands: introduce actions to perform deterministic reads

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  "readavailable" is useful as a debugging device to see what data is
  available on a pipe. But the mechanism isn't deterministic because
  what's available on a pipe is highly conditional on timing, system
  load, OS behavior, etc. This makes it not suitable for tests.
  
  We introduce "ereadline," "read," and "eread" for performing
  deterministic I/O operations (at least on blocking file descriptors).
  We convert uses of "readavailable" in tests to use the new
  primitives. This should hopefully make these tests deterministic.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2720

AFFECTED FILES
  mercurial/debugcommands.py
  tests/test-ssh-proto-unbundle.t
  tests/test-ssh-proto.t

CHANGE DETAILS

diff --git a/tests/test-ssh-proto.t b/tests/test-ssh-proto.t
--- a/tests/test-ssh-proto.t
+++ b/tests/test-ssh-proto.t
@@ -1118,7 +1118,8 @@
   > upgrade another-token proto=irrelevant\n
   > hello\n
   > readline
-  > readavailable
+  > ereadline
+  > ereadline
   > EOF
   using raw connection to peer
   i> write(153) -> 153:
@@ -1138,8 +1139,9 @@
   i> hello\n
   o> readline() -> 1:
   o> \n
-  e> read(-1) -> 42:
+  e> readline() -> 40:
   e> cannot upgrade protocols multiple times\n
+  e> readline() -> 2:
   e> -\n
 
 Malformed upgrade request line (not exactly 3 space delimited tokens)
@@ -1221,35 +1223,39 @@
   > upgrade token proto=exp-ssh-v2-0001\n
   > invalid\n
   > readline
-  > readavailable
+  > ereadline
+  > ereadline
   > EOF
   using raw connection to peer
   i> write(44) -> 44:
   i> upgrade token proto=exp-ssh-v2-0001\n
   i> invalid\n
   o> readline() -> 1:
   o> \n
-  e> read(-1) -> 46:
+  e> readline() -> 44:
   e> malformed handshake protocol: missing hello\n
+  e> readline() -> 2:
   e> -\n
 
   $ hg debugwireproto --localssh --peer raw << EOF
   > raw
   > upgrade token proto=exp-ssh-v2-0001\n
   > hello\n
   > invalid\n
   > readline
-  > readavailable
+  > ereadline
+  > ereadline
   > EOF
   using raw connection to peer
   i> write(50) -> 50:
   i> upgrade token proto=exp-ssh-v2-0001\n
   i> hello\n
   i> invalid\n
   o> readline() -> 1:
   o> \n
-  e> read(-1) -> 48:
+  e> readline() -> 46:
   e> malformed handshake protocol: missing between\n
+  e> readline() -> 2:
   e> -\n
 
   $ hg debugwireproto --localssh --peer raw << EOF
@@ -1259,7 +1265,8 @@
   > between\n
   > invalid\n
   > readline
-  > readavailable
+  > ereadline
+  > ereadline
   > EOF
   using raw connection to peer
   i> write(58) -> 58:
@@ -1269,8 +1276,9 @@
   i> invalid\n
   o> readline() -> 1:
   o> \n
-  e> read(-1) -> 49:
+  e> readline() -> 47:
   e> malformed handshake protocol: missing pairs 81\n
+  e> readline() -> 2:
   e> -\n
 
 Legacy commands are not exposed to version 2 of protocol
diff --git a/tests/test-ssh-proto-unbundle.t b/tests/test-ssh-proto-unbundle.t
--- a/tests/test-ssh-proto-unbundle.t
+++ b/tests/test-ssh-proto-unbundle.t
@@ -45,7 +45,7 @@
   > # This is "force" in hex.
   > heads 666f726365
   > PUSHFILE ../initial.v1.hg
-  > readavailable
+  > eread 115
   > EOF
   testing ssh1
   creating ssh peer from handshake results
@@ -93,7 +93,7 @@
   o> read(1) -> 1: 0
   result: 0
   remote output: 
-  e> read(-1) -> 115:
+  e> read(115) -> 115:
   e> abort: incompatible Mercurial client; bundle2 required\n
   e> (see https://www.mercurial-scm.org/wiki/IncompatibleClient)\n
   
@@ -143,7 +143,7 @@
   o> read(1) -> 1: 0
   result: 0
   remote output: 
-  e> read(-1) -> 115:
+  e> read(115) -> 115:
   e> abort: incompatible Mercurial client; bundle2 required\n
   e> (see https://www.mercurial-scm.org/wiki/IncompatibleClient)\n
 
@@ -212,7 +212,7 @@
   > # This is "force" in hex.
   > heads 666f726365
   > PUSHFILE ../initial.v1.hg
-  > readavailable
+  > eread 196
   > EOF
   testing ssh1
   creating ssh peer from handshake results
@@ -260,7 +260,7 @@
   o> read(1) -> 1: 0
   result: 0
   remote output: 
-  e> read(-1) -> 196:
+  e> read(196) -> 196:
   e> adding changesets\n
   e> adding manifests\n
   e> adding file changes\n
@@ -316,7 +316,7 @@
   o> read(1) -> 1: 0
   result: 0
   remote output: 
-  e> read(-1) -> 196:
+  e> read(196) -> 196:
   e> adding changesets\n
   e> adding manifests\n
   e> adding file changes\n
@@ -338,7 +338,7 @@
   > # This is "force" in hex.
   > heads 666f726365
   > PUSHFILE ../initial.v1.hg
-  > readavailable
+  > eread 218
   > EOF
   testing ssh1
   creating ssh peer from handshake results
@@ -386,7 +386,7 @@
   o> read(1) -> 1: 0
   result: 0
   remote output: 
-  e> read(-1) -> 218:
+  e> read(218) -> 218:
   e> adding changesets\n
   e> adding manifests\n
   e> adding file changes\n
@@ -443,7 +443,7 @@
   o> 

D2718: wireproto: declare permissions requirements in @wireprotocommand (API)

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  With the security patches from 4.5.2 merged into default, we now
  have a per-command attribute defining what permissions are needed
  to run that command. We now have a richer @wireprotocommand that
  can be extended to record additional command metadata. So we
  port the permissions mechanism to be based on @wireprotocommand.
  
  .. api::
  
hgweb_mod.perms and wireproto.permissions have been removed. Wire
protocol commands should declare their required permissions in the
@wireprotocommand decorator.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2718

AFFECTED FILES
  hgext/largefiles/uisetup.py
  mercurial/hgweb/hgweb_mod.py
  mercurial/wireproto.py
  mercurial/wireprotoserver.py
  tests/test-http-permissions.t

CHANGE DETAILS

diff --git a/tests/test-http-permissions.t b/tests/test-http-permissions.t
--- a/tests/test-http-permissions.t
+++ b/tests/test-http-permissions.t
@@ -21,12 +21,10 @@
   > @wireproto.wireprotocommand('customwritenoperm')
   > def customwritenoperm(repo, proto):
   > return b'write command no defined permissions\n'
-  > wireproto.permissions['customreadwithperm'] = 'pull'
-  > @wireproto.wireprotocommand('customreadwithperm')
+  > @wireproto.wireprotocommand('customreadwithperm', permission='pull')
   > def customreadwithperm(repo, proto):
   > return b'read-only command w/ defined permissions\n'
-  > wireproto.permissions['customwritewithperm'] = 'push'
-  > @wireproto.wireprotocommand('customwritewithperm')
+  > @wireproto.wireprotocommand('customwritewithperm', permission='push')
   > def customwritewithperm(repo, proto):
   > return b'write command w/ defined permissions\n'
   > EOF
diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -242,11 +242,7 @@
'over HTTP'))
 return []
 
-# Assume commands with no defined permissions are writes /
-# for pushes. This is the safest from a security perspective
-# because it doesn't allow commands with undefined semantics
-# from bypassing permissions checks.
-checkperm(wireproto.permissions.get(cmd, 'push'))
+checkperm(wireproto.commands[cmd].permission)
 
 rsp = wireproto.dispatch(repo, proto, cmd)
 
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -592,10 +592,12 @@
 
 class commandentry(object):
 """Represents a declared wire protocol command."""
-def __init__(self, func, args='', transports=None):
+def __init__(self, func, args='', transports=None,
+ permission='push'):
 self.func = func
 self.args = args
 self.transports = transports or set()
+self.permission = permission
 
 def _merge(self, func, args):
 """Merge this instance with an incoming 2-tuple.
@@ -605,7 +607,8 @@
 data not captured by the 2-tuple and a new instance containing
 the union of the two objects is returned.
 """
-return commandentry(func, args=args, transports=set(self.transports))
+return commandentry(func, args=args, transports=set(self.transports),
+permission=self.permission)
 
 # Old code treats instances as 2-tuples. So expose that interface.
 def __iter__(self):
@@ -643,7 +646,8 @@
 else:
 # Use default values from @wireprotocommand.
 v = commandentry(v[0], args=v[1],
- transports=set(wireprototypes.TRANSPORTS))
+ transports=set(wireprototypes.TRANSPORTS),
+ permission='push')
 else:
 raise ValueError('command entries must be commandentry instances '
  'or 2-tuples')
@@ -672,12 +676,8 @@
 
 commands = commanddict()
 
-# Maps wire protocol name to operation type. This is used for permissions
-# checking. All defined @wireiprotocommand should have an entry in this
-# dict.
-permissions = {}
-
-def wireprotocommand(name, args='', transportpolicy=POLICY_ALL):
+def wireprotocommand(name, args='', transportpolicy=POLICY_ALL,
+ permission='push'):
 """Decorator to declare a wire protocol command.
 
 ``name`` is the name of the wire protocol command being provided.
@@ -688,6 +688,12 @@
 ``transportpolicy`` is a POLICY_* constant denoting which transports
 this wire protocol command should be exposed to. By default, commands
 are exposed to all wire protocol transports.
+
+``permission`` defines the permission type needed to run this command.
+Can be ``push`` or ``pull``. These roughly map to read-write and read-only,
+respectively. Default is to assume command 

D2719: wireproto: formalize permissions checking as part of protocol interface

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Per the inline comment desiring to formalize permissions checking
  in the protocol interface, we do that.
  
  I'm not convinced this is the best way to go about things. I would love
  for there to e.g. be a better exception for denoting permissions
  problems. But it does feel strictly better than snipping attributes
  on the proto instance.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2719

AFFECTED FILES
  mercurial/hgweb/hgweb_mod.py
  mercurial/wireproto.py
  mercurial/wireprotoserver.py
  mercurial/wireprototypes.py
  tests/test-wireproto.py

CHANGE DETAILS

diff --git a/tests/test-wireproto.py b/tests/test-wireproto.py
--- a/tests/test-wireproto.py
+++ b/tests/test-wireproto.py
@@ -18,6 +18,9 @@
 names = spec.split()
 return [args[n] for n in names]
 
+def checkperm(self, perm):
+pass
+
 class clientpeer(wireproto.wirepeer):
 def __init__(self, serverrepo):
 self.serverrepo = serverrepo
diff --git a/mercurial/wireprototypes.py b/mercurial/wireprototypes.py
--- a/mercurial/wireprototypes.py
+++ b/mercurial/wireprototypes.py
@@ -146,3 +146,12 @@
 
 Returns a list of capabilities. The passed in argument can be returned.
 """
+
+@abc.abstractmethod
+def checkperm(self, perm):
+"""Validate that the client has permissions to perform a request.
+
+The argument is the permission required to proceed. If the client
+doesn't have that permission, the exception should raise or abort
+in a protocol specific manner.
+"""
diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -54,9 +54,10 @@
 return ''.join(chunks)
 
 class httpv1protocolhandler(wireprototypes.baseprotocolhandler):
-def __init__(self, req, ui):
+def __init__(self, req, ui, checkperm):
 self._req = req
 self._ui = ui
+self._checkperm = checkperm
 
 @property
 def name(self):
@@ -139,14 +140,17 @@
 
 return caps
 
+def checkperm(self, perm):
+return self._checkperm(perm)
+
 # This method exists mostly so that extensions like remotefilelog can
 # disable a kludgey legacy method only over http. As of early 2018,
 # there are no other known users, so with any luck we can discard this
 # hook if remotefilelog becomes a first-party extension.
 def iscmd(cmd):
 return cmd in wireproto.commands
 
-def parsehttprequest(repo, req, query):
+def parsehttprequest(rctx, req, query, checkperm):
 """Parse the HTTP request for a wire protocol request.
 
 If the current request appears to be a wire protocol request, this
@@ -156,6 +160,8 @@
 
 ``req`` is a ``wsgirequest`` instance.
 """
+repo = rctx.repo
+
 # HTTP version 1 wire protocol requests are denoted by a "cmd" query
 # string parameter. If it isn't present, this isn't a wire protocol
 # request.
@@ -174,13 +180,13 @@
 if not iscmd(cmd):
 return None
 
-proto = httpv1protocolhandler(req, repo.ui)
+proto = httpv1protocolhandler(req, repo.ui,
+  lambda perm: checkperm(rctx, req, perm))
 
 return {
 'cmd': cmd,
 'proto': proto,
-'dispatch': lambda checkperm: _callhttp(repo, req, proto, cmd,
-checkperm),
+'dispatch': lambda: _callhttp(repo, req, proto, cmd),
 'handleerror': lambda ex: _handlehttperror(ex, req, cmd),
 }
 
@@ -224,7 +230,7 @@
 opts = {'level': ui.configint('server', 'zliblevel')}
 return HGTYPE, util.compengines['zlib'], opts
 
-def _callhttp(repo, req, proto, cmd, checkperm):
+def _callhttp(repo, req, proto, cmd):
 def genversion2(gen, engine, engineopts):
 # application/mercurial-0.2 always sends a payload header
 # identifying the compression engine.
@@ -242,7 +248,7 @@
'over HTTP'))
 return []
 
-checkperm(wireproto.commands[cmd].permission)
+proto.checkperm(wireproto.commands[cmd].permission)
 
 rsp = wireproto.dispatch(repo, proto, cmd)
 
@@ -392,6 +398,9 @@
 def addcapabilities(self, repo, caps):
 return caps
 
+def checkperm(self, perm):
+pass
+
 class sshv2protocolhandler(sshv1protocolhandler):
 """Protocol handler for version 2 of the SSH protocol."""
 
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -731,13 +731,10 @@
 vals[unescapearg(n)] = unescapearg(v)
 func, spec = commands[op]
 
-# If the protocol supports permissions checking, perform that
-# checking on each batched command.
-# TODO formalize permission checking as part of protocol interface.
-if 

D2716: wireprotoserver: check if command available before calling it

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The previous behavior was just plain wrong. I have no clue how it
  landed. My guess is a merge conflict resolution gone wrong on my
  end a few weeks ago.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2716

AFFECTED FILES
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -235,14 +235,14 @@
 for chunk in gen:
 yield chunk
 
-rsp = wireproto.dispatch(repo, proto, cmd)
-
 if not wireproto.commands.commandavailable(cmd, proto):
 req.respond(HTTP_OK, HGERRTYPE,
 body=_('requested wire protocol command is not available '
'over HTTP'))
 return []
 
+rsp = wireproto.dispatch(repo, proto, cmd)
+
 if isinstance(rsp, bytes):
 req.respond(HTTP_OK, HGTYPE, body=rsp)
 return []



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2717: wireprotoserver: check permissions in main dispatch function

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The permissions checking code merged from stable is out of place
  in the refactored hgweb_mod module.
  
  This commit moves the main call to wireprotoserver. We still have
  some lingering code in hgweb_mod. This will get addressed later.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2717

AFFECTED FILES
  mercurial/hgweb/hgweb_mod.py
  mercurial/wireprotoserver.py

CHANGE DETAILS

diff --git a/mercurial/wireprotoserver.py b/mercurial/wireprotoserver.py
--- a/mercurial/wireprotoserver.py
+++ b/mercurial/wireprotoserver.py
@@ -179,7 +179,8 @@
 return {
 'cmd': cmd,
 'proto': proto,
-'dispatch': lambda: _callhttp(repo, req, proto, cmd),
+'dispatch': lambda checkperm: _callhttp(repo, req, proto, cmd,
+checkperm),
 'handleerror': lambda ex: _handlehttperror(ex, req, cmd),
 }
 
@@ -223,7 +224,7 @@
 opts = {'level': ui.configint('server', 'zliblevel')}
 return HGTYPE, util.compengines['zlib'], opts
 
-def _callhttp(repo, req, proto, cmd):
+def _callhttp(repo, req, proto, cmd, checkperm):
 def genversion2(gen, engine, engineopts):
 # application/mercurial-0.2 always sends a payload header
 # identifying the compression engine.
@@ -241,6 +242,12 @@
'over HTTP'))
 return []
 
+# Assume commands with no defined permissions are writes /
+# for pushes. This is the safest from a security perspective
+# because it doesn't allow commands with undefined semantics
+# from bypassing permissions checks.
+checkperm(wireproto.permissions.get(cmd, 'push'))
+
 rsp = wireproto.dispatch(repo, proto, cmd)
 
 if isinstance(rsp, bytes):
diff --git a/mercurial/hgweb/hgweb_mod.py b/mercurial/hgweb/hgweb_mod.py
--- a/mercurial/hgweb/hgweb_mod.py
+++ b/mercurial/hgweb/hgweb_mod.py
@@ -357,22 +357,15 @@
 protohandler = wireprotoserver.parsehttprequest(rctx.repo, req, query)
 
 if protohandler:
-cmd = protohandler['cmd']
 try:
 if query:
 raise ErrorResponse(HTTP_NOT_FOUND)
 
 # TODO fold this into parsehttprequest
-req.checkperm = lambda op: self.check_perm(rctx, req, op)
-protohandler['proto'].checkperm = req.checkperm
+checkperm = lambda op: self.check_perm(rctx, req, op)
+protohandler['proto'].checkperm = checkperm
 
-# Assume commands with no defined permissions are writes /
-# for pushes. This is the safest from a security perspective
-# because it doesn't allow commands with undefined semantics
-# from bypassing permissions checks.
-req.checkperm(perms.get(cmd, 'push'))
-
-return protohandler['dispatch']()
+return protohandler['dispatch'](checkperm)
 except ErrorResponse as inst:
 return protohandler['handleerror'](inst)
 



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2686: xdiff: add a preprocessing step that trims files

[quark (Jun Wu)]

quark updated this revision to Diff 6708.
quark edited the summary of this revision.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2686?vs=6664=6708

REVISION DETAIL
  https://phab.mercurial-scm.org/D2686

AFFECTED FILES
  mercurial/thirdparty/xdiff/xdiffi.c
  mercurial/thirdparty/xdiff/xprepare.c
  mercurial/thirdparty/xdiff/xtypes.h

CHANGE DETAILS

diff --git a/mercurial/thirdparty/xdiff/xtypes.h 
b/mercurial/thirdparty/xdiff/xtypes.h
--- a/mercurial/thirdparty/xdiff/xtypes.h
+++ b/mercurial/thirdparty/xdiff/xtypes.h
@@ -60,6 +60,7 @@
 
 typedef struct s_xdfenv {
xdfile_t xdf1, xdf2;
+   long nprefix, nsuffix;
 } xdfenv_t;
 
 
diff --git a/mercurial/thirdparty/xdiff/xprepare.c 
b/mercurial/thirdparty/xdiff/xprepare.c
--- a/mercurial/thirdparty/xdiff/xprepare.c
+++ b/mercurial/thirdparty/xdiff/xprepare.c
@@ -156,6 +156,78 @@
 }
 
 
+/*
+ * Trim common prefix from files.
+ *
+ * Note: trimming could have side effects on hunk shifting, but the performance
+ * benefit outweighs the possible shifting change.
+ */
+static void xdl_trim_files(mmfile_t *mf1, mmfile_t *mf2, long reserved,
+   xdfenv_t *xe, mmfile_t *out_mf1, mmfile_t *out_mf2) {
+   mmfile_t msmall, mlarge;
+   long plines = 0, pbytes = 0, slines = 0, sbytes = 0, i;
+   const char *pp1, *pp2, *ps1, *ps2;
+
+   /* reserved must >= 0 for the line boundary adjustment to work */
+   if (reserved < 0)
+   reserved = 0;
+
+   if (mf1->size < mf2->size) {
+   memcpy(, mf1, sizeof(mmfile_t));
+   memcpy(, mf2, sizeof(mmfile_t));
+   } else {
+   memcpy(, mf2, sizeof(mmfile_t));
+   memcpy(, mf1, sizeof(mmfile_t));
+   }
+
+   pp1 = msmall.ptr, pp2 = mlarge.ptr;
+   for (i = 0; i < msmall.size && *pp1 == *pp2; ++i) {
+   plines += (*pp1 == '\n');
+   pp1++, pp2++;
+   }
+
+   ps1 = msmall.ptr + msmall.size - 1, ps2 = mlarge.ptr + mlarge.size - 1;
+   for (; ps1 > pp1 && *ps1 == *ps2; ++i) {
+   slines += (*ps1 == '\n');
+   ps1--, ps2--;
+   }
+
+   /* Retract common prefix and suffix boundaries for reserved lines */
+   if (plines <= reserved + 1) {
+   plines = 0;
+   } else {
+   for (i = 0; i <= reserved;) {
+   pp1--;
+   i += (*pp1 == '\n');
+   }
+   /* The new mmfile starts at the next char just after '\n' */
+   pbytes = pp1 - msmall.ptr + 1;
+   plines -= reserved;
+   }
+
+   if (slines <= reserved + 1) {
+   slines = 0;
+   } else {
+   for (i = 0; i <= reserved;) {
+   ps1++;
+   i += (*ps1 == '\n');
+   }
+   /* The new mmfile includes this '\n' */
+   sbytes = msmall.ptr + msmall.size - ps1 - 1;
+   slines -= reserved;
+   if (msmall.ptr[msmall.size - 1] == '\n')
+   slines -= 1;
+   }
+
+   xe->nprefix = plines;
+   xe->nsuffix = slines;
+   out_mf1->ptr = mf1->ptr + pbytes;
+   out_mf1->size = mf1->size - pbytes - sbytes;
+   out_mf2->ptr = mf2->ptr + pbytes;
+   out_mf2->size = mf2->size - pbytes - sbytes;
+}
+
+
 static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, 
xpparam_t const *xpp,
   xdlclassifier_t *cf, xdfile_t *xdf) {
unsigned int hbits;
@@ -254,10 +326,13 @@
xdl_cha_free(>rcha);
 }
 
+/* Reserved lines for trimming, to leave room for shifting */
+#define TRIM_RESERVED_LINES 100
 
 int xdl_prepare_env(mmfile_t *mf1, mmfile_t *mf2, xpparam_t const *xpp,
xdfenv_t *xe) {
long enl1, enl2, sample;
+   mmfile_t tmf1, tmf2;
xdlclassifier_t cf;
 
memset(, 0, sizeof(cf));
@@ -270,12 +345,14 @@
if (xdl_init_classifier(, enl1 + enl2 + 1, xpp->flags) < 0)
return -1;
 
-   if (xdl_prepare_ctx(1, mf1, enl1, xpp, , >xdf1) < 0) {
+   xdl_trim_files(mf1, mf2, TRIM_RESERVED_LINES, xe, , );
+
+   if (xdl_prepare_ctx(1, , enl1, xpp, , >xdf1) < 0) {
 
xdl_free_classifier();
return -1;
}
-   if (xdl_prepare_ctx(2, mf2, enl2, xpp, , >xdf2) < 0) {
+   if (xdl_prepare_ctx(2, , enl2, xpp, , >xdf2) < 0) {
 
xdl_free_ctx(>xdf1);
xdl_free_classifier();
diff --git a/mercurial/thirdparty/xdiff/xdiffi.c 
b/mercurial/thirdparty/xdiff/xdiffi.c
--- a/mercurial/thirdparty/xdiff/xdiffi.c
+++ b/mercurial/thirdparty/xdiff/xdiffi.c
@@ -1062,6 +1062,7 @@
 static int xdl_call_hunk_func(xdfenv_t *xe, xdchange_t *xscr, xdemitcb_t *ecb,
  xdemitconf_t const *xecfg)
 {
+   long p = xe->nprefix, s = xe->nsuffix;
xdchange_t *xch, *xche;
 
if (!xecfg->hunk_func)
@@ -1073,23 

Re: [PATCH] Fix for Bug #5807

[Sascha Nemecek]

Am 2018-03-02 um 05:14 schrieb Yuya Nishihara:
> On Thu, 1 Mar 2018 11:06:59 +0100, Sascha Nemecek wrote:
>> # HG changeset patch
>> # User Sascha Nemecek 
>> # Date 1519831479 -3600
>> #  Wed Feb 28 16:24:39 2018 +0100
>> # Node ID 42ddf4ee4f91d76f19ca0c3efc4c8e4c1c6fa96c
>> # Parent  1bd132a021dd00f96604e33a8fb5306d37e56007
>> Don't close 'fp' (= 'ui.fout') stream to prevent 'ValueError: I/O 
>> operation on closed file' (Bug #5807).
>>
>> Regression of changeset 30261:6bed17ba00a1 
>> (https://www.mercurial-scm.org/repo/hg/rev/6bed17ba00a1)
>>
>> diff -r 1bd132a021dd -r 42ddf4ee4f91 hgext/convert/subversion.py
>> --- a/hgext/convert/subversion.pyWed Feb 21 14:36:42 2018 +0530
>> +++ b/hgext/convert/subversion.pyWed Feb 28 16:24:39 2018 +0100
>> @@ -149,7 +149,7 @@
>>   pickle.dump(str(inst), fp, protocol)
>>   else:
>>   pickle.dump(None, fp, protocol)
>> -fp.close()
>> +fp.flush()
>>   # With large history, cleanup process goes crazy and suddenly
>>   # consumes *huge* amount of memory. The output file being closed,
>>   # there is no need for clean termination.
> 
> I don't think fp.close() was the source of the problem. Here the process
> _exit()s so no cleanup would be run.

Of course, the problem might lie deeper. Empirically tested, the
fp.close() triggered the reported error output. From my observation, the
fp.close() affected stdout of the convert function and also pdb.
Therefore I came to the conclusion that the fp.close() / ui.fout.close()
was the culprit.


> I suspect that some hidden bug was disclosed by the change 3a4c0905f357,
> "util: always force line buffered stdout when stdout is a tty." Before,
> ui.fout.close() just called fflush() because sys.stdout.close() doesn't
> close the underlying file stream. This no longer applies to the current
> ui.fout as it is created by fdopen(1).

This sounds reasonable to me.

Since I'm not a python developer and lack the insight into the project,
I did not and could not dig deeper. So there might be a better fix than
my humble patch.


Best regards,
Sascha

PS: Thanks fly to Augie for fixing my damaged patch. I'll try to do
better next time.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2709: rebase: remove unused argument "state" from rebasenode()

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG0f3116c08e65: rebase: remove unused argument 
state from rebasenode() (authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2709?vs=6697=6704

REVISION DETAIL
  https://phab.mercurial-scm.org/D2709

AFFECTED FILES
  hgext/rebase.py

CHANGE DETAILS

diff --git a/hgext/rebase.py b/hgext/rebase.py
--- a/hgext/rebase.py
+++ b/hgext/rebase.py
@@ -485,8 +485,8 @@
 try:
 ui.setconfig('ui', 'forcemerge', opts.get('tool', ''),
  'rebase')
-stats = rebasenode(repo, rev, p1, base, self.state,
-   self.collapsef, dest, 
wctx=self.wctx)
+stats = rebasenode(repo, rev, p1, base, self.collapsef,
+   dest, wctx=self.wctx)
 if stats and stats[3] > 0:
 if self.wctx.isinmemory():
 raise error.InMemoryMergeConflictsError()
@@ -1108,7 +1108,7 @@
 repo.dirstate.setbranch(repo[newnode].branch())
 return newnode
 
-def rebasenode(repo, rev, p1, base, state, collapse, dest, wctx):
+def rebasenode(repo, rev, p1, base, collapse, dest, wctx):
 'Rebase a single revision rev on top of p1 using base as merge ancestor'
 # Merge phase
 # Update to destination and merge it with local



To: martinvonz, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2711: rebase: collapse two nested if-conditions

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGa835bf3fe40a: rebase: collapse two nested if-conditions 
(authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2711?vs=6699=6706

REVISION DETAIL
  https://phab.mercurial-scm.org/D2711

AFFECTED FILES
  hgext/rebase.py

CHANGE DETAILS

diff --git a/hgext/rebase.py b/hgext/rebase.py
--- a/hgext/rebase.py
+++ b/hgext/rebase.py
@@ -605,9 +605,8 @@
 hg.updaterepo(repo, newwd, False)
 
 collapsedas = None
-if not self.keepf:
-if self.collapsef:
-collapsedas = newnode
+if self.collapsef and not self.keepf:
+collapsedas = newnode
 clearrebased(ui, repo, self.destmap, self.state, self.skipped,
  collapsedas, self.keepf, fm=fm)
 



To: martinvonz, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2712: rebase: only store collapse message once

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG6dab3bdb1f00: rebase: only store collapse message once 
(authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2712?vs=6700=6707

REVISION DETAIL
  https://phab.mercurial-scm.org/D2712

AFFECTED FILES
  hgext/rebase.py

CHANGE DETAILS

diff --git a/hgext/rebase.py b/hgext/rebase.py
--- a/hgext/rebase.py
+++ b/hgext/rebase.py
@@ -478,7 +478,6 @@
  self.state, self.skipped,
  self.obsoletenotrebased)
 self.storestatus(tr=tr)
-storecollapsemsg(repo, self.collapsemsg)
 if len(repo[None].parents()) == 2:
 repo.ui.debug('resuming interrupted rebase\n')
 else:
@@ -846,6 +845,7 @@
 retcode = rbsrt._preparenewrebase(destmap)
 if retcode is not None:
 return retcode
+storecollapsemsg(repo, rbsrt.collapsemsg)
 
 tr = None
 



To: martinvonz, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2710: rebase: reduce scope of "dsguard" variables a bit

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG1004fd71810f: rebase: reduce scope of dsguard 
variables a bit (authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2710?vs=6698=6705

REVISION DETAIL
  https://phab.mercurial-scm.org/D2710

AFFECTED FILES
  hgext/rebase.py

CHANGE DETAILS

diff --git a/hgext/rebase.py b/hgext/rebase.py
--- a/hgext/rebase.py
+++ b/hgext/rebase.py
@@ -565,7 +565,6 @@
 editor = cmdutil.getcommiteditor(edit=editopt, editform=editform)
 revtoreuse = max(self.state)
 
-dsguard = None
 if self.inmemory:
 newnode = concludememorynode(repo, revtoreuse, p1,
 self.external,
@@ -575,6 +574,7 @@
 keepbranches=self.keepbranchesf,
 date=self.date, wctx=self.wctx)
 else:
+dsguard = None
 if ui.configbool('rebase', 'singletransaction'):
 dsguard = dirstateguard.dirstateguard(repo, 'rebase')
 with util.acceptintervention(dsguard):
@@ -849,7 +849,6 @@
 return retcode
 
 tr = None
-dsguard = None
 
 singletr = ui.configbool('rebase', 'singletransaction')
 if singletr:
@@ -861,6 +860,7 @@
 with util.acceptintervention(tr):
 # Same logic for the dirstate guard, except we don't create one 
when
 # rebasing in-memory (it's not needed).
+dsguard = None
 if singletr and not inmemory:
 dsguard = dirstateguard.dirstateguard(repo, 'rebase')
 with util.acceptintervention(dsguard):



To: martinvonz, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2708: rebase: delete obsolete internal "keepopen" option

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGf7e3fe95b663: rebase: delete obsolete internal 
keepopen option (authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2708?vs=6696=6703

REVISION DETAIL
  https://phab.mercurial-scm.org/D2708

AFFECTED FILES
  hgext/rebase.py

CHANGE DETAILS

diff --git a/hgext/rebase.py b/hgext/rebase.py
--- a/hgext/rebase.py
+++ b/hgext/rebase.py
@@ -174,9 +174,6 @@
 
 self.keepf = opts.get('keep', False)
 self.keepbranchesf = opts.get('keepbranches', False)
-# keepopen is not meant for use on the command line, but by
-# other extensions
-self.keepopen = opts.get('keepopen', False)
 self.obsoletenotrebased = {}
 self.obsoletewithoutsuccessorindestination = set()
 self.inmemory = inmemory
@@ -551,7 +548,7 @@
 repo, ui, opts = self.repo, self.ui, self.opts
 fm = ui.formatter('rebase', opts)
 fm.startitem()
-if self.collapsef and not self.keepopen:
+if self.collapsef:
 p1, p2, _base = defineparents(repo, min(self.state), self.destmap,
   self.state, self.skipped,
   self.obsoletenotrebased)



To: martinvonz, #hg-reviewers, indygreg
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2714: tests: add test for issue 5494 but with --collapse

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This was not fixed, so the test case currently demonstrates the
  breakage.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2714

AFFECTED FILES
  tests/test-rebase-interruptions.t

CHANGE DETAILS

diff --git a/tests/test-rebase-interruptions.t 
b/tests/test-rebase-interruptions.t
--- a/tests/test-rebase-interruptions.t
+++ b/tests/test-rebase-interruptions.t
@@ -463,3 +463,23 @@
   $ hg resolve --list
   $ test -d .hg/merge
   [1]
+Now try again with --collapse
+  $ hg unbundle -q .hg/strip-backup/fdaca8533b86-7fd70513-rebase.hg
+  $ hg rebase -s 2 -d 1 --noninteractive --collapse
+  rebasing 2:fdaca8533b86 "b" (tip)
+  merging a
+  warning: conflicts while merging a! (edit, then use 'hg resolve --mark')
+  unresolved conflicts (see hg resolve, then hg rebase --continue)
+  [1]
+  $ echo a > a
+  $ echo c >> a
+  $ hg resolve --mark a
+  (no more unresolved files)
+  continue: hg rebase --continue
+  $ hg rebase --continue
+  rebasing 2:fdaca8533b86 "b" (tip)
+  saved backup bundle to 
$TESTTMP/repo/.hg/strip-backup/fdaca8533b86-7fd70513-rebase.hg
+BROKEN: the merge state was not cleared
+  $ hg resolve --list
+  R a
+  $ test -d .hg/merge



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2713: tests: .hg/merge is a directory, so use `test -d`

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This part of test-rebase-interrupts.t would have passed before the fix
  in https://phab.mercurial-scm.org/rHGa580b2d65ded988da1b06e4300d05dcb0b95b352 
(rebase: make sure merge state is cleaned up for no-op
  rebases (issue5494), 2017-05-18).

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2713

AFFECTED FILES
  tests/test-rebase-interruptions.t

CHANGE DETAILS

diff --git a/tests/test-rebase-interruptions.t 
b/tests/test-rebase-interruptions.t
--- a/tests/test-rebase-interruptions.t
+++ b/tests/test-rebase-interruptions.t
@@ -461,5 +461,5 @@
   note: rebase of 1:fdaca8533b86 created no changes to commit
   saved backup bundle to 
$TESTTMP/repo/.hg/strip-backup/fdaca8533b86-7fd70513-rebase.hg
   $ hg resolve --list
-  $ test -f .hg/merge
-  [1]
+  $ test -d .hg/merge
+  [1]



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2712: rebase: only store collapse message once

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The message is determined by the user passing --message or --log when
  the rebase is started. There's no need to write it to a file for each
  rebased commit; writing it once at the start of the rebase is enough.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2712

AFFECTED FILES
  hgext/rebase.py

CHANGE DETAILS

diff --git a/hgext/rebase.py b/hgext/rebase.py
--- a/hgext/rebase.py
+++ b/hgext/rebase.py
@@ -478,7 +478,6 @@
  self.state, self.skipped,
  self.obsoletenotrebased)
 self.storestatus(tr=tr)
-storecollapsemsg(repo, self.collapsemsg)
 if len(repo[None].parents()) == 2:
 repo.ui.debug('resuming interrupted rebase\n')
 else:
@@ -846,6 +845,7 @@
 retcode = rbsrt._preparenewrebase(destmap)
 if retcode is not None:
 return retcode
+storecollapsemsg(repo, rbsrt.collapsemsg)
 
 tr = None
 



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2711: rebase: collapse two nested if-conditions

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Also change the order since it feel to me like it's more about
  --collapse than it is about --keep.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2711

AFFECTED FILES
  hgext/rebase.py

CHANGE DETAILS

diff --git a/hgext/rebase.py b/hgext/rebase.py
--- a/hgext/rebase.py
+++ b/hgext/rebase.py
@@ -605,9 +605,8 @@
 hg.updaterepo(repo, newwd, False)
 
 collapsedas = None
-if not self.keepf:
-if self.collapsef:
-collapsedas = newnode
+if self.collapsef and not self.keepf:
+collapsedas = newnode
 clearrebased(ui, repo, self.destmap, self.state, self.skipped,
  collapsedas, self.keepf, fm=fm)
 



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2710: rebase: reduce scope of "dsguard" variables a bit

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2710

AFFECTED FILES
  hgext/rebase.py

CHANGE DETAILS

diff --git a/hgext/rebase.py b/hgext/rebase.py
--- a/hgext/rebase.py
+++ b/hgext/rebase.py
@@ -565,7 +565,6 @@
 editor = cmdutil.getcommiteditor(edit=editopt, editform=editform)
 revtoreuse = max(self.state)
 
-dsguard = None
 if self.inmemory:
 newnode = concludememorynode(repo, revtoreuse, p1,
 self.external,
@@ -575,6 +574,7 @@
 keepbranches=self.keepbranchesf,
 date=self.date, wctx=self.wctx)
 else:
+dsguard = None
 if ui.configbool('rebase', 'singletransaction'):
 dsguard = dirstateguard.dirstateguard(repo, 'rebase')
 with util.acceptintervention(dsguard):
@@ -849,7 +849,6 @@
 return retcode
 
 tr = None
-dsguard = None
 
 singletr = ui.configbool('rebase', 'singletransaction')
 if singletr:
@@ -861,6 +860,7 @@
 with util.acceptintervention(tr):
 # Same logic for the dirstate guard, except we don't create one 
when
 # rebasing in-memory (it's not needed).
+dsguard = None
 if singletr and not inmemory:
 dsguard = dirstateguard.dirstateguard(repo, 'rebase')
 with util.acceptintervention(dsguard):



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2708: rebase: delete obsolete internal "keepopen" option

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The option was apparently introduced for use by the "pbranch", see
  https://phab.mercurial-scm.org/rHGf2558a8228be7dc016f03f08593c7c010cac1608 
(rebase: add option to not commit after a collapsing,
  2010-02-07). However, it doesn't seem like it was ever used by that
  extension (according to `hg grep` in a clone of
  https://bitbucket.org/parren/hg-pbranch/), so let's delete it.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2708

AFFECTED FILES
  hgext/rebase.py

CHANGE DETAILS

diff --git a/hgext/rebase.py b/hgext/rebase.py
--- a/hgext/rebase.py
+++ b/hgext/rebase.py
@@ -174,9 +174,6 @@
 
 self.keepf = opts.get('keep', False)
 self.keepbranchesf = opts.get('keepbranches', False)
-# keepopen is not meant for use on the command line, but by
-# other extensions
-self.keepopen = opts.get('keepopen', False)
 self.obsoletenotrebased = {}
 self.obsoletewithoutsuccessorindestination = set()
 self.inmemory = inmemory
@@ -551,7 +548,7 @@
 repo, ui, opts = self.repo, self.ui, self.opts
 fm = ui.formatter('rebase', opts)
 fm.startitem()
-if self.collapsef and not self.keepopen:
+if self.collapsef:
 p1, p2, _base = defineparents(repo, min(self.state), self.destmap,
   self.state, self.skipped,
   self.obsoletenotrebased)



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2709: rebase: remove unused argument "state" from rebasenode()

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2709

AFFECTED FILES
  hgext/rebase.py

CHANGE DETAILS

diff --git a/hgext/rebase.py b/hgext/rebase.py
--- a/hgext/rebase.py
+++ b/hgext/rebase.py
@@ -485,8 +485,8 @@
 try:
 ui.setconfig('ui', 'forcemerge', opts.get('tool', ''),
  'rebase')
-stats = rebasenode(repo, rev, p1, base, self.state,
-   self.collapsef, dest, 
wctx=self.wctx)
+stats = rebasenode(repo, rev, p1, base, self.collapsef,
+   dest, wctx=self.wctx)
 if stats and stats[3] > 0:
 if self.wctx.isinmemory():
 raise error.InMemoryMergeConflictsError()
@@ -1108,7 +1108,7 @@
 repo.dirstate.setbranch(repo[newnode].branch())
 return newnode
 
-def rebasenode(repo, rev, p1, base, state, collapse, dest, wctx):
+def rebasenode(repo, rev, p1, base, collapse, dest, wctx):
 'Rebase a single revision rev on top of p1 using base as merge ancestor'
 # Merge phase
 # Update to destination and merge it with local



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2255: releasenotes: replace abort with warning while parsing (issue5775)

[rishabhmadan96 (Rishabh Madan)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG658ed9c7442b: releasenotes: replace abort with warning 
while parsing (issue5775) (authored by rishabhmadan96, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2255?vs=6590=6694

REVISION DETAIL
  https://phab.mercurial-scm.org/D2255

AFFECTED FILES
  hgext/releasenotes.py
  tests/test-releasenotes-parsing.t

CHANGE DETAILS

diff --git a/tests/test-releasenotes-parsing.t 
b/tests/test-releasenotes-parsing.t
--- a/tests/test-releasenotes-parsing.t
+++ b/tests/test-releasenotes-parsing.t
@@ -177,3 +177,26 @@
   paragraph: Bullet item 1
 bullet point:
   paragraph: Bullet item 2
+
+Warn user in case of unexpected block while parsing
+
+  $ hg init relnotes-warn
+  $ cd relnotes-warn
+  $ touch feature1
+  $ hg -q commit -A -l - << EOF
+  > commit 1
+  > 
+  > .. feature::
+  > 
+  >new feature added.
+  > some words about the feature.
+  > EOF
+
+  $ hg releasenote -r .
+  unexpected block in release notes directive feature
+  New Features
+  
+  
+  * new feature added.  some words about the feature.
+
+  $ cd ..
diff --git a/hgext/releasenotes.py b/hgext/releasenotes.py
--- a/hgext/releasenotes.py
+++ b/hgext/releasenotes.py
@@ -325,8 +325,8 @@
 continue
 
 if pblock['type'] != 'paragraph':
-raise error.Abort(_('unexpected block in release notes '
-'directive %s') % directive)
+repo.ui.warn(_('unexpected block in release notes '
+'directive %s\n') % directive)
 
 if pblock['indent'] > 0:
 paragraphs.append(pblock['lines'])



To: rishabhmadan96, #hg-reviewers, durin42
Cc: durin42, pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2254: releasenotes: allow notes for multiple directives in a single changeset

[rishabhmadan96 (Rishabh Madan)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGa5891e94bfe1: releasenotes: allow notes for multiple 
directives in a single changeset (authored by rishabhmadan96, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2254?vs=6592=6695

REVISION DETAIL
  https://phab.mercurial-scm.org/D2254

AFFECTED FILES
  hgext/releasenotes.py
  tests/test-releasenotes-formatting.t

CHANGE DETAILS

diff --git a/tests/test-releasenotes-formatting.t 
b/tests/test-releasenotes-formatting.t
--- a/tests/test-releasenotes-formatting.t
+++ b/tests/test-releasenotes-formatting.t
@@ -457,3 +457,35 @@
   --
   
   First paragraph of fix 1.
+
+  $ cd ..
+
+Using multiple admonitions in same changeset
+
+  $ hg init relnotes-multiadmon
+  $ cd relnotes-multiadmon
+
+  $ touch file1
+  $ hg -q commit -A -l - << EOF
+  > commit 1
+  > 
+  > .. feature::
+  > 
+  >Details about new feature.
+  > 
+  > .. perf::
+  > 
+  >Improves the execution by 2x
+  > EOF
+
+  $ hg releasenotes -r . $TESTTMP/relnotes-multiple-admonitions
+  $ cat $TESTTMP/relnotes-multiple-admonitions
+  New Features
+  
+  
+  * Details about new feature.
+  
+  Performance Improvements
+  
+  
+  * Improves the execution by 2x
diff --git a/hgext/releasenotes.py b/hgext/releasenotes.py
--- a/hgext/releasenotes.py
+++ b/hgext/releasenotes.py
@@ -324,6 +324,9 @@
 if pblock['type'] == 'margin':
 continue
 
+if pblock['type'] == 'admonition':
+break
+
 if pblock['type'] != 'paragraph':
 repo.ui.warn(_('changeset %s: unexpected block in release '
 'notes directive %s\n') % (ctx, directive))



To: rishabhmadan96, #hg-reviewers, durin42
Cc: pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2624: perf: teach perfbdiff to call blocks() and to use xdiff

[indygreg (Gregory Szorc)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGd382344c69aa: perf: teach perfbdiff to call blocks() and to 
use xdiff (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2624?vs=6522=6691

REVISION DETAIL
  https://phab.mercurial-scm.org/D2624

AFFECTED FILES
  contrib/perf.py

CHANGE DETAILS

diff --git a/contrib/perf.py b/contrib/perf.py
--- a/contrib/perf.py
+++ b/contrib/perf.py
@@ -939,11 +939,16 @@
 timer(d)
 fm.end()
 
-def _bdiffworker(q, ready, done):
+def _bdiffworker(q, blocks, xdiff, ready, done):
 while not done.is_set():
 pair = q.get()
 while pair is not None:
-mdiff.textdiff(*pair)
+if xdiff:
+mdiff.bdiff.xdiffblocks(*pair)
+elif blocks:
+mdiff.bdiff.blocks(*pair)
+else:
+mdiff.textdiff(*pair)
 q.task_done()
 pair = q.get()
 q.task_done() # for the None one
@@ -954,6 +959,8 @@
 ('', 'count', 1, 'number of revisions to test (when using --startrev)'),
 ('', 'alldata', False, 'test bdiffs for all associated revisions'),
 ('', 'threads', 0, 'number of thread to use (disable with 0)'),
+('', 'blocks', False, 'test computing diffs into blocks'),
+('', 'xdiff', False, 'use xdiff algorithm'),
 ],
 
 '-c|-m|FILE REV')
@@ -969,14 +976,21 @@
 measure bdiffs for all changes related to that changeset (manifest
 and filelogs).
 """
+opts = pycompat.byteskwargs(opts)
+
+if opts['xdiff'] and not opts['blocks']:
+raise error.CommandError('perfbdiff', '--xdiff requires --blocks')
+
 if opts['alldata']:
 opts['changelog'] = True
 
 if opts.get('changelog') or opts.get('manifest'):
 file_, rev = None, file_
 elif rev is None:
 raise error.CommandError('perfbdiff', 'invalid arguments')
 
+blocks = opts['blocks']
+xdiff = opts['xdiff']
 textpairs = []
 
 r = cmdutil.openrevlog(repo, 'perfbdiff', file_, opts)
@@ -1007,15 +1021,21 @@
 if not withthreads:
 def d():
 for pair in textpairs:
-mdiff.textdiff(*pair)
+if xdiff:
+mdiff.bdiff.xdiffblocks(*pair)
+elif blocks:
+mdiff.bdiff.blocks(*pair)
+else:
+mdiff.textdiff(*pair)
 else:
 q = util.queue()
 for i in xrange(threads):
 q.put(None)
 ready = threading.Condition()
 done = threading.Event()
 for i in xrange(threads):
-threading.Thread(target=_bdiffworker, args=(q, ready, 
done)).start()
+threading.Thread(target=_bdiffworker,
+ args=(q, blocks, xdiff, ready, done)).start()
 q.join()
 def d():
 for pair in textpairs:



To: indygreg, #hg-reviewers, quark, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2253: releasenotes: mention changeset with warning and abort

[rishabhmadan96 (Rishabh Madan)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG3fff6f30bd7f: releasenotes: mention changeset with warning 
and abort (authored by rishabhmadan96, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2253?vs=6591=6693

REVISION DETAIL
  https://phab.mercurial-scm.org/D2253

AFFECTED FILES
  hgext/releasenotes.py
  tests/test-releasenotes-parsing.t

CHANGE DETAILS

diff --git a/tests/test-releasenotes-parsing.t 
b/tests/test-releasenotes-parsing.t
--- a/tests/test-releasenotes-parsing.t
+++ b/tests/test-releasenotes-parsing.t
@@ -193,7 +193,7 @@
   > EOF
 
   $ hg releasenote -r .
-  unexpected block in release notes directive feature
+  changeset a4251905c440: unexpected block in release notes directive feature
   New Features
   
   
diff --git a/hgext/releasenotes.py b/hgext/releasenotes.py
--- a/hgext/releasenotes.py
+++ b/hgext/releasenotes.py
@@ -311,8 +311,8 @@
 title = block['lines'][0].strip() if block['lines'] else None
 
 if i + 1 == len(blocks):
-raise error.Abort(_('release notes directive %s lacks content')
-  % directive)
+raise error.Abort(_('changeset %s: release notes directive %s '
+'lacks content') % (ctx, directive))
 
 # Now search ahead and find all paragraphs attached to this
 # admonition.
@@ -325,8 +325,8 @@
 continue
 
 if pblock['type'] != 'paragraph':
-repo.ui.warn(_('unexpected block in release notes '
-'directive %s\n') % directive)
+repo.ui.warn(_('changeset %s: unexpected block in release '
+'notes directive %s\n') % (ctx, directive))
 
 if pblock['indent'] > 0:
 paragraphs.append(pblock['lines'])



To: rishabhmadan96, #hg-reviewers, durin42
Cc: av6, pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2704: archival: fileit should not use atomictemp, causes major performance regression

[vincent.parrett (Vincent Parrett)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGa148c67d8b09: archival: fileit should not use atomictemp, 
causes performance regression (authored by vincent.parrett, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D2704?vs=6680=6692

REVISION DETAIL
  https://phab.mercurial-scm.org/D2704

AFFECTED FILES
  mercurial/archival.py

CHANGE DETAILS

diff --git a/mercurial/archival.py b/mercurial/archival.py
--- a/mercurial/archival.py
+++ b/mercurial/archival.py
@@ -272,7 +272,7 @@
 if islink:
 self.opener.symlink(data, name)
 return
-f = self.opener(name, "w", atomictemp=True)
+f = self.opener(name, "w", atomictemp=False)
 f.write(data)
 f.close()
 destfile = os.path.join(self.basedir, name)



To: vincent.parrett, #hg-reviewers, durin42
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2394: histedit: make histedit's commands accept revsets (issue5746)

[sangeet259 (Sangeet Kumar Mishra)]

sangeet259 added a comment.


  @durin42  Here is the output! https://bpaste.net/show/7ccf4a9fea50

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2394

To: sangeet259, durin42, #hg-reviewers
Cc: tom.prince, krbullock, rishabhmadan96, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2588: commit: adds multiline commit message support(issue5616)

[durin42 (Augie Fackler)]

durin42 added inline comments.

INLINE COMMENTS

> pulkit wrote in test-commit.t:847
> starting with a newline seems awkward.

agreed

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2588

To: sangeet259, #hg-reviewers
Cc: durin42, tom.prince, yuja, pulkit, jeffpc, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2588: commit: adds multiline commit message support(issue5616)

[sangeet259 (Sangeet Kumar Mishra)]

sangeet259 added a comment.


  @pulkit Yeah, this rather a not so elegant hack. How can it be parsed in hg 
commit? 
  As the first message passed is overwritten by the second one at this step 
only.  :/
  
  @yuja I think maybe subclassing the customopt may be a fair choice.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2588

To: sangeet259, #hg-reviewers
Cc: tom.prince, yuja, pulkit, jeffpc, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2394: histedit: make histedit's commands accept revsets (issue5746)

[durin42 (Augie Fackler)]

durin42 added a comment.


  In https://phab.mercurial-scm.org/D2394#43612, @sangeet259 wrote:
  
  > @durin42  Even I thought of this construct before as it appears more 
elegant. But it is failing some tests!
  >  Specifically, the error messages are changed in this one.
  >
  >   Failed test-histedit-arguments.t: output changed
  >   Failed test-histedit-edit.t: output changed
  >   # Ran 14 tests, 0 skipped, 2 failed.
  >   python hash seed: 3573457086
  >  
  >
  
  
  Huh. Can you pastebin those failures so I can see what the changes were like?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2394

To: sangeet259, durin42, #hg-reviewers
Cc: tom.prince, krbullock, rishabhmadan96, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D2394: histedit: make histedit's commands accept revsets (issue5746)

[sangeet259 (Sangeet Kumar Mishra)]

sangeet259 added a comment.


  @durin42  Even I thought of this construct before as it appears more elegant. 
But it is failing some tests!
  Specifically, the error messages are changed in this one.
  
Failed test-histedit-arguments.t: output changed
Failed test-histedit-edit.t: output changed
# Ran 14 tests, 0 skipped, 2 failed.
python hash seed: 3573457086

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2394

To: sangeet259, durin42, #hg-reviewers
Cc: tom.prince, krbullock, rishabhmadan96, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH evolve-ext resend] obsdiscovery: include units in ui.progress() calls (issue5773)

[Anton Shestakov]

# HG changeset patch
# User Anton Shestakov 
# Date 1519388837 -28800
#  Fri Feb 23 20:27:17 2018 +0800
# Node ID 7ce1b90dbc7d6bce36398f0fab6f6db708beee0c
# Parent  1f0c88a9dd1cdf11d9eb8db48cc5e6e70022230b
obsdiscovery: include units in ui.progress() calls (issue5773)

diff --git a/hgext3rd/evolve/obsdiscovery.py b/hgext3rd/evolve/obsdiscovery.py
--- a/hgext3rd/evolve/obsdiscovery.py
+++ b/hgext3rd/evolve/obsdiscovery.py
@@ -95,7 +95,8 @@ def findcommonobsmarkers(ui, local, remo
 common = set()
 undecided = set(probeset)
 totalnb = len(undecided)
-ui.progress(_("comparing with other"), 0, total=totalnb)
+ui.progress(_("comparing with other"), 0, total=totalnb,
+unit=_("changesets"))
 _takefullsample = setdiscovery._takefullsample
 if remote.capable('_evoext_obshash_1'):
 getremotehash = remote.evoext_obshash1
@@ -114,7 +115,7 @@ def findcommonobsmarkers(ui, local, remo
 
 roundtrips += 1
 ui.progress(_("comparing with other"), totalnb - len(undecided),
-total=totalnb)
+total=totalnb, unit=_("changesets"))
 ui.debug("query %i; still undecided: %i, sample size is: %i\n"
  % (roundtrips, len(undecided), len(sample)))
 # indices between sample and externalized version must match
@@ -175,7 +176,8 @@ def findmissingrange(ui, local, remote, 
 
 local.obsstore.rangeobshashcache.update(local)
 querycount = 0
-ui.progress(_("comparing obsmarker with other"), querycount)
+ui.progress(_("comparing obsmarker with other"), querycount,
+unit=_("obsmarkers"))
 overflow = []
 while sample or overflow:
 if overflow:
@@ -230,7 +232,8 @@ def findmissingrange(ui, local, remote, 
 addentry(new)
 assert nbsample == nbreplies
 querycount += 1
-ui.progress(_("comparing obsmarker with other"), querycount)
+ui.progress(_("comparing obsmarker with other"), querycount,
+unit=_("obsmarkers"))
 ui.progress(_("comparing obsmarker with other"), None)
 local.obsstore.rangeobshashcache.save(local)
 duration = timer() - starttime
@@ -771,7 +774,8 @@ def _obsrelsethashtree(repo, encodeonema
 cache = []
 unfi = repo.unfiltered()
 markercache = {}
-repo.ui.progress(_("preparing locally"), 0, total=len(unfi))
+repo.ui.progress(_("preparing locally"), 0, total=len(unfi),
+ unit=_("changesets"))
 for i in unfi:
 ctx = unfi[i]
 entry = 0
@@ -801,7 +805,8 @@ def _obsrelsethashtree(repo, encodeonema
 cache.append((ctx.node(), sha.digest()))
 else:
 cache.append((ctx.node(), node.nullid))
-repo.ui.progress(_("preparing locally"), i, total=len(unfi))
+repo.ui.progress(_("preparing locally"), i, total=len(unfi),
+ unit=_("changesets"))
 repo.ui.progress(_("preparing locally"), None)
 return cache
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel