Re: [PATCH 1 of 4] debug: add a 'debugdownload' command

2018-01-06 Thread Augie Fackler 
On Wed, Jan 03, 2018 at 10:27:13AM +, Gábor STEFANIK wrote:
> > -Original Message-
> > From: Mercurial-devel [mailto:mercurial-devel-boun...@mercurial-scm.org]
> > On Behalf Of Boris Feld
> > Sent: Tuesday, January 2, 2018 11:40 AM
> > To: mercurial-devel@mercurial-scm.org
> > Subject: [PATCH 1 of 4] debug: add a 'debugdownload' command
> >
> > # HG changeset patch
> > # User Boris Feld <boris.f...@octobus.net> # Date 1513326616 -3600
> > #  Fri Dec 15 09:30:16 2017 +0100
> > # Node ID 2bd6c705949fae0b4477759479e9a0a905788ca4
> > # Parent  2c47986505ff1f9c9c77117eca584347dbd1d89b
> > # EXP-Topic largefile-url
> > # Available At https://bitbucket.org/octobus/mercurial-devel/
> > #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
> > 2bd6c705949f
> > debug: add a 'debugdownload' command

I like where this is headed, but please send a v2 that addresses
Gabor's comments, especially around magic numbers.

> >
> > This command resolve and fetch and URL through the Mercurial logic.
> > Mercurial logic add various headers (including authentication) while 
> > resolving
> > an URL so the commands helps with building the same request Mercurial
> > would be doing.
> >
> > A new test file is created because we'll add more logic regarding Mercurial
> > download logic and it will grow to a reasonable size.
> >
> > diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
> > --- a/mercurial/debugcommands.py
> > +++ b/mercurial/debugcommands.py
> > @@ -69,6 +69,7 @@ from . import (
> >  templater,
> >  treediscovery,
> >  upgrade,
> > +url as urlmod,
> >  util,
> >  vfs as vfsmod,
> >  )
> > @@ -786,6 +787,25 @@ def debugdiscovery(ui, repo, remoteurl="
> >  localrevs = opts['rev']
> >  doit(localrevs, remoterevs)
> >
> > +@command('debugdownload',
> > +[
> > +('o', 'output', '', _('URL')),
> > +],
> > +norepo=True)
> > +def debugdownload(ui, url, output=None, **opts):
> > +"""Download a ressource using Mercurial logic and config
>
> typo: resource
>
> > +"""
> > +fh = urlmod.open(ui, url, output)
> > +
> > +dest = ui
> > +if output:
> > +dest = open(output, "wb", 4<<10)
>
> Repeatedly appearing magic number, please name it instead.
>
> > +
> > +data = fh.read(4<<10)
> > +while data:
> > +dest.write(data)
> > +data = fh.read(4<<10)
> > +
> >  @command('debugextensions', cmdutil.formatteropts, [], norepo=True)
> > def debugextensions(ui, **opts):
> >  '''show information about active extensions'''
> > diff --git a/tests/test-completion.t b/tests/test-completion.t
> > --- a/tests/test-completion.t
> > +++ b/tests/test-completion.t
> > @@ -85,6 +85,7 @@ Show debug commands if there are no othe
> >debugdeltachain
> >debugdirstate
> >debugdiscovery
> > +  debugdownload
> >debugextensions
> >debugfileset
> >debugformat
> > @@ -263,6 +264,7 @@ Show all commands + options
> >debugdeltachain: changelog, manifest, dir, template
> >debugdirstate: nodates, datesort
> >debugdiscovery: old, nonheads, rev, ssh, remotecmd, insecure
> > +  debugdownload: output
> >debugextensions: template
> >debugfileset: rev
> >debugformat: template
> > diff --git a/tests/test-help.t b/tests/test-help.t
> > --- a/tests/test-help.t
> > +++ b/tests/test-help.t
> > @@ -919,6 +919,8 @@ Test list of internal help commands
> >   show the contents of the current dirstate
> > debugdiscovery
> >   runs the changeset discovery protocol in isolation
> > +   debugdownload
> > + Download a ressource using Mercurial logic and config
>
> typo: resource
>
> > debugextensions
> >   show information about active extensions
> > debugfileset  parse and apply a fileset specification diff --git 
> > a/tests/test-
> > url-download.t b/tests/test-url-download.t new file mode 100644
> > --- /dev/null
> > +++ b/tests/test-url-download.t
> > @@ -0,0 +1,36 @@
> > +#require serve
> > +
> > +  $ hg init server
> > +  $ hg serve -R server -p $HGPORT -d --pid-file=hg1.pid -E ../error.log
> > + $ cat hg1.pid >> $DAEMON_PIDS
> > +
> > +Check basic fetching
> > +
>

RE: [PATCH 1 of 4] debug: add a 'debugdownload' command

2018-01-03 Thread Gábor STEFANIK 
> -Original Message-
> From: Mercurial-devel [mailto:mercurial-devel-boun...@mercurial-scm.org]
> On Behalf Of Boris Feld
> Sent: Tuesday, January 2, 2018 11:40 AM
> To: mercurial-devel@mercurial-scm.org
> Subject: [PATCH 1 of 4] debug: add a 'debugdownload' command
>
> # HG changeset patch
> # User Boris Feld <boris.f...@octobus.net> # Date 1513326616 -3600
> #  Fri Dec 15 09:30:16 2017 +0100
> # Node ID 2bd6c705949fae0b4477759479e9a0a905788ca4
> # Parent  2c47986505ff1f9c9c77117eca584347dbd1d89b
> # EXP-Topic largefile-url
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
> 2bd6c705949f
> debug: add a 'debugdownload' command
>
> This command resolve and fetch and URL through the Mercurial logic.
> Mercurial logic add various headers (including authentication) while resolving
> an URL so the commands helps with building the same request Mercurial
> would be doing.
>
> A new test file is created because we'll add more logic regarding Mercurial
> download logic and it will grow to a reasonable size.
>
> diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
> --- a/mercurial/debugcommands.py
> +++ b/mercurial/debugcommands.py
> @@ -69,6 +69,7 @@ from . import (
>  templater,
>  treediscovery,
>  upgrade,
> +url as urlmod,
>  util,
>  vfs as vfsmod,
>  )
> @@ -786,6 +787,25 @@ def debugdiscovery(ui, repo, remoteurl="
>  localrevs = opts['rev']
>  doit(localrevs, remoterevs)
>
> +@command('debugdownload',
> +[
> +('o', 'output', '', _('URL')),
> +],
> +norepo=True)
> +def debugdownload(ui, url, output=None, **opts):
> +"""Download a ressource using Mercurial logic and config

typo: resource

> +"""
> +fh = urlmod.open(ui, url, output)
> +
> +dest = ui
> +if output:
> +dest = open(output, "wb", 4<<10)

Repeatedly appearing magic number, please name it instead.

> +
> +data = fh.read(4<<10)
> +while data:
> +dest.write(data)
> +data = fh.read(4<<10)
> +
>  @command('debugextensions', cmdutil.formatteropts, [], norepo=True)
> def debugextensions(ui, **opts):
>  '''show information about active extensions'''
> diff --git a/tests/test-completion.t b/tests/test-completion.t
> --- a/tests/test-completion.t
> +++ b/tests/test-completion.t
> @@ -85,6 +85,7 @@ Show debug commands if there are no othe
>debugdeltachain
>debugdirstate
>debugdiscovery
> +  debugdownload
>debugextensions
>debugfileset
>debugformat
> @@ -263,6 +264,7 @@ Show all commands + options
>debugdeltachain: changelog, manifest, dir, template
>debugdirstate: nodates, datesort
>debugdiscovery: old, nonheads, rev, ssh, remotecmd, insecure
> +  debugdownload: output
>debugextensions: template
>debugfileset: rev
>debugformat: template
> diff --git a/tests/test-help.t b/tests/test-help.t
> --- a/tests/test-help.t
> +++ b/tests/test-help.t
> @@ -919,6 +919,8 @@ Test list of internal help commands
>   show the contents of the current dirstate
> debugdiscovery
>   runs the changeset discovery protocol in isolation
> +   debugdownload
> + Download a ressource using Mercurial logic and config

typo: resource

> debugextensions
>   show information about active extensions
> debugfileset  parse and apply a fileset specification diff --git 
> a/tests/test-
> url-download.t b/tests/test-url-download.t new file mode 100644
> --- /dev/null
> +++ b/tests/test-url-download.t
> @@ -0,0 +1,36 @@
> +#require serve
> +
> +  $ hg init server
> +  $ hg serve -R server -p $HGPORT -d --pid-file=hg1.pid -E ../error.log
> + $ cat hg1.pid >> $DAEMON_PIDS
> +
> +Check basic fetching
> +
> +  $ hg debugdownload "http://localhost:$HGPORT/?cmd=lookup=tip;
> +  1 
> +  $ hg debugdownload  -o null.txt
> "http://localhost:$HGPORT/?cmd=lookup=null;
> +  $ cat null.txt
> +  1 
> +
> +Check the request is seens as coming from Mercurial (rev details, give

typo... what exactly was intended here? "is seen as coming from Mercurial"? 
"seems as if coming from Mercurial"?

> +different content if the request has a Mercurial user agent)
> +
> +  $ get-with-headers.py --headeronly "localhost:$HGPORT" "rev/tip"
> + content-type
> +  200 Script output follows
> +  content-type: text/html; char

[PATCH 1 of 4] debug: add a 'debugdownload' command

2018-01-02 Thread Boris Feld 
# HG changeset patch
# User Boris Feld 
# Date 1513326616 -3600
#  Fri Dec 15 09:30:16 2017 +0100
# Node ID 2bd6c705949fae0b4477759479e9a0a905788ca4
# Parent  2c47986505ff1f9c9c77117eca584347dbd1d89b
# EXP-Topic largefile-url
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
2bd6c705949f
debug: add a 'debugdownload' command

This command resolve and fetch and URL through the Mercurial logic. Mercurial
logic add various headers (including authentication) while resolving an URL so
the commands helps with building the same request Mercurial would be doing.

A new test file is created because we'll add more logic regarding Mercurial
download logic and it will grow to a reasonable size.

diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -69,6 +69,7 @@ from . import (
 templater,
 treediscovery,
 upgrade,
+url as urlmod,
 util,
 vfs as vfsmod,
 )
@@ -786,6 +787,25 @@ def debugdiscovery(ui, repo, remoteurl="
 localrevs = opts['rev']
 doit(localrevs, remoterevs)
 
+@command('debugdownload',
+[
+('o', 'output', '', _('URL')),
+],
+norepo=True)
+def debugdownload(ui, url, output=None, **opts):
+"""Download a ressource using Mercurial logic and config
+"""
+fh = urlmod.open(ui, url, output)
+
+dest = ui
+if output:
+dest = open(output, "wb", 4<<10)
+
+data = fh.read(4<<10)
+while data:
+dest.write(data)
+data = fh.read(4<<10)
+
 @command('debugextensions', cmdutil.formatteropts, [], norepo=True)
 def debugextensions(ui, **opts):
 '''show information about active extensions'''
diff --git a/tests/test-completion.t b/tests/test-completion.t
--- a/tests/test-completion.t
+++ b/tests/test-completion.t
@@ -85,6 +85,7 @@ Show debug commands if there are no othe
   debugdeltachain
   debugdirstate
   debugdiscovery
+  debugdownload
   debugextensions
   debugfileset
   debugformat
@@ -263,6 +264,7 @@ Show all commands + options
   debugdeltachain: changelog, manifest, dir, template
   debugdirstate: nodates, datesort
   debugdiscovery: old, nonheads, rev, ssh, remotecmd, insecure
+  debugdownload: output
   debugextensions: template
   debugfileset: rev
   debugformat: template
diff --git a/tests/test-help.t b/tests/test-help.t
--- a/tests/test-help.t
+++ b/tests/test-help.t
@@ -919,6 +919,8 @@ Test list of internal help commands
  show the contents of the current dirstate
debugdiscovery
  runs the changeset discovery protocol in isolation
+   debugdownload
+ Download a ressource using Mercurial logic and config
debugextensions
  show information about active extensions
debugfileset  parse and apply a fileset specification
diff --git a/tests/test-url-download.t b/tests/test-url-download.t
new file mode 100644
--- /dev/null
+++ b/tests/test-url-download.t
@@ -0,0 +1,36 @@
+#require serve
+
+  $ hg init server
+  $ hg serve -R server -p $HGPORT -d --pid-file=hg1.pid -E ../error.log
+  $ cat hg1.pid >> $DAEMON_PIDS
+
+Check basic fetching
+
+  $ hg debugdownload "http://localhost:$HGPORT/?cmd=lookup=tip;
+  1 
+  $ hg debugdownload  -o null.txt 
"http://localhost:$HGPORT/?cmd=lookup=null;
+  $ cat null.txt
+  1 
+
+Check the request is seens as coming from Mercurial
+(rev details, give different content if the request has a Mercurial user agent)
+
+  $ get-with-headers.py --headeronly "localhost:$HGPORT" "rev/tip" content-type
+  200 Script output follows
+  content-type: text/html; charset=ascii
+  $ hg debugdownload "http://localhost:$HGPORT/rev/tip;
+  
+  # HG changeset patch
+  # User 
+  # Date 0 0
+  # Node ID 
+  
+  
+  
+  
+
+Check other kind of compatible url
+
+  $ hg debugdownload ./null.txt
+  1 
+
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel