Re: [PATCH 1 of 4] debug: add a 'debugdownload' command

2018-01-06 Thread Augie Fackler 
On Wed, Jan 03, 2018 at 10:27:13AM +, Gábor STEFANIK wrote:
> > -Original Message-
> > From: Mercurial-devel [mailto:mercurial-devel-boun...@mercurial-scm.org]
> > On Behalf Of Boris Feld
> > Sent: Tuesday, January 2, 2018 11:40 AM
> > To: mercurial-devel@mercurial-scm.org
> > Subject: [PATCH 1 of 4] debug: add a 'debugdownload' command
> >
> > # HG changeset patch
> > # User Boris Feld  # Date 1513326616 -3600
> > #  Fri Dec 15 09:30:16 2017 +0100
> > # Node ID 2bd6c705949fae0b4477759479e9a0a905788ca4
> > # Parent  2c47986505ff1f9c9c77117eca584347dbd1d89b
> > # EXP-Topic largefile-url
> > # Available At https://bitbucket.org/octobus/mercurial-devel/
> > #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
> > 2bd6c705949f
> > debug: add a 'debugdownload' command

I like where this is headed, but please send a v2 that addresses
Gabor's comments, especially around magic numbers.

> >
> > This command resolve and fetch and URL through the Mercurial logic.
> > Mercurial logic add various headers (including authentication) while 
> > resolving
> > an URL so the commands helps with building the same request Mercurial
> > would be doing.
> >
> > A new test file is created because we'll add more logic regarding Mercurial
> > download logic and it will grow to a reasonable size.
> >
> > diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
> > --- a/mercurial/debugcommands.py
> > +++ b/mercurial/debugcommands.py
> > @@ -69,6 +69,7 @@ from . import (
> >  templater,
> >  treediscovery,
> >  upgrade,
> > +url as urlmod,
> >  util,
> >  vfs as vfsmod,
> >  )
> > @@ -786,6 +787,25 @@ def debugdiscovery(ui, repo, remoteurl="
> >  localrevs = opts['rev']
> >  doit(localrevs, remoterevs)
> >
> > +@command('debugdownload',
> > +[
> > +('o', 'output', '', _('URL')),
> > +],
> > +norepo=True)
> > +def debugdownload(ui, url, output=None, **opts):
> > +"""Download a ressource using Mercurial logic and config
>
> typo: resource
>
> > +"""
> > +fh = urlmod.open(ui, url, output)
> > +
> > +dest = ui
> > +if output:
> > +dest = open(output, "wb", 4<<10)
>
> Repeatedly appearing magic number, please name it instead.
>
> > +
> > +data = fh.read(4<<10)
> > +while data:
> > +dest.write(data)
> > +data = fh.read(4<<10)
> > +
> >  @command('debugextensions', cmdutil.formatteropts, [], norepo=True)
> > def debugextensions(ui, **opts):
> >  '''show information about active extensions'''
> > diff --git a/tests/test-completion.t b/tests/test-completion.t
> > --- a/tests/test-completion.t
> > +++ b/tests/test-completion.t
> > @@ -85,6 +85,7 @@ Show debug commands if there are no othe
> >debugdeltachain
> >debugdirstate
> >debugdiscovery
> > +  debugdownload
> >debugextensions
> >debugfileset
> >debugformat
> > @@ -263,6 +264,7 @@ Show all commands + options
> >debugdeltachain: changelog, manifest, dir, template
> >debugdirstate: nodates, datesort
> >debugdiscovery: old, nonheads, rev, ssh, remotecmd, insecure
> > +  debugdownload: output
> >debugextensions: template
> >debugfileset: rev
> >debugformat: template
> > diff --git a/tests/test-help.t b/tests/test-help.t
> > --- a/tests/test-help.t
> > +++ b/tests/test-help.t
> > @@ -919,6 +919,8 @@ Test list of internal help commands
> >   show the contents of the current dirstate
> > debugdiscovery
> >   runs the changeset discovery protocol in isolation
> > +   debugdownload
> > + Download a ressource using Mercurial logic and config
>
> typo: resource
>
> > debugextensions
> >   show information about active extensions
> > debugfileset  parse and apply a fileset specification diff --git 
> > a/tests/test-
> > url-download.t b/tests/test-url-download.t new file mode 100644
> > --- /dev/null
> > +++ b/tests/test-url-download.t
> > @@ -0,0 +1,36 @@
> > +#require serve
> > +
> > +  $ hg init server
> > +  $ hg serve -R server -p $HGPORT -d --pid-file=hg1.pid -E ../error.log
> > + $ cat hg1.pid >> $DAEMON_PIDS
> > +
> > +Check basic fetching
> > +
> > +  $ hg debugdownload "http://localhost:$HGPORT/?cmd=lookup=tip;
> > +  1 
> > +  $ hg debugdownload  -o null.txt
> > "http://localhost:$HGPORT/?cmd=lookup=null;
> > +  $ cat null.txt
> > +  1 
> > +
> > +Check the request is seens as coming from Mercurial (rev details, give
>
> typo... what exactly was intended here? "is seen as coming from Mercurial"? 
> "seems as if coming from Mercurial"?
>
> > +different content if the request has a Mercurial user agent)
> > +
> > +  $ get-with-headers.py --headeronly "localhost:$HGPORT" "rev/tip"
> > + content-type
> > +  200 Script output follows
> > +  content-type: text/html; charset=ascii  $ hg debugdownload
> > +

RE: [PATCH 1 of 4] debug: add a 'debugdownload' command

2018-01-03 Thread Gábor STEFANIK 
> -Original Message-
> From: Mercurial-devel [mailto:mercurial-devel-boun...@mercurial-scm.org]
> On Behalf Of Boris Feld
> Sent: Tuesday, January 2, 2018 11:40 AM
> To: mercurial-devel@mercurial-scm.org
> Subject: [PATCH 1 of 4] debug: add a 'debugdownload' command
>
> # HG changeset patch
> # User Boris Feld  # Date 1513326616 -3600
> #  Fri Dec 15 09:30:16 2017 +0100
> # Node ID 2bd6c705949fae0b4477759479e9a0a905788ca4
> # Parent  2c47986505ff1f9c9c77117eca584347dbd1d89b
> # EXP-Topic largefile-url
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
> 2bd6c705949f
> debug: add a 'debugdownload' command
>
> This command resolve and fetch and URL through the Mercurial logic.
> Mercurial logic add various headers (including authentication) while resolving
> an URL so the commands helps with building the same request Mercurial
> would be doing.
>
> A new test file is created because we'll add more logic regarding Mercurial
> download logic and it will grow to a reasonable size.
>
> diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
> --- a/mercurial/debugcommands.py
> +++ b/mercurial/debugcommands.py
> @@ -69,6 +69,7 @@ from . import (
>  templater,
>  treediscovery,
>  upgrade,
> +url as urlmod,
>  util,
>  vfs as vfsmod,
>  )
> @@ -786,6 +787,25 @@ def debugdiscovery(ui, repo, remoteurl="
>  localrevs = opts['rev']
>  doit(localrevs, remoterevs)
>
> +@command('debugdownload',
> +[
> +('o', 'output', '', _('URL')),
> +],
> +norepo=True)
> +def debugdownload(ui, url, output=None, **opts):
> +"""Download a ressource using Mercurial logic and config

typo: resource

> +"""
> +fh = urlmod.open(ui, url, output)
> +
> +dest = ui
> +if output:
> +dest = open(output, "wb", 4<<10)

Repeatedly appearing magic number, please name it instead.

> +
> +data = fh.read(4<<10)
> +while data:
> +dest.write(data)
> +data = fh.read(4<<10)
> +
>  @command('debugextensions', cmdutil.formatteropts, [], norepo=True)
> def debugextensions(ui, **opts):
>  '''show information about active extensions'''
> diff --git a/tests/test-completion.t b/tests/test-completion.t
> --- a/tests/test-completion.t
> +++ b/tests/test-completion.t
> @@ -85,6 +85,7 @@ Show debug commands if there are no othe
>debugdeltachain
>debugdirstate
>debugdiscovery
> +  debugdownload
>debugextensions
>debugfileset
>debugformat
> @@ -263,6 +264,7 @@ Show all commands + options
>debugdeltachain: changelog, manifest, dir, template
>debugdirstate: nodates, datesort
>debugdiscovery: old, nonheads, rev, ssh, remotecmd, insecure
> +  debugdownload: output
>debugextensions: template
>debugfileset: rev
>debugformat: template
> diff --git a/tests/test-help.t b/tests/test-help.t
> --- a/tests/test-help.t
> +++ b/tests/test-help.t
> @@ -919,6 +919,8 @@ Test list of internal help commands
>   show the contents of the current dirstate
> debugdiscovery
>   runs the changeset discovery protocol in isolation
> +   debugdownload
> + Download a ressource using Mercurial logic and config

typo: resource

> debugextensions
>   show information about active extensions
> debugfileset  parse and apply a fileset specification diff --git 
> a/tests/test-
> url-download.t b/tests/test-url-download.t new file mode 100644
> --- /dev/null
> +++ b/tests/test-url-download.t
> @@ -0,0 +1,36 @@
> +#require serve
> +
> +  $ hg init server
> +  $ hg serve -R server -p $HGPORT -d --pid-file=hg1.pid -E ../error.log
> + $ cat hg1.pid >> $DAEMON_PIDS
> +
> +Check basic fetching
> +
> +  $ hg debugdownload "http://localhost:$HGPORT/?cmd=lookup=tip;
> +  1 
> +  $ hg debugdownload  -o null.txt
> "http://localhost:$HGPORT/?cmd=lookup=null;
> +  $ cat null.txt
> +  1 
> +
> +Check the request is seens as coming from Mercurial (rev details, give

typo... what exactly was intended here? "is seen as coming from Mercurial"? 
"seems as if coming from Mercurial"?

> +different content if the request has a Mercurial user agent)
> +
> +  $ get-with-headers.py --headeronly "localhost:$HGPORT" "rev/tip"
> + content-type
> +  200 Script output follows
> +  content-type: text/html; charset=ascii  $ hg debugdownload
> + "http://localhost:$HGPORT/rev/tip;
> +
> +  # HG changeset patch
> +  # User
> +  # Date 0 0
> +  # Node ID 
> +
> +
> +
> +
> +
> +Check other kind of compatible url
> +
> +  $ hg debugdownload ./null.txt
> +  1 
> +
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
>