Re: [PATCH 1 of 3] hgweb: add a hook for processing LFS Batch API requests

2018-03-05 Thread Matt Harbison 

On Mon, 26 Feb 2018 08:47:01 -0500, Yuya Nishihara  wrote:


On Thu, 22 Feb 2018 01:02:41 -0500, Matt Harbison wrote:

# HG changeset patch
# User Matt Harbison 
# Date 1519274700 18000
#  Wed Feb 21 23:45:00 2018 -0500
# Node ID d38f7cc80f9dc453e7968fdb594e0a1119003d14
# Parent  c8891cc3fa9ec855a3bdefd3dd759d19927c6b85
hgweb: add a hook for processing LFS Batch API requests

There really isn't a clean way to give LFS a crack at intercepting the  
requests
without hardcoding some LFS knowledge in the core.  The rationale for  
this URI
is that the spec for the Batch API[1] defines the URL as the LFS server  
url +

'/objects/batch'.  The default git URLs are:

Git remote: https://git-server.com/foo/bar
LFS server: https://git-server.com/foo/bar.git/info/lfs
Batch API: https://git-server.com/foo/bar.git/info/lfs/objects/batch

'.git/' seems like it's not something a user would normally track.  If  
we adhere
to how git defines the URLs, then the hg-git extension should be able  
to talk to

a git based server without any additional work.

[1] https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md

diff --git a/mercurial/hgweb/hgweb_mod.py b/mercurial/hgweb/hgweb_mod.py
--- a/mercurial/hgweb/hgweb_mod.py
+++ b/mercurial/hgweb/hgweb_mod.py
@@ -95,6 +95,12 @@
 urlel = os.path.dirname(urlel)
 return reversed(breadcrumb)

+def _processlfsbatchreq(repo, req):
+"""A hook for the LFS extension to wrap that handles requests to  
the Batch

+API, and returns the appropriate JSON response.
+"""
+raise ErrorResponse(HTTP_NOT_FOUND)
+
 class requestcontext(object):
 """Holds state/context for an individual request.

@@ -371,6 +377,11 @@

 return protohandler['dispatch']()

+# Route LFS Batch API requests to the appropriate handler
+
+if req.env[r'PATH_INFO'] == '/.git/info/lfs/objects/batch':
+return _processlfsbatchreq(rctx.repo, req)


(CC: indygreg as web expert)

I'm not pretty sure, but given we do "'PATH_INFO' in req.env" before,
req.env['PATH_INFO'] could be missing. And maybe we'll need some  
check_perm().


Disregard this series.  I've added a user agent check, and try/catch  
around these.  But I need to rebase a huge series over the recent  
changes.  I'm still curious what to do without PATH_INFO.


We can probably check_perm() the Batch API here, but the check on the  
actual upload/download needs to be done in the extension.  (The  
upload/download command is packed in the JSON, and implies 'push' or  
'pull' check here.)


Something strange I noticed is that hgweb.common.checkauthz() throws a 401  
if the user is on the deny_read list, or not in allow_read.  Shouldn't it  
be 403?  The git docs say it will attempt authentication and retry on a  
401 (though it seems unlikely that a git client will ever talk to this  
server).


https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md#response-errors

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 3] hgweb: add a hook for processing LFS Batch API requests

2018-02-26 Thread Matt Harbison 

> On Feb 26, 2018, at 8:47 AM, Yuya Nishihara  wrote:
> 
>> On Thu, 22 Feb 2018 01:02:41 -0500, Matt Harbison wrote:
>> # HG changeset patch
>> # User Matt Harbison 
>> # Date 1519274700 18000
>> #  Wed Feb 21 23:45:00 2018 -0500
>> # Node ID d38f7cc80f9dc453e7968fdb594e0a1119003d14
>> # Parent  c8891cc3fa9ec855a3bdefd3dd759d19927c6b85
>> hgweb: add a hook for processing LFS Batch API requests
>> 
>> There really isn't a clean way to give LFS a crack at intercepting the 
>> requests
>> without hardcoding some LFS knowledge in the core.  The rationale for this 
>> URI
>> is that the spec for the Batch API[1] defines the URL as the LFS server url +
>> '/objects/batch'.  The default git URLs are:
>> 
>>Git remote: https://git-server.com/foo/bar
>>LFS server: https://git-server.com/foo/bar.git/info/lfs
>>Batch API: https://git-server.com/foo/bar.git/info/lfs/objects/batch
>> 
>> '.git/' seems like it's not something a user would normally track.  If we 
>> adhere
>> to how git defines the URLs, then the hg-git extension should be able to 
>> talk to
>> a git based server without any additional work.
>> 
>> [1] https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md
>> 
>> diff --git a/mercurial/hgweb/hgweb_mod.py b/mercurial/hgweb/hgweb_mod.py
>> --- a/mercurial/hgweb/hgweb_mod.py
>> +++ b/mercurial/hgweb/hgweb_mod.py
>> @@ -95,6 +95,12 @@
>> urlel = os.path.dirname(urlel)
>> return reversed(breadcrumb)
>> 
>> +def _processlfsbatchreq(repo, req):
>> +"""A hook for the LFS extension to wrap that handles requests to the 
>> Batch
>> +API, and returns the appropriate JSON response.
>> +"""
>> +raise ErrorResponse(HTTP_NOT_FOUND)
>> +
>> class requestcontext(object):
>> """Holds state/context for an individual request.
>> 
>> @@ -371,6 +377,11 @@
>> 
>> return protohandler['dispatch']()
>> 
>> +# Route LFS Batch API requests to the appropriate handler
>> +
>> +if req.env[r'PATH_INFO'] == '/.git/info/lfs/objects/batch':
>> +return _processlfsbatchreq(rctx.repo, req)
> 
> (CC: indygreg as web expert)
> 
> I'm not pretty sure, but given we do "'PATH_INFO' in req.env" before,
> req.env['PATH_INFO'] could be missing.

One of the things I’m unsure of is how fundamental things like that go missing, 
and what to do in that case.

> And maybe we'll need some check_perm().

I’m a bit unclear on how the permission stuff works, because there’s a bit of 
auth stuff in lfs too.  I figured the lfs layer might need to do something, but 
that might be clearer after some of this stuff is in place. I can put this 
under an experimental config if there’s concern it might get forgotten about 
before the release.

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 3] hgweb: add a hook for processing LFS Batch API requests

2018-02-26 Thread Yuya Nishihara 
On Thu, 22 Feb 2018 01:02:41 -0500, Matt Harbison wrote:
> # HG changeset patch
> # User Matt Harbison 
> # Date 1519274700 18000
> #  Wed Feb 21 23:45:00 2018 -0500
> # Node ID d38f7cc80f9dc453e7968fdb594e0a1119003d14
> # Parent  c8891cc3fa9ec855a3bdefd3dd759d19927c6b85
> hgweb: add a hook for processing LFS Batch API requests
> 
> There really isn't a clean way to give LFS a crack at intercepting the 
> requests
> without hardcoding some LFS knowledge in the core.  The rationale for this URI
> is that the spec for the Batch API[1] defines the URL as the LFS server url +
> '/objects/batch'.  The default git URLs are:
> 
> Git remote: https://git-server.com/foo/bar
> LFS server: https://git-server.com/foo/bar.git/info/lfs
> Batch API: https://git-server.com/foo/bar.git/info/lfs/objects/batch
> 
> '.git/' seems like it's not something a user would normally track.  If we 
> adhere
> to how git defines the URLs, then the hg-git extension should be able to talk 
> to
> a git based server without any additional work.
> 
> [1] https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md
> 
> diff --git a/mercurial/hgweb/hgweb_mod.py b/mercurial/hgweb/hgweb_mod.py
> --- a/mercurial/hgweb/hgweb_mod.py
> +++ b/mercurial/hgweb/hgweb_mod.py
> @@ -95,6 +95,12 @@
>  urlel = os.path.dirname(urlel)
>  return reversed(breadcrumb)
>  
> +def _processlfsbatchreq(repo, req):
> +"""A hook for the LFS extension to wrap that handles requests to the 
> Batch
> +API, and returns the appropriate JSON response.
> +"""
> +raise ErrorResponse(HTTP_NOT_FOUND)
> +
>  class requestcontext(object):
>  """Holds state/context for an individual request.
>  
> @@ -371,6 +377,11 @@
>  
>  return protohandler['dispatch']()
>  
> +# Route LFS Batch API requests to the appropriate handler
> +
> +if req.env[r'PATH_INFO'] == '/.git/info/lfs/objects/batch':
> +return _processlfsbatchreq(rctx.repo, req)

(CC: indygreg as web expert)

I'm not pretty sure, but given we do "'PATH_INFO' in req.env" before,
req.env['PATH_INFO'] could be missing. And maybe we'll need some check_perm().
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 3] hgweb: add a hook for processing LFS Batch API requests

2018-02-21 Thread Matt Harbison 
# HG changeset patch
# User Matt Harbison 
# Date 1519274700 18000
#  Wed Feb 21 23:45:00 2018 -0500
# Node ID d38f7cc80f9dc453e7968fdb594e0a1119003d14
# Parent  c8891cc3fa9ec855a3bdefd3dd759d19927c6b85
hgweb: add a hook for processing LFS Batch API requests

There really isn't a clean way to give LFS a crack at intercepting the requests
without hardcoding some LFS knowledge in the core.  The rationale for this URI
is that the spec for the Batch API[1] defines the URL as the LFS server url +
'/objects/batch'.  The default git URLs are:

Git remote: https://git-server.com/foo/bar
LFS server: https://git-server.com/foo/bar.git/info/lfs
Batch API: https://git-server.com/foo/bar.git/info/lfs/objects/batch

'.git/' seems like it's not something a user would normally track.  If we adhere
to how git defines the URLs, then the hg-git extension should be able to talk to
a git based server without any additional work.

[1] https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md

diff --git a/mercurial/hgweb/hgweb_mod.py b/mercurial/hgweb/hgweb_mod.py
--- a/mercurial/hgweb/hgweb_mod.py
+++ b/mercurial/hgweb/hgweb_mod.py
@@ -95,6 +95,12 @@
 urlel = os.path.dirname(urlel)
 return reversed(breadcrumb)
 
+def _processlfsbatchreq(repo, req):
+"""A hook for the LFS extension to wrap that handles requests to the Batch
+API, and returns the appropriate JSON response.
+"""
+raise ErrorResponse(HTTP_NOT_FOUND)
+
 class requestcontext(object):
 """Holds state/context for an individual request.
 
@@ -371,6 +377,11 @@
 
 return protohandler['dispatch']()
 
+# Route LFS Batch API requests to the appropriate handler
+
+if req.env[r'PATH_INFO'] == '/.git/info/lfs/objects/batch':
+return _processlfsbatchreq(rctx.repo, req)
+
 # translate user-visible url structure to internal structure
 
 args = query.split('/', 2)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel