Re: [PATCH 1 of 5] test-lfs: add tests to force server error path coverage

2018-04-14 Thread Matt Harbison 

On Sat, 14 Apr 2018 03:28:49 -0400, Yuya Nishihara  wrote:


On Fri, 13 Apr 2018 18:04:33 -0400, Matt Harbison wrote:

# HG changeset patch
# User Matt Harbison 
# Date 1523119701 14400
#  Sat Apr 07 12:48:21 2018 -0400
# Node ID 1d394ac0efd4aa4f61f428fbac140fe57398f0b8
# Parent  bfdd20d22a86edc318493b4da84a1d7ff4ef98f2
test-lfs: add tests to force server error path coverage


The test failed on my machine. Can you take a look?


It crossed paths with 330ada7e8ea5.  Will resend.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 5] test-lfs: add tests to force server error path coverage

2018-04-14 Thread Yuya Nishihara 
On Fri, 13 Apr 2018 18:04:33 -0400, Matt Harbison wrote:
> # HG changeset patch
> # User Matt Harbison 
> # Date 1523119701 14400
> #  Sat Apr 07 12:48:21 2018 -0400
> # Node ID 1d394ac0efd4aa4f61f428fbac140fe57398f0b8
> # Parent  bfdd20d22a86edc318493b4da84a1d7ff4ef98f2
> test-lfs: add tests to force server error path coverage

The test failed on my machine. Can you take a look?


--- /home/yuya/work/hghacks/mercurial-review/tests/test-lfs-serve-access.t
+++ /home/yuya/work/hghacks/mercurial-review/tests/test-lfs-serve-access.t.err
@@ -270,14 +270,12 @@
   $LOCALIP - - [$LOGDATE$] "GET /?cmd=listkeys HTTP/1.1" 200 - 
x-hgarg-1:namespace=phases x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ 
partial-pull (glob)
   $LOCALIP - - [$LOGDATE$] "GET /?cmd=listkeys HTTP/1.1" 200 - 
x-hgarg-1:namespace=bookmarks x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ 
partial-pull (glob)
   $LOCALIP - - [$LOGDATE$] "GET /?cmd=branchmap HTTP/1.1" 200 - 
x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob)
-  $LOCALIP - - [$LOGDATE$] "GET /?cmd=branchmap HTTP/1.1" 200 - 
x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob)
   $LOCALIP - - [$LOGDATE$] "GET /?cmd=listkeys HTTP/1.1" 200 - 
x-hgarg-1:namespace=bookmarks x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ 
partial-pull (glob)
   $LOCALIP - - [$LOGDATE$] "POST /.git/info/lfs/objects/batch HTTP/1.1" 200 - 
(glob)
   $LOCALIP - - [$LOGDATE$] "GET /?cmd=capabilities HTTP/1.1" 200 - (glob)
   $LOCALIP - - [$LOGDATE$] "GET /?cmd=batch HTTP/1.1" 200 - 
x-hgarg-1:cmds=heads+%3Bknown+nodes%3D392c05922088bacf8e68a6939b480017afbf245d 
x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob)
   $LOCALIP - - [$LOGDATE$] "GET /?cmd=listkeys HTTP/1.1" 200 - 
x-hgarg-1:namespace=phases x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ 
partial-pull (glob)
   $LOCALIP - - [$LOGDATE$] "GET /?cmd=listkeys HTTP/1.1" 200 - 
x-hgarg-1:namespace=bookmarks x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ 
partial-pull (glob)
-  $LOCALIP - - [$LOGDATE$] "GET /?cmd=branchmap HTTP/1.1" 200 - 
x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob)
   $LOCALIP - - [$LOGDATE$] "GET /?cmd=branchmap HTTP/1.1" 200 - 
x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob)
   $LOCALIP - - [$LOGDATE$] "GET /?cmd=listkeys HTTP/1.1" 200 - 
x-hgarg-1:namespace=bookmarks x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ 
partial-pull (glob)
   $LOCALIP - - [$LOGDATE$] "POST /.git/info/lfs/objects/batch HTTP/1.1" 200 - 
(glob)

ERROR: test-lfs-serve-access.t output changed
!
Failed test-lfs-serve-access.t: output changed
# Ran 1 tests, 0 skipped, 1 failed.
python hash seed: 3873177365
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 5] test-lfs: add tests to force server error path coverage

2018-04-13 Thread Matt Harbison 
# HG changeset patch
# User Matt Harbison 
# Date 1523119701 14400
#  Sat Apr 07 12:48:21 2018 -0400
# Node ID 1d394ac0efd4aa4f61f428fbac140fe57398f0b8
# Parent  bfdd20d22a86edc318493b4da84a1d7ff4ef98f2
test-lfs: add tests to force server error path coverage

The tests are somewhat fragile in that the extension that forces the errors is
counting how many times some of the functions are being called, so it depends
heavily on the content of the repo.  Maybe we can do something clever like load
an extension on the client, and have it send over instructions in the HTTP
header how to fail.  (I'm trying to avoid killing and restarting the server,
because Windows seems to have issues with doing that a lot.)  But I'd rather fix
issues than polish tests before the freeze.

diff --git a/tests/test-lfs-serve-access.t b/tests/test-lfs-serve-access.t
--- a/tests/test-lfs-serve-access.t
+++ b/tests/test-lfs-serve-access.t
@@ -4,7 +4,6 @@
   > [extensions]
   > lfs=
   > [lfs]
-  > url=http://localhost:$HGPORT/.git/info/lfs
   > track=all()
   > [web]
   > push_ssl = False
@@ -149,3 +148,189 @@ Blob URIs are correct when --prefix is u
   $LOCALIP - - [$LOGDATE$] "GET /subdir/mount/point?cmd=getbundle HTTP/1.1" 
200 - 
x-hgarg-1:bookmarks=1=HG20%2Cbundle2%3DHG20%250Abookmarks%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Aphases%253Dheads%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps%250Arev-branch-cache%250Astream%253Dv2=1==525251863cad618e55d483555f3d00a2ca99597e=bookmarks=1
 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ partial-pull (glob)
   $LOCALIP - - [$LOGDATE$] "POST 
/subdir/mount/point/.git/info/lfs/objects/batch HTTP/1.1" 200 - (glob)
   $LOCALIP - - [$LOGDATE$] "GET 
/subdir/mount/point/.hg/lfs/objects/f03217a32529a28a42d03b1244fe09b6e0f9fd06d7b966d4d50567be2abe6c0e
 HTTP/1.1" 200 - (glob)
+
+  $ cat >> $TESTTMP/lfsstoreerror.py < import errno
+  > from hgext.lfs import blobstore
+  > 
+  > _numverifies = 0
+  > _readerr = True
+  > 
+  > def reposetup(ui, repo):
+  > # Nothing to do with a remote repo
+  > if not repo.local():
+  > return
+  > 
+  > store = repo.svfs.lfslocalblobstore
+  > class badstore(store.__class__):
+  > def download(self, oid, src):
+  > '''Called in the server to handle reading from the client in a
+  > PUT request.'''
+  > origread = src.read
+  > def _badread(nbytes):
+  > # Simulate bad data/checksum failure from the client
+  > return b'0' * len(origread(nbytes))
+  > src.read = _badread
+  > super(badstore, self).download(oid, src)
+  > 
+  > def _read(self, vfs, oid, verify):
+  > '''Called in the server to read data for a GET request, and 
then
+  > calls self._verify() on it before returning.'''
+  > global _readerr
+  > # One time simulation of a read error
+  > if _readerr:
+  > _readerr = False
+  > raise IOError(errno.EIO, '%s: I/O error' % oid)
+  > # Simulate corrupt content on client download
+  > blobstore._verify(oid, 'dummy content')
+  > 
+  > def verify(self, oid):
+  > '''Called in the server to populate the Batch API response,
+  > letting the client re-upload if the file is corrupt.'''
+  > # Fail verify in Batch API for one clone command and one push
+  > # command with an IOError.  Then let it through to access other
+  > # functions.  Checksum failure is tested elsewhere.
+  > global _numverifies
+  > _numverifies += 1
+  > if _numverifies <= 2:
+  > raise IOError(errno.EIO, '%s: I/O error' % oid)
+  > return super(badstore, self).verify(oid)
+  > 
+  > store.__class__ = badstore
+  > EOF
+
+  $ rm -rf `hg config lfs.usercache`
+  $ rm -f $TESTTMP/access.log $TESTTMP/errors.log
+  $ hg --config "lfs.usercache=$TESTTMP/servercache" \
+  >--config extensions.lfsstoreerror=$TESTTMP/lfsstoreerror.py \
+  >-R server serve -d \
+  >-p $HGPORT1 --pid-file=hg.pid -A $TESTTMP/access.log -E 
$TESTTMP/errors.log
+  $ cat hg.pid >> $DAEMON_PIDS
+
+Test an I/O error in localstore.verify() (Batch API) with GET
+
+  $ hg clone http://localhost:$HGPORT1 httpclone2
+  requesting all changes
+  adding changesets
+  adding manifests
+  adding file changes
+  added 1 changesets with 1 changes to 1 files
+  new changesets 525251863cad
+  updating to branch default
+  abort: LFS server error for "lfs.bin": Internal server error!
+  [255]
+
+Test an I/O error in localstore.verify() (Batch API) with PUT
+
+  $ echo foo > client/lfs.bin
+  $ hg -R client ci -m 'mod lfs'
+  $