Re: [Zope-dev] Linux x86_64 [Was: Zope Tests: 3 OK, 5 Failed]
On Tue, Jul 21, 2009 at 18:40, Tres Seaver tsea...@palladion.com wrote:
File /home/tseaver/projects/Zope-trunk/src/OFS/tests/testRanges.py,
line 332, in testMultipleRangesBigFileOutOfOrder
(7, 80001)])
File /home/tseaver/projects/Zope-trunk/src/OFS/tests/testRanges.py,
line 209, in expectMultipleRanges
str(len(body)), rsp.getHeader('content-length')))
File /opt/Python-2.6.2/lib/python2.6/unittest.py, line 321, in
failIf
if expr: raise self.failureException, msg
AssertionError: Incorrect Content-Length is set! Expected 20425,
got 20426.
I don't grok the range support code at all: probably Martijn Pieters is
the only person in the world who does. The tests are quite obscure to
me, so I can't diagnose even whether the failure is a testing artifact
or a real bug.
Here's my response to Tres; for some reason my mail reader didn't
include zope-dev. I can add now that I'll be at a sprint this weekend
where I'll probably have time to take a look myself:
8--
Sorry you find them obscure; I followed unit test expectations as best
as I could.
The test that fails does the following:
1) Create a big file (uploadBigFile, enough random data to be more
than one pdata chunk)
2) Request for multiple HTTP ranges (slices) of the file to be sent.
These slices should be returned a) in the same order, b) of the
correct length, and of course c) the correct slice of the big file
created in 1.
In this case line 330 specifies that bytes 10-15 (inclusive), the last
1 bytes, and bytes 7-8 (inclusive) are are to be returned.
The call to expectMultipleRanges includes the expected slices (python
syntax, so end value is exclusive).
3) On line 209 of expectMultipleRanges it then tests if the resulting
response has the correct content length header for the whole response
set. The failure reported is that the Content-Length header reports 1
byte more than what was actually sent, so the body was 20425 bytes
long, while the content-header claims it would be 20426 bytes.
Because this is a multi-part range request, Image.py line 277 and
onwards handle this request. The response ends up looking something
like this:
Content-length: size we pre-calculate
Content-type: multipart/byteranges; boundary=boundarystring
More headers
--boundarystring
Content-type: image/whatever
Content-range: bytes start-end/total-file-size
data ...
--boundarystring
... Another section
... etc.
--boundarystring--
There are 2 ways the failure could be caused.
Something could go wrong with the size calculation on lines 280-287;
the total body length is the length of the range data plus the length
of the headers per multi-part plus the length of the MIME boundaries
between them. There is thus a length per part plus the length of the
last boundary.
Or something goes wrong when writing the range data in lines 309-358.
Because there is only one byte difference between what was calculated
on lines 280-287 and what is produced here this is probably a newline
somewhere.
I do find it puzzling this only fails on 64 bit platforms. It could be
that the pdata chunk handling is borken and has a off-by-one error
when running on 64-bit platforms.
Hope this clarifies things a little. I have little time right now to
chase this myself right now, sorry!
8--
--
Martijn Pieters
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.testing 3.8 fails in mysterious ways
hi, just fyi: the bug is also present in zope.testing 3.7.7 (showing up when testing plone.z3cform). would it perhaps be possible to release a fixed 3.7.8 soon, so that our buildout stops complaining again? :) cheers, andi Godefroid Chapelle wrote: Sidnei da Silva wrote: On Thu, Jul 30, 2009 at 9:33 PM, Martin Aspelioptilude+li...@gmail.com wrote: Unfortunately, I've got other packages that depend on a newer zope.testing (specifically, collective.testcaselayer). But I thought zope.testing aimed to be able to run any valid tests, so it sounds like a bug in zope.testing regardless, at least since every other test I've run in the same instance work fine. The traceback you pasted shows an UnboundLocalError. What about looking at the source and figuring out why that local variable is not defined? -- Sidnei I bumped into the same bug while working on the same package... I allowed myself to add the missing assignment. However, I'd like someone to check over my shoulder. http://mail.zope.org/pipermail/checkins/2009-July/036586.html Further, this bug is triggered by another one happening when running tests with buildout in plone.z3cform current trunk (r102411). The test runner is trying to spawn a subprocess with the --resume-layer argument. However, the --resume-layer argument is not accepted by the subprocess. This produces an error in a format not foreseen by the error parser. (which then triggered the UnboundLocalError). It would be nice if someone with a better understanding of the testrunner could take a look and understand why the --resume-layer stuff happens. Christian ? -- zeidler it consulting - http://zitc.de/ - i...@zitc.de friedelstraße 31 - 12047 berlin - telefon +49 30 25563779 pgp key at http://zitc.de/pgp - http://wwwkeys.de.pgp.net/ plone 3.3rc4 released! -- http://plone.org/products/plone/ ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope Tests: 4 OK, 4 Failed
Summary of messages to the zope-tests list. Period Sun Aug 2 12:00:00 2009 UTC to Mon Aug 3 12:00:00 2009 UTC. There were 8 messages: 8 from Zope Tests. Test failures - Subject: FAILED (failures=1) : Zope-2.12 Python-2.6.2 : Linux From: Zope Tests Date: Sun Aug 2 20:50:59 EDT 2009 URL: http://mail.zope.org/pipermail/zope-tests/2009-August/012210.html Subject: FAILED (failures=1) : Zope-2.12-alltests Python-2.6.2 : Linux From: Zope Tests Date: Sun Aug 2 20:54:59 EDT 2009 URL: http://mail.zope.org/pipermail/zope-tests/2009-August/012212.html Subject: FAILED (failures=1) : Zope-trunk Python-2.6.2 : Linux From: Zope Tests Date: Sun Aug 2 20:56:59 EDT 2009 URL: http://mail.zope.org/pipermail/zope-tests/2009-August/012213.html Subject: FAILED (failures=1) : Zope-trunk-alltests Python-2.6.2 : Linux From: Zope Tests Date: Sun Aug 2 20:58:59 EDT 2009 URL: http://mail.zope.org/pipermail/zope-tests/2009-August/012214.html Tests passed OK --- Subject: OK : Zope-2.10 Python-2.4.6 : Linux From: Zope Tests Date: Sun Aug 2 20:44:58 EDT 2009 URL: http://mail.zope.org/pipermail/zope-tests/2009-August/012207.html Subject: OK : Zope-2.11 Python-2.4.6 : Linux From: Zope Tests Date: Sun Aug 2 20:46:58 EDT 2009 URL: http://mail.zope.org/pipermail/zope-tests/2009-August/012208.html Subject: OK : Zope-2.12 Python-2.4.6 : Linux From: Zope Tests Date: Sun Aug 2 20:48:59 EDT 2009 URL: http://mail.zope.org/pipermail/zope-tests/2009-August/012209.html Subject: OK : Zope-2.12-alltests Python-2.4.6 : Linux From: Zope Tests Date: Sun Aug 2 20:52:59 EDT 2009 URL: http://mail.zope.org/pipermail/zope-tests/2009-August/012211.html ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Linux x86_64 [Was: Zope Tests: 3 OK, 5 Failed]
On 03.08.09 09:04, Martijn Pieters wrote:
On Tue, Jul 21, 2009 at 18:40, Tres Seaver tsea...@palladion.com wrote:
File /home/tseaver/projects/Zope-trunk/src/OFS/tests/testRanges.py,
line 332, in testMultipleRangesBigFileOutOfOrder
(7, 80001)])
File /home/tseaver/projects/Zope-trunk/src/OFS/tests/testRanges.py,
line 209, in expectMultipleRanges
str(len(body)), rsp.getHeader('content-length')))
File /opt/Python-2.6.2/lib/python2.6/unittest.py, line 321, in
failIf
if expr: raise self.failureException, msg
AssertionError: Incorrect Content-Length is set! Expected 20425,
got 20426.
I don't grok the range support code at all: probably Martijn Pieters is
the only person in the world who does. The tests are quite obscure to
me, so I can't diagnose even whether the failure is a testing artifact
or a real bug.
Here's my response to Tres; for some reason my mail reader didn't
include zope-dev. I can add now that I'll be at a sprint this weekend
where I'll probably have time to take a look myself:
Oki.
Tnx,
Andreas
begin:vcard
fn:Andreas Jung
n:Jung;Andreas
org:ZOPYX Ltd. Co. KG
adr;quoted-printable:;;Charlottenstr. 37/1;T=C3=BCbingen;;72070;Germany
email;internet:i...@zopyx.com
title:CEO
tel;work:+49-7071-793376
tel;fax:+49-7071-7936840
tel;home:+49-7071-793257
x-mozilla-html:FALSE
url:www.zopyx.com
version:2.1
end:vcard
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.index 3.5.2 broken
On Sun, Aug 02, 2009 at 08:48:24PM +0200, Andreas Jung wrote: Hi, the doctests for zope.index 3.5.2 - as used in Zope 2.12 - fail badly: File /home/ajung/.buildout/eggs/zope.index-3.5.2-py2.6-linux-x86_64.egg/zope/index/text/tests/../textindex.txt, line 143, in textindex.txt Failed example: [(k, %.4f % v) for (k, v) in index2.apply(Zorro).items()] == result I'm assuming items() returns a plain Python dictionary with string keys. Python's string hashes return different valuesfor half of all the strings on 64-bit machines. This influences the ordering of dictionary keys and some other things too (such as the sequence of random numbers you get if you use a string as the seed). Add a sorted() on both sides and the test should pass. Marius Gedminas -- http://pov.lt/ -- Zope 3 consulting and development signature.asc Description: Digital signature ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.index 3.5.2 broken
Marius Gedminas wrote: On Sun, Aug 02, 2009 at 08:48:24PM +0200, Andreas Jung wrote: Hi, the doctests for zope.index 3.5.2 - as used in Zope 2.12 - fail badly: File /home/ajung/.buildout/eggs/zope.index-3.5.2-py2.6-linux-x86_64.egg/zope/index/text/tests/../textindex.txt, line 143, in textindex.txt Failed example: [(k, %.4f % v) for (k, v) in index2.apply(Zorro).items()] == result I'm assuming items() returns a plain Python dictionary with string keys. Python's string hashes return different valuesfor half of all the strings on 64-bit machines. This influences the ordering of dictionary keys and some other things too (such as the sequence of random numbers you get if you use a string as the seed). Add a sorted() on both sides and the test should pass. Actually, those tests were plain insane and I've fixed them on the trunk. I intend to make a new zope.index release today. How insane were these tests? Well, the author of the tests noticed that the C optimization produces different scores than the Python version, and compensated for that in a way that dramatically reduced readability. Furthermore, the C code was producing incorrect results due to an unsafe optimization that accidentally took 32 bits of a Python float and pretended it was an integer. The breakage only became obvious when we started taking 64 bits instead, leading to completely different results. Shane ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.testing 3.8 fails in mysterious ways
On Mon, Aug 03, 2009 at 01:17:41PM +0200, Andreas Zeidler wrote: just fyi: the bug is also present in zope.testing 3.7.7 (showing up when testing plone.z3cform). would it perhaps be possible to release a fixed 3.7.8 soon, so that our buildout stops complaining again? :) I looked at the bug and Godefroid's checkin today. Things seem to be complicated. Short summary: collective.recipe.z2testrunner doesn't support running tests in a subprocess. Long summary: https://launchpad.net/bugs/407916 As a quick workaround, instead of a single 'run tests' step that does bin/test do separate 'run unit tests' and 'run functional tests' steps bin/test -u bin/test -f in your buildbot config. Marius Gedminas -- http://pov.lt/ -- Zope 3 consulting and development signature.asc Description: Digital signature ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.index 3.5.2 broken
On 8/3/09 1:07 PM, Shane Hathaway wrote: Marius Gedminas wrote: On Sun, Aug 02, 2009 at 08:48:24PM +0200, Andreas Jung wrote: Hi, the doctests for zope.index 3.5.2 - as used in Zope 2.12 - fail badly: File /home/ajung/.buildout/eggs/zope.index-3.5.2-py2.6-linux-x86_64.egg/zope/index/text/tests/../textindex.txt, line 143, in textindex.txt Failed example: [(k, %.4f % v) for (k, v) in index2.apply(Zorro).items()] == result I'm assuming items() returns a plain Python dictionary with string keys. Python's string hashes return different valuesfor half of all the strings on 64-bit machines. This influences the ordering of dictionary keys and some other things too (such as the sequence of random numbers you get if you use a string as the seed). Add a sorted() on both sides and the test should pass. Actually, those tests were plain insane and I've fixed them on the trunk. I intend to make a new zope.index release today. How insane were these tests? Well, the author of the tests noticed that the C optimization produces different scores than the Python version, and compensated for that in a way that dramatically reduced readability. /me hangs head in shame. - C ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.index 3.5.2 broken
On 03.08.09 20:15, Chris McDonough wrote: On 8/3/09 1:07 PM, Shane Hathaway wrote: Marius Gedminas wrote: On Sun, Aug 02, 2009 at 08:48:24PM +0200, Andreas Jung wrote: Hi, the doctests for zope.index 3.5.2 - as used in Zope 2.12 - fail badly: File /home/ajung/.buildout/eggs/zope.index-3.5.2-py2.6-linux-x86_64.egg/zope/index/text/tests/../textindex.txt, line 143, in textindex.txt Failed example: [(k, %.4f % v) for (k, v) in index2.apply(Zorro).items()] == result I'm assuming items() returns a plain Python dictionary with string keys. Python's string hashes return different valuesfor half of all the strings on 64-bit machines. This influences the ordering of dictionary keys and some other things too (such as the sequence of random numbers you get if you use a string as the seed). Add a sorted() on both sides and the test should pass. Actually, those tests were plain insane and I've fixed them on the trunk. I intend to make a new zope.index release today. How insane were these tests? Well, the author of the tests noticed that the C optimization produces different scores than the Python version, and compensated for that in a way that dramatically reduced readability. Is there a buildbot for the zope.* or ZTK packages testing them under Linux 32bit and 64 bit? Andreas begin:vcard fn:Andreas Jung n:Jung;Andreas org:ZOPYX Ltd. Co. KG adr;quoted-printable:;;Charlottenstr. 37/1;T=C3=BCbingen;;72070;Germany email;internet:i...@zopyx.com title:CEO tel;work:+49-7071-793376 tel;fax:+49-7071-7936840 tel;home:+49-7071-793257 x-mozilla-html:FALSE url:www.zopyx.com version:2.1 end:vcard ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.index 3.5.2 broken
Chris McDonough wrote: On 8/3/09 1:07 PM, Shane Hathaway wrote: How insane were these tests? Well, the author of the tests noticed that the C optimization produces different scores than the Python version, and compensated for that in a way that dramatically reduced readability. /me hangs head in shame. That was you? Well, it could have been worse. :-) Shane ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.index 3.5.2 broken
On 8/3/09 2:34 PM, Shane Hathaway wrote: Chris McDonough wrote: On 8/3/09 1:07 PM, Shane Hathaway wrote: How insane were these tests? Well, the author of the tests noticed that the C optimization produces different scores than the Python version, and compensated for that in a way that dramatically reduced readability. /me hangs head in shame. That was you? Well, it could have been worse. :-) I didn't write the original tests but I did paper over the okascore result differences by making the tests compare true or false vs. the expected output. Ten lashes. - C ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.index 3.5.2 broken
* Andreas Jung li...@zopyx.com [2009-08-03 20:21]: On 03.08.09 20:15, Chris McDonough wrote: On 8/3/09 1:07 PM, Shane Hathaway wrote: Marius Gedminas wrote: On Sun, Aug 02, 2009 at 08:48:24PM +0200, Andreas Jung wrote: the doctests for zope.index 3.5.2 - as used in Zope 2.12 - fail badly: Python's string hashes return different valuesfor half of all the strings on 64-bit machines. This influences the ordering of dictionary keys and some other things too (such as the sequence of random numbers you get if you use a string as the seed). Is there a buildbot for the zope.* or ZTK packages testing them under Linux 32bit and 64 bit? AFAICS all three buildbots listed on http://docs.zope.org/zopetoolkit/process/tools.html do. Wolfgang ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
