Bug#836585: celery: FTBFS in testing (failing tests)
On Wed, 14 Sep 2016, Michael Fladischer wrote: > > So: It is really so much work to add a versioned build-depends? > > (I see that a lot of them are already versioned). > > > > We really *never* use versioned build-depends to avoid FTBFS bugs > > as some people claim? > > To me, versioned dependencies were used to prevent FTBFS in unstable. > But you have a point here, I'll probably add a version constraint to > python-openssl in my next upload to unstable. Hi. This is probably not necessary anymore as there is a better solution proposed by Scott Kitterman in -devel. Thanks.
Bug#836585: celery: FTBFS in testing (failing tests)
On Wed, Sep 14, 2016 at 01:06:43PM +0200, Michael Fladischer wrote: > You do realize that I'm a volunteer and therefore offer up my spare time > to work on those packages. One of the implications of this is that I'm > not always able to jump immediately when someone files a bugreport, > regardless if it's RC or not. Daily job and personal life comes first. > Please be patient with people. Sure, of course! If I expected this to be looked at sooner, it's only in a "statistical" sense, never in the duty or obligation sense. Thanks.
Bug#836585: celery: FTBFS in testing (failing tests)
On 2016-09-14 12:08, Santiago Vila wrote: > (The "some hint" thing I surely did: I clearly put "testing" in the > subject and "stretch" in the body). I did notice this and as I commented when lowering the severity, I used a testing buildroot with cowbuilder accordingly. > For completeness, here is a build log that was created the same day I > reported this, so you can check that this problem was indeed real. > > I didn't include it in the initial report for two reasons: > > * The bug was very easy to reproduce in testing and it was not a > random FTBFS bug at all. > > * I naively thought, being a FTBFS bug, that you would try to > reproduce it sooner. I reported this on 2016-09-04 and the first reply > from Paul was on 2016-09-13. You do realize that I'm a volunteer and therefore offer up my spare time to work on those packages. One of the implications of this is that I'm not always able to jump immediately when someone files a bugreport, regardless if it's RC or not. Daily job and personal life comes first. Please be patient with people. > This was already diagnosed by Paul and it was due to the > python-openssl package in testing at the time not being ok to build > the celery package in testing at the time. > > So this problem happened in stretch at least between 2016-09-04 and > 2016-09-09, the last date being the time pyopenssl 16.1.0-1 migrated > to testing. > > In case you still want to verify the problem, you can use > snapshot.debian.org and try building with python-openssl 16.0.0-2. I don't think anyone thought that your report was not based on hard facts, it was just that once Paul did his rebuild, a newer python-openssl has already migrated to testing. I could have reassigned the bug to python-openssl and annoy its maintainer with a bugreport that would have to be closed there. But I decided not to. > So: It is really so much work to add a versioned build-depends? > (I see that a lot of them are already versioned). > > We really *never* use versioned build-depends to avoid FTBFS bugs > as some people claim? To me, versioned dependencies were used to prevent FTBFS in unstable. But you have a point here, I'll probably add a version constraint to python-openssl in my next upload to unstable. > Agreed, this does not happen anymore in stretch (I assure you that it > did when I reported it), but the source package claims buildability > with the build-dependencies shown in the build-depends field in the > control file, so it may be argued this is still a bug in the source > package. What about Ubuntu or any other Debian-derived distro which > forks unstable and not testing? Someone on debian-devel mentioned checking for passing builds before migration packages to testing … maybe this is the way to go. Cheers, Michael signature.asc Description: OpenPGP digital signature
Bug#836585: celery: FTBFS in testing (failing tests)
On Wed, Sep 14, 2016 at 01:06:43PM +0200, Michael Fladischer wrote: > Someone on debian-devel mentioned checking for passing builds before > migration packages to testing … maybe this is the way to go. Yes, Matthias. I think he was really joking, or trying to make a point by using sarcasm. I replied to him and explained why it would not work: It's packages entering testing all the time breaking other packages that were already in testing, and we can't detect that by rebuilding only the packages which are about to enter testing. Thanks a lot.
Bug#836585: celery: FTBFS in testing (failing tests)
On Wed, 14 Sep 2016, Michael Fladischer wrote: > close 836585 > thanks > > Santiago, please provide full build logs and some hint on how you tried your > rebuild the next time you do some sort of MBF. This would have spared > maintainers > a lot of trouble and time. You are right and I'm sorry if the lack of a build log caused you any trouble. (The "some hint" thing I surely did: I clearly put "testing" in the subject and "stretch" in the body). For completeness, here is a build log that was created the same day I reported this, so you can check that this problem was indeed real. I didn't include it in the initial report for two reasons: * The bug was very easy to reproduce in testing and it was not a random FTBFS bug at all. * I naively thought, being a FTBFS bug, that you would try to reproduce it sooner. I reported this on 2016-09-04 and the first reply from Paul was on 2016-09-13. This was already diagnosed by Paul and it was due to the python-openssl package in testing at the time not being ok to build the celery package in testing at the time. So this problem happened in stretch at least between 2016-09-04 and 2016-09-09, the last date being the time pyopenssl 16.1.0-1 migrated to testing. In case you still want to verify the problem, you can use snapshot.debian.org and try building with python-openssl 16.0.0-2. So: It is really so much work to add a versioned build-depends? (I see that a lot of them are already versioned). We really *never* use versioned build-depends to avoid FTBFS bugs as some people claim? Agreed, this does not happen anymore in stretch (I assure you that it did when I reported it), but the source package claims buildability with the build-dependencies shown in the build-depends field in the control file, so it may be argued this is still a bug in the source package. What about Ubuntu or any other Debian-derived distro which forks unstable and not testing? Thanks a lot. celery_3.1.23-5_amd64-20160904T091402Z.gz Description: application/gzip
Bug#836585: celery: FTBFS in testing (failing tests)
On Tue, 13 Sep 2016, Paul Wise wrote: > On Sun, 4 Sep 2016 11:54:01 +0200 (CEST) Santiago Vila wrote: > > > I tried to build this package in stretch with "dpkg-buildpackage -A" > > (which is what the "Arch: all" autobuilder would do to build it) > > but it failed: > > I can't reproduce this build failure using `pdebuild --debbuildopts -A`. > > Which version of python-openssl did you have installed? On 2016-09-04, when I reported the bug, it was python-openssl 16.0.0-2. I'm having a deja vu... Now I can think of two ways to handle this: A. celery now builds in stretch, nothing to do, this was really a bug in python-openssl and it's already fixed. or B. Policy says "it must be possible to build the package when the build-dependencies are met". Updating the build-depends on python-openssl and make it versioned takes very little. I suggested B on another package but (in addition to being called a lot of things) I was told this: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=836508#24 which, as I explained later in the same bug, does not sound very convincing. So, if you choose A please be consistent at least and reassign the bug to python-openssl and use "affects" on "src:celery". Thanks.
Bug#836585: celery: FTBFS in testing (failing tests)
On Sun, 4 Sep 2016 11:54:01 +0200 (CEST) Santiago Vila wrote: > I tried to build this package in stretch with "dpkg-buildpackage -A" > (which is what the "Arch: all" autobuilder would do to build it) > but it failed: I can't reproduce this build failure using `pdebuild --debbuildopts -A`. Which version of python-openssl did you have installed? I have version python-openssl 16.1.0-1 installed. It looks like this issue was fixed by skipping PyOpenSSL tests on Python 3: $ egrep -iC3 'test_unmatched_key_cert|test_unknown_source' ../*.build test_self_send (celery.tests.security.test_serialization.test_SecureSerializer) ... ok test_separate_ends (celery.tests.security.test_serialization.test_SecureSerializer) ... ok test_serialize (celery.tests.security.test_serialization.test_SecureSerializer) ... ok test_unknown_source (celery.tests.security.test_serialization.test_SecureSerializer) ... ok test_unmatched_key_cert (celery.tests.security.test_serialization.test_SecureSerializer) ... ok test_AsyncResult_when_not_registered (celery.tests.tasks.test_canvas.test_Signature) ... ok test_INVERT (celery.tests.tasks.test_canvas.test_Signature) ... ok test_OR (celery.tests.tasks.test_canvas.test_Signature) ... ok -- test_self_send (celery.tests.security.test_serialization.test_SecureSerializer) ... SKIP: PyOpenSSL does not work on Python 3 test_separate_ends (celery.tests.security.test_serialization.test_SecureSerializer) ... SKIP: PyOpenSSL does not work on Python 3 test_serialize (celery.tests.security.test_serialization.test_SecureSerializer) ... SKIP: PyOpenSSL does not work on Python 3 test_unknown_source (celery.tests.security.test_serialization.test_SecureSerializer) ... SKIP: PyOpenSSL does not work on Python 3 test_unmatched_key_cert (celery.tests.security.test_serialization.test_SecureSerializer) ... SKIP: PyOpenSSL does not work on Python 3 test_AsyncResult_when_not_registered (celery.tests.tasks.test_canvas.test_Signature) ... ok test_INVERT (celery.tests.tasks.test_canvas.test_Signature) ... ok test_OR (celery.tests.tasks.test_canvas.test_Signature) ... ok -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Bug#836585: celery: FTBFS in testing (failing tests)
Package: src:celery
Version: 3.1.23-5
Severity: serious
Dear maintainer:
I tried to build this package in stretch with "dpkg-buildpackage -A"
(which is what the "Arch: all" autobuilder would do to build it)
but it failed:
[...]
debian/rules build-indep
dh build-indep --with python2,python3,sphinxdoc --buildsystem=pybuild
dh_testdir -i -O--buildsystem=pybuild
dh_update_autotools_config -i -O--buildsystem=pybuild
dh_auto_configure -i -O--buildsystem=pybuild
I: pybuild base:184: python2.7 setup.py config
running config
I: pybuild base:184: python3.5 setup.py config
running config
debian/rules override_dh_auto_build
make[1]: Entering directory '/<>'
PYTHONPATH=.:$PYTHONPATH sphinx-build -b html -d .build/.doctrees -N docs
.build/html
Running Sphinx v1.4.5
[... snipped ...]
ERROR: test_unknown_source
(celery.tests.security.test_serialization.test_SecureSerializer)
--
Traceback (most recent call last):
File "/<>/celery/tests/security/test_serialization.py", line 42,
in test_unknown_source
s1.deserialize, s1.serialize('foo'))
File "/<>/celery/security/serialization.py", line 55, in
serialize
signer=self._cert.get_id())
File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
self.gen.throw(type, value, traceback)
File "/<>/celery/security/utils.py", line 35, in reraise_errors
sys.exc_info()[2])
File "/<>/celery/security/utils.py", line 31, in reraise_errors
yield
File "/<>/celery/security/serialization.py", line 54, in
serialize
signature=self._key.sign(body, self._digest),
File "/<>/celery/security/key.py", line 27, in sign
return crypto.sign(self._key, ensure_bytes(data), digest)
File "/usr/lib/python2.7/dist-packages/OpenSSL/crypto.py", line 2563, in sign
md_ctx = _ffi.new("EVP_MD_CTX*")
SecurityError: Unable to serialize: TypeError("cannot instantiate ctype
'EVP_MD_CTX' of unknown size",)
==
ERROR: test_unmatched_key_cert
(celery.tests.security.test_serialization.test_SecureSerializer)
--
Traceback (most recent call last):
File "/<>/celery/tests/security/test_serialization.py", line 36,
in test_unmatched_key_cert
s.deserialize, s.serialize('foo'))
File "/<>/celery/security/serialization.py", line 55, in
serialize
signer=self._cert.get_id())
File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
self.gen.throw(type, value, traceback)
File "/<>/celery/security/utils.py", line 35, in reraise_errors
sys.exc_info()[2])
File "/<>/celery/security/utils.py", line 31, in reraise_errors
yield
File "/<>/celery/security/serialization.py", line 54, in
serialize
signature=self._key.sign(body, self._digest),
File "/<>/celery/security/key.py", line 27, in sign
return crypto.sign(self._key, ensure_bytes(data), digest)
File "/usr/lib/python2.7/dist-packages/OpenSSL/crypto.py", line 2563, in sign
md_ctx = _ffi.new("EVP_MD_CTX*")
SecurityError: Unable to serialize: TypeError("cannot instantiate ctype
'EVP_MD_CTX' of unknown size",)
--
Ran 1710 tests in 32.248s
FAILED (errors=7, skipped=49)
E: pybuild pybuild:276: test: plugin custom failed with: exit code=1: python2.7
setup.py test
dh_auto_test: pybuild --test --test-nose -i python{version} -p 2.7
--system=custom --test-args={interpreter} setup.py test returned exit code 13
debian/rules:25: recipe for target 'override_dh_auto_test' failed
make[1]: *** [override_dh_auto_test] Error 25
make[1]: Leaving directory '/<>'
debian/rules:10: recipe for target 'build-indep' failed
make: *** [build-indep] Error 2
dpkg-buildpackage: error: debian/rules build-indep gave error exit status 2
The failure does not seem related to using "dpkg-buildpackage -A".
Thanks.
