Bug#732945: Debian-Bug#732945: python-csb autopkg test always fails
Hi Ivan, you were right on target. I've added the build path to the PYTHONPATH and the tests are now running. Andreas, I'll push my changes to the repo right away. Could you please check if everything seems to be fine also on your side? Cheers! Tomás On 28/12/13 00:32, Ivan Kalev wrote: Hi Andreas, Nothing has really changed on our end. Is /your/path/to/CSB in $PYTHONPATH? This error typically means that the package csb is not importable, i.e. the python interpreter does not know where to find it. Here is an example: $ hg clone https://hg.codeplex.com/csb ~/build $ export PYTHONPATH=~/build:$PYTHONPATH $ python ~/build/csb/test/app.py -g /tmp Best regards, Ivan -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#732945: Debian-Bug#732945: python-csb autopkg test always fails
Hi Andreas, Here is an alternative solution: in version 1.2.2 you can override the default cache directory with -g: $ python csb/test/app.py --generated-resources /tmp What do you think? Version 1.2.2 can be downloaded from PyPI or CodePlex Best regards, Ivan On Dec 25, 2013 10:02 AM, Andreas Tille andr...@an3as.eu wrote: Hi Ivan, On Wed, Dec 25, 2013 at 08:56:02AM +, Ivan Kalev wrote: The problem with /var/lib or cache is that it won't work on windows. The advantage of ~ is that python resolves it to user's home directory on any platform. Transparent portability is somehow essential in csb. OK, sorry for my *NIX centric view. :-) But there are other options as well: 1) Use a temp directory and setup/teardown the text fixture every time. 2) Skip some tests at runtime if the resources they depend on are missing. These tests are important during development, but not at all essential during installation. So it is perfectly okay to skip them. Both solutions are very easy to implement. IMHO both solutions sound reasonable for our purpose. I trust you to pick the better one from your position. Kind regards Andreas. -- http://fam-tille.de
Bug#732945: Debian-Bug#732945: python-csb autopkg test always fails
Hi Ivan, On Fri, Dec 27, 2013 at 11:07:57PM +0100, Ivan Kalev wrote: Hi Andreas, Here is an alternative solution: in version 1.2.2 you can override the default cache directory with -g: $ python csb/test/app.py --generated-resources /tmp What do you think? Version 1.2.2 can be downloaded from PyPI or CodePlex In principle this sounds very good, but $ PYTHONWARNINGS=d python csb/test/app.py --generated-resources /tmp Traceback (most recent call last): File csb/test/app.py, line 5, in module from csb.test import Console ImportError: No module named csb.test What new dependency is needed here to run the test successfully? Kind regards Andreas. -- http://fam-tille.de -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#732945: Debian-Bug#732945: python-csb autopkg test always fails
Hi Andreas, Nothing has really changed on our end. Is /your/path/to/CSB in $PYTHONPATH? This error typically means that the package csb is not importable, i.e. the python interpreter does not know where to find it. Here is an example: $ hg clone https://hg.codeplex.com/csb ~/build $ export PYTHONPATH=~/build:$PYTHONPATH $ python ~/build/csb/test/app.py -g /tmp Best regards, Ivan On Dec 27, 2013 11:18 PM, Andreas Tille ti...@debian.org wrote: Hi Ivan, On Fri, Dec 27, 2013 at 11:07:57PM +0100, Ivan Kalev wrote: Hi Andreas, Here is an alternative solution: in version 1.2.2 you can override the default cache directory with -g: $ python csb/test/app.py --generated-resources /tmp What do you think? Version 1.2.2 can be downloaded from PyPI or CodePlex In principle this sounds very good, but $ PYTHONWARNINGS=d python csb/test/app.py --generated-resources /tmp Traceback (most recent call last): File csb/test/app.py, line 5, in module from csb.test import Console ImportError: No module named csb.test What new dependency is needed here to run the test successfully? Kind regards Andreas. -- http://fam-tille.de
Bug#732945: Debian-Bug#732945: python-csb autopkg test always fails
Hi Andreas,
The problem with /var/lib or cache is that it won't work on windows. The
advantage of ~ is that python resolves it to user's home directory on any
platform. Transparent portability is somehow essential in csb.
But there are other options as well:
1) Use a temp directory and setup/teardown the text fixture every time.
2) Skip some tests at runtime if the resources they depend on are missing.
These tests are important during development, but not at all essential
during installation. So it is perfectly okay to skip them.
Both solutions are very easy to implement.
Best regards,
Ivan
On Dec 24, 2013 4:24 PM, Andreas Tille andr...@an3as.eu wrote:
Hi Ivan,
On Tue, Dec 24, 2013 at 08:41:38AM +, Ivan Kalev wrote:
Just to make sure that I understood everything correctly:
- you run the build as root
Well, it is actually fakeroot (so faking to be root but without
permission to write to /usr).
- then you run the test suite again as root and it passes, right?
No, the testsuite does not pass.
- this bug appears only when you install the built package and then run
the
test suite as normal user?
In both cases it does not run.
Is this all correct?
Here is some background. Our project supports several workflows:
1) csb developer: checkout/update from the repo, run all tests, write
code,
run tests, commit, etc. The first time a developer runs the test suite,
some files get written to csb/test/data and remain there (virtually
forever). Next time the developer runs the test suite to verify their
code,
these files will be read directly from csb/test/data. This is what I mean
by future use.
2) csb nightly build process: this is pretty much your workflow as well.
Checkout the repo in an empty directory, run the test suite, build
package,
publish artifacts, wipe the entire checkout directory. The moment the
build
bot runs the test suite, some files get generated in csb/test/data. These
are needed by some test cases and will get deleted along with everything
else once the checkout dir gets wiped at the end of the build.
3) csb user: install package, use package. No test suites are necessary
and
no files in csb/test/data need to get generated, stored or used. If the
user decided to run the test suite for some reason (e.g. to verify that
all
dependencies are properly installed), csb will attempt to generate the
missing files in csb/test/data. But as you observed, this is only
possible
if csb/test/data is a writable directory. One workaround is to install
csb
from PyPI with normal privileges and a prefix:
$ python setup.py install --prefix alt location
Another possibility is to modify our code to write these files in
csb/test/data, but if that is not possible --- to ~/.csb-test-files or
something similar.
According to FHS it would be reasonable to write to
/var/{lib,cache}/csb
I think this would fit your workflow as well, isn't it?
Many thanks for your help!
Thanks for supporting our packaging by responding very quickly and helpful
Andreas.
--
http://fam-tille.de
Bug#732945: Debian-Bug#732945: python-csb autopkg test always fails
Hi Ivan, On Wed, Dec 25, 2013 at 08:56:02AM +, Ivan Kalev wrote: The problem with /var/lib or cache is that it won't work on windows. The advantage of ~ is that python resolves it to user's home directory on any platform. Transparent portability is somehow essential in csb. OK, sorry for my *NIX centric view. :-) But there are other options as well: 1) Use a temp directory and setup/teardown the text fixture every time. 2) Skip some tests at runtime if the resources they depend on are missing. These tests are important during development, but not at all essential during installation. So it is perfectly okay to skip them. Both solutions are very easy to implement. IMHO both solutions sound reasonable for our purpose. I trust you to pick the better one from your position. Kind regards Andreas. -- http://fam-tille.de -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#732945: Debian-Bug#732945: python-csb autopkg test always fails
Hi Andreas, Just to make sure that I understood everything correctly: - you run the build as root - then you run the test suite again as root and it passes, right? - this bug appears only when you install the built package and then run the test suite as normal user? Is this all correct? Here is some background. Our project supports several workflows: 1) csb developer: checkout/update from the repo, run all tests, write code, run tests, commit, etc. The first time a developer runs the test suite, some files get written to csb/test/data and remain there (virtually forever). Next time the developer runs the test suite to verify their code, these files will be read directly from csb/test/data. This is what I mean by future use. 2) csb nightly build process: this is pretty much your workflow as well. Checkout the repo in an empty directory, run the test suite, build package, publish artifacts, wipe the entire checkout directory. The moment the build bot runs the test suite, some files get generated in csb/test/data. These are needed by some test cases and will get deleted along with everything else once the checkout dir gets wiped at the end of the build. 3) csb user: install package, use package. No test suites are necessary and no files in csb/test/data need to get generated, stored or used. If the user decided to run the test suite for some reason (e.g. to verify that all dependencies are properly installed), csb will attempt to generate the missing files in csb/test/data. But as you observed, this is only possible if csb/test/data is a writable directory. One workaround is to install csb from PyPI with normal privileges and a prefix: $ python setup.py install --prefix alt location Another possibility is to modify our code to write these files in csb/test/data, but if that is not possible --- to ~/.csb-test-files or something similar. Many thanks for your help! Best regards, Ivan On Dec 23, 2013 4:00 PM, Andreas Tille andr...@an3as.eu wrote: Hi Ivan, many thanks for your quick reply. On Mon, Dec 23, 2013 at 12:39:46PM +, Ivan Kalev wrote: Michael's address has changed indeed --- the correct one is mhab...@gwdg.de. Thanks - updated the address in our packaging metadata. The problem here is that these are not intermediate results, but rather static test files that need to be installed once and cached for future use. So a temp directory won't work here I guess. What do you think about storing these files in ~/csb-test-files when the install dir is not write-accessible? We can implement that easily. Hmmm, I'm not sure what you mean by cached for future use. We would like to run the test-suite in the package build process which is usually done in a minimalistic chroot environment and there is only root as user. This environment is deleted once the package is built (and the test has passed). I'm not really sure how this fits your imagination of data used in the future. By the way, this is probably an issue only when the PyPI package is installed with sudo (which is not at all necessary when doing this type of installation) and the test suite is useful mostly for developers, who are supposed to checkout the repository anyway (rather than installing the package). So I hope that the impact of this bug is not big. No, it is not really big. We could even skip the test completely. However, we are always keen on verifying the functionality of some software we are bundling as Debian packages. Thus we try to run any test that is available and makes sense. If you think it does not make any sense this is no problem and we will disable it (which would close the bug report as well). Best regards, Ivan Thanks again for your response Andreas. -- http://fam-tille.de
Bug#732945: Debian-Bug#732945: python-csb autopkg test always fails
Hi Ivan,
On Tue, Dec 24, 2013 at 08:41:38AM +, Ivan Kalev wrote:
Just to make sure that I understood everything correctly:
- you run the build as root
Well, it is actually fakeroot (so faking to be root but without
permission to write to /usr).
- then you run the test suite again as root and it passes, right?
No, the testsuite does not pass.
- this bug appears only when you install the built package and then run the
test suite as normal user?
In both cases it does not run.
Is this all correct?
Here is some background. Our project supports several workflows:
1) csb developer: checkout/update from the repo, run all tests, write code,
run tests, commit, etc. The first time a developer runs the test suite,
some files get written to csb/test/data and remain there (virtually
forever). Next time the developer runs the test suite to verify their code,
these files will be read directly from csb/test/data. This is what I mean
by future use.
2) csb nightly build process: this is pretty much your workflow as well.
Checkout the repo in an empty directory, run the test suite, build package,
publish artifacts, wipe the entire checkout directory. The moment the build
bot runs the test suite, some files get generated in csb/test/data. These
are needed by some test cases and will get deleted along with everything
else once the checkout dir gets wiped at the end of the build.
3) csb user: install package, use package. No test suites are necessary and
no files in csb/test/data need to get generated, stored or used. If the
user decided to run the test suite for some reason (e.g. to verify that all
dependencies are properly installed), csb will attempt to generate the
missing files in csb/test/data. But as you observed, this is only possible
if csb/test/data is a writable directory. One workaround is to install csb
from PyPI with normal privileges and a prefix:
$ python setup.py install --prefix alt location
Another possibility is to modify our code to write these files in
csb/test/data, but if that is not possible --- to ~/.csb-test-files or
something similar.
According to FHS it would be reasonable to write to
/var/{lib,cache}/csb
I think this would fit your workflow as well, isn't it?
Many thanks for your help!
Thanks for supporting our packaging by responding very quickly and helpful
Andreas.
--
http://fam-tille.de
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#732945: python-csb autopkg test always fails
Sorry about missing the previous related bug and for my delay answering this one. I will contact upstream right away to figure out a fix. Best regards, Tomás Date: Sun, 22 Dec 2013 23:44:55 +0100 From: Matthias Klose d...@debian.org - Body: Package: python-csb Version: 1.2.1+dfsg-2 Severity: serious Tags: sid, jessie the python-csb autopkg test always fails, trying to write test data into the installed location. adt-run:
Bug#732945: Debian-Bug#732945: python-csb autopkg test always fails
Hi Ivan, many thanks for your quick reply. On Mon, Dec 23, 2013 at 12:39:46PM +, Ivan Kalev wrote: Michael's address has changed indeed --- the correct one is mhab...@gwdg.de. Thanks - updated the address in our packaging metadata. The problem here is that these are not intermediate results, but rather static test files that need to be installed once and cached for future use. So a temp directory won't work here I guess. What do you think about storing these files in ~/csb-test-files when the install dir is not write-accessible? We can implement that easily. Hmmm, I'm not sure what you mean by cached for future use. We would like to run the test-suite in the package build process which is usually done in a minimalistic chroot environment and there is only root as user. This environment is deleted once the package is built (and the test has passed). I'm not really sure how this fits your imagination of data used in the future. By the way, this is probably an issue only when the PyPI package is installed with sudo (which is not at all necessary when doing this type of installation) and the test suite is useful mostly for developers, who are supposed to checkout the repository anyway (rather than installing the package). So I hope that the impact of this bug is not big. No, it is not really big. We could even skip the test completely. However, we are always keen on verifying the functionality of some software we are bundling as Debian packages. Thus we try to run any test that is available and makes sense. If you think it does not make any sense this is no problem and we will disable it (which would close the bug report as well). Best regards, Ivan Thanks again for your response Andreas. -- http://fam-tille.de -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#732945: python-csb autopkg test always fails
Package: python-csb
Version: 1.2.1+dfsg-2
Severity: serious
Tags: sid, jessie
the python-csb autopkg test always fails, trying to write test data into the
installed location.
adt-run: dsc0t-build: [
- Finished parsing the build-deps
Traceback (most recent call last):
File test/app.py, line 11, in module
main('csb.test.cases.*')
File /usr/lib/python2.7/dist-packages/csb/test/__init__.py, line 804, in
__init__
self.run()
File /usr/lib/python2.7/dist-packages/csb/test/__init__.py, line 850, in run
Config().ensureDataConsistency()
File /usr/lib/python2.7/dist-packages/csb/test/__init__.py, line 312, in
ensureDataConsistency
self.updateDataFiles()
File /usr/lib/python2.7/dist-packages/csb/test/__init__.py, line 330, in
updateDataFiles
Pickle.dump(ensemble, open(os.path.join(self.data, '1nz9.full.pickle'),
'wb'))
IOError: [Errno 13] Permission denied:
'/usr/lib/python2.7/dist-packages/csb/test/data/1nz9.full.pickle'
adt-run1: testbed executing test finished with exit status 1
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
