Re: running tests against installed version of package

2016-04-06 Thread Piotr Ożarowski
[Thomas Goirand, 2016-04-06]
> Don't use py.test-FOO, as this is deprecated. Instead, use something
> like this:
> 
> PYTHON3S:=$(shell py3versions -vr)
> 
> override_dh_auto_test:
> ifeq (,$(findstring nocheck, $(DEB_BUILD_OPTIONS)))
> @echo "===> Running tests"
> set -e ; set -x ; for i in 2.7 $(PYTHON3S) ; do \
>   PYTHONPATH=. python$$i -m pytest ; \
>   done
> endif
> 
> Note that you can hardcode 2.7, as we will keep this version for Python2
> basically forever (or until Python 2 completely dies), so it doesn't
> really mater.

hardcoding 2.7 is OK

> Having multiple version of Py3 supported also helps backporting from Sid
> to Jessie, and also supports future version of Py3 (as Brian wrote).
> 
> Last, as Piotr wrote, PYTHONPATH=. isn't enough for non pure-python
> stuff. In this type of case, there are other workarounds (which I can

again, testing sources, even without extensions is not something we
should do. If testing if all files are installed doesn't convince you,
how about 2to3 changes not applied while testing Python 3.X?

> tell if you need them). What I personally do is that I get things built
> and installed within debian/tmp. That's the most easy way, a lot more
> than double-guessing the build/* folder names. This also has the
> advantage that the egg-info gets (re-)generated in the correct place. In
> such case, you will want to do something like this:
> 
> python setup.py install --install-layout=deb \
>   --root=$(CURDIR)/debian/tmp
> PYTHONPATH=$(CURDIR)/debian/tmp/usr/lib/python2.7/dist-packages \
>   PYTHONPATH=. python -m pytest
> 
> Combine this with the loop for multiple version of Python above, and you
> got (nearly) everything covered.

... and you will end up in debian/rules that is 5 times bigger than
before and once pytest's interface changes, your package FTBFS - that's
OK if you have time to update all your packages, but if you, like me,
are lazy, then you can use "not very helpful for running tests" pybuild
that does all above and more for you.
-- 
Piotr Ożarowski Debian GNU/Linux Developer
www.ozarowski.pl  www.griffith.cc   www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645



Re: running tests against installed version of package

2016-04-05 Thread Thomas Goirand
On 04/02/2016 09:41 AM, Tiago Ilieve wrote:
> Thomas,
> 
> On 31 March 2016 at 18:49, Thomas Goirand  wrote:
>> Most of the time, you get by this doing:
>> PYTHONPATH=$(CURDIR) python -m pytest tests
> 
> Awesome tip! Just stumbled up on this one when imports where changed
> from relative to absolute and this tip properly fixed the matter.
> 
> Piotr,
> 
> On 31 March 2016 at 19:13, Piotr Ożarowski  wrote:
>> this will test python2.7 only and will most probably ignore extensions, etc.
> 
> I've used the following workaround for this.
> 
> override_dh_auto_test:
> PYTHONPATH=$(CURDIR) py.test
> PYTHONPATH=$(CURDIR) py.test-3

Don't use py.test-FOO, as this is deprecated. Instead, use something
like this:

PYTHON3S:=$(shell py3versions -vr)

override_dh_auto_test:
ifeq (,$(findstring nocheck, $(DEB_BUILD_OPTIONS)))
@echo "===> Running tests"
set -e ; set -x ; for i in 2.7 $(PYTHON3S) ; do \
PYTHONPATH=. python$$i -m pytest ; \
done
endif

Note that you can hardcode 2.7, as we will keep this version for Python2
basically forever (or until Python 2 completely dies), so it doesn't
really mater.

Having multiple version of Py3 supported also helps backporting from Sid
to Jessie, and also supports future version of Py3 (as Brian wrote).

Last, as Piotr wrote, PYTHONPATH=. isn't enough for non pure-python
stuff. In this type of case, there are other workarounds (which I can
tell if you need them). What I personally do is that I get things built
and installed within debian/tmp. That's the most easy way, a lot more
than double-guessing the build/* folder names. This also has the
advantage that the egg-info gets (re-)generated in the correct place. In
such case, you will want to do something like this:

python setup.py install --install-layout=deb \
--root=$(CURDIR)/debian/tmp
PYTHONPATH=$(CURDIR)/debian/tmp/usr/lib/python2.7/dist-packages \
PYTHONPATH=. python -m pytest

Combine this with the loop for multiple version of Python above, and you
got (nearly) everything covered.

Hoping this helps,

Cheers,

Thomas Goirand (zigo)



Re: running tests against installed version of package

2016-04-02 Thread Brian May
Tiago Ilieve  writes:
> I see. Actually I've removed the "override_dh_auto_test"[1] when I
> found out[2] that Pybuild can to do this by itself.
>
> Now I wonder whether this is enough to fit the use case of multiple
> Python 3 versions that you mentioned...


Yes, I believe that is sufficient. The default dh_auto_test will do the
correct thing.
-- 
Brian May 



Re: running tests against installed version of package

2016-04-02 Thread Tiago Ilieve
Hi Brian,

On 2 April 2016 at 22:32, Brian May  wrote:
> This will only test the current version of Python 3. Which is OK at the
> moment, there is only Python 3.5
>
> However it was very useful to have packages run tests against Python 3.5
> while Python 3.4 was still the default.
>
> I imagine the same thing will happen when Python 3.6 is released.

I see. Actually I've removed the "override_dh_auto_test"[1] when I
found out[2] that Pybuild can to do this by itself.

Now I wonder whether this is enough to fit the use case of multiple
Python 3 versions that you mentioned...

Regards,
Tiago.

[1]: 
https://anonscm.debian.org/git/collab-maint/python-path-and-address.git/commit/?h=debian/unstable=9b57cbf
[2]: https://wiki.debian.org/Python/LibraryStyleGuide#Overriding

-- 
Tiago "Myhro" Ilieve
Blog: https://blog.myhro.info/
GitHub: https://github.com/myhro
LinkedIn: https://br.linkedin.com/in/myhro
Montes Claros - MG, Brasil



Re: running tests against installed version of package

2016-04-02 Thread Brian May
Tiago Ilieve  writes:

> override_dh_auto_test:
> PYTHONPATH=$(CURDIR) py.test
> PYTHONPATH=$(CURDIR) py.test-3

This will only test the current version of Python 3. Which is OK at the
moment, there is only Python 3.5

However it was very useful to have packages run tests against Python 3.5
while Python 3.4 was still the default.

I imagine the same thing will happen when Python 3.6 is released.
-- 
Brian May 



Re: running tests against installed version of package

2016-04-02 Thread Piotr Ożarowski
[Thomas Goirand, 2016-04-02]
> > this will test python2.7 only
> 
> Running tests with multiple version of Python was out of scope of my
> message.

it might be out of scope of your message, but during package's build all
interpreters ought to be tested

> > and will most probably ignore extensions, etc.
> 
> What do you mean? What extensions? Suff like pypy?

no, I mean Python extensions (.so files). distutils builds them by
default in ./build/lib.linux-x86_64-2.7/foo (where "lib.linux-x86_64-2.7"
is sometimes random) and this dir is not in sys.path, so you're not
testing built files - you're testing source files only (.so files are not
tested, if any .py file is not installed, your tests will not detect
that, etc.). Testing sources is good, but we can do better.
-- 
Piotr Ożarowski Debian GNU/Linux Developer
www.ozarowski.pl  www.griffith.cc   www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645



Re: running tests against installed version of package

2016-04-02 Thread Thomas Goirand
On 04/01/2016 12:13 AM, Piotr Ożarowski wrote:
>> Most of the time, you get by this doing:
>> PYTHONPATH=$(CURDIR) python -m pytest tests
> 
> this will test python2.7 only

Running tests with multiple version of Python was out of scope of my
message.

> and will most probably ignore extensions, etc.

What do you mean? What extensions? Suff like pypy?

Cheers,

Thomas Goirand (zigo)



Re: running tests against installed version of package

2016-04-02 Thread Tiago Ilieve
Thomas,

On 31 March 2016 at 18:49, Thomas Goirand  wrote:
> Most of the time, you get by this doing:
> PYTHONPATH=$(CURDIR) python -m pytest tests

Awesome tip! Just stumbled up on this one when imports where changed
from relative to absolute and this tip properly fixed the matter.

Piotr,

On 31 March 2016 at 19:13, Piotr Ożarowski  wrote:
> this will test python2.7 only and will most probably ignore extensions, etc.

I've used the following workaround for this.

override_dh_auto_test:
PYTHONPATH=$(CURDIR) py.test
PYTHONPATH=$(CURDIR) py.test-3

... As Python's "unittest discover" weren't working anyway.

Regards,
Tiago.

-- 
Tiago "Myhro" Ilieve
Blog: https://blog.myhro.info/
GitHub: https://github.com/myhro
LinkedIn: https://br.linkedin.com/in/myhro
Montes Claros - MG, Brasil



Re: running tests against installed version of package

2016-03-31 Thread Piotr Ożarowski
> > By default pybuild runs tests I think using the source tree.

pybuild tells build system to build .py/.so files in temporary
directory (with a _stable_ name), copies tests dir there as well,
changes current directory to that dir (so that sys.path[0] doesn't point
to sources), runs tests and removes them (so that install target doesn't
pick them up)

> In most cases, Pybuild isn't very helpful for running tests and fails in
> the most common traps (like the one I'm solving for you below), which is
> why I don't really think it's useful at all. YMMV...

file bugs please. Works fine for me.

> > However I have a package where the tests require the entry points from
> > setup.py to be configured, the tests fail without this.
> 
> Most of the time, you get by this doing:
> PYTHONPATH=$(CURDIR) python -m pytest tests

this will test python2.7 only and will most probably ignore extensions, etc.
-- 
Piotr Ożarowski Debian GNU/Linux Developer
www.ozarowski.pl  www.griffith.cc   www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645



Re: running tests against installed version of package

2016-03-31 Thread Thomas Goirand
On 03/25/2016 08:17 AM, Brian May wrote:
> Hello,
> 
> By default pybuild runs tests I think using the source tree.

In most cases, Pybuild isn't very helpful for running tests and fails in
the most common traps (like the one I'm solving for you below), which is
why I don't really think it's useful at all. YMMV...

> However I have a package where the tests require the entry points from
> setup.py to be configured, the tests fail without this.

Most of the time, you get by this doing:
PYTHONPATH=$(CURDIR) python -m pytest tests

If this doesn't work (maybe because you need the egg-info which isn't in
the tarball, or otherwise...), then get the package installed first:

python setup.py install --install-layout=deb \
--root $(CURDIR)/debian/tmp
PYTHONPATH=$(CURDIR)/debian/tmp python -m pytest tests

In the case of apscheduler, simply defining PYTHONPATH=. is enough, then
unit tests failing are those trying to connect to a local redis server
which may be fixed by manually starting redis-server from your
debian/rules (take care: you *must* start it on a non-standard port or
unix file-socket, and get your unit tests to use that non-standard port,
otherwise you'll be in trouble either in sbuild, or in non-sbuild).

By doing the above, I was able to run *all* of the unit tests of
apscheduler (ie: 404 tests, minus the 7 tests which got automatically
skipped).

I hope this helps, even though you probably will not maintain this
package. Hopefully, it will help you for the next one! :)

Cheers,

Thomas Goirand (zigo)



Re: running tests against installed version of package

2016-03-31 Thread Barry Warsaw
On Mar 26, 2016, at 01:49 PM, Brian May wrote:

>Barry Warsaw  writes:
>
>> In some cases, I've just taken to adding DEP-8 tests for those.  
>
>Do you have an example I can look at?

Hi Brian.  Take a look at tox for example.

Cheers,
-Barry



Re: running tests against installed version of package

2016-03-25 Thread Ben Finney
Brian May  writes:

> Ben Finney  writes:
>
> > As another check, you can test the resulting URL with a ‘git clone’ to a
> > temporary target directory.
>
> It doesn't seem to like me today:

Alioth is not responding at all, for me at the moment. And not just for me:



You'll need to do the test when Alioth can actually pass the test :-)

-- 
 \  “If you were going to shoot a mime, would you use a silencer?” |
  `\—Steven Wright |
_o__)  |
Ben Finney



Re: running tests against installed version of package

2016-03-25 Thread Brian May
Ben Finney  writes:

> As another check, you can test the resulting URL with a ‘git clone’ to a
> temporary target directory.

It doesn't seem to like me today:

% git clone 
https://anonscm.debian.org/git/python-modules/packages/apscheduler.git
Cloning into 'apscheduler'...
[ relatively long timeout ]
fatal: unable to access 
'https://anonscm.debian.org/git/python-modules/packages/apscheduler.git/': 
gnutls_handshake() failed: Error in the pull function.

Is this URL correct?

I also tried git from unstable in case the version in Jessie is broken,
however the results are the same
-- 
Brian May 



Re: running tests against installed version of package

2016-03-25 Thread Ben Finney
Brian May  writes:

> Ben Finney  writes:
>
> > The “git:” URL method is not encrypted. You should specify the encrypted
> > “https:” method in the “VCS-Git” field
>
> So just to double check, this should solve that right?
>
> sed 's=git://anonscm.debian.org/=https://anonscm.debian.org/git/=' 
> */debian/control

That looks right to me, for the case you presented. I make no promises
for what will result on other texts :-)

> ...I notice a number of packages I maintain have this problem and I
> really want to make sure I get this right first time!

As another check, you can test the resulting URL with a ‘git clone’ to a
temporary target directory.

-- 
 \ “A child of five could understand this. Fetch me a child of |
  `\  five.” —Groucho Marx |
_o__)  |
Ben Finney



Re: running tests against installed version of package

2016-03-25 Thread Brian May
Ben Finney  writes:

> The “git:” URL method is not encrypted. You should specify the encrypted
> “https:” method in the “VCS-Git” field:
>
> Vcs-Git:
> https://anonscm.debian.org/git/python-modules/packages/apscheduler.git

So just to double check, this should solve that right?

sed 's=git://anonscm.debian.org/=https://anonscm.debian.org/git/=' 
*/debian/control

...I notice a number of packages I maintain have this problem and I
really want to make sure I get this right first time!
-- 
Brian May 



Re: running tests against installed version of package

2016-03-25 Thread Barry Warsaw
On Mar 25, 2016, at 06:17 PM, Brian May wrote:

>However I have a package where the tests require the entry points from
>setup.py to be configured, the tests fail without this.
>
>So what is the appropriate way to modify debian/rules to get the tests
>to run from the installed version with the entry points available?

In some cases, I've just taken to adding DEP-8 tests for those.

Cheers,
-Barry



Re: running tests against installed version of package

2016-03-25 Thread Ben Finney
Brian May  writes:

> Vcs-Git: git://anonscm.debian.org/python-modules/packages/apscheduler.git
> Vcs-Browser: 
> https://anonscm.debian.org/cgit/python-modules/packages/apscheduler.git

The “git:” URL method is not encrypted. You should specify the encrypted
“https:” method in the “VCS-Git” field:

Vcs-Git: 
https://anonscm.debian.org/git/python-modules/packages/apscheduler.git

-- 
 \  “That's all very good in practice, but how does it work in |
  `\ *theory*?” —anonymous |
_o__)  |
Ben Finney