Re: Compiled bytecode installed, because test suite was run

2016-04-13 Thread Piotr Ożarowski
[Ben Finney, 2016-04-13]
> package_share_dir = /usr/share/${PACKAGE_NAME}
> export PYBUILD_INSTALL_ARGS ?= \
>--install-lib=${package_share_dir}/ \
>--install-scripts=${package_share_dir}/
> 
> So with Pybuild configured correctly, that should mean I don't need to
> also fiddle manually with ‘dh_python3’. Yes?

No. pybuild and dh_python3 are two separate things. I do my best to
separate them (so that you can replace any of them at any time and the
other one will still work - f.e. you can use dh_py{support,central}
instead of dh_python2 or dh_install instead of pybuild).
This is why you have to pass '--with python3' to dh even though you
already have '--buildsystem=pybuild'.

If you configure pybuild to install into some custom location that is
not checked by dh_python3 - you have to tell dh_python3 to handle that
dir.

Why it doesn't check all dirs?  well, if you had a good reason to
install into custom dir (a plugin for different app, an example file, a
documentation, ...), dh_python3 will not mess with your files and let
tools that handle that dir do their job (note that it's possible that
it still will be handled later by the same tools dh_python3 uses:
py3compile/py3clean but via other package's trigger)
-- 
Piotr Ożarowski Debian GNU/Linux Developer
www.ozarowski.pl  www.griffith.cc   www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645



Re: Compiled bytecode installed, because test suite was run

2016-04-13 Thread Ben Finney
Piotr Ożarowski  writes:

> [Ben Finney, 2016-04-13]
> > One complication may have been that the directory name to which the
> > library was installed (with the “--install-lib” option) does not
> > contain the Debian package name.
>
> […] By changing --install-lib you modified destination directory
> anyway (and you should point dh_python3 to the same dir - it doesn't
> know about pybuild's existence)

I'm not invoking ‘dh_python3’ directly, I am telling ‘dh’ to do it:

dh $@ --with python3 --buildsystem=pybuild

To my mind that should cause Pybuild to tell ‘dh-python3’ everything it
needs to know, because I've configured Pybuild:

package_share_dir = /usr/share/${PACKAGE_NAME}
export PYBUILD_INSTALL_ARGS ?= \
   --install-lib=${package_share_dir}/ \
   --install-scripts=${package_share_dir}/

So with Pybuild configured correctly, that should mean I don't need to
also fiddle manually with ‘dh_python3’. Yes?

-- 
 \ “Capitalism has destroyed our belief in any effective power but |
  `\  that of self interest backed by force.” —George Bernard Shaw |
_o__)  |
Ben Finney



Re: Compiled bytecode installed, because test suite was run

2016-04-13 Thread Piotr Ożarowski
> PYBUILD_NAME and PYBUILD_DESTDIR are pybuild related, dh_python3 doesn't
> use them. By changing --install-lib you modified destination directory
> anyway (and you should point dh_python3 to the same dir - it doesn't
> know about pybuild's existence)

FTR: PYBUILD_DESTDIR is like DESTDIR but allows to set different dirs for
each interpreter and PYBUILD_NAME is something that can be used if all
your PYBUILD_DESTDIRs follow standards (so instead of defining destrid
for Python 3, Python 2, debug interpreters, pypy, etc. you can just
define a name that will be used to set destdirs correcly - all this is
optional, you can use dh_install if you want)

> That said, removing __pycache__ everywhere is harmless and dh_python3
> should remove it even if you don't tell it to check given dir. I will
> change that.

actually that should be the case already so I don't know why it didn't
work for you. My guess is you didn't run dh_python3 at all
-- 
Piotr Ożarowski Debian GNU/Linux Developer
www.ozarowski.pl  www.griffith.cc   www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645



Re: Compiled bytecode installed, because test suite was run

2016-04-13 Thread Piotr Ożarowski
[Ben Finney, 2016-04-13]
> Piotr Ożarowski  writes:
> 
> > [Ben Finney, 2016-04-12]
> > > * Pybuild runs the upstream test suite
> > > * which imports the Python packages
> > > * which creates the compiled bytecode files
> > > * which remain in the package directories
> > > * which are scooped up by Pybuild for installation
> >
> > * which should be removed by dh_python3
> 
> How does ‘dh_python3’ know to do that? Are there settings I should look
> at to tell ‘dh_python3’ what to do when it misses some files?

dh_python3 is checking dist-packages and few private dirs by default,
see "private dirs" section in dh_python3.1

> > > * which leads to Lintian correctly complaining
> > >   “package-installs-python-pycache-dir”.
> > > 
> > > How do I convince Pybuild to run the tests as part of the package build
> > > (as normal), but clean up the compiled files *after* the tests and
> > > *before* gathering the installed files?
> >
> > does your package name start with python3- or do you have
> > ${python3:Depends} in Depends?
> 
> The problem is no longer occurring, because I have tweaked some of the
> directories and package names around.
> 
> One complication may have been that the directory name to which the
> library was installed (with the “--install-lib” option) does not contain
> the Debian package name.
> 
> When I change that, it works; I don't know why.
> 
> The documentation for ‘PYBUILD_NAME’ and ‘PYBUILD_DESTDIR’ don't help me
> to understand what is happening when things go wrong. The latter,
> especially, makes no sense to me and the man page doesn't help me know
> what effect setting that variable will have.

PYBUILD_NAME and PYBUILD_DESTDIR are pybuild related, dh_python3 doesn't
use them. By changing --install-lib you modified destination directory
anyway (and you should point dh_python3 to the same dir - it doesn't
know about pybuild's existence)

That said, removing __pycache__ everywhere is harmless and dh_python3
should remove it even if you don't tell it to check given dir. I will
change that.
-- 
Piotr Ożarowski Debian GNU/Linux Developer
www.ozarowski.pl  www.griffith.cc   www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645



Re: Compiled bytecode installed, because test suite was run

2016-04-12 Thread Ben Finney
Piotr Ożarowski  writes:

> [Ben Finney, 2016-04-12]
> > * Pybuild runs the upstream test suite
> > * which imports the Python packages
> > * which creates the compiled bytecode files
> > * which remain in the package directories
> > * which are scooped up by Pybuild for installation
>
> * which should be removed by dh_python3

How does ‘dh_python3’ know to do that? Are there settings I should look
at to tell ‘dh_python3’ what to do when it misses some files?

> > * which leads to Lintian correctly complaining
> >   “package-installs-python-pycache-dir”.
> > 
> > How do I convince Pybuild to run the tests as part of the package build
> > (as normal), but clean up the compiled files *after* the tests and
> > *before* gathering the installed files?
>
> does your package name start with python3- or do you have
> ${python3:Depends} in Depends?

The problem is no longer occurring, because I have tweaked some of the
directories and package names around.

One complication may have been that the directory name to which the
library was installed (with the “--install-lib” option) does not contain
the Debian package name.

When I change that, it works; I don't know why.

The documentation for ‘PYBUILD_NAME’ and ‘PYBUILD_DESTDIR’ don't help me
to understand what is happening when things go wrong. The latter,
especially, makes no sense to me and the man page doesn't help me know
what effect setting that variable will have.

-- 
 \ “Geeks like to think that they can ignore politics. You can |
  `\leave politics alone, but politics won't leave you alone.” |
_o__) —Richard M. Stallman, 2002-07-26 |
Ben Finney



Re: Compiled bytecode installed, because test suite was run

2016-04-12 Thread Piotr Ożarowski
[Ben Finney, 2016-04-12]
> Piotr Ożarowski  writes:
> 
> > does your package name start with python3- or do you have
> > ${python3:Depends} in Depends?
> 
> Both, yes. The “Depends” field contains “${python3:Depends}”.
> 
> Is there anything I can do to diagnose this more precisely? Some part of
> the ‘sbuild’ output?

if it's open source, then pointing us to the sources would definitely
help
-- 
Piotr Ożarowski Debian GNU/Linux Developer
www.ozarowski.pl  www.griffith.cc   www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645



Re: Compiled bytecode installed, because test suite was run

2016-04-12 Thread Ben Finney
Piotr Ożarowski  writes:

> does your package name start with python3- or do you have
> ${python3:Depends} in Depends?

Both, yes. The “Depends” field contains “${python3:Depends}”.

Is there anything I can do to diagnose this more precisely? Some part of
the ‘sbuild’ output?

-- 
 \  “[I]t is impossible for anyone to begin to learn that which he |
  `\thinks he already knows.” —Epictetus, _Discourses_ |
_o__)  |
Ben Finney



Re: Compiled bytecode installed, because test suite was run

2016-04-12 Thread Piotr Ożarowski
[Ben Finney, 2016-04-12]
> I am packaging a code base, which has a test suite. The compiled byte
> code from the test suite run, is being collected when Pybuild discovers
> the resulting Python packages.
> 
> As far as I can determine, these steps are occurring:
> 
> * Pybuild runs the upstream test suite
> * which imports the Python packages
> * which creates the compiled bytecode files
> * which remain in the package directories
> * which are scooped up by Pybuild for installation

* which should be removed by dh_python3

> * which leads to Lintian correctly complaining
>   “package-installs-python-pycache-dir”.
> 
> How do I convince Pybuild to run the tests as part of the package build
> (as normal), but clean up the compiled files *after* the tests and
> *before* gathering the installed files?

does your package name start with python3- or do you have
${python3:Depends} in Depends?

(yeah, something dh_python* should be more verbose about, will be fixed
soon)
-- 
Piotr Ożarowski Debian GNU/Linux Developer
www.ozarowski.pl  www.griffith.cc   www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645



Compiled bytecode installed, because test suite was run

2016-04-11 Thread Ben Finney
Howdy all,

I am packaging a code base, which has a test suite. The compiled byte
code from the test suite run, is being collected when Pybuild discovers
the resulting Python packages.

As far as I can determine, these steps are occurring:

* Pybuild runs the upstream test suite
* which imports the Python packages
* which creates the compiled bytecode files
* which remain in the package directories
* which are scooped up by Pybuild for installation
* which leads to Lintian correctly complaining
  “package-installs-python-pycache-dir”.

How do I convince Pybuild to run the tests as part of the package build
(as normal), but clean up the compiled files *after* the tests and
*before* gathering the installed files?

-- 
 \   “The great thing about science is we can test our ideas.… But |
  `\   until we do, until we have data, it is just one more proposal.” |
_o__) —Darren Saunders, 2015-12-02 |
Ben Finney