Installing python modules in a non-standard way

2015-10-25 Thread Onur Aslan
Hi. I asked this in IRC channel but didn't get a clear answer.

I have a library package written in C and it's also providing
python wrappers.

You can see the package in here:
https://anonscm.debian.org/cgit/collab-maint/gumbo-parser.git/

I can't use --buildsystem=pybuild to install python wrappers,
this package is not providing a configure script or Makefile. pybuild
is not invoking autoreconf to generate a configure script. I have
to build C library manually if I use pybuild buildsystem
for this package.

I've been using $(py3versions -rv debian/control) in
override_dh_auto_install to install python3 package. But since
python3-all became depended on python3.4 and python3.5,
py3versions started to return two python versions and my override
is no longer working.

I decided to use $(py3versions -d) but I am not sure this is good
idea. It's skipping X-Python3-Version defined in d/control.

Piotr suggested me to make a loop to install python3.4 and python3.5
versions of package. But is it really necessary?

I am wondering what is proper way to install python modules in this
package?



Re: Installing python modules in a non-standard way

2015-10-25 Thread Andrey Rahmatullin
On Sun, Oct 25, 2015 at 10:37:13AM +0200, Onur Aslan wrote:
> I can't use --buildsystem=pybuild to install python wrappers,
> this package is not providing a configure script or Makefile. pybuild
> is not invoking autoreconf to generate a configure script. I have
> to build C library manually if I use pybuild buildsystem
> for this package.
Nothing is invoking autoreconf unless you use --with autoreconf. Are you
saying that with pybuild this option has no effect?

> Piotr suggested me to make a loop to install python3.4 and python3.5
> versions of package. But is it really necessary?
If you want to install modules for all supported versions then yes.

-- 
WBR, wRAR


signature.asc
Description: PGP signature


Re: Installing python modules in a non-standard way

2015-10-25 Thread Ben Finney
Onur Aslan  writes:

> I can't use --buildsystem=pybuild to install python wrappers, this
> package is not providing a configure script or Makefile. pybuild is
> not invoking autoreconf to generate a configure script.

Can you add a target to ‘debian/rules’ that will invoke ‘autoreconf’, as
a pre-requisite for the ‘override_dh_build’ target?

> Piotr suggested me to make a loop to install python3.4 and python3.5
> versions of package.

I don't use a loop, but rather a pattern-matching target and a Makefile
substitution variable to generate targets for all the supported Python
versions. It amounts to the same result, but is not “a loop”.

> But is it really necessary?

In the absence of Pybuild, yes I've found that explicitly making
separate targets for each supported Python version is necessary. This is
one of the main benefits of migrating to Pybuild.

> I am wondering what is proper way to install python modules in this
> package?

My suggestion: work hard to get the package to provide what Pybuild
needs, and let Pybuild do its job from there.

-- 
 \  “Our products just aren't engineered for security.” —Brian |
  `\ Valentine, senior vice-president of Microsoft Windows |
_o__)development, 2002 |
Ben Finney



Re: git-dpm: ERROR: 'upstream' does not contain previously recorded revision

2015-10-25 Thread Brian May
Ludovic Rousseau  writes:

>> git push --all
>>
>
> Ah.
> Maybe this should be added in the documentation
> https://wiki.debian.org/Python/GitPackaging

Yes. I notice that the git-dpm man page is also fails to mention
this. At least in the version on Jessie. I imagine the version in sid
will be the same.

> I now just did:

Oops. Sounds like while what you did worked, all you needed to fix the
problem - in this particular case - was:

git checkout upstream
git merge --ff-only 8234fc2de2b89d65661233b357f922494590b5aa
git checkout master

You may have also needed the pristine-tar command to add the version.

As the commit still existed - it was part of the master branch, it just
wasn't part of the upstream branch.

Sorry, I probably should have mentioned this. I tested it, but didn't
think of saying it worked :-(.

In any case it sounds like you have a working repository, and that is
the important thing. It all looks good to me. Don't run the above
commands now.

> With my changes above I now have:
> $ git-dpm tag
> Creating new tag 'upstream/1.3.1'...
> Creating new tag 'patched/1.3.1-1'...
> Creating new tag 'debian/1.3.1-1'...

That is also pretty good. I think git-dpm has its own format for these
tags and gets upset if it sees an existing tag that is different. Even
though it points to the same version.

Wish it was possible to get git-dpm to create the upstream tag, but not
the other two. e.g. when you have imported new source, but not ready to
make a Debian release.

> I looks like the repo is back in order.
> I will see if I have problems the next time I update the package.

Yes, I imagine it should be fine now.


I think the important things with git-dpm are:

1. Don't push until you are absolutely happy. That way even if you stuff
it up entirely you can just delete, reclone, and retry. I have done this
on one occasion when I imported a new source version and didn't like
some of the decisions I had made.

You could also find a temporary location somewhere (dare I say github?
or gitlab?) push there and ask for help if you are worried. Then delete
it when done.

2. Remember to push the "upstream" and "prestine-tar" branches, as well
as the "master" branch. In fact, remember the tags too. I need to use:

git push --all
git push --tags

As sometimes "git push --all" won't push the tags - I think it won't
push the tag if the changeset is already on the server.
-- 
Brian May 



Re: Does “${python3:Depends}” reliably generate correct dependencies?

2015-10-25 Thread Scott Kitterman


On October 25, 2015 1:17:17 AM EDT, Ben Finney  
wrote:
>Howdy all,
>
>If we set “Depends: ${python3:Depends}” in the binary package, and use
>‘dh_python3’, is that all that is needed to ensure the correct
>dependencies on Python 3 versions?
>
>Recently I received this bug report (Bug#802889):
>
>please don't depend on all python3 versions, forcing the
>installation of all python versions shouldn't be enforced for a
>module.
>
>$ apt-cache show python3-coverage | grep Depends
>   Depends: python3-pkg-resources, python3 (<< 3.6), python3 (>= 3.4~),
>python3.4, python3.5, libc6 (>= 2.4)
>
>
>
>So far as I can tell, these dependencies are generated because
>“Depends: […] ${python3:Depends}” is in that “Package” definition.
>
>On a Debian Sid system today, building a package for Python 3, along
>with its extension modules, means building it for all supported Python
>3
>versions. Are the above dependencies wrong for that?
>
>If the dependencies are incorrect, what should be used in the Depends
>field instead of “${python3:Depends}”?
>
>If the dependencies are correct, exactly what should I say in
>explanation to the reporter of Bug#802889?

Usually you only get the dependency on the specific python3 interpreters if 
there's a script with a shebang using that version.  I'd check for that.

It could either be in the original source or due to dh_python3 doing something 
interesting in shebang rewriting (IIRC there's a known issue that P1otr is 
working on).

Scott K



Re: Does “${python3:Depends}” reliably generate correct dependencies?

2015-10-25 Thread Dmitry Shachnev
On Sun, 25 Oct 2015 05:05:08 -0400, Scott Kitterman wrote:
> Usually you only get the dependency on the specific python3 interpreters if
> there's a script with a shebang using that version.  I'd check for that.
>
> It could either be in the original source or due to dh_python3 doing something
> interesting in shebang rewriting (IIRC there's a known issue that P1otr is
> working on).

I think this is intended because:

  $ head -n1 

signature.asc
Description: OpenPGP digital signature


Re: Does “${python3:Depends}” reliably generate correct dependencies?

2015-10-25 Thread Piotr Ożarowski
[Scott Kitterman, 2015-10-25]
> Usually you only get the dependency on the specific python3
> interpreters if there's a script with a shebang using that version.
> I'd check for that.

right

> It could either be in the original source or due to dh_python3 doing
> something interesting in shebang rewriting (IIRC there's a known issue
> that P1otr is working on).

dh_python3 does the right thing

What I wanted to improve is pybuild which calls default interpreter as
python3 but if the default version is <= than other supported ones, it
it not called last (so until 3.5 is the default, it's possible that 3.5
is overwriting /usr/bin/python3 shebangs)
-- 
Piotr Ożarowski Debian GNU/Linux Developer
www.ozarowski.pl  www.griffith.cc   www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645



Re: Installing python modules in a non-standard way

2015-10-25 Thread Andrey Rahmatullin
On Sun, Oct 25, 2015 at 12:39:44PM +0200, Onur Aslan wrote:
> > Nothing is invoking autoreconf unless you use --with autoreconf. Are you
> > saying that with pybuild this option has no effect?
> 
> I am already using --with autoreconf but when I use:
> 
> dh $@ --with autoreconf,python2,python3 --buildsystem=pybuild
> 
> It's running autoreconf to generate configure script and thats all.
> It's not running ./configure or make to build library.
Indeed.
In this case I'd use the usual autotools buildsystem and do everything
else manually, without using pybuild. But maybe there are better options.

-- 
WBR, wRAR


signature.asc
Description: PGP signature


Re: Installing python modules in a non-standard way

2015-10-25 Thread Onur Aslan
On 2015-10-05,  Andrey Rahmatullin wrote:
> Nothing is invoking autoreconf unless you use --with autoreconf. Are you
> saying that with pybuild this option has no effect?

I am already using --with autoreconf but when I use:

dh $@ --with autoreconf,python2,python3 --buildsystem=pybuild

It's running autoreconf to generate configure script and thats all.
It's not running ./configure or make to build library.

'dh_auto_configure -O--buildsystem=pybuild' is skipping ./configure
and only configuring python modules. I believe if I use
--buildsystem=pybuild I have to configure and build C library
manually. That is the reason I want to avoid pybuild in this package.

You can see full build log in attached file.
dpkg-buildpackage: source package gumbo-parser
dpkg-buildpackage: source version 0.10.1+dfsg-3
dpkg-buildpackage: source distribution UNRELEASED
dpkg-buildpackage: source changed by Onur Aslan 
 dpkg-source --before-build gumbo-parser
dpkg-buildpackage: host architecture amd64
 fakeroot debian/rules clean
dh clean --with autoreconf,python2,python3 --buildsystem=pybuild
   dh_testdir -O--buildsystem=pybuild
   debian/rules override_dh_auto_clean
make[1]: Entering directory '/home/onur/code/debian/libgumbo/gumbo-parser'
dh_auto_clean
pybuild --clean -i python{version} -p 2.7 --dir .
I: pybuild base:170: python2.7 setup.py clean 
running clean
removing 
'/home/onur/code/debian/libgumbo/gumbo-parser/.pybuild/pythonX.Y_2.7/build' 
(and everything under it)
'build/bdist.linux-x86_64' does not exist -- can't clean it
'build/scripts-2.7' does not exist -- can't clean it
pybuild --clean -i python{version} -p "3.5 3.4" --dir .
I: pybuild base:170: python3.5 setup.py clean 
running clean
removing 
'/home/onur/code/debian/libgumbo/gumbo-parser/.pybuild/pythonX.Y_3.5/build' 
(and everything under it)
'build/bdist.linux-x86_64' does not exist -- can't clean it
'build/scripts-3.5' does not exist -- can't clean it
I: pybuild base:170: python3.4 setup.py clean 
running clean
removing 
'/home/onur/code/debian/libgumbo/gumbo-parser/.pybuild/pythonX.Y_3.4/build' 
(and everything under it)
'build/bdist.linux-x86_64' does not exist -- can't clean it
'build/scripts-3.4' does not exist -- can't clean it
rm -rf .pybuild/
find . -name \*.pyc -exec rm {} \;
rm -f -rfv build/
rm -f -rfv docs/
rm -f -rfv python/gumbo.egg-info/
removed ‘python/gumbo.egg-info/not-zip-safe’
removed ‘python/gumbo.egg-info/PKG-INFO’
removed ‘python/gumbo.egg-info/SOURCES.txt’
removed ‘python/gumbo.egg-info/dependency_links.txt’
removed ‘python/gumbo.egg-info/top_level.txt’
removed directory: ‘python/gumbo.egg-info/’
rm -f -rfv Makefile.in aclocal.m4 compile config.guess config.sub \
   configure depcomp install-sh ltmain.sh m4/ missing test-driver
removed ‘Makefile.in’
removed ‘aclocal.m4’
removed ‘compile’
removed ‘config.guess’
removed ‘config.sub’
removed ‘configure’
removed ‘depcomp’
removed ‘install-sh’
removed ‘ltmain.sh’
removed ‘m4/ltoptions.m4’
removed ‘m4/lt~obsolete.m4’
removed ‘m4/ltversion.m4’
removed ‘m4/libtool.m4’
removed ‘m4/ltsugar.m4’
removed directory: ‘m4/’
removed ‘missing’
removed ‘test-driver’
make[1]: Leaving directory '/home/onur/code/debian/libgumbo/gumbo-parser'
   dh_autoreconf_clean -O--buildsystem=pybuild
rm -f -- ./depcomp ./config.sub ./config.guess ./Makefile.in 
./configure ./install-sh ./compile ./m4/ltoptions.m4 ./m4/lt\~obsolete.m4 
./m4/ltversion.m4 ./m4/libtool.m4 ./m4/ltsugar.m4 ./aclocal.m4 ./missing 
./ltmain.sh ./test-driver ./autom4te.cache/output.1 ./autom4te.cache/output.2 
./autom4te.cache/traces.1 ./autom4te.cache/traces.2 ./autom4te.cache/traces.0 
./autom4te.cache/output.0 ./autom4te.cache/requests
rm -f debian/autoreconf.before debian/autoreconf.after
   dh_clean -O--buildsystem=pybuild
rm -f debian/libgumbo-dev.substvars
rm -f debian/libgumbo-dev.*.debhelper
rm -rf debian/libgumbo-dev/
rm -f debian/libgumbo1.substvars
rm -f debian/libgumbo1.*.debhelper
rm -rf debian/libgumbo1/
rm -f debian/python-gumbo.substvars
rm -f debian/python-gumbo.*.debhelper
rm -rf debian/python-gumbo/
rm -f debian/python3-gumbo.substvars
rm -f debian/python3-gumbo.*.debhelper
rm -rf debian/python3-gumbo/
rm -rf debian/.debhelper/
rm -f debian/*.debhelper.log
rm -f debian/files
find .  \( \( \
\( -path .\*/.git -o -path .\*/.svn -o -path .\*/.bzr -o -path 
.\*/.hg -o -path .\*/CVS \) -prune -o -type f -a \
\( -name '#*#' -o -name '.*~' -o -name '*~' -o -name DEADJOE \
 -o -name '*.orig' -o -name '*.rej' -o -name '*.bak' \
 -o -name '.*.orig' -o -name .*.rej -o -name '.SUMS' \
 -o -name TAGS -o \( -path '*/.deps/*' -a -name '*.P' \) \
\) -exec rm -f {} + \) -o \
\( -type d -a -name autom4te.cache -prune -exec rm -rf {} + 

Re: Does “${python3:Depends}” reliably generate correct dependencies?

2015-10-25 Thread Piotr Ożarowski
[Ben Finney, 2015-10-25]
> Howdy all,
> 
> If we set “Depends: ${python3:Depends}” in the binary package, and use
> ‘dh_python3’, is that all that is needed to ensure the correct
> dependencies on Python 3 versions?

you also have to add correct Build-Dependencies (if not for something
else, then because you need them for tests)

> Recently I received this bug report (Bug#802889):
> 
> please don't depend on all python3 versions, forcing the
> installation of all python versions shouldn't be enforced for a
> module.

my bet is you have versioned interpreter in shebang (which dh_python3
doesn't rewrite unless you tell it to via --shebang=/usr/bin/python3)
-- 
Piotr Ożarowski Debian GNU/Linux Developer
www.ozarowski.pl  www.griffith.cc   www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645



Re: Request to Join DPMT

2015-10-25 Thread Afif Elghraoui
Because I got no response to my request from ten days ago (below), I am
maintaining python-avro within Debian Med. I was sponsored there and it
has already been accepted by the ftp-masters.

Afif

-- 
Afif Elghraoui | عفيف الغراوي
http://afif.ghraoui.name

On الخميس 15 تشرين الأول 2015 00:20, Afif Elghraoui wrote:
> Hello,
> A project I wanted to package as part of Debian Med has a dependency on
> the python modules for Apache Avro. I'd like to join the DPMT and
> package python-avro as part of this team. While I could just as well do
> it within Debian Med, I think DPMT is a better home for this package's
> development.
> 
> I am a DM and my alioth account username is afif-guest. I've read the
> modules team policy [1] and agree with it. I also saw the message
> recently posted to -devel-announce [2] regarding the git transition you
> all made.
> 
> I already maintain some biology-related python packages as part of
> Debian Med-- these are python-cobra, python-pbcore, and
> python-pbh5tools. I also have two more python packages currently in
> NEW-- pbalign and pbgenomicconsensus.
> 
> Many thanks and regards
> Afif
> 
> 
> 1. http://python-modules.alioth.debian.org/python-modules-policy.html
> 2. https://lists.debian.org/debian-devel-announce/2015/10/msg0.html
> 




signature.asc
Description: OpenPGP digital signature


Re: Installing python modules in a non-standard way

2015-10-25 Thread Dmitry Shachnev
On Sun, Oct 25, 2015 at 04:45:17PM +0500, Andrey Rahmatullin wrote:
> Indeed.
> In this case I'd use the usual autotools buildsystem and do everything
> else manually, without using pybuild. But maybe there are better options.

Or maybe try to use both build systems, something like this:

  override_dh_auto_build:
  dh_auto_build
  dh_auto_build -O--buildsystem=pybuild

(and the same for install/clean/etc).

--
Dmitry Shachnev