Re: Python BoF at DebConf15 - summary
Thanks for the summary Piotr! I was really sorry I couldn't make Debconf this
year.
On Aug 24, 2015, at 11:08 PM, Piotr Ożarowski wrote:
Python 3.5 as supported
===
python3-defaults in experimental already has 3.5 as supported. Ubuntu did
this change as well and reported back that it's not as problematic as many
previous transitions we had in the past. Some concerns were raised that many
upstreams didn't test their code with 3.5 as its final version is still not
released (and services like Travis do not enable it). In some places the
packaging assumes all python3 to be available, but the build dependency on
python3-all{,-dev} is missing.
I've made very good progress here, with one notable exception (see below),
although of course there is a long tail of packages which need fixing for
Python 3.5 support. Ubuntu 15.10 will have 3.5 supported but not default.
Maybe by the time 16.04 opens, we'll have already made the switch in Stretch.
I'd certainly like the next Debian release, and the next Ubuntu LTS to be 3.5
by default, with 3.4 (possibly? probably?) dropped.
Wherever possible, I've tried to upload fixes to unstable, even though we
won't notice them until 3.5 is supported there too. Anyone interested in
helping out should take a look at the ubuntu-devel@ mailing list threads, and
take a look at buildlogs from the Ubuntu PPA I am keeping up to date:
https://launchpad.net/~pythoneers/+archive/ubuntu/py35asdefault/+packages
Django is the notable exception. I had hoped to be able to backport enough
Python 3.5 support to Django 1.7.9, but that just wasn't feasible. We'll need
to get 1.8.4 out of experimental, but there are a bunch of packages that also
need updating or fixing, and I don't exactly know how compatible they all are
with both Python 3.5 and Django 1.8.
http://tinyurl.com/qz6rorh
If you can help with any of the Django packages, please get in touch.
Cheers,
-Barry
pgppMzPq3X85B.pgp
Description: OpenPGP digital signature
Re: Removing some python3-* packages
Just a quick follow-up I've been meaning to send. On Jul 02, 2015, at 03:55 PM, Barry Warsaw wrote: As part of the 3.5 test rebuild I noticed an incompatibility with python3-enum, which I reported upstream. The response was: there's actually no reason to have a Python 3 version of enum in any version = Python 3.4. Since that's all we have now, maybe it makes more sense to just remove the python3-enum package from Debian. Which is what I'm going to do. I've interacted with upstream and there's just no reason for us to be shipping a Python 3 version of enum34, especially in Stretch given our current and future supported Python 3 versions. I'll be uploading a new version of enum34 that deletes python3-enum34 asap. py3porters, please keep this in mind and don't add it back. It'll ftbfs anyway and I'll close any related bug as won't fix. In general, I think we'll just have to take similar situations on a case-by-case basis. For packages which provide additional functionality not in stdlib Python 3, it may still make sense to provide a python3- version of the package. Cheers, -Barry pgp_2BLSpo3X4.pgp Description: OpenPGP digital signature
Re: Removing some python3-* packages
On Aug 19, 2015, at 06:41 PM, Matthias Klose wrote: As a Debian developer you are duplicating code, and no, I don't think that providing this code under a different name is different enough to rectify distribution of this code in Debian. In some cases however, the standalone library moves ahead of the stdlib, because they have hugely different release cycles. They may eventually catch up every 18 months (when a new Python is released), but in the meantime the divergence might be needed by dependent packages. So I'm not sure a blanket policy is useful. By all means, if there isn't any significant difference between a standalone package and what's available in the current supported Python 3 version, let's not ship unnecessary binary packages. Cheers, -Barry
Python BoF at DebConf15 - summary
Last Tuesday (2015-08-18) we had a Python BoF at DebConf15 discussing
various Debian Python related topics. You can download a recording of
it from here¹ or here² thanks to our amazing Video Team.
Short summary:
removing python-support from the archive
We discussed what needs to be done in order to remove deprecated
dh_pysupport from Stretch. There were almost 300 packages still using
it³ so we organized a sprint to convert those and in next few days fixed
more than 100 packages (some prepared by our new contributors: Ana,
Larissa and Valerie - how about that, Debian Women? :)
Many are still not fixed, so if you want to help: we are coordinating
our efforts on whiteboard⁴, please join, instructions are here⁵.
Once all packages are converted, dh_pysupport will start returning 1
exit code so that no new packages can be built with it (packages already
built with it outside Debian main will continue to work in Stretch as
all needed triggers / byte compilation scripts will be available)
removing dh-python from python3's Depends
=
python3 package depends on dh-python to make sure all packages that were
using dh_python3 provided by this package are still buildable. dh-python
package is not really needed for everyone who installs python3 so Piotr
would like to remove it at some point (hopefully before releasing Stretch).
We discussed how to raise awareness of this issue.
Lintian already checks that, we need fixes.
It's not a big problem though compared to...
removing old dh_python2 from python package
===
dh-python's dh_python2 is written in Python 3 so python3's solution
(dependency on dh-python) cannot be used here (as python install would
pull in python3 and we don't want to do that, even if replacing python
with python3 is our goal). We discussed what can we do to get rid of
this older version of dh_python2. The fix is trivial (add dh-python to
Build-Depends) but there are ~500 packages that prevent us from removing
it.
There's a mass bug filling proposal⁶ on this mailing list (debian-devel one
will follow soon). The goal is to get rid of old dh_python2 before releasing
Stretch. Piotr wants to remove dh_python2 from python in January 2016 (or
sooner, depending on the number of package that will FTBFS if it's done).
This is a great way for newcomers to start contributing, BTW.
It's easier than pysupport one and Piotr will happily check/sponsor
everyone who wants to prepare a NMU.
Python 3.5 as supported
===
python3-defaults in experimental already has 3.5 as supported.
Ubuntu did this change as well and reported back that it's not as problematic
as many previous transitions we had in the past. Some concerns were
raised that many upstreams didn't test their code with 3.5 as its final
version is still not released (and services like Travis do not enable
it). In some places the packaging assumes all python3 to be available,
but the build dependency on python3-all{,-dev} is missing.
We will upload 3.5 as supported to unstable once GCC transition is
(almost?) done and upstream releases final version - expect it around end
of September. 3.5 will be marked as default and 3.4 removed from
supported once most of our packages are working fine with new version.
more python3-foo packages
=
Python3Port team⁷ is doing a great job, but more help is needed. Please
join if you want to help porting Debian tools or packages to Python 3.
Yet another tool that Piotr wants to take over the world with was demonstrated.
It was written during Debconf but Piotr still wants to polish it a bit
before pushing sources somewhere (hopefully he will find some time to
do it this weekend).
Piotr briefly talked with ftp-master about it and most probably
a PPAEXT repo will be provided with most of pypi.python.org available as Debian
packages. PPAEXT will not be available this year, so pypi.debian.net
will be used in the meantime (Marcin already promised to help with setting
up infrastructure on Amazon to build them).
Compared to stdeb this new tool generates Build-Depends, copyright file,
pypy- and -doc packages (with rebuilt Sphinx docs) and has some hidden
options (like --dpmt or --papt). It also generates XB-Python-Egg-Name
to make package names translations easier (still not sure about
XB-Python-Egg-Version with raw upstream version). More useful metadata
from PyPI would be useful, though.
other topics
* stripping -dbg packages - dh_strip does that by default, how can we
change that? The best option would be to patch dh_strip to not do that
if package name ends with -dbg, but there are no volunteers to do that
yet. Anyone interested? There is a bug for lintian open (#795261) to
at least warn about stripped -dbg packages).
* backporting Python 3's stdlib modules to Python 2 - avoid if possible.
* improving Pybuild's documentation -
Re: Removing some python3-* packages
On 25 August 2015 at 09:57, Barry Warsaw ba...@debian.org wrote: ... By all means, if there isn't any significant difference between a standalone package and what's available in the current supported Python 3 version, let's not ship unnecessary binary packages. Even at the cost of having to patch upstream projects? -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud
Re: Removing some python3-* packages
On Aug 25, 2015, at 10:03 AM, Robert Collins wrote: On 25 August 2015 at 09:57, Barry Warsaw ba...@debian.org wrote: ... By all means, if there isn't any significant difference between a standalone package and what's available in the current supported Python 3 version, let's not ship unnecessary binary packages. Even at the cost of having to patch upstream projects? Sorry, I'm not sure what the question means. Cheers, -Barry pgpcXi3cKf4sQ.pgp Description: OpenPGP digital signature
Re: Removing some python3-* packages
On Aug 25, 2015, at 10:45 AM, Robert Collins wrote: Lets take Ironic. While it supports Python 2.7+ and 3.4+ it will depend on 'mock' for unit testing. That's not unreasonable, and different upstreams will have different policies, but if it was *my* upstream package, I'd probably do a conditional import so that unittest.mock were used if available, falling back to mock if not. If it's easy to patch upstream to do this, then it seems an acceptable approach to take. I wouldn't say it's *required* (which is why I'm not sure a blanket policy is appropriate for us). - patch Ironic to use unittest.mock on Python 3.5 That would be my first preference, and I'd also submit the patch to upstream. - patch the stdlib to make 'mock' be an alias to unittest.mock Nope. - include 'python3-mock' as a binary package That's not unreasonable for Debian either. I'll note another thing that perhaps makes the enum34 case different. It exports the package under the same name in both the standalone version and in the stdlib, so there *really* is no difference between the two. That's not the case with unittest.mock/mock. - not run the Ironic unit tests. Nope. But anyway, that's MHO. Cheers, -Barry pgpkdZ2ym1xdB.pgp Description: OpenPGP digital signature
Re: Removing some python3-* packages
On 25 August 2015 at 11:49, Thomas Kluyver tho...@kluyver.me.uk wrote: On Mon, Aug 24, 2015, at 04:30 PM, Robert Collins wrote: c) write convoluted tricky code to workaround the bugs and differing behaviour on 3.4 vs 3.5. I use unittest.mock from Python 3.4 on several packages, and it has not required convoluted code. I would be very surprised if that code breaks when run under Python 3.5. If it works with mock 1.3 (in stretch) and with unittest.mock from 3.4, then you're fine. Very much depends on the size and variety of your codebase, whether you will have an issue or not. But it doesn't take 'works for me' to invalidate the point: it requires 'works for *everyone*' - and while you haven't experienced the faults yourself that in no way invalidates the experience of folk that have experienced it. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud
Re: Removing some python3-* packages
On Aug 25, 2015, at 11:30 AM, Robert Collins wrote: Except that that will break: mock in 3.4 vs 3.5 are different. Then they aren't the same wink. So it sounds like it doesn't make sense to remove python3-mock from Debian. Cheers, -Barry pgpV2khUPaCtA.pgp Description: OpenPGP digital signature
Re: Removing some python3-* packages
On Mon, Aug 24, 2015, at 04:30 PM, Robert Collins wrote: c) write convoluted tricky code to workaround the bugs and differing behaviour on 3.4 vs 3.5. I use unittest.mock from Python 3.4 on several packages, and it has not required convoluted code. I would be very surprised if that code breaks when run under Python 3.5.
Re: Removing some python3-* packages
On 25 August 2015 at 10:37, Barry Warsaw ba...@debian.org wrote: On Aug 25, 2015, at 10:03 AM, Robert Collins wrote: On 25 August 2015 at 09:57, Barry Warsaw ba...@debian.org wrote: ... By all means, if there isn't any significant difference between a standalone package and what's available in the current supported Python 3 version, let's not ship unnecessary binary packages. Even at the cost of having to patch upstream projects? Sorry, I'm not sure what the question means. Lets take Ironic. While it supports Python 2.7+ and 3.4+ it will depend on 'mock' for unit testing. If Debian has only Python 2.7 and 3.5 and wants to build Ironic for only Python 3.5 - which is a reasonable thing, then 'mock' is needed (because Ironic depends on 'mock'). We could: - patch Ironic to use unittest.mock on Python 3.5 - patch the stdlib to make 'mock' be an alias to unittest.mock - include 'python3-mock' as a binary package - not run the Ironic unit tests. Perhaps to you 'thing we package X uses library Y' implies 'binary package X is uncessary' - but the thing is that 'binary package X' may not be a significant difference vs the Python version in Debian at the time - even if there are significant differences vs older still upstream-supported Python versions - which is what the developer ecosystem will be referencing. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud
