Re: pybuild-autopkgtest (was: Notes from the DC22 Python Team BoF)

2022-07-26 Thread Scott Kitterman



On July 26, 2022 2:50:19 PM UTC, Antonio Terceiro  wrote:
>On Mon, Jul 25, 2022 at 09:04:27AM +0100, Julian Gilbey wrote:
>> On Sat, Jul 23, 2022 at 07:52:19PM +0200, Louis-Philippe Véronneau wrote:
>> > == pybuild improvements ==
>> > 
>> > getting the autopkgtest MR in would be great
>> > 
>> > https://salsa.debian.org/python-team/tools/dh-python/-/merge_requests/27
>> > 
>> > We need people to test this MR some more, although it seems fairly mature.
>> > 
>> > It might be a good idea to have a line in d/control to let us migrate from
>> > the existing autopkgtests running unit tests to the new automated MR.
>> 
>> I've been looking at this a bit more.  I'm not sure what this last
>> paragraph means: the new "automated" autopkgtest will only be run if
>> the maintainer explicitly adds:
>> 
>> Testsuite: autopkgtest-pkg-pybuild
>> 
>> to debian/control (see the autodep8 MR at
>> https://salsa.debian.org/ci-team/autodep8/-/merge_requests/27/diffs -
>> it will never automatically detect a pybuild package).
>> And a maintainer would presumably only add that if they are also
>> removing their existing debian/tests/control (or want to run the
>> pybuild tests in addition).
>> 
>> An alternative would be for the autodep8 patch to try to determine
>> whether to run pybuild-autopkgtest.  One approach could be:
>> 
>> if the package would run autopkgtest-pkg-python:
>>   if debian/control does not contain an override_dh_auto_test stanza:
>> run pybuild-autopkgtest
>> 
>> Note, though, that if autodep8 is called, it will run all of the
>> detected tests.  (At least that is what I believe happens from reading
>> /usr/bin/autodep8; I haven't double-checked this.)  So, for example,
>> if a package specifies
>> 
>> Testsuite: autopkgtest-pkg-python
>> 
>> it will also run the autopkgtest-pkg-pybuild suite as it will be
>> detected as being a Python package, and vice versa.  That is a
>> possible reason *not* to use the above suggestion, as it would
>> potentially run pybuild-autopkgtest even if not desired.
>
>I think the notes did not capture the consensus correctly. The point was
>that it should be possible to automate updating the `Testsuite:` field
>to run tests with pybuild-autopkgtest, and that we should probably do
>that across team packages with the help of some scripting.

I suppose this is fine as long as whoever does it fixes all the resulting RC 
bugs.

Scott K



Re: Help for python-subby needed

2022-02-01 Thread Scott Kitterman
On Tuesday, February 1, 2022 6:01:22 AM EST Andreas Tille wrote:
> Am Sat, Jan 29, 2022 at 01:50:22AM +0500 schrieb Andrey Rahmatullin:
> > On Fri, Jan 28, 2022 at 09:48:00PM +0100, Andreas Tille wrote:
> > > Hi,
> > > 
> > > I need python-subby as a new dependency for some Debian Med package.
> > > 
> > > Unfortunately it does not build easily[1]:
> > >dh_auto_build -O--buildsystem=pybuild
> > > 
> > > E: pybuild pybuild:367: build: plugin flit failed with: Neither
> > > [project] nor [tool.flit.metadata] found in pyproject.toml E: pybuild
> > > pybuild:367: build: plugin flit failed with: Neither [project] nor
> > > [tool.flit.metadata] found in pyproject.toml dh_auto_build: error:
> > > pybuild --build -i python{version} -p "3.10 3.9" returned exit code 13> 
> > It indeed uses poetry, not flit, but you have export PYBUILD_SYSTEM=flit.
> 
> Where can I find an example how to use poetry?

Add build-depends on pybuild-plugin-pyproject.

Drop build-depends on flit.

Drop the "export PYBUILD_SYSTEM=flit" line in d/rules.

The poetry build-depends I'm a little uncertain on. Subby has:

> [build-system]
> requires = ["poetry>=0.12"]
> build-backend = "poetry.masonry.api"

What I recall seeing is:

> [build-system]
> requires = ["poetry>=0.12"]
> build-backend = "poetry.core.masonry.api"

I am not familiar enough to know if that means that you need python3-poetry 
(which you have) or if the upstream pyproject.toml is wrong and you need 
poetry-core.  Maybe someone else knows or you can experiment.

That should get you close.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: Reaching team consensus on usage of py3versions -r and X-Python3-Version in Lintian

2022-01-18 Thread Scott Kitterman
On Tuesday, January 18, 2022 3:17:31 AM EST Neil Williams wrote:
> On Mon, 17 Jan 2022 12:47:44 -0500
> 
> Louis-Philippe Véronneau  wrote:
> > Hey folks,
> > 
> > I'm following up on bug #1001677 [1] on the DPT's list to try to reach
> > consensus, as I think the Lintian tags that were created to fix this
> > bug are not recommending the proper thing.
> > 
> > As a TL;DR for those of you who don't want to read the whole BTS
> > thread, jdg saw that a bunch of packages were using `py3versions -r`
> > in autopkgtests, and this fails when there's no X-Python3-Version
> > variable in d/control.
> > 
> > The fix that Lintian now proposes for packages that use `py3versions
> > -r` in autopkgtests is to set X-Python3-Version.
> > 
> > I think the proper fix would be to ask people to move away from
> > `py3versions -r` if there is no X-Python3-Version, and use`py3versions
> > -s` instead.
> 
> Just as a general point, can I ask that - especially in a TL;DR - that
> long option names are used or the context of each option is included as
> the difference between -r and -s is not obvious from this email & not
> everyone on the list uses py3versions routinely.

TIL py3versions has long option names ...

  -d, --defaultprint the default python3 version
  -s, --supported  print the supported python3 versions
  -r, --requested  print the python3 versions requested by a build; the
   argument is either the name of a control file or the value
   of the X-Python3-Version attribute
  -i, --installed  print the installed supported python3 versions

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: Cleaning up the Salsa DPT landing page

2022-01-17 Thread Scott Kitterman
On Monday, January 17, 2022 12:59:53 PM EST Louis-Philippe Véronneau wrote:
> On 2022-01-17 12 h 31, Scott Kitterman wrote:
> > On Monday, January 17, 2022 12:20:59 PM EST Louis-Philippe Véronneau 
wrote:
> >> Hey folks,
> >> 
> >> The merger between the DPMT and the PAPT into a single entity has been
> >> pretty successful IMO and I think it's time to cleanup the Salsa DPT
> >> landing page.
> >> 
> >> Looking at https://salsa.debian.org/python-team, I would propose the
> >> following:
> >> 
> >> 1. Delete the empty DPMT sub-group at
> >> https://salsa.debian.org/python-team/modules
> >> 
> >> 2. Delete the empty PAPT sub-group at
> >> https://salsa.debian.org/python-team/applications
> > 
> > I don't have an opinion on #3 and #4.
> 
> I mostly care about #3 in #4 :P
> 
> > Might it be better to leave these with a description that explains where
> > they went?  There's lots of things that refer to DPMT/PAPT and I don't
> > think all the packages have been uploaded with the correct Vcs-* data
> > yet.  It doesn't hurt to leave them there and if they explain where to
> > look instead, I think the chances of someone being confused later are
> > reduced.
> 
> The following lintian tags flag packages using the old Vcs-* data:
> 
> https://lintian.debian.org/tags/old-papt-vcs (11 packages)
> https://lintian.debian.org/tags/old-dpmt-vcs (431 packages)
> 
> Those packages have been fixed in git though, as Ondřej ran a script to
> fix all of them a while ago already.
> 
> Someone correct me if I'm wrong, but I don't think keeping empty dirs
> does anything to the Salsa redirects though.

I don't know, but I wasn't primarily thinking about people working internal to 
Debian (who might look at lintian output).  I was more thinking about users 
(both ours and downstream).  If they apt source  their debian/control 
will have the obsolete Vcs-Browser information.  I think there should at least 
be a tombstone there for them to understand where the team went.

I think this is much less relevant for #3 and #4 since they are more 
internally focused so some expectation that people will keep up with changes 
is more reasonable.

Scott K

Scott K




Re: Cleaning up the Salsa DPT landing page

2022-01-17 Thread Scott Kitterman
On Monday, January 17, 2022 12:59:53 PM EST Louis-Philippe Véronneau wrote:
> On 2022-01-17 12 h 31, Scott Kitterman wrote:
> > On Monday, January 17, 2022 12:20:59 PM EST Louis-Philippe Véronneau 
wrote:
> >> Hey folks,
> >> 
> >> The merger between the DPMT and the PAPT into a single entity has been
> >> pretty successful IMO and I think it's time to cleanup the Salsa DPT
> >> landing page.
> >> 
> >> Looking at https://salsa.debian.org/python-team, I would propose the
> >> following:
> >> 
> >> 1. Delete the empty DPMT sub-group at
> >> https://salsa.debian.org/python-team/modules
> >> 
> >> 2. Delete the empty PAPT sub-group at
> >> https://salsa.debian.org/python-team/applications
> > 
> > I don't have an opinion on #3 and #4.
> 
> I mostly care about #3 in #4 :P
> 
> > Might it be better to leave these with a description that explains where
> > they went?  There's lots of things that refer to DPMT/PAPT and I don't
> > think all the packages have been uploaded with the correct Vcs-* data
> > yet.  It doesn't hurt to leave them there and if they explain where to
> > look instead, I think the chances of someone being confused later are
> > reduced.
> 
> The following lintian tags flag packages using the old Vcs-* data:
> 
> https://lintian.debian.org/tags/old-papt-vcs (11 packages)
> https://lintian.debian.org/tags/old-dpmt-vcs (431 packages)
> 
> Those packages have been fixed in git though, as Ondřej ran a script to
> fix all of them a while ago already.
> 
> Someone correct me if I'm wrong, but I don't think keeping empty dirs
> does anything to the Salsa redirects though.

I don't know, but I wasn't primarily thinking about people working internal to 
Debian (who might look at lintian output).  I was more thinking about users 
(both ours and downstream).  If they apt source  their debian/control 
will have the obsolete Vcs-Browser information.  I think there should at least 
be a tombstone there for them to understand where the team went.

I think this is much less relevant for #3 and #4 since they are more 
internally focused so some expectation that people will keep up with changes 
is more reasonable.

Scott K

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: Cleaning up the Salsa DPT landing page

2022-01-17 Thread Scott Kitterman
On Monday, January 17, 2022 12:20:59 PM EST Louis-Philippe Véronneau wrote:
> Hey folks,
> 
> The merger between the DPMT and the PAPT into a single entity has been
> pretty successful IMO and I think it's time to cleanup the Salsa DPT
> landing page.
> 
> Looking at https://salsa.debian.org/python-team, I would propose the
> following:
> 
> 1. Delete the empty DPMT sub-group at
> https://salsa.debian.org/python-team/modules
> 
> 2. Delete the empty PAPT sub-group at
> https://salsa.debian.org/python-team/applications

I don't have an opinion on #3 and #4.

Might it be better to leave these with a description that explains where they 
went?  There's lots of things that refer to DPMT/PAPT and I don't think all 
the packages have been uploaded with the correct Vcs-* data yet.  It doesn't 
hurt to leave them there and if they explain where to look instead, I think 
the chances of someone being confused later are reduced.

Scott K

signature.asc
Description: This is a digitally signed message part.


New python-nacl

2022-01-11 Thread Scott Kitterman
I've just uploaded the new (version 1.5.0) python-nacl to experimental to give 
rdepends a chance to test.  Being a python extension there's not a formal 
transition for this, but I wanted to do some testing first, so I thought I'd 
put it in experimental so others can too.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: DPT repositories checks/"violations" report

2022-01-03 Thread Scott Kitterman
On Monday, January 3, 2022 3:13:23 PM EST Louis-Philippe Véronneau wrote:
> On 2022-01-03 01 h 30, Scott Kitterman wrote:
> 
> > Since this is all about team Git repositories, someone should just fix
> > them 
 (modulo the one about using pypi, which I think we mostly agree
> > isn't something someone unfamiliar with the package can 'fix').
> 
> But that doesn't prevent future errors from popping up and doesn't make
> maintainers aware of their errors (so they can learn from it).
> 
> I think the "perfect" solution would be to have an automated tool (aka
> janitor) fixing these automatically, but this would require more work.

The first step if we think these things should be the standard for team 
repositories is to document it.  Personally, I would look here:

https://wiki.debian.org/Python/GitPackaging#Creating_new_repositories

At least for me, a good explanation of what to do in the documentation would 
be much more valuable than a bug report.  Approximately all  of the team 
repositories I've created are "wrong" because I followed the documentation.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: DPT repositories checks/"violations" report

2022-01-02 Thread Scott Kitterman
On Sunday, January 2, 2022 7:50:02 PM EST Louis-Philippe Véronneau wrote:
> On 2021-12-12 01 h 23, Sandro Tosi wrote:
> 
> >> I think there's still one point we need to figure out: how to make
> >> these remarks known to the packages maintainers, instead of all of
> >> them just being in a text file.
> > 
> > 
> > This is still an open point, and i welcome ideas
> 
> 
> Is there a reason why we shouldn't just file bugs in the BTS? I get it's
> not the perfect tool for that, but it would certainly help reach the
> maintainers.
> 
> Using a common usertag would also make it easier to find and fix these
> issues en masse ;)

Since this is all about team Git repositories, someone should just fix them 
(modulo the one about using pypi, which I think we mostly agree isn't 
something someone unfamiliar with the package can 'fix').

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: New pybuild plugin for PEP-517 pyproject.toml projects

2021-12-29 Thread Scott Kitterman



On December 29, 2021 11:33:26 PM UTC, Michel Alexandre Salim 
 wrote:
>Hi Stefano,
>
>On Wed, Dec 29, 2021 at 10:17:57PM +, stefa...@debian.org wrote:
>> Hi debian-python (2021.12.17_23:52:14_+)
>> > - Build-depend on `dh-python-pep517` as well as any build tools
>> >   specified by upstream in pyproject.toml (in build-system.requires)
>> >   such as python3-setuptools, flit, or python3-poetry-core
>> 
>> At upstream's suggestion, this is renamed to pybuild-plugin-pyproject.
>> We're providing dh-python-pep517 for compatibility, but hope to drop it
>> ASAP.
>>
>Is there a way to see what packages currently build-depend on this?
>
>This does not seem to be supported:
>$ apt-rdepends -r --build-depends dh-python-pep517 
>E: Reverse build-dependencies are not supported

In the ubuntu-dev-tools package, reverse-depends -b $PACKAGE.

Scott K



Re: Packaging request for pygrib

2021-12-28 Thread Scott Kitterman



Not a fool, just learning.  Debian is complicated and the best way to learn is 
to dive in and try things.  As long as you learn along the way, it's all good.

Scott K

On December 28, 2021 11:46:49 PM UTC, Kyle Lawlor-Bagcal  wrote:
>Oh, I see. That is indeed the same package. I don't think there is any 
>work to do then. I'm just a fool who didn't know how to search well 
>enough ;-)
>
>$ sudo apt-cache show python3-grib
>Package: python3-grib
>Architecture: amd64
>Version: 2.0.4-2build2
>Priority: optional
>Section: universe/python
>Source: pygrib
>Origin: Ubuntu
>Maintainer: Ubuntu Developers 
>Original-Maintainer: Alastair McKinstry 
>Bugs: https://bugs.launchpad.net/ubuntu/+filebug
>Installed-Size: 1130
>Depends: libc6 (>= 2.4), libeccodes0 (>= 2.16.0), libeccodes-data, 
>python3-numpy (>= 1:1.16.0~rc1), python3-numpy-abi9, python3 (<< 3.9), 
>python3 (>= 3.8~), python3:any, python3-pyproj, libeccodes-dev, 
>libgrib2c-dev
>Recommends: python-grib-doc
>Breaks: python-grib (<< 2.0.0-1), python-grib-doc (<< 2.0.0-2)
>Replaces: python-grib (<< 2.0.0-1), python-grib-doc (<< 2.0.0-2)
>Filename: pool/universe/p/pygrib/python3-grib_2.0.4-2build2_amd64.deb
>Size: 305204
>MD5sum: de8d54eaaa0cd448b2aa8b35e6b15d3a
>SHA1: 343647462ced180a318e18d63b99808135eb5632
>SHA256: 74b23dbda57b1f2ab01af66935a62666066c04f8996a4b46e16572d4b2c88370
>Homepage: https://github.com/jswhit/pygrib
>Description-en: Python 3 module for reading and writing GRIB files
>  Python 3 module for reading and writing GRIB (editions 1 and 2) files.
>  GRIB is the World Meterological Organization standard for
>  distributing gridded data. The module is a Python 3 interface
>  to the GRIB API C library from the
>  European Centre for Medium-Range Weather Forecasts (ECMWF).
>  .
>  This package also contains the cnvgrib1to2, grib_list, grib_repack, and
>  cnvgrib2to1 scripts.
>Description-md5: 914b7563eb5a65791173632367a72e64
>



Re: Packaging request for pygrib

2021-12-28 Thread Scott Kitterman



On December 28, 2021 11:34:01 PM UTC, Kyle Lawlor-Bagcal  wrote:
>On 12/28/21 6:18 PM, Scott Kitterman wrote:
>> For The Debian Python team, if the team is listed as the maintainer for an
>> existing package, then you can go ahead.  If an individual is listed at
>> Maintainer and the team is listed in Uploaders, then you should check with 
>> the
>> identified maintainer before doing stuff.
>>
>> For new packages, you don't need to wait for permission to prepare the
>> package.
>
>Thanks Scott. I assume that this is the place to look for that info:
>
>https://tracker.debian.org/teams/python/+table/general/
>
>I'm not seeing the pygrib package there, so I can start working.
>
There's an existing package of that name maintained outside the team:

https://tracker.debian.org/pkg/pygrib

Assuming it's the same package, you should contact the maintainer about 
contributing.

Scott K



Re: Packaging request for pygrib

2021-12-28 Thread Scott Kitterman
On Tuesday, December 28, 2021 6:14:20 PM EST Kyle Lawlor wrote:
> On Tue, Dec 28, 2021 at 5:59 PM, Geert Stappers 
> wrote:
> 
> https://wiki.debian.org/Packaging
> 
> 
> Geert Stappers
> Could not resist to ignore the "tell me how to do it", settled for a short
> reply. Thank you for the link, I will pursue this and reach out when I have
> questions or progress to share.
> 
> --
> Silence is hard to parse

For The Debian Python team, if the team is listed as the maintainer for an 
existing package, then you can go ahead.  If an individual is listed at 
Maintainer and the team is listed in Uploaders, then you should check with the 
identified maintainer before doing stuff.

For new packages, you don't need to wait for permission to prepare the 
package.

Scott K


signature.asc
Description: This is a digitally signed message part.


Use of flit as a build-backend in pybuild

2021-11-29 Thread Scott Kitterman
The versions of pybuild in stable, testing, and unstable all support flit as a 
build-backend for packages built upstream using flit.  You can tell if your 
package is built using flit if it has a pyproject.toml file and it contains a 
paragraph like:

[build-system]
requires = ["flit_core >=3.2.0,<4"]
build-backend = "flit_core.buildapi"

Of the packages I've looked at, they either have no setup.py at all or a very 
rudimentary generated setup.py.  Although the generated setup.py will work, 
it's only going to provide limited Python metadata for the package.  Under the 
hood, using flit provides a nicer package.

To use the flit backend for pybuild all you should need to do is add flit to 
build-depends and drop whichever of python3-setuptools or python3-distutils 
you are using now.  Flit will be autodetected (and you will see it mentioned 
in the build log).

Flit 3.0 is the lowest version in Debian, so if the requires version is 3.0 or 
less, then an unversioned build-depends on flit is appropriate.  For the 
example above it would be:

flit >= 3.2.0

There is one issue at present, currently if upstream provides optional depends 
in  PEP 621 metadata (there is a [project.optional-dependencies] paragraph in 
the pyproject.toml) these optional dependencies are added to the binary 
depends through the python3:Depends expansion variable.  I expect this to be 
fixed in the next dh-python upload.  In the meantime this can be disabled by:

override_dh_python3:
dh_python3 --no-guessing-deps

As I'm going through updating my packages, I'm finding flit use is much more 
common than I was expecting.  You may be able to use this more than you 
thought.  I certainly am.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: DPT repositories checks/"violations" report

2021-11-27 Thread Scott Kitterman



On November 27, 2021 6:01:08 AM UTC, Sandro Tosi  wrote:
>Hello,
>while working on something else[1], i noticed how many of the
>repositories in the DPT salsa group are in poor shape:
>
>* missing branches
>* changes not pushed to salsa
>* general misalignment in configuration/setup/organization
>* many other small nuances
>
>[1] https://github.com/sandrotosi/debian-python-team-tracker
>
>That makes it hard to make mass work as in [1], so I thought it would
>be interesting to have a tool to evaluate the amount of issues our
>repos have, and identify such "violations" so that they can be
>addressed.
>
>That information is now available at [2].
>
>[2] https://github.com/sandrotosi/dpt-repos-check/blob/main/violations.txt
>
>please take the content with caution, as it's still an early, raw
>draft (i spot-checked some of the reported issues, but there could be
>bugs/issues) and it contains data that can be outdated (see below re
>caching); the fact that the report indicates only 43 repos are without
>violations is a bit disarming, but i think the tool tries to err on
>the side of verbosity and pedantry, with 2 level of violations (ERROR
>and WARNING) to mark which ones are the most important that require
>immediate attention vs the nice-to-have ones.
>
>The checks are under-documented, although they should be somehow
>self-explanatory. While the current format is just a text file, I plan
>on getting it converted to markdown, although I'm still not sure how
>to convey that amount of information effectively.
>
>The report is extremely intensive to generate, as it needs to make 10+
>API calls to salsa.d.o for each repository, and it gets throttled
>quite early in the run (i got away in dev by installing a cache, so
>that i could test it without hitting salsa every time, but that makes
>some info stale); for that reason, and if we decide is valuable to
>generate it periodically, i don't expect to be able to run it more
>than once a month (maybe once a week? TBD).
>
>Comments, critics and improvements are welcome.

I don't think the pypi tarball "issue" should be presumed to be a problem at 
all.  I wasn't paying attention to Debian when that discussion happened, but in 
my experience there was a lot wrong with the idea.  A properly constructed 
sdist is exactly what we want to build a package from.  That's almost never 
found on GitHub.

Scott K



Re: Downgrading dnspython back to 1.16.0 to fix Eventlet

2021-02-05 Thread Scott Kitterman
On Thursday, February 4, 2021 4:49:21 AM EST Thomas Goirand wrote:
> On 2/2/21 7:46 PM, Thomas Goirand wrote:
> > Both Eventlet and DNSPython are monkey patching the standard SSL library
> > in potentially conflicting ways
> 
> After checking, that's *NOT* the case. Though Eventlet is doing
> monkey-patching of dnspython, in a possible not-compatible with 2.x.
> 
> Anyways, looks like this small patch fixes Eventlet with dnspython 2:
> 
> https://github.com/tipabu/eventlet/commit/2f9b7969f9a66a75e72908454246b88bf5
> 7fe58a
> 
> I've uploaded Debian release 0.26.1-5, and when it reaches the mirrors,
> I'll try again to make OpenStack work, and see how it goes. If it fixes
> everything, then we're good to go. Otherwise, my questioning about
> downgrading dnspython to 1.16.0 still stand. I'll let you know.
> 
> Cheers,
> 
> Thomas Goirand (zigo)
> 
> P.S: Thanks to Tim Burke for this patch

If Eventlet is monkey patching DNSPython and it doesn't work, I think that's 
totally Eventlet's problem.  Hopefully your patch works.  I do not think 
downgrading DNSPython for this is a good idea.

Scott K


signature.asc
Description: This is a digitally signed message part.


Re: How to watch pypi.org

2020-10-04 Thread Scott Kitterman
On Sunday, October 4, 2020 10:24:22 PM EDT Paul Wise wrote:
> On Sun, Oct 4, 2020 at 3:29 PM Fioddor Superconcentrado wrote:
> > I've packaged a project provided via https://pipi.org and I wanted to
> > create a debian/watch file but pipi.org publishes the tarball behind a
> > strange url like
> I would suggest using the upstream git repo instead of the PyPi tarballs.

I think that's a different argument.

There is a pypi redirector for Debian watch files.  Something like this works 
(this is from the pyspf package):

https://pypi.debian.net/pyspf/pyspf-([0-9][0-9t\.\-]*).tar.gz

This currently works, but no guarantee for how long:

https://pypi.python.org/packages/source/x/@PACKAGE@/
@PACKAGE@@ANY_VERSION@@ARCHIVE_EXT@

Scott K




Upcoming dnspython 2.0.0 upload (new and improved with incompatibilities)

2020-08-09 Thread Scott Kitterman
I've started working on updating dnspython to 2.0.0.  The package itself is 
basically ready to go, but there is at least one incompatibility I've found 
relative to the current 1.16.0 version.

As far as I can tell, this only affects DNS results for type TXT lookups.  For 
those, the data returned is a tuple instead of a list.  There is also a 
deprecation warning for the dns.resolver.query class:

DeprecationWarning: please use dns.resolver.resolve() instead

That will, eventually, affect all DNS lookups, so it's worth discussing with 
upstream developers.  For projects still maintaining python2 compatibility it 
will be slightly more complicated to deal with since dnspython 2.0.0 is 
python3 only and dns.resolver.resolve does not exist in the last python2 
version.

I suspect usage of DNS TXT lookups is somewhat of a niche thing.  I've already 
fixed pyspf, dkimpy, and dkimpy-milter to work with both the old and new 
dnspython (still using the deprecated dns.resolver.query class).  If anyone 
else know of packages that use dnspython to make DNS TXT lookups and would 
like help fixing their package, please let me know and I'll be glad to help as 
I have time (which has been very limited lately).

If I don't hear from anyone I' plan on uploading dnspython 2.0.0 to unstable 
in the coming days.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: Python2 packages for bullseye

2020-07-09 Thread Scott Kitterman
On Thursday, July 9, 2020 7:21:45 AM EDT Matthias Klose wrote:
> The removal of packages still depending on Python2 looks good [1], however
> we have a bunch of packages that still require Python2, and where
> maintainers explicitly asked to keep those in the distro [2].  Among those
> are pypy and pypy3 which need Python2 for bootstrapping.  I'm going to keep
> the Python2 packages for bullseye, and having those just as build
> dependencies shouldn't really effect any end-user.  A different thing might
> be the Python2 usage at runtime, however I'm not very passionate to remove
> all of those packages.
> 
> What still should be done for bullseye is the removal of the unversioned
> python. I'm removing the packages python-minimal, python, python-dev,
> python-dbg, python-doc, and stop shipping the /usr/bin/python symlink, so
> that packages are required to either use python2 or python3 explicitly. 
> Planning that change for late August / early September.  That should give
> plenty of time to address any unversioned python usage before the release
> freeze.

Are you going to keep python-setuptools?  If you do, it seems likely we'll be 
able to keep pip and virtualenv so they support running python2 in a 
virtualenv in bullseye, which seems the best way to do it for those that need 
to.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: Maintaining all of the testing-cabal packages under the OpenStack team

2020-06-29 Thread Scott Kitterman
On Monday, June 29, 2020 7:53:46 AM EDT Thomas Goirand wrote:
> On 6/29/20 12:58 PM, Scott Kitterman wrote:
> > On June 29, 2020 10:12:49 AM UTC, Thomas Goirand  wrote:
> >> On 6/29/20 8:34 AM, Ondrej Novy wrote:
> >>> nope, this is not true. Using the newest debhelper compat level is
> >>> recommended, see man page. There is no reason to __not__ upgrade
> >>> debhelper compat level. I will always upgrade debhelper in my
> >> 
> >> packages
> >> 
> >>> to the newest debhelper as soon as possible. Please newer downgrade
> >>> debhelper in my packages again without asking.
> >> 
> >> I don't agree this is best practice when backports are to be expected.
> > 
> > I'm substantially less enthusiastic about bumping compat levels than
> > Ondrej, but since debhelper 13 is available in buster-backports,>
> > backporting is unrelated to whether it's a good idea or not.
> I'm not maintaining OpenStack through the official backports channel,
> because OpenStack users need to have access to all versions of OpenStack
> backported to the current Stable. These backports are available through
> a debian.net channel (available using extrepo).
> 
> Therefore, the debhelper backport is not available in my build
> environment unless I explicitly do some work to make this happen (and
> Ondrej is aware of that). Just bumping the debhelper version (and
> without a good reason to do so) just add some unnecessary work
> maintaining the debhelper backport for me. By all means, let's bump to
> version 12. But why version 13 if you don't need the added features?
> This makes no sense to me.

Since you are maintaining an external backports repository, I think it's 
perfectly reasonable to expect packages that would work with Debian Backports 
to be supported.  One debhelper upload per compat level doesn't seem like 
enough work to be worth all this complaining.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: Maintaining all of the testing-cabal packages under the OpenStack team

2020-06-29 Thread Scott Kitterman
On Monday, June 29, 2020 10:17:57 AM EDT Scott Talbert wrote:
> On Mon, 29 Jun 2020, Scott Kitterman wrote:
> >>> More over, mock debhelper was upgraded to 13, for no apparent
> >> 
> >> reason
> >> 
> >>> (yet another "cosmetic fix" that isn't helping?). I'd like to
> >> 
> >> remind
> >> 
> >>> everyone that, increasing debhelper compat version to a number
> >> 
> >> that
> >> 
> >>> isn't in stable, without a specific reason (like the need of a
> >> 
> >> specific
> >> 
> >>> feature that wasn't there before) is just annoying for anyone
> >>> maintaining backports. That's truth even for when debhelper
> >> 
> >> itself is
> >> 
> >>> backported to oldstable (it's always nicer to be able to build a
> >>> backport without requiring another backport at build time).
> >>> 
> >>> nope, this is not true. Using the newest debhelper compat level is
> >>> recommended, see man page. There is no reason to __not__ upgrade
> >>> debhelper compat level. I will always upgrade debhelper in my
> >> 
> >> packages
> >> 
> >>> to the newest debhelper as soon as possible. Please newer downgrade
> >>> debhelper in my packages again without asking.
> >> 
> >> I don't agree this is best practice when backports are to be expected.
> > 
> > I'm substantially less enthusiastic about bumping compat levels than
> > Ondrej, but since debhelper 13 is available in buster-backports,
> > backporting is unrelated to whether it's a good idea or not.
> 
> Can you elaborate on other reasons not the upgrade the compat levels?
> 
> Scott

This is a matter of personal preference, but since the behavior of debhelper 
changes between compat versions, I prefer no to change it unless I have time 
to thoroughly QA changes in the package.  This generally means I don't change 
it often.

Unless there are issues with a specific compat level (hello compat 11) or the 
compat level has been deprecated, I tend to not to do it, but I'm generally 
pretty minimalist in my package updates.  That doesn't mean someone else is 
wrong to do so if they've checked that package is correct after the change.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: Maintaining all of the testing-cabal packages under the OpenStack team

2020-06-29 Thread Scott Kitterman



On June 29, 2020 10:12:49 AM UTC, Thomas Goirand  wrote:
>On 6/29/20 8:34 AM, Ondrej Novy wrote:
...
>> More over, mock debhelper was upgraded to 13, for no apparent
>reason
>> (yet another "cosmetic fix" that isn't helping?). I'd like to
>remind
>> everyone that, increasing debhelper compat version to a number
>that
>> isn't in stable, without a specific reason (like the need of a
>specific
>> feature that wasn't there before) is just annoying for anyone
>> maintaining backports. That's truth even for when debhelper
>itself is
>> backported to oldstable (it's always nicer to be able to build a
>> backport without requiring another backport at build time).
>> 
>> nope, this is not true. Using the newest debhelper compat level is
>> recommended, see man page. There is no reason to __not__ upgrade
>> debhelper compat level. I will always upgrade debhelper in my
>packages
>> to the newest debhelper as soon as possible. Please newer downgrade
>> debhelper in my packages again without asking.
>
>I don't agree this is best practice when backports are to be expected.

I'm substantially less enthusiastic about bumping compat levels than Ondrej, 
but since debhelper 13 is available in buster-backports, backporting is 
unrelated to whether it's a good idea or not.

Scott K



Re: Asking for help Poetry

2020-06-28 Thread Scott Kitterman



On June 29, 2020 3:08:53 AM UTC, Emmanuel Arias  wrote:
>Hi,
>
>I'm working on poetry packaging, I created on salsa de repository [0].
>
>Note that there are many packages that are not in Debian like cachy,
>cleo,
>etc.
>
>I RFS python-cachy [1] and now I'm working on cleo, which depends on
>clikit
>that is not on Debian (if I search correctly).
>
>Bastian Venthur note me that pienv, pip have vendors folder with the
>dependencies
>but looking on poetry that _vendor folder is empty.
>
>So, looking for I more experienced opinion, do you think that we would
>try
>convince
>to upstream to make available vendors on the release (if that is
>necessary)
>or
>we need to package all the missing dependencies?

Definitely they should be packaged.

At least for pip none of the vendored libs are used in Debian's package.  
Fortunately upstream supports use of system libs with only minor patching.

Scott K



Re: Maintaining all of the testing-cabal packages under the OpenStack team

2020-06-28 Thread Scott Kitterman
On Sunday, June 28, 2020 1:59:08 PM EDT Sandro Tosi wrote:
> 5. consolidating packages *into* the DPMT/PAPT gives a lot of
> benefits, f.e. people basically got "free" handling of the py2removal
> process; moving packages out is actually detrimental for the python
> ecosystem (at least that's my opinion).

Definitely this.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: Future of PyPy (not PyPy3) in Debian

2020-06-04 Thread Scott Kitterman
On Thursday, June 4, 2020 8:00:30 PM EDT Sandro Tosi wrote:
> Hello all,
> it looks like i started a process that would require the removal of
> several PyPy (as in pypy-* depending on the `pypy` package) packages
> from the archive.
> 
> I'm now wondering: what should we do with the entire pypy ecosystem?
> should we treat pypy-* packages like python-* ones and remove then as
> part of py2removal? do we need/want to track this effort with the same
> usertag? is there a (even if only hypothetical for now) programmed
> transition from pypy- to pypy3-?
> 
> PS: if i made mistakes, just call me out, and i'll do my best to fix
> them or revert them; i have no problem in being told i did something
> wrong (let's keep the discussion in this thread on topic, so pypy etc)

My understanding is that unlike python2.7, pypy is not going away.  If nothing 
else it's needed to build pypy3.

Tumbleweed and I recently did some integration work to make sure pip and 
virtualenv for Bullseye will support pypy and pypy3.  In fact, a pypy 
virtualenv may be a decent solution for people with python2 code that isn't 
ported to python3 yet.  I'd hate to mess that up.

I think we need to keep some of pypy.  The question is how much.

Since Tumbleweed is the one most familiar with the environment, I defer to him 
on the specifics about what we need to keep.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: PEP-517/PEP-518 Support In Debian

2020-06-04 Thread Scott Kitterman
On Thursday, June 4, 2020 7:39:59 PM EDT Nicholas D Steeves wrote:
> Hi,
> 
> Scott Kitterman  writes:
> > On Monday, April 13, 2020 5:18:53 AM EDT Dmitry Shachnev wrote:
> >> Hi Scott!
> >> 
> >> On Sun, Apr 12, 2020 at 06:31:57PM -0400, Scott Kitterman wrote:
> >> > This being roughly the mid-point in the development cycle, I thought it
> >> > might be good to see where we are in terms of future Python packaging
> >> > developments.
> >> > 
> >> > As I understand it, PEP-517 and PEP-518 are 'the future'.
> 
> While importing the latest release of Fissix (a grammar parser used by
> Bowler) I found that it had a pyproject.toml but no setup.py.
> 
> >> > I took a look at the presence of pyproject.toml files in the Debian
> >> > archive. There are currently 99 packages.  Of those, only 28 specify a
> >> > 'build-backend', which is required by 517/518 to be useful for building
> >> > a
> >> > package.
> >> > 
> >> > 25 specify: build-backend = "setuptools.build_meta" (including
> >> > setuptools)
> >> > 3 specify: build-backend = "flit_core.buildapi" (including flit)
> >> 
> >> pyqt5 and pyqt5webengine specify: build-backend = "sipbuild.api".
> 
> Is there existing documentation around if/how build-backend can be used
> to work around the absence of setup.py?
> 
> > So they do.  They didn't show up in my codesearch.d.n results, that makes
> > me wonder what else I missed.  If anyone has an idea of a better way to
> > track this, please speak up.
> > 
> >> > If build-backend is not specified, the build system has to fall back to
> >> > setup.py.
> >> > 
> >> > I've recently package flit (just arrived in Testing) and written a flit
> >> > plugin for pybuild that's pending review for merging[1].  I also
> >> > packaged
> >> > pep517 for our pip package, so that's available to support future
> >> > Debian
> >> > tool development in this area.
> > 
> > P1otr merged the flit plugin a little while ago, so it'll be in the next
> > dh- python upload.
> 
> I think this is probably what is needed to unblock work on fissix (and
> thus Bowler), because its Makefile has:
> 
> python -m pip install -r requirements.txt
> python -m pip install -r requirements-dev.txt
> python -m flit install --symlink
> 
> [snip]
> 
> Any advise for existing workarounds would be appreciated.  So far the
> only idea I've had is writing or generating a setup.py, and then adding
> it as a quilt patch.

That is probably the best approach for now.

Ultimately we have two choices:

1.  Add per-backend capability into pybuild as I have done with flit.  
Depending on how many of them there are, this may or may not be a reasonable 
approach.

2.  Use the pep517 module with pip in our package build system.  I think 
Fedora has gone this direction.

I prefer #1 because we want to produce a traditional bdist, not wheels, but 
we'll have to do the work.  I don't think we want an impenetrable morass of 
zip files in /usr/lib/python3/dist-packages.

Scott K 

signature.asc
Description: This is a digitally signed message part.


Re: packaging DiscoDOS - a cli tool for vinyl DJs

2020-05-17 Thread Scott Kitterman
On Sunday, May 17, 2020 7:44:38 PM EDT Steve Langasek wrote:
> On Sat, May 16, 2020 at 11:09:33AM +0800, Paul Wise wrote:
> > On Fri, 2020-05-15 at 19:56 -0700, Steve Langasek wrote:
> > > FTR, UbuntuStudio is an official Ubuntu flavor, not a derivative ;)
> > 
> > Woops. Did that change at some point or did I mix them up with another
> > distro or just make a stupid mistake?
> 
> Couldn't say, sorry.  It's certainly been that way for several years
> already.

Although it predates the terminology "official flavor" by some years, it's 
effectively been one since it was first released with 7.04 (Feisty Fawn, IIRC).

Scott K


signature.asc
Description: This is a digitally signed message part.


Re: Issues running pytest from within .pybuild

2020-05-17 Thread Scott Kitterman
On Sunday, May 17, 2020 6:26:11 PM EDT Christian Kastner wrote:
> In a recent version of src:scikit-learn, a workaround was added to
> resolve an ImportMismatchError regarding one of the conftest.py files
> used by pytest.
> 
> I hit a new instance of this while preparing a new upstream release, so
> I inquired [1] with pytest upstream as to why this could be happening.
> 
> Apparently, the error is caused by the .pybuild directory being located
> as a subfolder of the source directory. This leads to pytest finding two
> separate sklearn/conftest.py files
>   * $SRCDIR/sklearn/conftest.py
>   * $SRCDIR/.pybuild/.../sklearn/conftest.py)
> which triggers the ImportMisMatchError.
> 
> Now, I've only seen this happen with src:scikit-learn, and (oddly
> enough) only with 2 of 4 conftest.py files. The other two do not trigger
> the error, nor do other packages of mine which use pytest.
> 
> Has anyone seen something similar with their packages, or is anyone
> aware of any recipes that can be used to resolve the issue properly?
> 
> >From the GitHub issue, it seems as if the current practice of testing
> 
> from within .pybuild is not supported by pytest, but I'm inclined to
> believe that it is my approach that is flawed.
> 
> [1] https://github.com/pytest-dev/pytest/issues/7223
> 
> Christian

I think this would only happen if sys.path had been extended to include the 
main source directory.  You may need to cd to $SRCDIR/.pybuild/.../sklearn 
before calling pytest or something may be inserting the source directory into 
sys.path.

Scott K

signature.asc
Description: This is a digitally signed message part.


Disparaging people's motivation to contribute to Debian was: Re: packaging DiscoDOS - a cli tool for vinyl DJs

2020-05-15 Thread Scott Kitterman
On Friday, May 15, 2020 4:36:52 PM EDT Thomas Goirand wrote:
> On 5/15/20 7:09 PM, Scott Kitterman wrote:
> > On Friday, May 15, 2020 12:55:48 PM EDT Thomas Goirand wrote:
> >> On 5/15/20 5:43 PM, jojo wrote:
> >>> Hi,
> >>> 
> >>> I'd like to join the list because I think my software is a valuable
> >>> addition to the debian universe, my ultimate goal would be to bring it
> >>> into Ubuntu Studio because it is music-related.
> >> 
> >> I really think it's a shame that people join Debian just because of
> >> Ubuntu... :(
> > 
> > Thomas,
> > 
> > Ask yourself if you are modelling being a member of a welcoming community
> > here?
> > 
> > There are lots of examples of people who initially became interested in
> > Debian via Ubuntu and are good Debian contributors and project members.
> > 
> > You are free to think whatever you want, but I don't think this kind of
> > sentiment has any place on Debian lists.
> > 
> > Scott K
> 
> This was kind of rhetorical, and it is my believe that if it is the way
> it is, *we* are at fault, globally in Debian. I'm just not sure how to
> fix that. I BTW don't agree with you, and IMO, this has some place on
> the Debian lists. Having Debian (directly) appealing to everyone is very
> important topic.
> 
> Of course, Jojo is very much welcome, and I'm sorry that you take it
> this way. I've pointed at many docs to help, so I very much believe he
> knows I warmly welcome him.
> 
> Cheers,
> 
> Thomas Goirand (zigo)

If you don't actually think it's a shame he wants to participate in Debian, it 
might be better not to say so.  I think your response it logically disjoint 
from your original mail, so I don't see any point in further conversation on 
the matter.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: packaging DiscoDOS - a cli tool for vinyl DJs

2020-05-15 Thread Scott Kitterman
On Friday, May 15, 2020 12:55:48 PM EDT Thomas Goirand wrote:
> On 5/15/20 5:43 PM, jojo wrote:
> > Hi,
> > 
> > I'd like to join the list because I think my software is a valuable
> > addition to the debian universe, my ultimate goal would be to bring it
> > into Ubuntu Studio because it is music-related.
> 
> I really think it's a shame that people join Debian just because of
> Ubuntu... :(

Thomas,

Ask yourself if you are modelling being a member of a welcoming community 
here?

There are lots of examples of people who initially became interested in Debian 
via Ubuntu and are good Debian contributors and project members.

You are free to think whatever you want, but I don't think this kind of 
sentiment has any place on Debian lists.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: Request to join PAPT

2020-05-11 Thread Scott Kitterman
On Thursday, April 23, 2020 10:32:42 PM EDT Doug Torrance wrote:
> Hello!
> 
> I am interested in joining the Python Applications Packaging Team.  I'm
> experienced in packaging for Debian and am an active member of the Window
> Maker and Science teams.  However, I don't have much experience with Python
> packaging yet and am looking for some guidance and extra sets of eyes on
> any packages that I might maintain under the PAPT umbrella.
> 
> Currently, I maintain one package which I think would be a good fit for the
> team, git-big-picture [1].  It's currently out of testing with RC bug
> #936615.  However, I've packaged a new upstream version which fixes it, and
> I think it's almost ready for review and sponsorship.
> 
> My Salsa username is dtorrance.  I have read and agree to the PAPT
> Policy.
> 
> Thanks!
> Doug Torrance
> 
> [1] https://salsa.debian.org/dtorrance/git-big-picture

Also quite late, but welcome to the team.  Note I did add you with your 
current username.

Scott K




Re: Joining PAPT

2020-05-11 Thread Scott Kitterman
On Thursday, December 5, 2019 11:42:02 AM EDT Scott Talbert wrote:
> Hi,
> 
> I'm already a member of DPMT, I would like to join PAPT as well so I can
> also contribute to those packages.  I have read the policy and accept it.
> 
> Salsa ID: swt2c-guest
> 
> Thanks,
> Scott

I know this is late, but welcome to the team.  Sorry for the delay.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: joining the PAPT

2020-05-11 Thread Scott Kitterman
On Wednesday, May 6, 2020 2:05:09 PM EDT Hans-Christoph Steiner wrote:
> Hey all,
> 
> I'm a DD and a long time member of PMPT  I would like to join the PAPT.
>  I am packaging https://github.com/cryptax/droidlysis and I think it
> fits best in PAPT.  I am also willing to be a sponsor for packages I
> know something about or are simple enough that I can understand them.
> 
> I have read and accept the PAPT policy at
> https://salsa.debian.org/python-team/tools/python-apps/-/blob/c0eb83b9/polic
> y.rst
> 
> My salsa username is eighthave, same as PMPT.
> 
> .hc

Welcome to the team.  Sorry for the bother.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: Best practice on how to package a python module along with a c++ program

2020-05-08 Thread Scott Kitterman
On Friday, May 8, 2020 6:59:08 AM EDT Uwe Steinmann wrote:
> Hi,
> 
> not sure if this is the right list, but it was my closes guess.
> 
> I'm packaging a c++ program (horizon-eda) which also contains a
> python module written in c++. Upstream's Makefile has a target 'pymodule'
> and 'make pymodule' creates horizon.so in the build directory.
> According to upstream this has to be copied іnto python's sys.path
> 
> I thought about overriding dh_auto_build with
> 
> override_dh_auto_build:
>   make pymodule
>   dh_auto_build
> 
> which should at least build the module. But it still needs to be
> installed at the right place. What would be the preferred way?

First, you don't specify, but assuming this is python3:

The preferred mechanism would be to install the .so file into:

usr/lib/python3.X/dist-packages/ (where X is the python3 version you are 
building with)

Then use dh_python3 and ${python3:Depends} to install it in the correct final 
location and generate appropriate Python interpreter dependencies.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: joining the DPMT.

2020-05-07 Thread Scott Kitterman
On Thursday, April 30, 2020 5:32:39 PM EDT peter green wrote:
> I would like to join the DPMT, there are a couple of reasons for this.
> 
> Firstly I have been making an effort to try and get broken
> build-dependencies in testing fixed, and this often ends up involving
> python module packages. It would be easier to fix such packages as a member
> of the team than working through patches and NMUs as I have done so far.
> 
> Secondly I maintain a couple of python modules, which it may make sense to
> bring into team maintainership, though I would have to figure out how to
> restructure the git repositories to fit the dpmt policy.
> 
> I have read and accept the DPMT policy at
> https://salsa.debian.org/python-team/tools/python-modules/blob/master/polic
> y.rst
> 
> my salsa username is plugwash

Welcome to the team.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: Python Wheel Related Policy Change

2020-05-01 Thread Scott Kitterman
On Thursday, April 30, 2020 6:21:48 PM EDT Scott Kitterman wrote:
> On Thursday, April 30, 2020 5:49:20 PM EDT Stefano Rivera wrote:
> > Hi Scott (2020.04.30_20:33:59_+)
> > 
> > > > That seems reasonable, although if we're going down that road, it
> > > > probably makes no sense for any of them to be universal.
> > > 
> > > If we were talking about maintaining this for multiple release cycles
> > > with
> > > lots of version divergence, I would agree.  Let's not do more than we
> > > have
> > > to until python2 is gone (whether it is before bullseye or after).
> > 
> > I suspect pypy (2.7) will probably be around post-bullseye, unless
> > somebody funds pypy to migrate rpython to python 3.
> > 
> > But yeah, we can change strategy later, if appropriate.
> 
> Well, we have also talked about pypy vendoring as much of the python2.7
> package as it needs to build itself so we don't have to support it in the
> archive as an active interpreter, but that's a different discussion.
> 
> > > As a result, one could add some kind of py2 flag to dirtbike to tell it
> > > to
> > > look in /usr/lib/python2.7/dist-packages and make a 'universal' wheel
> > > from there. It could then rename the files from py2.py3-none-any.whl to
> > > py2-none-any.whl. The bonus Tag in the WHEEL file shouldn't hurt
> > > anything.
> > > 
> > > They key is I think we can do this without actually running python2, we
> > > just need to be able to install python-setuptools/pkg-resources.  That
> > > should be supportable ~as long as python2.7 is in the archive.
> > 
> > Yep, that sounds good.
> > 
> > > Agreed.  As long as pip supports python2, we should try.  Worst case we
> > > can
> > > tell people to find their own interpreter and do a --download install
> > > with
> > > setuptools pinned to 44.0.0 (that approach doesn't need ipaddr because
> > > they'll get the upstream pip in the virtualenv too.
> > 
> > When we get to the download route: pip can parse Requires-Python, and it
> > looks like virtualenv uses the system pip to download pip, so the
> > pinning shouldn't be necessary, I think.
> 
> I haven't really tested that yet, so I don't doubt that.  It was an
> assumption.  My first run at virtualenv ran aground on trying to have
> different package lists depending on if the install invoked --download or
> not (since a --download install with the upstream pip doesn't need all the
> unvendored wheels).  So that's something to sort later.  We'll want to
> document how to do it even if we get a working solution that doesn't
> require it.
> 
> > > > So, the options are:
> > > > 1. pin pip dependencies at versions that support py2, forever.
> > > > Obviously
> > > > 
> > > >this is a no-go.
> > > > 
> > > > 2. Ship py2 and py3 wheels.
> > > > 
> > > >2.1. package the py2 weels with dirtbike. This requires bringing
> > > >back
> > > >
> > > > python-wheel, or more work on dirtbike.
> > > >
> > > >2.2. Ship static py2 wheels (like we did pre-dirtbike). It's not
> > > >like
> > > >
> > > > upstream is continuing development...
> > > > 
> > > > 3. Let virtualenv always download pip from pypi for py2. Only ship py3
> > > > 
> > > >wheels.
> > > > 
> > > > The easy options here are 2.2 and 3.
> > > 
> > > 2.1a Where needed package 'py2' wheels with dirtbike running python3.
> > 
> > Ah, yeah.
> > 
> > Still requires keeping py2 source packages whenever a library in the pip
> > dependency stack drops py2 support. But if they move slowly, that's
> > fine.
> 
> So far setuptools is the only one.  It might pick up though now that they've
> done it.  We've split packages into a source for python2 and a source for
> python3 in a number of cases, so I think this scales reasonably.  We can
> just yank all the python2 sources the same time pip drops python2 or we do.
> > > 2.2 is using sourceless binaries, so I think it's got DFSG issues.  We
> > > need
> > > the preferred form of modification and a bdist_wheel is not that.
> > 
> > They could be bdist_wheeled at build time, within a single source
> > package.
> > 
> > > I think 2.1a would be nice if someone can manage it.  We'll need to
> > > support 3 eventually.  I plan to implement 3 as an op

Re: Python Wheel Related Policy Change

2020-04-30 Thread Scott Kitterman
On Thursday, April 30, 2020 5:49:20 PM EDT Stefano Rivera wrote:
> Hi Scott (2020.04.30_20:33:59_+)
> 
> > > That seems reasonable, although if we're going down that road, it
> > > probably makes no sense for any of them to be universal.
> > 
> > If we were talking about maintaining this for multiple release cycles with
> > lots of version divergence, I would agree.  Let's not do more than we have
> > to until python2 is gone (whether it is before bullseye or after).
> 
> I suspect pypy (2.7) will probably be around post-bullseye, unless
> somebody funds pypy to migrate rpython to python 3.
> 
> But yeah, we can change strategy later, if appropriate.

Well, we have also talked about pypy vendoring as much of the python2.7 
package as it needs to build itself so we don't have to support it in the 
archive as an active interpreter, but that's a different discussion.

> > As a result, one could add some kind of py2 flag to dirtbike to tell it to
> > look in /usr/lib/python2.7/dist-packages and make a 'universal' wheel
> > from there. It could then rename the files from py2.py3-none-any.whl to
> > py2-none-any.whl. The bonus Tag in the WHEEL file shouldn't hurt
> > anything.
> > 
> > They key is I think we can do this without actually running python2, we
> > just need to be able to install python-setuptools/pkg-resources.  That
> > should be supportable ~as long as python2.7 is in the archive.
> 
> Yep, that sounds good.
> 
> > Agreed.  As long as pip supports python2, we should try.  Worst case we
> > can
> > tell people to find their own interpreter and do a --download install with
> > setuptools pinned to 44.0.0 (that approach doesn't need ipaddr because
> > they'll get the upstream pip in the virtualenv too.
> 
> When we get to the download route: pip can parse Requires-Python, and it
> looks like virtualenv uses the system pip to download pip, so the
> pinning shouldn't be necessary, I think.

I haven't really tested that yet, so I don't doubt that.  It was an 
assumption.  My first run at virtualenv ran aground on trying to have different 
package lists depending on if the install invoked --download or not (since a 
--download install with the upstream pip doesn't need all the unvendored 
wheels).  So that's something to sort later.  We'll want to document how to do 
it even if we get a working solution that doesn't require it.

> > > So, the options are:
> > > 1. pin pip dependencies at versions that support py2, forever. Obviously
> > > 
> > >this is a no-go.
> > > 
> > > 2. Ship py2 and py3 wheels.
> > > 
> > >2.1. package the py2 weels with dirtbike. This requires bringing back
> > >
> > > python-wheel, or more work on dirtbike.
> > >
> > >2.2. Ship static py2 wheels (like we did pre-dirtbike). It's not like
> > >
> > > upstream is continuing development...
> > > 
> > > 3. Let virtualenv always download pip from pypi for py2. Only ship py3
> > > 
> > >wheels.
> > > 
> > > The easy options here are 2.2 and 3.
> > 
> > 2.1a Where needed package 'py2' wheels with dirtbike running python3.
> 
> Ah, yeah.
> 
> Still requires keeping py2 source packages whenever a library in the pip
> dependency stack drops py2 support. But if they move slowly, that's
> fine.

So far setuptools is the only one.  It might pick up though now that they've 
done it.  We've split packages into a source for python2 and a source for 
python3 in a number of cases, so I think this scales reasonably.  We can just 
yank all the python2 sources the same time pip drops python2 or we do.

> > 2.2 is using sourceless binaries, so I think it's got DFSG issues.  We
> > need
> > the preferred form of modification and a bdist_wheel is not that.
> 
> They could be bdist_wheeled at build time, within a single source
> package.
> 
> > I think 2.1a would be nice if someone can manage it.  We'll need to
> > support 3 eventually.  I plan to implement 3 as an option when I upload
> > pip 20.1 and virtualenv 2.0.18.
> 
> Not necessarily. 2.2 is equivalent to 3, but without hitting PyPI.

Let's focus on 2.1a so we don't have to have that argument ...

I might actually have a little time tomorrow to work on it.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: Python Wheel Related Policy Change

2020-04-30 Thread Scott Kitterman
On Thursday, April 30, 2020 2:32:24 PM EDT Stefano Rivera wrote:
> Hi Scott (2020.04.30_16:23:41_+)
> 
> > Making dirtbike build a py3 wheel for setuptools (and pkg_resources),
> > which is technically accurate at this point since dirtbike can't build
> > wheels from a> 
> > python2 package was trivial to do:
> >  dirtbike/__init__.py | 6 +-
> >  1 file changed, 5 insertions(+), 1 deletion(-)
> > 
> > Three of the five insertions were comment.
> 
> That seems reasonable, although if we're going down that road, it
> probably makes no sense for any of them to be universal.

If we were talking about maintaining this for multiple release cycles with 
lots of version divergence, I would agree.  Let's not do more than we have to 
until python2 is gone (whether it is before bullseye or after).

> > If someone wants to provide a patch to dirtbike so it can also build
> > wheels
> > from python-setuptools and python-pkg-resources, then we could add then to
> > build-depends and provide both a set of py2 wheels and py3 wheels for as
> > long as they are in the archive.
> 
> The window for doing this is probably closing, too.
> 
> I hacked this in Ubuntu 20.04, just before release:
> https://launchpadlibrarian.net/475592029/python-pip_20.0.2-5_20.0.2-5ubuntu1
> .diff.gz
> 
> As you can see, python-wheel is no longer available, and some of the
> other libraries dirtbike uses are probably heading that way.
> 
> So, to make that approach work, we have to introduce more py2-only
> libraries again, or allow dirtbike to build py2 wheels while running
> under python3.

I had in mind the latter.  I looked and there's not a lot of difference in the 
metadata of a pure python wheel that's universal and one that's not.  The 
universal one has tags for both versions:

$ cat WHEEL 
Wheel-Version: 1.0
Generator: bdist_wheel (0.34.2)
Root-Is-Purelib: true
Tag: py2-none-any
Tag: py3-none-any

As a result, one could add some kind of py2 flag to dirtbike to tell it to look 
in /usr/lib/python2.7/dist-packages and make a 'universal' wheel from there.  
It could then rename the files from py2.py3-none-any.whl to py2-none-any.whl.  
The bonus Tag in the WHEEL file shouldn't hurt anything.

They key is I think we can do this without actually running python2, we just 
need to be able to install python-setuptools/pkg-resources.  That should be 
supportable ~as long as python2.7 is in the archive.

> > It still needs the Python policy change since they aren't universal, but
> > it
> > would allow setuptools to keep up for python3 in pip without dropping the
> > local wheels from python2.  We also have ipaddr back in the archive to
> > support wheeling it so pip will run with python2.
> 
> But, without the py2 setuptools wheels, I don't know much that buys us.

Technically I think it's solvable and is supportable until the python2 
interpreter goes away.

> Long-term: pypy (2.7) is going to continue to be around, until rpython
> moves to python3 (which is still officially never, but people are
> starting to poke at it).
> https://doc.pypy.org/en/latest/faq.html#how-long-will-pypy-support-python2
> 
> If we have a python 2 interpreter, it's really nice to have working
> virtualenv for it. Makes it a lot more useful for users, esp. as we
> don't have many debian libraries packaged for it.

Agreed.  As long as pip supports python2, we should try.  Worst case we can 
tell people to find their own interpreter and do a --download install with 
setuptools pinned to 44.0.0 (that approach doesn't need ipaddr because they'll 
get the upstream pip in the virtualenv too.  

> So, the options are:
> 1. pin pip dependencies at versions that support py2, forever. Obviously
>this is a no-go.
> 2. Ship py2 and py3 wheels.
>2.1. package the py2 weels with dirtbike. This requires bringing back
> python-wheel, or more work on dirtbike.
>2.2. Ship static py2 wheels (like we did pre-dirtbike). It's not like
> upstream is continuing development...
> 3. Let virtualenv always download pip from pypi for py2. Only ship py3
>wheels.
> 
> The easy options here are 2.2 and 3.

2.1a Where needed package 'py2' wheels with dirtbike running python3.

2.2 is using sourceless binaries, so I think it's got DFSG issues.  We need 
the preferred form of modification and a bdist_wheel is not that.

I think 2.1a would be nice if someone can manage it.  We'll need to support 3 
eventually.  I plan to implement 3 as an option when I upload pip 20.1 and 
virtualenv 2.0.18.

Scott K


signature.asc
Description: This is a digitally signed message part.


Re: Python Wheel Related Policy Change

2020-04-30 Thread Scott Kitterman
On Thursday, April 30, 2020 5:53:33 AM EDT Scott Kitterman wrote:
> On April 30, 2020 7:21:39 AM UTC, Matthias Klose  wrote:
> >On 4/30/20 5:28 AM, Scott Kitterman wrote:
> >> I think Python policy changes should be discussed.  I accidentally
> >committed a
> >> change to git [1] (I didn't realize I still had access, I thought it
> >would be
> >> a merge request) to allow Python 3 only wheels for packages that
> >require
> >> wheels, but that no longer support Python 2.  This is going to happen
> >the next
> >> time I upload python-pip since the current version of setuptools is
> >python3
> >> only.
> >
> >this is not true. python-setuptools is still built from the source
> >packages
> >python-setuptools.  Is there a reason that you cannot use it?
> 
> In the short run, the blocker is that dirtbike doesn't support it.  In the
> longer run I think adding back python2 dependencies is not a good idea.
> 
> At best it's a short-term avoidance mechanism.
> 
> >> Hopefully this won't be controversial.  There's really no way to
> >> avoid it.
> >
> >well, better don't assume that.
> 
> It's true that there is a short-term alternative, but I think it's a waste
> of effort to try and implement it.  At most it only buys you until the next
> pip vendored lib that we unbundle as a wheel goes python3 only.
> 
> Are you volunteering to rewrite dirtbike so it can make a universal wheel
> from python-setuptools?
> 
> Personally, I'm not interested.  If you add the word reasonable to my
> original statement, I stand by it.

Making dirtbike build a py3 wheel for setuptools (and pkg_resources), which is 
technically accurate at this point since dirtbike can't build wheels from a 
python2 package was trivial to do:

 dirtbike/__init__.py | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

Three of the five insertions were comment.

If someone wants to provide a patch to dirtbike so it can also build wheels 
from python-setuptools and python-pkg-resources, then we could add then to 
build-depends and provide both a set of py2 wheels and py3 wheels for as long 
as they are in the archive.

It still needs the Python policy change since they aren't universal, but it 
would allow setuptools to keep up for python3 in pip without dropping the 
local wheels from python2.  We also have ipaddr back in the archive to support 
wheeling it so pip will run with python2.

I think that would be the best technical approach for now, but I don't have 
time at the moment to proceed further.  Contributions welcome.  

I'd like to resolve this one way or the other because as it stands, the next 
python-pip upload will make wheels from python3-setuptools and python3-pkg-
resources that are formatted as universal wheels, but aren't in terms of 
content. 

I've started working on packaging pip 20.1.  Once I have it ready to go, I'll 
upload dirtbike either as it stands or with support for making python2 wheels 
if someone provides it.  Just uploading pip using the current dirtbike is not 
appropriate.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: Python Wheel Related Policy Change

2020-04-30 Thread Scott Kitterman



On April 30, 2020 7:21:39 AM UTC, Matthias Klose  wrote:
>On 4/30/20 5:28 AM, Scott Kitterman wrote:
>> I think Python policy changes should be discussed.  I accidentally
>committed a 
>> change to git [1] (I didn't realize I still had access, I thought it
>would be 
>> a merge request) to allow Python 3 only wheels for packages that
>require 
>> wheels, but that no longer support Python 2.  This is going to happen
>the next 
>> time I upload python-pip since the current version of setuptools is
>python3 
>> only.
>
>this is not true. python-setuptools is still built from the source
>packages
>python-setuptools.  Is there a reason that you cannot use it?

In the short run, the blocker is that dirtbike doesn't support it.  In the 
longer run I think adding back python2 dependencies is not a good idea.

At best it's a short-term avoidance mechanism.

>> Hopefully this won't be controversial.  There's really no way to
>> avoid it.
>
>well, better don't assume that.

It's true that there is a short-term alternative, but I think it's a waste of 
effort to try and implement it.  At most it only buys you until the next pip 
vendored lib that we unbundle as a wheel goes python3 only.

Are you volunteering to rewrite dirtbike so it can make a universal wheel from 
python-setuptools?

Personally, I'm not interested.  If you add the word reasonable to my original 
statement, I stand bye it.  

Scott K



Python Wheel Related Policy Change

2020-04-29 Thread Scott Kitterman
I think Python policy changes should be discussed.  I accidentally committed a 
change to git [1] (I didn't realize I still had access, I thought it would be 
a merge request) to allow Python 3 only wheels for packages that require 
wheels, but that no longer support Python 2.  This is going to happen the next 
time I upload python-pip since the current version of setuptools is python3 
only.

Hopefully this won't be controversial.  There's really no way to avoid it.

Scott K

[1] https://salsa.debian.org/cpython-team/python3-defaults/-/commit/
92e10578994c1950e0c98387a60fdcfa4c69e1d4

signature.asc
Description: This is a digitally signed message part.


Re: issues installing psutil with pip in virtual environment

2020-04-26 Thread Scott Kitterman



On April 26, 2020 4:06:00 PM UTC, Anil F Duggirala  
wrote:
>hello,
>I am running into an issue installing psutil: pip3 install psutil, in a
>virtual environment. I have upgraded my pip and setuptools with no
>avail. I am getting this error: https://pastebin.com/2Xb7UN9g
>Some have suggested installing the python3-dev package. Saying that I
>require "header" files (don't know what those are). So this means
>installing that package and creating a new venv, where those files are
>available. Is there a way to make this install work without installing
>that package? Is that package really necessary? Does this mean my
>virtual environments are somehow subject to what libraries are
>available in my system python installation? Is there some pip
>installabel package that provides these files?
>thank you,

No.  No.  Yes.  Yes.  No.

Pip doesn't provide the python interpreter.  The solution is in the traceback 
you posted:

("sudo apt-get install gcc python%s-dev" % py3)
  File "/tmp/pip-install-b88905i2/psutil/setup.py", line 116, in missdeps
s = hilite("C compiler or Python headers are not installed ", ok=False)

So install gcc and python3-dev and try again.

Scott K



Re: best practice when installing python packages

2020-04-25 Thread Scott Kitterman
On Saturday, April 25, 2020 1:10:37 PM EDT Anil F Duggirala wrote:
> hello,
> Im having an issue while installing a piece of software with pip3:
> pip3 install --user -r contrib/requirements/requirements-binaries.txt
> 
> Command "python setup.py egg_info" failed with error code 1 in
> /tmp/pip-install-48wlqr39/PyQt5/
> 
> I had previously installed: apt install python3-pyqt5. Now, pyqt5 is
> listed in the requirements-binaries.txt list, however, it requires a
> newer version than the one installed by apt.
> 
> Online I find many people getting a similar error and suggesting using
> pip3 upgrade to upgrade the "pip" and the "setuptools" modules. And
> this brings me to my question. What happens if I use pip3 to upgrade a
> module that was installed via apt?
> 
> Now pip3 list is listing both the pip and setuptools modules (which
> were installed using apt). It even tells me about a newer version
> available (via pip). However pip3 list is not listing the pyqt5 module
> as installed, why is it not showing this module, which is installed?
> Does this have to do with the fact that my pip or setuptools modules
> are outdated? (both have much newer versions available via pip)
> 
> please reply to my email, I am not subscribed to this list,
> thank you,

On Debian pip/pip3 does a user install by default, so if you do an 'upgrade' 
of a system installed module, it should have no system wide effect, only for 
the current user.

The Buster (Debian 10) version of PyQt5 does not install the Python packaging 
related metadata, so it not being listed by pip3 is not a surprise (for the 
next release it is provided).

For cases like this, I think the best practice is to work inside a virtualenv 
where you can upgrade pip and install whatever you need via pip with no impact 
on either your user or system python.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: mailing-list-obsolete-in-debian-infrastructure warning

2020-04-23 Thread Scott Kitterman



On April 24, 2020 12:26:27 AM UTC, "Louis-Philippe Véronneau" 
 wrote:
>On 20-04-23 17 h 54, Scott Kitterman wrote:
>> On Thursday, April 23, 2020 5:21:49 PM EDT Rebecca N. Palmer wrote:
>>> This warning was added to Lintian today, after being requested in
>>> #958182.  It appears on thousands of packages:
>>>
>https://lintian.debian.org/tags/mailing-list-obsolete-in-debian-infrastructu
>>> re.html
>> 
>> Thanks.  I mailed the bug to point out this tag is currently
>problematic.
>
>FWIW, I proposed we merge the DPMT and the PAPT a few months ago [1]
>and
>migrate to team+pyt...@tracker.debian.org.
>
>That would also solve the current issue with lintian :)
>
>[1]:
>https://salsa.debian.org/python-team/tools/python-modules/-/merge_requests/10

And I don't think the response was particularly positive.  My recollection was 
that there's a preference for two teams because packages and modules have 
different needs.

I don't think we should let an artificial sense of urgency because lintian push 
us in to anything (apologies if I'm remembering the wrong instance of this 
being suggested - it's happened before).

Scott K



Re: mailing-list-obsolete-in-debian-infrastructure warning

2020-04-23 Thread Scott Kitterman
On Thursday, April 23, 2020 5:21:49 PM EDT Rebecca N. Palmer wrote:
> This warning was added to Lintian today, after being requested in
> #958182.  It appears on thousands of packages:
> https://lintian.debian.org/tags/mailing-list-obsolete-in-debian-infrastructu
> re.html

Thanks.  I mailed the bug to point out this tag is currently problematic.

Scott K




Re: todoman_3.7.0-2_source.changes ACCEPTED into unstable

2020-04-23 Thread Scott Kitterman
This one has the same issue.

Scott K

On Thursday, April 23, 2020 1:05:23 PM EDT Debian FTP Masters wrote:
> Accepted:
> 
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA512
> 
> Format: 1.8
> Date: Thu, 23 Apr 2020 18:39:20 +0200
> Source: todoman
> Architecture: source
> Version: 3.7.0-2
> Distribution: unstable
> Urgency: medium
> Maintainer: Debian Python Team 
> Changed-By: Félix Sipma 
> Changes:
>  todoman (3.7.0-2) unstable; urgency=medium
>  .
>* move the package to Debian Python Team
>* bump Standards-Version to 4.5.0 (no change required)
>* make use of dh_sphinxdoc
> Checksums-Sha1:
>  c9a5a6c19949ea901dd18fb506f443dc815f744a 1905 todoman_3.7.0-2.dsc
>  a7d48bdd9e3704782b607f79426dea4cb0e74b51 8792 todoman_3.7.0-2.debian.tar.xz
> Checksums-Sha256:
>  2c37c319760bf289400d8897a4028470b464539585bb463c9a4af34832c57640 1905
> todoman_3.7.0-2.dsc
> d9c343a2dea2f39a77505820cfb169de104d4ca2efa0d8691c070114ab92a1db 8792
> todoman_3.7.0-2.debian.tar.xz Files:
>  ecc64aedae81de7c17e81acb0b8e71fb 1905 utils optional todoman_3.7.0-2.dsc
>  a512649e075c04c069d0b89a5f7b854c 8792 utils optional
> todoman_3.7.0-2.debian.tar.xz
> 
> -BEGIN PGP SIGNATURE-
> 
> iHUEARYKAB0WIQR6zeIsS8L0XLQfqiQBpfxHUdFE8AUCXqHF1AAKCRABpfxHUdFE
> 8PDNAQCP/CRZt73P/FkX+uyurE9roJULZbLHtJb3aPZmktRAoQEAhgihrb3OY7qs
> fF8bbfnzAy6uyQ62FvzWQGdI/85k5QQ=
> =fWLV
> -END PGP SIGNATURE-
> 
> 
> Thank you for your contribution to Debian.






Re: khard_0.16.1-1_source.changes ACCEPTED into unstable

2020-04-23 Thread Scott Kitterman
On Thursday, April 23, 2020 1:06:42 PM EDT Félix Sipma wrote:
> Note that changing the address triggers a lintian warning:
> 
> W: khard source: mailing-list-obsolete-in-debian-infrastructure Python
> Applications Packaging Team 

Python Applications Packaging Team  
is correct.

Scott K




Re: [felix+deb...@gueux.org: Bug#955862: RFS: sphinx-autoapi/1.2.1-1 [ITP]]

2020-04-19 Thread Scott Kitterman



On April 19, 2020 3:24:17 PM UTC, "Félix Sipma"  wrote:
>Hi Thomas,
>
>Thanks for the review, and it's nice to see you found a number of 
>problems! I have to admit I did not prepare a new package since a long 
>time, probably forgot a lot of things, and copied others from other 
>packages of mine...
>
>On 2020-04-19 14:09+0200, Thomas Goirand wrote:
>>Hi Felix,
>>
>>Thanks for working on this. Here's my comments. I'm sorry that there's
>a
>>lot to say on what you did... :/
>>
>>On 4/19/20 11:53 AM, Félix Sipma wrote:
>>>  dget -x 
>   
> https://mentors.debian.net/debian/pool/main/s/sphinx-autoapi/sphinx-autoapi_1.2.1-1.dsc
>>
>>Looking at your debian/copyright, I'd strongly suggest releasing the
>>Debian packaging under the same license. For me that's a blocker for
>>sponsoring the package (it may be ok for others to sponsor).
>
>OK. I prefer using GPL-3+, but nevermind, I would really like to get
>this package in sid soon. Other sponsors are still to be found, so I 
>turned it to Expat.

As a member of the FTP Team, I see this situation from time to time.  It's 
usually not a serious problem, but I agree with Zigo on this.

Where it becomes problematic is when code in debian/ becomes intermingled with 
the installed upstream code.  One has to look at the license of both upstream 
and debian/ to determine the effective license of the package.  If the licenses 
are incompatible, then the binary isn't distributable.  I have seen this happen.

Aligning the license of the packaging with the upstream license makes it all 
much simpler.

...
>>Also, please remove:
>>
>>[import-orig]
>>merge-mode = replace
>>
>>from debian/gbp.conf. This is typically something that goes into
>>~/.gbp.conf rather than on individual packages.
>
>I don't agree with you on this one. To me, this kind of setting should 
>be put in the package, to have an uniform way of updating/building/etc.
>
>a given package, whoever the uploader could be.
>
...
>fixed the issues you found and that you will agree with my argument for
>the debian/gbp.conf setting.

If you intend to maintain this package in DPMT or PAPT, we have a standard 
gbp.conf for the teams that doesn't include this, so it would be inappropriate. 
 In any case, it's the default for a format 3.0(Quilt) source package, so 
there's no need for it anyway:

https://manpages.debian.org/testing/git-buildpackage/gbp-import-orig.1.en.html

Scott K



Re: Please grant me access to PAPT

2020-04-14 Thread Scott Kitterman
On Tuesday, April 14, 2020 11:33:29 AM EDT Alex Mestiashvili wrote:
> On 4/14/20 4:02 PM, Scott Kitterman wrote:
> > On Tuesday, April 14, 2020 8:25:36 AM EDT Alex Mestiashvili wrote:
> >> Hi,
> >> could please somebody who have enough privileges remove
> >> 
> >>  https://salsa.debian.org/python-team/modules/authprogs.git
> >> 
> >> It is an application and should be located in python-team/applications.
> > 
> > I've moved it:
> > 
> > https://salsa.debian.org/python-team/applications/authprogs
> > 
> > Scott K
> 
> Thanks! Would you also mind granting me access to PAPT as well?
> I'd like to maintain authprogs.
> I've read and accept the PAPT policy:
> https://salsa.debian.org/python-team/tools/python-apps/blob/master/policy.rs
> t and my salsa login is: mestia
> 
> I have requested to join PAPT some time ago:
> https://lists.debian.org/debian-python/2020/04/msg00032.html

Done.

Welcome to the team.

Scott K

signature.asc
Description: This is a digitally signed message part.


Python3.8 Transition Lessons Learned

2020-04-14 Thread Scott Kitterman
In the course of dropping python3.7 from supported versions binNMUs were 
scheduled before the python3-defaults update dropping python3.7 migrated to 
testing.

These binNMUs migrated to Testing immediately.  This resulted in a case where 
in Testing, python3.7 and python3.8 were supported versions, but packages had 
lost their python3.7 support.  This caused autopkgtest failures which appeared 
as regressions.

In the future, I think it would be better to upload python3-defaults dropping 
the old version, let it migrate to Testing, and then schedule binNMUs.  That 
would result in less churn.

This is sent to more than one list, so please pay attention to which list you 
are replying.

Scott K
P. S. I am not subscribed to debian-release, so please cc me on any replies 
for that list.

signature.asc
Description: This is a digitally signed message part.


Re: Please remove python-team/modules/authprogs.git

2020-04-14 Thread Scott Kitterman
On Tuesday, April 14, 2020 8:25:36 AM EDT Alex Mestiashvili wrote:
> Hi,
> could please somebody who have enough privileges remove
> 
>  https://salsa.debian.org/python-team/modules/authprogs.git
> 
> It is an application and should be located in python-team/applications.

I've moved it:

https://salsa.debian.org/python-team/applications/authprogs

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: PEP-517/PEP-518 Support In Debian

2020-04-13 Thread Scott Kitterman
On Monday, April 13, 2020 5:18:53 AM EDT Dmitry Shachnev wrote:
> Hi Scott!
> 
> On Sun, Apr 12, 2020 at 06:31:57PM -0400, Scott Kitterman wrote:
> > This being roughly the mid-point in the development cycle, I thought it
> > might be good to see where we are in terms of future Python packaging
> > developments.
> > 
> > As I understand it, PEP-517 and PEP-518 are 'the future'.
> > 
> > I took a look at the presence of pyproject.toml files in the Debian
> > archive. There are currently 99 packages.  Of those, only 28 specify a
> > 'build-backend', which is required by 517/518 to be useful for building a
> > package.
> > 
> > 25 specify: build-backend = "setuptools.build_meta" (including setuptools)
> > 3 specify: build-backend = "flit_core.buildapi" (including flit)
> 
> pyqt5 and pyqt5webengine specify: build-backend = "sipbuild.api".

So they do.  They didn't show up in my codesearch.d.n results, that makes me 
wonder what else I missed.  If anyone has an idea of a better way to track 
this, please speak up.

> > If build-backend is not specified, the build system has to fall back to
> > setup.py.
> > 
> > I've recently package flit (just arrived in Testing) and written a flit
> > plugin for pybuild that's pending review for merging[1].  I also packaged
> > pep517 for our pip package, so that's available to support future Debian
> > tool development in this area.

P1otr merged the flit plugin a little while ago, so it'll be in the next dh-
python upload.

> > For the moment, I guess we are in reasonable shape, but it might be useful
> > to have a pybuild plugin to use PEP517/setuptools.build_meta in lieu of
> > setup.py with setuptools/distutils when available.  In the future, this
> > will be the primary API and the sooner we start to use it, the better.
> 
> One issue that comes to mind: how will we specify the install location in a
> way that will work with any backend? In other words, what is the replacement
> for distutils' --install-layout=deb?

I think the answer to that question is going to be build-backend specific.

The good news is that the legacy processing for setup.py with setuptools/
distutils will be around approximately forever, so a complete answer to the 
question isn't urgent.

One related question is if we are willing to bring pip into our package build 
process?  I took a quick look at python3-pep517 to see how hard it would be to 
add support for build-backend = "setuptools.build_meta"  to pybuild.  My quick 
look answer is not too hard if we are willing to bring pip into the process. 
If we aren't willing to involve pip (which does bring a lot of complexity), I 
think there will be substantially more Debian specific code required to support 
it.

I expect build-backend = "sipbuild.api" will need custom support as well, but 
I haven't looked into it at all.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: PEP-517/PEP-518 Support In Debian

2020-04-12 Thread Scott Kitterman



On April 13, 2020 3:44:31 AM UTC, Paul Wise  wrote:
>On Sun, Apr 12, 2020 at 10:32 PM Scott Kitterman wrote:
>
>> I took a look at the presence of pyproject.toml files in the Debian
>archive.
>> There are currently 99 packages.  Of those, only 28 specify a
>'build-backend',
>> which is required by 517/518 to be useful for building a package.
>
>Is there a tool that can report problems in pyproject.toml files?
>Either way I think that lintian needs to detect this particular issue.

Not that I know of, but I don't think it's time yet.

They're non-build uses for pyproject.toml, so if build-backend isn't present, 
it just means that it isn't used for build reasons.

The pep517 package (python3-pep517) can validate metadata generated based on 
pyproject.toml, but it spins up it's own environment using pip and does a 
package sdist build to do so.  I don't think that's what you have in mind.

In the pybuild plugin I did for flit, I do check for the correct build-backend 
when autodetecting if the plugin should be used.  Using python3-toml it's a one 
liner.  If we really want to head down rlthe path of checking, it ought to be 
simple enough to write.

Scott K



Re: py2removal: drop python-pytest by not running tests for python2 packages

2020-04-12 Thread Scott Kitterman
On Sunday, April 12, 2020 9:56:01 PM EDT Sandro Tosi wrote:
> Hello,
> python-pytest is blocking 18 packages from removal, most of them would
> be leaves once python-pytest is gone.
> 
> so i was playing with the idea of tackling python-pytest removal by
> updating all its rdeps and not run unittests for the python2 binary
> (so dropping pytest and the other b-d* only used for tests).
> 
> I know it's suboptimal (some python2 packages can still see updates
> until we're ready to drop them and running tests can still have
> value), but i think the cost/benefit ratio points towards removing
> python-pytest soon rather than wait.
> 
> There are only 25 packages that would need updating, and most of them
> are in DPMT/PAPT.

Go for it.

Scott K

signature.asc
Description: This is a digitally signed message part.


PEP-517/PEP-518 Support In Debian

2020-04-12 Thread Scott Kitterman
This being roughly the mid-point in the development cycle, I thought it might 
be good to see where we are in terms of future Python packaging developments.

As I understand it, PEP-517 and PEP-518 are 'the future'.

I took a look at the presence of pyproject.toml files in the Debian archive.  
There are currently 99 packages.  Of those, only 28 specify a 'build-backend', 
which is required by 517/518 to be useful for building a package.

25 specify: build-backend = "setuptools.build_meta" (including setuptools)
3 specify: build-backend = "flit_core.buildapi" (including flit)

If build-backend is not specified, the build system has to fall back to 
setup.py.

I've recently package flit (just arrived in Testing) and written a flit plugin 
for pybuild that's pending review for merging[1].  I also packaged pep517 for 
our pip package, so that's available to support future Debian tool development 
in this area.

For the moment, I guess we are in reasonable shape, but it might be useful to 
have a pybuild plugin to use PEP517/setuptools.build_meta in lieu of setup.py 
with setuptools/distutils when available.  In the future, this will be the 
primary API and the sooner we start to use it, the better.

Scott K

[1] https://salsa.debian.org/python-team/tools/dh-python/-/merge_requests/12

signature.asc
Description: This is a digitally signed message part.


Re: Fwd: [covid] New Contributor for biohackathon

2020-04-08 Thread Scott Kitterman
On Wednesday, April 8, 2020 11:56:22 PM EDT Olek Wojnar wrote:
> Dear DPMT members,
> 
> You may be aware that the Debian Med team is in the middle of a
> biohackathon [1] to provide better FOSS tools to the medical community for
> their work on the COVID-19 pandemic. We just had a new contributor package
> a Python module that we need for one of our packages. Since time is of the
> essence in these efforts, would you be willing/able to accept this package
> under your team and do a quick review? If you are unable to do a quick
> review, would you agree to have one of the DDs on the Debian Med team
> review the package and upload it with you as the maintainer? Thanks in
> advance for anything that you can do to support our efforts!
> 
> -Olek
> 
> PS I am subscribed to Debian Med but not to Debian Python
> 
> [1]
> https://salsa.debian.org/med-team/community/2020-covid19-hackathon/-/wikis/C
> ovid-19-hackathon

I took care of the request, so they are in the team now and can push the 
package to the team repo.  Once that's done they can request sponsorship via 
our usual process (either RFS mail to debian-python@l.d.o or put the name of 
the package in /topic on #debian-python).  The team is usually pretty 
responsive about sponsoring.

I will try to stay away from sponsoring it since if I do that, I won't be able 
to process it through New.

Scott K

> -- Forwarded message -
> From: fancycade 
> Date: Wed, Apr 8, 2020 at 10:42 PM
> Subject: [covid] New Contributor for biohackathon
> To: debian-...@lists.debian.org 
> 
> 
> Hi there!
> 
> I would like to help out with the efforts of the biohackathon. Sadly I see
> I missed the jitsi meetings. I was busy making my first debian package. I
> have a pretty good understanding of the debian policy, but I'm sure there
> is room for improvement.
> 
> I was following the need for packaging the CHIME project which requires the
> streamlit package. After taking a look at the dependencies for streamlit I
> saw there were only a few missing packages. One of them being validators.
> Since this was a simple python module, I figured it would be a good first
> package to try my hand.
> 
> upstream: https://github.com/kvesteri/validators
> 
> I have submitted an ITP to the bug tracker
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=956082
> 
> And have a building package on salsa:
> https://salsa.debian.org/fancycade-guest/validators
> 
> At this point I think I need a review and someone to sponsor it. I have
> also submitted a request to join the Debian Python Modules team, but have
> not gotten a reply yet.
> 
> 
> Thanks!
> 
> - Harley Swick



signature.asc
Description: This is a digitally signed message part.


Re: Request to join Python Modules Team

2020-04-08 Thread Scott Kitterman
On Saturday, April 4, 2020 5:53:30 PM EDT fancycade wrote:
> Hi there!
> 
> I would like to join the Python Modules Team. For the time being I am
> interested in maintaining packages related to the upcoming covid
> biohackathon. Specifically I am trying to get my feet wet by packaging this
> module https://github.com/kvesteri/validators which is needed by the
> streamlit package.
> 
> I am familiar with Debian packaging, and already have a package that builds.
> 
> My salsa login is: @fancycade-guest
> 
> I have read the Python Modules policy.
> 
> Thanks!

Welcome to the team.

Scott K

signature.asc
Description: This is a digitally signed message part.


Bug#956184: RFP: python3-sphinx-better-theme -- modified version of the default Sphinx theme

2020-04-07 Thread Scott Kitterman
Package: wnpp
Severity: wishlist

* Package name: python3-sphinx-better-theme
  Version : 0.1.5
  Upstream Author : Steve Johnson 
* URL : https://pypi.org/project/sphinx-better-theme
* License : Expat
  Programming Lang: Python
  Description : modified version of the default Sphinx theme

This theme is now used by psycopg2 for its documentation, so it would be
nice to see it in Debian so we could use it (instead of patching to use
the default Sphinx Theme.

 sphinx-better-theme is an update to the default Sphinx theme with the
 following goals:
 .
  - Remove frivolous colors, especially hard-coded ones
  - Improve readability by limiting width and using more whitespace
  - Encourage visual customization through CSS, not themeconf
Use semantic markup



Re: Telepathy-gabble and python2-rm

2020-04-06 Thread Scott Kitterman
On Monday, April 6, 2020 8:17:53 PM EDT Sandro Tosi wrote:
> On Fri, Apr 3, 2020 at 5:04 PM Scott Kitterman  wrote:
> > There was a telepathy-gabble upload this week that took out a bunch of
> > python2 build-deps, but it looks like telepathy-gabble-tests retained
> > many of them as depends:
> > 
> > https://packages.debian.org/unstable/telepathy-gabble-tests
> > 
> > In particular, this is the last package in Testing that depends on python-
> > twisted.  If this could go away, then a bunch of stuff could get
> > decrufted.
> 
> that's just been taken care of, sorry it took so long

Thanks,

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: Python3 modules not built for all supported Python versions

2020-04-06 Thread Scott Kitterman
On Monday, April 6, 2020 9:24:35 AM EDT Jochen Sprickerhof wrote:
> Hi Emilio,
> 
> * Emilio Pozuelo Monfort  [2020-03-30 17:47]:
> >I've heard pybuild now has a cmake backend, so theoretically you could do
> >something like
> >
> >%:
> > dh $@ --buildsystem=pybuild --system=cmake
> 
> [..]
> 
> >I don't know if I'm missing an argument to dh_python3 so that it knows the
> >python version, or even if there's a better workaround. But perhaps pybuild
> >should be doing this automatically between the dh_auto_install calls so
> >that this kind of workarounds aren't necessary.
> 
> Thanks for looking into this and providing a patch!
> I polished it a little:
> 
> https://salsa.debian.org/science-team/ros-geometry2/-/commit/11739091abe5800
> 7485772492fddbdc1a12a59c6
> 
> and uploaded a working version to the archive. Will fix all ros-*
> packages now.

You may have noticed that the pybuild cmake plug-in is somewhat under 
documented.  While this is fresh in your mind, it might be useful for you to 
consider what documentation would have made this easier for you to figure out.

https://salsa.debian.org/python-team/tools/dh-python

Scott K

signature.asc
Description: This is a digitally signed message part.


Telepathy-gabble and python2-rm

2020-04-03 Thread Scott Kitterman
There was a telepathy-gabble upload this week that took out a bunch of python2 
build-deps, but it looks like telepathy-gabble-tests retained many of them as 
depends:

https://packages.debian.org/unstable/telepathy-gabble-tests

In particular, this is the last package in Testing that depends on python-
twisted.  If this could go away, then a bunch of stuff could get decrufted.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: py2removal: proposal to increase apps popcon limit

2020-03-31 Thread Scott Kitterman
On Tuesday, March 31, 2020 9:54:41 AM EDT Sandro Tosi wrote:
> >> Maybe you can paste the list of 20 affected apps, though?
> > 
> > I'm more interested in the 18 that are above 1000.  Could you please just
> > list them all 38, sorted by popocon?
> here's the list:
> 
> cvs2svn - popcon = 303,
> chirp - popcon = 350,
> gnat-gps - popcon = 354,
> cfv - popcon = 387,
> fretsonfire-game - popcon = 394,
> neopi - popcon = 398,
> kcachegrind-converters - popcon = 414,
> virt-goodies - popcon = 430,
> freeorion - popcon = 433,
> syncthing-gtk - popcon = 435,
> postnews - popcon = 557,
> pssh - popcon = 562,
> displaycal - popcon = 562,
> archivemail - popcon = 674,
> syslog-summary - popcon = 692,
> xboxdrv - popcon = 696,
> trash-cli - popcon = 753,
> fsl-5.0-core - popcon = 822,
> denyhosts - popcon = 852,
> geda-utils - popcon = 938,
> mirage - popcon = 1006,
> gnumeric-plugins-extra - popcon = 1110,
> getmail - popcon = 1191,
> mailman - popcon = 1344,
> smart-notifier - popcon = 1362,
> xchat - popcon = 1606,
> mysql-workbench - popcon = 1688,
> calligra-libs - popcon = 2296,
> mysql-utilities - popcon = 2448,
> lokalize - popcon = 2598,
> libboost-mpi-python1.67.0 - popcon = 2691,
> bleachbit - popcon = 2773,
> qbittorrent - popcon = 3402,
> libapache2-mod-python - popcon = 3778,
> nagios-plugins-contrib - popcon = 4059,
> playonlinux - popcon = 4781,
> sugar-browse-activity - popcon = 6421,
> ipython - popcon = 8296,

Thanks.  I believe ipython will be gone shortly.

I do not see any problems with bumping to 1,000 now.

For the ones > 1,000, I think we should do some assessment of what needs to 
happen on a case by case basis.  Some of those, I suspect, only use python2 
for very limited purposes that could either be dropped or updated to python3.

Scott K





Re: Python3.8 as default final status

2020-03-28 Thread Scott Kitterman



On March 28, 2020 5:10:42 AM UTC, Sergio Durigan Junior  
wrote:
>On Friday, March 27 2020, Håvard Flaget Aasen wrote:
>
>> On 27.03.2020 20:09, Sergio Durigan Junior wrote:
>>> On Friday, March 27 2020, Scott Kitterman wrote:
>>> 
>>>> The python3-defaults with python3.8 as the default python3 has
>migrated to 
>>>> Testing thanks to the release team hammering things around until it
>went.
>>> 
>>> Thanks for this.
>>> 
>>>> Most of the outstanding autipkgtest failures with python3.8 were
>fixed either 
>>>> in unstable or in git/BTS.  Here are the remaining issues that
>someone (who 
>>>> isn't me) should have a look at:
>>>>
>>>> celery/4.2.1-5: #952217 autorm 4/13
>>> 
>>> FWIW, I looked at this a little bit, but could not make much
>progress.
>>> I'm very interested in fixing this since it impacts pagure.  I'll
>try to
>>> investigate more this weekend, but if someone else wants to take a
>look
>>> (and let me know), you're more than welcome!
>>> 
>>
>> I believe I already fixed that package, it's waiting for someone to
>> review and upload it. Did you look at the repository in salsa?
>
>I had looked at the repository when I was working with the package.
>I see you pushed your changes 2 days ago, but the last time I looked at
>the package was at least 7 days ago.
>
>Anyhow, I thank you for letting me know, but I am not sure I am
>satisfied with the solution.  You basically disabled the test on Python
>3.8, which obviously works, but doesn't really tell me whether there
>was
>indeed a problem with the package/testcase or not.

I completely agree.  It's just papering over the problem.  It's not in the 
spirit of the Debian Social Contract (#3).

>My approach (failed, so far) was to try and figure out what was
>happening, and then devise a proper fix for it.  My next step was going
>to be to involve upstream in this.
>
>Would you like to follow up with them and check if they're are aware of
>the failure?  Maybe they already have a proper solution for it.

Upstream should definitely be involved.

Scott K



Re: Python3.8 as default final status

2020-03-27 Thread Scott Kitterman
On Friday, March 27, 2020 3:39:30 PM EDT Scott Kitterman wrote:
> On Friday, March 27, 2020 3:21:38 PM EDT Håvard Flaget Aasen wrote:
> > On 27.03.2020 20:09, Sergio Durigan Junior wrote:
> > > On Friday, March 27 2020, Scott Kitterman wrote:
> > >> The python3-defaults with python3.8 as the default python3 has migrated
> > >> to
> > >> Testing thanks to the release team hammering things around until it
> > >> went.
> > > 
> > > Thanks for this.
> > > 
> > >> Most of the outstanding autipkgtest failures with python3.8 were fixed
> > >> either in unstable or in git/BTS.  Here are the remaining issues that
> > >> someone (who isn't me) should have a look at:
> > >> 
> > >> celery/4.2.1-5: #952217 autorm 4/13
> > > 
> > > FWIW, I looked at this a little bit, but could not make much progress.
> > > I'm very interested in fixing this since it impacts pagure.  I'll try to
> > > investigate more this weekend, but if someone else wants to take a look
> > > (and let me know), you're more than welcome!
> > 
> > I believe I already fixed that package, it's waiting for someone to
> > review and upload it. Did you look at the repository in salsa?
> > 
> > 
> > Håvard
> 
> I didn't.  The ones I knew about having fixed in git or patches in the BTS,
> I didn't include.  I didn't go through and check them all.
> 
> Scott K

I should read more carefully.  I failed to note that this wasn't directed at 
me.  Nevermind.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: Python3.8 as default final status

2020-03-27 Thread Scott Kitterman
On Friday, March 27, 2020 3:21:38 PM EDT Håvard Flaget Aasen wrote:
> On 27.03.2020 20:09, Sergio Durigan Junior wrote:
> > On Friday, March 27 2020, Scott Kitterman wrote:
> >> The python3-defaults with python3.8 as the default python3 has migrated
> >> to
> >> Testing thanks to the release team hammering things around until it went.
> > 
> > Thanks for this.
> > 
> >> Most of the outstanding autipkgtest failures with python3.8 were fixed
> >> either in unstable or in git/BTS.  Here are the remaining issues that
> >> someone (who isn't me) should have a look at:
> >> 
> >> celery/4.2.1-5: #952217 autorm 4/13
> > 
> > FWIW, I looked at this a little bit, but could not make much progress.
> > I'm very interested in fixing this since it impacts pagure.  I'll try to
> > investigate more this weekend, but if someone else wants to take a look
> > (and let me know), you're more than welcome!
> 
> I believe I already fixed that package, it's waiting for someone to
> review and upload it. Did you look at the repository in salsa?
> 
> 
> Håvard

I didn't.  The ones I knew about having fixed in git or patches in the BTS, I 
didn't include.  I didn't go through and check them all.

Scott K

signature.asc
Description: This is a digitally signed message part.


Python3.8 as default final status

2020-03-27 Thread Scott Kitterman
The python3-defaults with python3.8 as the default python3 has migrated to 
Testing thanks to the release team hammering things around until it went.

Most of the outstanding autipkgtest failures with python3.8 were fixed either 
in unstable or in git/BTS.  Here are the remaining issues that someone (who 
isn't me) should have a look at:

celery/4.2.1-5: #952217 autorm 4/13
circlator/1.5.6-1: 954403 against joblib
fades/8.1-1: #950858 autorm 3/29
fdroidserver/1.1.6-1: #954395 
meson/0.52.1-1: #952610
natsort/6.0.0-1.2: #950221 new upstream needs packaging autorm 4/13
python-icecream/1.3.1-1: #950847 needs new upstream autorm 3/29
qiime/2019.10.0-1: #950842 autorm 4/13
spades/3.13.1+dfsg-2: 954403 against joblib
vulture/0.21-1.1: #950861 needs new upstream autorm 4/13
xapers/0.8.2-1.1: #954805

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: python3-openvdb: build against the default python3 version

2020-03-27 Thread Scott Kitterman
Pybuild has a cmake plugin that I think is relatively new.  It may handle the 
complexity for you, but I've never tried it.  We used to do this with pykde4, 
which is available on snapshot..d.o.

Scott K

On Friday, March 27, 2020 8:48:33 AM EDT Mathieu Malaterre wrote:
> Hi all,
> 
> Could anyone point me at a Debian package (possibly using dh) where
> multiple compilation are done (one against python3.7 and one against
> python3.8). I'd like to avoid re-inventing the wheel for building a
> cmake package (c++) which ships a single python module (so I need to
> do one configure+build+install per python version).
> 
> Thanks,
> 
> On Fri, Mar 27, 2020 at 12:57 PM Emilio Pozuelo Monfort
> 
>  wrote:
> > Control: reopen -1 7.0.0-1
> > Control: retitle -1 python3-openvdb: build against the default python3
> > version> 
> > On Mon, 24 Feb 2020 11:10:49 +0100 Mathieu Malaterre  
wrote:
> > > Control: tags -1 + patch
> > > Control: retitle -1 replace python3-all-dev with python3.7-dev
> > 
> > Err, that's not really the solution.
> > 
> > The not ideal solution was to build for the default python version, i.e.
> > build-depend on python3-dev and use python3. That would have built against
> > python3.7 when that was the default, and against python3.8 now that it has
> > changed. And with a binNMU from the release team, you wouldn't have even
> > noticed.
> > 
> > The ideal solution is to build against python3-all-dev and build for *all*
> > supported python versions. So that when python3.7 and python3.8 are both
> > supported, you build the python extension for both of them.
> > 
> > Cheers,
> > Emilio



signature.asc
Description: This is a digitally signed message part.


Re: Python help needed for test suite in multiqc

2020-03-26 Thread Scott Kitterman



On March 26, 2020 7:02:20 AM UTC, Andreas Tille  wrote:
>Hi Andrey,
>
>On Thu, Mar 26, 2020 at 12:34:11AM +0500, Andrey Rahmatullin wrote:
>> http://bugs.debian.org/954640
>
>Thanks a lot.  Multiqc builds with humanfriendly from Git.
>
>Kind regards
>
>   Andreas.

The bug is fixed in Unstable now.

Scott K



Re: Autopkgtest issues blocking python3.8 as default transition

2020-03-24 Thread Scott Kitterman
On Monday, March 23, 2020 12:30:15 PM EDT Scott Kitterman wrote:
> On Sunday, March 22, 2020 1:10:35 AM EDT Scott Kitterman wrote:
> > On Saturday, March 21, 2020 2:28:15 PM EDT Scott Kitterman wrote:
> > > On Saturday, March 21, 2020 5:44:15 AM EDT Graham Inggs wrote:
> > > > Hi Scott
> > > > 
> > > > On Sat, 21 Mar 2020 at 06:41, Scott Kitterman 
> > 
> > wrote:
> > > > > I've gone through and statused the issues currently listed on the
> > > > > package
> > > > > tracker for python3-defaults migration that are autopkgtest related
> > > > > [1].
> > > > > I
> > > > > stuffed it into a pastebin [2].
> > > > 
> > > > Thanks for your work on this!
> > > > 
> > > > I filed bug #954395 against fdroidserver
> > > > I filed bug #954403 against joblib affecting circlator and spades
> > > > siconos already has a FTBFS bug #952601
> > > > I marked sagemath FTBFS bug #950147 affecting brial and sagetex
> > > > 
> > > > Regards
> > > > 
> > > > Graham
> > > 
> > > Thank you.  Updated http://paste.debian.net/1135879/
> > > 
> > > Scott K
> > 
> > Updated again: http://paste.debian.net/1135939/
> > 
> > There was just a new python3-defaults upload, so everything shows as in
> > progress against hte new python3-defaults revision.  I'd be surprised if
> > any of the ones shown here don't come back once it's all processed.
> > 
> > Scott K
> 
> Things are mostly sorted out from the last python3-defaults upload.  I've
> updated the list again:
> 
> http://paste.debian.net/1136190/
> 
> The ones marked NEW need someone to look at the logs and figure out what the
> issue is.  Several fell of the list due to have been fixed, so the list is
> not as much longer as the number of NEW issues would indicate.

Update for today:

http://paste.debian.net/1136336/

Many of the new issues from yesterday dropped off.  Presumably they were 
transients of some kind.  I've investigated the remaining issues, so no more 
unknowns.  Plenty of bugs to fix though ...

Scott K


signature.asc
Description: This is a digitally signed message part.


Re: Autopkgtest issues blocking python3.8 as default transition

2020-03-23 Thread Scott Kitterman
On Sunday, March 22, 2020 1:10:35 AM EDT Scott Kitterman wrote:
> On Saturday, March 21, 2020 2:28:15 PM EDT Scott Kitterman wrote:
> > On Saturday, March 21, 2020 5:44:15 AM EDT Graham Inggs wrote:
> > > Hi Scott
> > > 
> > > On Sat, 21 Mar 2020 at 06:41, Scott Kitterman 
> 
> wrote:
> > > > I've gone through and statused the issues currently listed on the
> > > > package
> > > > tracker for python3-defaults migration that are autopkgtest related
> > > > [1].
> > > > I
> > > > stuffed it into a pastebin [2].
> > > 
> > > Thanks for your work on this!
> > > 
> > > I filed bug #954395 against fdroidserver
> > > I filed bug #954403 against joblib affecting circlator and spades
> > > siconos already has a FTBFS bug #952601
> > > I marked sagemath FTBFS bug #950147 affecting brial and sagetex
> > > 
> > > Regards
> > > 
> > > Graham
> > 
> > Thank you.  Updated http://paste.debian.net/1135879/
> > 
> > Scott K
> 
> Updated again: http://paste.debian.net/1135939/
> 
> There was just a new python3-defaults upload, so everything shows as in
> progress against hte new python3-defaults revision.  I'd be surprised if any
> of the ones shown here don't come back once it's all processed.
> 
> Scott K

Things are mostly sorted out from the last python3-defaults upload.  I've 
updated the list again:

http://paste.debian.net/1136190/

The ones marked NEW need someone to look at the logs and figure out what the 
issue is.  Several fell of the list due to have been fixed, so the list is not 
as much longer as the number of NEW issues would indicate.

Scott K


signature.asc
Description: This is a digitally signed message part.


Re: Autopkgtest issues blocking python3.8 as default transition

2020-03-21 Thread Scott Kitterman
On Saturday, March 21, 2020 2:28:15 PM EDT Scott Kitterman wrote:
> On Saturday, March 21, 2020 5:44:15 AM EDT Graham Inggs wrote:
> > Hi Scott
> > 
> > On Sat, 21 Mar 2020 at 06:41, Scott Kitterman  
wrote:
> > > I've gone through and statused the issues currently listed on the
> > > package
> > > tracker for python3-defaults migration that are autopkgtest related [1].
> > > I
> > > stuffed it into a pastebin [2].
> > 
> > Thanks for your work on this!
> > 
> > I filed bug #954395 against fdroidserver
> > I filed bug #954403 against joblib affecting circlator and spades
> > siconos already has a FTBFS bug #952601
> > I marked sagemath FTBFS bug #950147 affecting brial and sagetex
> > 
> > Regards
> > 
> > Graham
> 
> Thank you.  Updated http://paste.debian.net/1135879/
> 
> Scott K

Updated again: http://paste.debian.net/1135939/

There was just a new python3-defaults upload, so everything shows as in 
progress against hte new python3-defaults revision.  I'd be surprised if any 
of the ones shown here don't come back once it's all processed.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: Autopkgtest issues blocking python3.8 as default transition

2020-03-21 Thread Scott Kitterman
On Saturday, March 21, 2020 5:44:15 AM EDT Graham Inggs wrote:
> Hi Scott
> 
> On Sat, 21 Mar 2020 at 06:41, Scott Kitterman  wrote:
> > I've gone through and statused the issues currently listed on the package
> > tracker for python3-defaults migration that are autopkgtest related [1]. 
> > I
> > stuffed it into a pastebin [2].
> 
> Thanks for your work on this!
> 
> I filed bug #954395 against fdroidserver
> I filed bug #954403 against joblib affecting circlator and spades
> siconos already has a FTBFS bug #952601
> I marked sagemath FTBFS bug #950147 affecting brial and sagetex
> 
> Regards
> 
> Graham

Thank you.  Updated http://paste.debian.net/1135879/

Scott K

signature.asc
Description: This is a digitally signed message part.


Autopkgtest issues blocking python3.8 as default transition

2020-03-20 Thread Scott Kitterman
I've gone through and statused the issues currently listed on the package 
tracker for python3-defaults migration that are autopkgtest related [1].  I 
stuffed it into a pastebin [2].  I'm open to suggestions about where better to 
maintain such a list.

If you ever want python3.8 as default python3, please have a look and fix 
stuff.  
The ones labled py3versions -i are likely related to a common error I wrote to 
on debian-devel recently [3].  They are easy to fix.

If you update this, please also update /topic in #debian-python as it is 
listed there also.  This was painful to do.  I'm unlikely to do it again 
(things like this are one of the reason I no longer co-maintain python3-
defualts).  It'll make everyone's life easier if we document our progress and 
don't stumble over each other.

Scott K

[1] https://tracker.debian.org/pkg/python3-defaults
[2] http://paste.debian.net/1135796/ 
[3] https://lists.debian.org/debian-devel/2020/03/msg00280.html

signature.asc
Description: This is a digitally signed message part.


Re: Bug#952952: RM: src:ipykernel-py2 -- RoM for Python 2 removal

2020-03-02 Thread Scott Kitterman
On Monday, March 2, 2020 9:14:52 AM EST Sandro Tosi wrote:
> > The src:ipykernel-py2 package provides python-ipykernel, which is to be
> > removed for the Python 2 removal transition.
> 
> Cc-ing Gordon explicitly: the last time we spoke, Gordon wanted to
> keep the python2 stack of ipython/ikernel/jupyther for people to still
> have a py2 infrastructure to run their notebooks. Is this changed? are
> we ready to remove them all?
> 
> Regards,

Personally, I think it should go, but I agree that's what Gordon said he 
wanted.  I've tagged the bug moreinfo so it won't be processed by the FTP Team 
while this is being sorted out.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: Request to (re)-join DPMT and PAPT

2020-03-01 Thread Scott Kitterman
On Friday, February 28, 2020 2:56:24 PM EST Scott Talbert wrote:
> Hi,
> 
> I am currently a member as swt2c-guest, but I recently became a DD, so can
> I please be re-added to DPMT and PAPT under swt2c.
> 
> I still accept all policies.  :)
> 
> Thanks,
> Scott

Done.  I set an expiration of a week from now for swt2c-guest in case you 
still need it for some reason in the transition.

Scott K


signature.asc
Description: This is a digitally signed message part.


Re: Request to join DPMT

2020-03-01 Thread Scott Kitterman
On Friday, February 28, 2020 2:28:29 PM EST Luca Boccassi wrote:
> Hello,
> 
> I'd like to join the DPMT group on Salsa, please.
> 
> I currently maintain 15 Python modules uploaded (repos are in the
> general debian group) that I'd like to move to the group, and most
> importantly I have accepted to help with existing Azure modules
> (python-azure, which is in the DPMT group).
> 
> I have read and accept the policy.rst - if accepted, I will update the
> branch policy of my modules to match the policy (mainly
> s|debian/sid|debian/master|) and update the Maintainer field,
> everything else already matches.
> 
> Thank you!

Welcome to the team.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: Joining the team (again)

2020-03-01 Thread Scott Kitterman
On Sunday, March 1, 2020 2:53:13 PM EST Georg Faerber wrote:
> Dear all,
> 
> I was part of the team in the past as 'georg-guest', I would like to
> join again as 'georg'. I do maintain anorack and mwic.
> 
> I've read the team policy and accept it.
> 
> Thanks,
> Georg

Welcome back.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: Separate lib and executable packages?

2020-02-08 Thread Scott Kitterman
On Saturday, February 8, 2020 2:50:54 PM EST Gordon Ball wrote:
> On Sat, Feb 08, 2020 at 02:42:11AM +0000, Scott Kitterman wrote:
> > On February 7, 2020 3:59:46 PM UTC, Mattia Rizzolo  
wrote:
> > >On Fri, Feb 07, 2020 at 11:36:00AM +, Gordon Ball wrote:
> > >> I wonder if this split really makes sense; it feels like adding the
> > >> overhead of an extra binary package to avoid not having a very small
> > >> file in /usr/bin if you're only planning to use the library.
> > >> 
> > >> Does it seem reasonable to drop the executable package and just make
> > >
> > >it
> > >
> > >> a Provides: of the library package? (and is there any potential
> > >
> > >breakage
> > >
> > >> here that I'm overlooking)
> > >
> > >Maybe not for ipython3, since that's very much tied to
> > >python3-ipython3.
> > >
> > >BUT, as a user (even forgetting I'm also a DD) I was hurt many times by
> > >executables in python-foo but wanting to use python3-foo instead, or by
> > >executables that moved from python-foo to python3-foo and I had to fix
> > >my own deployments, and whatnot.
> > >
> > >We are not going to have a python4 anytime soon (hopefully never), but
> > >I
> > >think that keeping a separate binary package makes sense for almost all
> > >cases I can think of packages shipping executables under /usr/bin/.
> > 
> > Except: Every time we add a binary to Debian it impacts everyone.  The
> > packages file gets bigger and so updates get slower.  Although there's no
> > firm rule, the FTP Masters discourage addition of trivially small
> > packages and so your package might be rejected.
> Perhaps this is worth making an explicit recommendation for new packages
> of this type, given that anything new _should_ be python3-only and we
> hope for at least some time before another python major version becomes
> an issue. "Packages which are primarily libraries but include an
> executable should avoid multiple binary packages without good reason"? -
> At least in the above linked style guide, since it's not normative
> policy in any case.

Sounds reasonable to me.  It's a wiki, so I'd say go for it.

> For packages already split (eg, ipython3, python3-ipython), having
> thought a bit more I suspect it's probably more trouble than it is worth
> to drop a handful of binary packages - I'm wondering about eg, what
> happens if the executable package is manually installed and the library
> hence flagged as automatically installed; if the former than becomes a
> Provides: of the latter, is it immediately a candidate for autoremoval?

Yes.  Removing the existing cases where it might not really be needed can be 
problematic.  I think it generally makes sense to leave them be.

Scott K

signature.asc
Description: This is a digitally signed message part.


Re: Separate lib and executable packages?

2020-02-08 Thread Scott Kitterman



On February 8, 2020 12:38:54 PM UTC, Ondrej Novy  wrote:
>Hi,
>
>so 8. 2. 2020 v 4:11 odesílatel Scott Kitterman 
>napsal:
>
>> Except: Every time we add a binary to Debian it impacts everyone. 
>The
>> packages file gets bigger and so updates get slower.  Although
>there's no
>> firm rule, the FTP Masters discourage addition of trivially small
>packages
>> and so your package might be rejected.
>>
>
>so let's fix infrastructure?

That's what everyone always suggests, but no one ever does.  AFAIK, there's no 
agreed design for secure package updates that doesn't require a local copy of 
the packages file.

If Debian could quit worrying about adding lots of little binaries, it would 
help in many areas.

Scott K



Re: Separate lib and executable packages?

2020-02-07 Thread Scott Kitterman



On February 7, 2020 3:59:46 PM UTC, Mattia Rizzolo  wrote:
>On Fri, Feb 07, 2020 at 11:36:00AM +, Gordon Ball wrote:
>> I wonder if this split really makes sense; it feels like adding the
>> overhead of an extra binary package to avoid not having a very small
>> file in /usr/bin if you're only planning to use the library.
>> 
>> Does it seem reasonable to drop the executable package and just make
>it
>> a Provides: of the library package? (and is there any potential
>breakage
>> here that I'm overlooking)
>
>
>Maybe not for ipython3, since that's very much tied to
>python3-ipython3.
>
>BUT, as a user (even forgetting I'm also a DD) I was hurt many times by
>executables in python-foo but wanting to use python3-foo instead, or by
>executables that moved from python-foo to python3-foo and I had to fix
>my own deployments, and whatnot.
>
>We are not going to have a python4 anytime soon (hopefully never), but
>I
>think that keeping a separate binary package makes sense for almost all
>cases I can think of packages shipping executables under /usr/bin/.

Except: Every time we add a binary to Debian it impacts everyone.  The packages 
file gets bigger and so updates get slower.  Although there's no firm rule, the 
FTP Masters discourage addition of trivially small packages and so your package 
might be rejected.

As an example, the dkimpy module that I maintain provides some scripts.  
Originally they were provided in python-dkim.  Now they are provided in 
python3-dkim (and since python-dkim is gone in unstable/testing, there's no 
more chance of users being confused about which one to install to get the 
scripts).

I recall a few questions by people that didn't know where to find the scripts.  
I could have had both packages provide it using update-alternatives, but the 
problem never seemed severe enough to be worth the work.

While splitting things out as suggested makes sense from the perspective of a 
single package, it has negative implications for the distro as a whole.  
Maintainers: please be mindful of the cumulative impact of the addition many 
small packages that aren't strictly needed.

Scott K



RE:pyqt5 and entrypoint

2020-02-02 Thread Scott Kitterman



On February 2, 2020 9:58:45 PM UTC, PICCA Frederic-Emmanuel 
 wrote:
>Hello
>
>on unstable it works but on buster (and we need to make it work for
>buster)
>we have this message
>
>Python 3.7.3 (default, Apr  3 2019, 05:39:12) 
>[GCC 8.3.0] on linux
>Type "help", "copyright", "credits" or "license" for more information.
 import pkg_resources
 pkg_resources.get_distribution('PyQt5')
>Traceback (most recent call last):
>  File "", line 1, in 
>File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line
>481, in get_distribution
>dist = get_provider(dist)
>File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line
>357, in get_provider
>   return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
>File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line
>900, in require
>needed = self.resolve(parse_requirements(requirements))
>File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line
>786, in resolve
>raise DistributionNotFound(req, requirers)
>pkg_resources.DistributionNotFound: The 'PyQt5' distribution was not
>found and is required by the application

Since you know that the dependency is satisfied due to your package 
dependencies in debian/control, you should be able to patch away the requires 
PyQt5 in setup.py and it'll work fine.

Scott K



Re: Updating pip

2020-01-24 Thread Scott Kitterman
On Friday, January 24, 2020 2:00:36 PM EST Scott Kitterman wrote:
> The next step is to deal with all the packages that pip vendors and we
> unvendor.  There are quite a few packages that need updating to a sufficient
> version for the new pip:
> 
> Needs packaging:
> pep517

I am taking care of this.

...
> 
> I would suggest that people who decide to work on one of the above reply to
> this message so we don't end up with people doing duplicate works.

Scott K

signature.asc
Description: This is a digitally signed message part.


Updating pip

2020-01-24 Thread Scott Kitterman
I started looking into updating pip to the current release because I knew it 
would take awhile and we need to do it at some point.  Merging our changes 
into the new upstream release turned out to be somewhat less challenging that 
I had feared (alternately it wasn't and I messed it up).  I've pushed the 
results of that effort to salsa.

The next step is to deal with all the packages that pip vendors and we 
unvendor.  There are quite a few packages that need updating to a sufficient 
version for the new pip:

Needs packaging:
pep517

Needs upgrade:
CacheControl
contextlib2
distro
ipaddress (python2 only)
msgpack
packaging
progress
pyparsing
pytoml
idna
urllib3

I'm not going to be able to deal with all that by myself, so this is a request 
for team members to step up and look at updating these packages.  I have not 
checked which of these are DPMT maintained, so in some cases the action may be 
to work with the maintainer to update their package.

I would suggest that people who decide to work on one of the above reply to 
this message so we don't end up with people doing duplicate works.

Thanks,

Scott K


Here's the full list:

appdirs==1.4.3 have 1.4.3
CacheControl==0.12.6 have 11.7.2
colorama==0.4.3 have 0.4.3
contextlib2==0.6.0 have 0.5.5
distlib==0.3.0 have 0.3.0
distro==1.4.0 have 1.3.0
html5lib==1.0.1 have 1.0.1
ipaddress==1.0.23 have 1.0.17  # Only needed on 2.6 and 2.7
msgpack==0.6.2 have 0.5.6
packaging==20.1 have 19.1.2
pep517==0.7.0 Missing (not packaged)
progress==1.5 have 1.2
pyparsing==2.4.6 have 2.4.2
pytoml==0.1.21 have 0.1.2
requests==2.22.0 have 2.22.0
certifi==2019.11.28 have 2019.11.28
chardet==3.0.4 have 3.0.4
idna==2.8 have 2.6
urllib3==1.25.7 have 1.25.6
retrying==1.3.3 have 1.3.3
setuptools==44.0.0 have 44.0.0
six==1.14.0 have 1.14.0
webencodings==0.5.1 have 0.5.1

signature.asc
Description: This is a digitally signed message part.


Re: Merge Requests

2019-12-06 Thread Scott Kitterman
It looks like a lot of this is lintian-brush generated and of questionable 
value, IMO.  As an example, bumping compat to 12 is trivial to do.  

The hard part is checking if it affects the package content, so the bot has 
produced a proposed change that will take far longer to verify than it would to 
produce.  If I had time to do the checking, I don't need a merge request for 
the trivial bit of changing one number.

I'd caution against just accepting such changes without careful review.

Scott K

On December 6, 2019 9:34:59 AM UTC, Jonathan Carter  wrote:
>Hey debian python team
>
>We currently have a few merge requests open:
>
>== tools ==
>
>Count: 6
>
>https://salsa.debian.org/groups/python-team/tools/-/merge_requests
>
>== papt ==
>
>Count: 8
>
>https://salsa.debian.org/groups/python-team/applications/-/merge_requests
>
>== dpmt ==
>
>Count: 31
>
>https://salsa.debian.org/groups/python-team/modules/-/merge_requests
>
>
>I merged a bunch of trivial ones yesterday, but even then it seems like
>we have some problems which might need some update in our policy in
>dealing with merge requests.
>
>I noticed that one MR fixed some typos but did it in the upstream
>source
>directly, which isn't all that useful to us.
>
>For other MRs, I noticed that many small changes in the packaging
>didn't
>have an associated changelog entry with it, so I had to dch to add a
>changelog entry. I think for small changes I'd prefer the person who
>submits the MR to add them. For larger ones it probably makes sense not
>to do that since it might take longer.
>
>Any suggestions? How about we draft some MR policy in gobby and get it
>added to the PAPT/DPMT policies?
>
>-Jonathan



Re: autopkgtest-pkg-python fails if package name is python-pyMODULENAME (Was: Bug#945768: python-pypubsub: autopkgtest failure: No module named 'pypubsub')

2019-11-28 Thread Scott Kitterman



On November 28, 2019 5:27:53 PM UTC, Simon McVittie  wrote:
>On Thu, 28 Nov 2019 at 11:15:31 -0500, Sandro Tosi wrote:
>> if you install `pubsub` as top-level module, your package must be
>> named pythonN-pubsub, if not it violates the policy and it's RC
>buggy.
>
>That's what I had thought, but I've also seen people asserting that the
>Debian package name ought to reflect the egg name in cases where it
>differs from the top-level Python module name.
>
>Some examples of where the difference between egg name and module name
>matters:
>
>- this one:
>  - module: pubsub (-> python3-pubsub)
>  - egg: pypubsub-*.egg-info (-> python3-pypubsub)
>  - is actually python3-pypubsub (named for the egg)
>
>- src:dbus-python:
>  - module: dbus (-> python3-dbus)
>  - egg: dbus_python-1.2.14.egg-info (-> python3-dbus-python)
>  - is actually python3-dbus (named for the module)
>
>- src:pygobject:
>  - module: gi (-> python3-gi) and pygtkcompat
>  - egg: PyGObject-3.34.0.egg-info (-> python3-pygobject)
>  - is actually python3-gi (named for the module)
>
>(Maybe python3-gi should also have Provides: python3-pygtkcompat?)
>
>Is there consensus that the top-level module name is what matters, and
>not
>following the recommendation is a bug?
>https://www.debian.org/doc/packaging-manuals/python-policy/module_packages.html
>says "The binary package for module foo should preferably be named
>python3-foo, if the module name allows" and "import foo should import
>the module", which suggests that it is indeed the name of the top-level
>importable module, and not the name of the egg, that matters (which
>would
>imply that -dbus and -gi are correct, and -pypubsub is not).
>
>Is there consensus that not following this recommendation is a *RC*
>bug?
>The bits I quoted above say "should" rather than "must".
>
>Thanks,
>smcv

Python Policy is not Debian Policy.  It's not a "policy" violation in the sense 
of a serious bug.  That said, it would be better to change it.

Scott K



Re: python3-dev dh-python dependency

2019-11-18 Thread Scott Kitterman



On November 18, 2019 11:14:03 AM UTC, Thibaut Paumard  
wrote:
>Hi,
>
>I've noted that python3-dev's last upload last week dropped the
>dependency on dh-python. I find this change surprising in the context
>of
>the current Python 3.8 transition.
>
>This makes "my" package yp-svipc FTBFS. I'm guessing many packages may
>be in the same case, there are 161 packages in buster main that
>build-depend (transitively) on python3-dev but do not depend (directly)
>on dh-python.
>
>So, should I fix yp-svipc now or wait for the end of the Python 3.8
>transition?
>
>Should this change in python3-dev be reverted until the end of the
>transition?
>
>Kind regards, Thibaut.

My recommendation would be to fix it.  As I recall, the depends was never meant 
to be permanent.  Any package that uses dh_python3 should build-depend on 
dh-python.

Scott K



Re: Policy About Maintainer and Uploaders Fields (was: PAPT: join request)

2019-11-11 Thread Scott Kitterman



On November 10, 2019 10:09:57 PM UTC, Thomas Goirand  wrote:
>On 11/10/19 1:20 AM, Sandro Tosi wrote:
>> is there any public trace of these "many voices"?
>
>Just like when we discussed moving away from SVN to Git, we can't know
>the exact number unless we have a kind of poll/vote (but we don't
>actually *have* to start such poll... I'm just saying it's hard to know
>otherwise).
>
>However, I can account for maybe 5 people (it's hard to remember) who
>told me (face to face) they hate this policy, and it felt like there
>was
>a broad consensus that it was really against a team spirit, plus it
>made
>little sense to have a package team maintained with strong ownership. I
>wouldn't name the persons I have in mind publicly, because they haven't
>granted me the permission to do so, and therefore, it wouldn't be nice
>to do so.
>
>All of this is just feelings I had during conversations with others,
>and
>of course, cannot be accounted as just plain facts, so just take it as
>it is: just a report of the feeling I had when talking with others,
>that
>it looks like I'm far from the only one disliking this policy because
>of
>the above mentioned reasons.
>
>As Louis-Philippe wrote, it's very much ok to delay this conversation,
>but sooner or later, it will come back on the table.

Personally, I've been judicious in putting myself as Maintainer in DPMT and 
PAPT packages.  If we were to ditch the current policy, my immediate response 
would be to remove DPMT/PAPT from uploaders and maintain them outside the team. 
 It's about 1/4 of my DPMT/PAPT packages.

I suspect I'm not the only one.

Your proposal is not cost free for the teams.

Scott K



Re: package_data in python package

2019-10-28 Thread Scott Kitterman
On Monday, October 28, 2019 11:02:40 AM EDT Félix Sipma wrote:
> On 2019-10-28 10:44-0400, Scott Kitterman wrote:
> > It's the way setuptools works.  Looking at the current upstream
> > recommendations:
> > 
> > https://python-packaging.readthedocs.io/en/latest/non-code-files.html
> > 
> > your solution seems correct.
> 
> Hi Scott,
> 
> Thanks for your message. I'm not sure I get why upstream can't reproduce
> the problem in this case... setuptools, as used in the setup.py seems to
> install the data dir just fine, and without my patch.
> 
> That's why I thought that it was maybe a limitation of dh-python...

It does depend considerably on how you do the install.  For example, to 
replicate what pip does (to get the data files installed) you have to do:

python setup.py install --single-version-externally-managed --record=/dev/null

I don't find I need to do that with pybuild though.  I find setuptools handling 
of data files highly mysterious and counter-intuitive, so I doubt I'll have 
more specific advice.

Scott K


signature.asc
Description: This is a digitally signed message part.


Re: package_data in python package

2019-10-28 Thread Scott Kitterman
On Monday, October 28, 2019 10:25:44 AM EDT Félix Sipma wrote:
> Hi,
> 
> I have a problem with package_data files in a package ("khard", which is
> not currently under the debian-python umbrella but it is on my todo list
> to transfer it, I just didn't find the time yet) not getting installed.
> 
> I submitted an error upstream,
> https://github.com/scheibler/khard/issues/231 but they can't reproduce
> the problem.
> 
> I "fixed" the issue by adding a patch to add "khard/data/" to
> MANIFEST.in (with "recursive-include khard/data/ *"), but I don't get
> why this directory doesn't get installed by default... Is it because of
> a limitation of dh-python? Have I forgot an option somewhere?
> 
> Link to the patch and the source of the package:
> https://sources.debian.org/src/khard/0.15.0-2/debian/patches/0003-add-khard-> 
> data-dir-to-MANIFEST.in.patch/
> 
> Thanks for your help!

It's the way setuptools works.  Looking at the current upstream 
recommendations:

https://python-packaging.readthedocs.io/en/latest/non-code-files.html

your solution seems correct.

Scott K





Re: Request to (re)join the team

2019-10-15 Thread Scott Kitterman
On Monday, October 14, 2019 3:45:06 PM EDT Nick Morrott wrote:
> Hi everyone,
> 
> I'd like to migrate my current membership of the debian-python DPMT
> and PAPT teams on salsa to use my shiny, new DD account \o/. My salsa
> account is nickm.
> 
> Thank you for your help, guidance, mentorship and sponsored uploads so far!

Done and congratulations.  Once you're sure it all works with the new account, 
don't forget to get rid of the old one.

Scott K




Re: Raising severity to serious for some Python 2 leaf packages with no Python 3 support upstream

2019-10-13 Thread Scott Kitterman
On Sunday, October 13, 2019 10:52:17 PM EDT Thomas Goirand wrote:
> Hi,
> 
> In some cases I've seen, particularly in the med or science team,
> switching some packages to Python 3 requires a significant effort.
> 
> For example, today I looked into removing Python 2 from python-cogent.
> Running sixer on all files lead to a huge log of problems to solve by
> hand. There's no upstream support for Python 3 on that one.
> 
> For this kind of package, I see no way out except:
> - Upstream works on Python 3 support
> - Someone in Debian makes the effort
> 
> But in both cases, it's going to take a very long time. Do we really
> want to get stuck on these packages for like forever, or would it feel
> ok to raise the severity to serious, so that the package gets
> auto-removed and then we can work on removing Python 2 from its
> dependencies?

There are two python2 only packages that I maintain.  I don't intend to keep 
them in bullseye, so I filed "should not be in bullseye" RC bugs against them.  
They and their rdepends will be out of Testing shortly. 

If you are the maintainer of a package, I think that's something that doesn't 
need to wait.

In the case of things that aren't ported to python3 yet they are mostly dead 
upstream.  For the dead ones, I don't think anyone in Debian should port them 
unless they are willing to take over as upstream (I've done this in a few 
cases).  If there's no upstream, they ought to just be removed.  The sooner 
the better.

Scott K





Re: Request to join the Debian Python Modules Team

2019-10-07 Thread Scott Kitterman
On Wednesday, October 2, 2019 11:37:59 AM EDT Santiago Ruano Rincón wrote:
> Hi there,
> 
> I would like to request to be included in the Debian Python Modules
> Team.
> 
> * Why you want to join the team: I would like to maintain altair (See
>   #941599) within the team, as any other future python module that I
>   would be interested in.
> * Your Salsa login: santiago
> * I state I have read
> https://salsa.debian.org/python-team/tools/python-modules/blob/master/polic
> y.rst and I accept it.

Added.  Welcome to the team.

Scott K




Re: Joining DPMT

2019-10-07 Thread Scott Kitterman
On Monday, October 7, 2019 1:56:18 AM EDT Stuart Prescott wrote:
> Hi folks
> 
> Could you please add me to DPMT.
> 
> I've been looking at upgrading plastex (which is team maintained but with no
> active maintainer) to a newer upstream version; upgrading it to a newer
> upstream version gets rid of another Python 2-only package.
> 
> My salsa login is 'stuart'.
> 
> I've read the DPMT policy and agree to it.

Welcome to the team,

Scott K




Re: What is the process to update the DPMT and PAPT policies?

2019-09-15 Thread Scott Kitterman



On September 15, 2019 11:59:08 PM UTC, "Louis-Philippe Véronneau" 
 wrote:
>Hi!
>
>What is the process to update the DPMT and PAPT policies? I feel the
>DPMT policy is pretty good and I feel the PAPT policy could copy a
>bunch
>of stuff from there.
>
>For example, the PAPT policy doesn't include a "Git Procedures"
>section.
>
>I'm guessing the way to go is to clearly propose a draft modification
>on
>the mailing list and see if it reaches consensus. Would that be
>acceptable?

My recollection is that is what we've done in the past.

Scott K



Re: Re: Bug#920127: Removed package(s) from unstable

2019-09-08 Thread Scott Kitterman



On September 9, 2019 3:25:43 AM UTC, Paul Wise  wrote:
>On Mon, Sep 9, 2019 at 4:11 AM Scott Kitterman wrote:
>
>> This was sent to the FTP Team, but it seems like someone with some
>bandwidth
>> to assist from DPMT/PAPT would be a better audience.  Note that the
>removal's
>> already been done, but if someone wants to sponsor a python3 update
>to the
>> package, they can ping me for a quick trip through New to bring it
>back.
>
>It seems like a little bit more due diligence is needed before filing
>removals.

In theory yes, but in practice because this is a multi-thousand package 
transition we're trying to get through, I think it's better to move fast even 
if we break a few things than to be super careful and slow.  Things like this 
can be fixed.

It's better to push hard early in the development cycle so there's time to 
recover.

Scott K



Fwd: Re: Bug#920127: Removed package(s) from unstable

2019-09-08 Thread Scott Kitterman
This was sent to the FTP Team, but it seems like someone with some bandwidth 
to assist from DPMT/PAPT would be a better audience.  Note that the removal's 
already been done, but if someone wants to sponsor a python3 update to the 
package, they can ping me for a quick trip through New to bring it back.

Scott K

-  Forwarded Message  --

Subject: Re: Bug#920127: Removed package(s) from unstable
Date: Sunday, September 8, 2019, 3:05:59 PM EDT
From: John Eikenberry 
To: Debian FTP Masters 

Hello,

I am up upstream maintainer for colortest-python and wanted to communicate 
that
it is not dead and it fully supports running with python3. I responded with 
the
same message to the ticket about this (#936320). I'd be happy to help maintain
it the package if necessary.

Thanks.

Debian FTP Masters wrote:

> We believe that the bug you reported is now fixed; the following
> package(s) have been removed from unstable:
> 
> colortest-python |  2.2-1 | source, all
> 
> --- Reason ---
> python2-only; orphaned; dead upstream; low popcon
> --
> 
> Note that the package(s) have simply been removed from the tag
> database and may (or may not) still be in the pool; this is not a bug.
> The package(s) will be physically removed automatically when no suite
> references them (and in the case of source, when no binary references
> it).  Please also remember that the changes have been done on the
> master archive and will not propagate to any mirrors until the next
> dinstall run at the earliest.
> 
> Packages are usually not removed from testing by hand. Testing tracks
> unstable and will automatically remove packages which were removed
> from unstable when removing them from testing causes no dependency
> problems. The release team can force a removal from testing if it is
> really needed, please contact them if this should be the case.
> 
> We try to close bugs which have been reported against this package
> automatically. But please check all old bugs, if they were closed
> correctly or should have been re-assigned to another package.
> 
> Thank you for reporting the bug, which will now be closed.  If you
> have further comments please address them to 920...@bugs.debian.org.
> 
> The full log for this bug can be viewed at https://bugs.debian.org/920127
> 
> This message was generated automatically; if you believe that there is
> a problem with it please contact the archive administrators by mailing
> ftpmas...@ftp-master.debian.org.
> 
> Debian distribution maintenance software
> pp.
> Scott Kitterman (the ftpmaster behind the curtain)
> 

-- 

John Eikenberry
[ j...@zhar.net - http://zhar.net ]

"Perfection is attained, not when no more can be added, but when no more
 can be removed." -- Antoine de Saint-Exupery


-




Re: Python 3 transition question

2019-09-01 Thread Scott Kitterman



On September 2, 2019 4:00:53 AM UTC, Sandro Tosi  wrote:
>> I would just stop building these.  And if the reverse dependencies
>have a
>> py2removal bug itself, then comment in these issues that the
>> suggested/recommended package gets removed.  If they don't have a
>py2removal
>> bug, please file the bugs for these packages.
>
>i dont believe this is a sensible approach; for example i maintain
>python-mpmath, that would be rendered uninstallable the moment
>python-gmp2 is removed. Now, python-mpmath has 3 external
>reverse-dependencies (just to name a couple, sagemath and simpy) that
>would be then uninstallable, and so on and so forth for all their
>rdeps.
>
>Martin, i think for now the only option is to keep the py2 packages
>around until we're ready to drop them (ie they have 0 rdeps).

I just checked on packages.d.o and according to it, python-gmp2 is a Suggests.  
Suggests aren't installed with packages.  Unless I'm missing something, 
python-mpmath wouldn't become uninstallable.

IIRC, policy doesn't even require Suggests packages to exist.

I agree about keeping packages as long as they have reverse Recommends, but I 
think Suggests is going too far (although AIUI, missing Recommends don't make 
the package uninstallable either).

Scott K



Re: Webpage to track py2removal bugs & packages

2019-08-31 Thread Scott Kitterman
On Saturday, August 31, 2019 11:09:33 AM EDT Andrey Rahmatullin wrote:
> On Sat, Aug 31, 2019 at 10:20:32AM -0400, Scott Kitterman wrote:
> > I definitely think it will be helpful.  Thanks for putting it together. 
> > It's the best thing I've seen yet for answering the question of what can
> > we try to kill off now.
> > 
> > Please keep it updated.
> 
> Please note though that all python modules with QA or DPMT in Maintainer
> and without revdeps are already fixed.

I'm not sure how you checked that.  I just now fixed pycxx which did have an 
rdepend, but only one from the same source package.  Depending on your 
methodology, there may be others.

Scott K




Re: Webpage to track py2removal bugs & packages

2019-08-31 Thread Scott Kitterman
On Saturday, August 31, 2019 2:31:36 AM EDT Sandro Tosi wrote:
> Hello,
> i've prepared a small website,
> http://sandrotosi.me/debian/py2removal/index.html, to keep track of
> the bugs user-tagged `py2removal`.
> 
> * i'm sure there are bugs
> * it's pretty brutal html, but should provide useful information
> * it tries to account for crufted binary packages
> * it ignores the package if the py2removal bug is closed (that means
> you can remove a pkg with rdeps and not be identified by this page)
> * since py2removal bugs are against src pkg, the script gets the
> binary package and work only on the ones depending on python2 packages
> * it's sorted with packages with 0 or few r-deps at the top, which
> ideally are packages "easy" to get py2 removed from
> * there's a graph too, which gives a more visual representation of the
> rdeps; it's currently at level=1 (ie only the target package + the
> direct reverse-dependencies are shown), i'll consider expand it to
> level=3/4
> * for now it's juts a single snapshot; if considered useful i'll to to
> keep it updated regularly (but my laptop is not always on anyway)
> 
> let me know if you consider it helpful and/or it should include more
> information to make it more interesting.

I definitely think it will be helpful.  Thanks for putting it together.  It's 
the best thing I've seen yet for answering the question of what can we try to 
kill off now.

Please keep it updated.

Scott K




Re: py2-rm: a few leaf packages to work on

2019-08-25 Thread Scott Kitterman
On Sunday, August 25, 2019 10:55:55 AM EDT Matthias Klose wrote:
> On 24.08.19 07:03, Scott Kitterman wrote:
> > On Thursday, August 15, 2019 8:08:41 AM EDT Thomas Goirand wrote:
> >> Hi there!
> >> 
> >> According to the daily graph I built here:
> >> http://py2graph.infomaniak.ch/py2.7.deps.svg
> >> 
> >> we can work on Python 2 removal for the below packages. Note that I have
> >> *not* checked for reverse dependencies, please do so before working on a
> >> package. The list isn't exhaustive at all, and didn't check if a package
> >> is just a remaining curft, though it's hopefully still helpful as a TODO
> >> list.
> > 
> > The arch all decruft is caught up now, so it ought to be easier now to
> > tell
> > what still has rdepends left.
> 
> is this now done automatically, or just a manual effort?

The DAK change to make it automatic hasn't been merged yet, so it's not fully 
automatic.  What we do get now is an automatically generated list of what 
needs removing so it's about 98% easier to do than it was before.

Scott K




  1   2   3   4   5   6   7   >