Re: sponsor uploading python-discogs-client_2.3.5-1

2020-12-15 Thread Thomas Goirand
On 12/14/20 11:43 PM, jojo wrote:
> On 11.12.20 10:03, jojo wrote:
>> Hi,
>> I updated a team-maintained package and am looking for an upload
>> sponsor. This one:
>> https://salsa.debian.org/python-team/packages/python3-discogs-client/-/tree/debian/2.3.5-1.
>> I already uploaded the package to my webserver:
>> https://jojo.peek-a-boo.at/E950513F/python-discogs-client_2.3.5-1.dsc
>>
>> I tried to find a sponsor on the irc channel #debian-python already. I
>> posted it in the channel and have been told to better just add it to
>> the topic of that channel. I didn't do that because I am an irc noob
>> and use the irc bridge on matrix.org to access irc rooms and didn't
>> find a way to do it yet. Just so you know.
> 
> I finally found out how to properly edit the topic via my matrix client
> and just have added python-discogs-client to the RFS list :-)

Uploaded.

Thanks for your contribution to Debian.

Cheers,

Thomas Goirand (zigo)



Re: Should Binaries provided by python-babel have a "python3-" prefix?

2020-11-27 Thread Thomas Goirand
On 11/27/20 1:13 PM, Simon McVittie wrote:
> A good way to decide this is to think about what we would do if we had a
> Python 4 that is incompatible with Python 3 (which I assume will happen
> eventually, although hopefully not for a few years).

It is very likely that you're wrongly guessing. Numerous times, the
Python upstream people wrote that the py2 to py3 transition was a bad
idea, and they will not do the same kind of mistake again, and that
there probably wont be a Python 4 anytime.

Which is why I thought merging the content of python3-babel and
python3-babel-localedata would be a good idea.

Cheers,

Thomas Goirand (zigo)



Re: Should Binaries provided by python-babel have a "python3-" prefix?

2020-11-26 Thread Thomas Goirand
On 11/26/20 1:16 PM, Nilesh Patra wrote:
> Hi,
> 
> Currently src:python-babel provides 3 binaries:
> 
> * python3-babel
> * python-babel-doc
> * python-babel-localedata
> 
> of which python3-babel is the main binary, -babel-doc is for the
> documentation and -babel-localedata is for storing locale data files
> used by python3-babel.
> 
> Should this be renamed to a "python3-" prefix for both binaries? They do
> not contain any actual code though
> 
> BTW this also has a RC bug, and I pushed the fix to salsa. If it needs a
> renaming, it'd be great if someone could upload it to NEW.
> 
> If not, I'll do a source-only-upload.
> Please let me know what you think of this.
> 
> Kind regards
> Nilesh

Hi,

It used to be that we had a python-babel that also needed the data in
python-babel-localedata, which was shared by both the Python 2 and 3
packages. Now, we don't have it anymore. So probably, what could be
done, is move all the files in python3-babel, and get rid of
python-babel-localedata completely.

python-babel-doc is still the correct package name for the doc (unless
the amount of doc is small and could be integrated in python3-babel as
well).

If that's the way to go, then python3-babel needs a Breaks+Replaces:
python-babel-localedata.

Cheers,

Thomas Goirand (zigo)



Re: Joining the team

2020-11-23 Thread Thomas Goirand
On 11/23/20 10:10 PM, Sandro Tosi wrote:
>>> First, an apology: it seems I misremembered being in the team, and uploaded 
>>> to
>>> NEW a bunch of packages with the team in `Uploaders`.
>>
>> Please put the team as Maintainer, and yourself as Uploaders.
> 
> why? that's not a requirement:
> https://salsa.debian.org/python-team/tools/python-modules/blob/master/policy.rst#maintainership

Because joining a team, putting packages in them, and enforcing strong
ownership, is not logic at all. I know you like to do this way, but this
shouldn't be what we advise for new comers.

Cheers,

Thomas Goirand (zigo)



Re: Joining the team

2020-11-23 Thread Thomas Goirand
On 11/23/20 2:15 PM, nicoo wrote:
> Hi everyone!
> 
> First, an apology: it seems I misremembered being in the team, and uploaded to
> NEW a bunch of packages with the team in `Uploaders`.

Please put the team as Maintainer, and yourself as Uploaders.

Cheers,

Thomas Goirand (zigo)



Re: [Python-modules-team] Bug#954381: marked as done (python3-kubernetes: New upstream version available)

2020-11-21 Thread Thomas Goirand
On 11/21/20 3:36 AM, Sandro Tosi wrote:
>>* Use git to generate upstream tarball, as the PyPi module doesn't include
>>  the test folder. Using the gen-orig-xz in debian/rules, as using the
>>  repack function of debian/watch doesn't make sense (why downloading a
>>  tarball that would be later on discarded? I'm open to a better solution
>>  which would be uscan compatible though...). Switch d/watch to the github
>>  tag therefore.
> 
> you can track the github project instead of pypi (man uscan has the
> details); this is was i'm doing recently, as most of the time PyPI
> releases dont have all the files we need (tests, or test data, or
> documentation, or a combination of that)

Hi.

Thanks, I know that. However, that's not my problem. The issue is that
uscan --download will download the tarball from github, and I'd like to
replace that by what I'm doing in debian/rules, which is using git and
git submodule, to fetch things using git, and create a tarball. Sure, I
could use a repack script in debian/watch, but then uscan will continue
to first download the archive from github, and *then* only, I can
discard what's been downloaded, and fetch stuff from github with git.

Is there a solution here, so that uscan uses a repack script directly
without attempting to download first?

Cheers,

Thomas Goirand (zigo)



Re: Python 3.9 for bullseye

2020-11-09 Thread Thomas Goirand
On 11/9/20 10:19 AM, Matthias Klose wrote:
> https://bugs.debian.org/973239 src:python-fixtures

Does anyone else than me think it's probably OK to just disable to 2
broken tests?

Cheers,

Thomas Goirand (zigo)



Re: "pytest" command is missing

2020-11-05 Thread Thomas Goirand
On 11/4/20 9:27 PM, Novy, Ondrej wrote:
> Hi,
> 
> Antonio Terceiro píše v St 04. 11. 2020 v 14:01 -0300:
>> Could you ellaborate? Maybe we should have a discussion in the Python
>> team so that we implement consistent practices. For example, `gunicorn`
>> and `pip` now point to their python3 versions, but you are saying that
>> pytest will not do that, what maybe creates more confusion given Debian
>> bullseye will not support any other Python.
> 
> "pytest" in buster now points to python2 version of pytest and
> "pytest-3" points to python3 version. To prevent confusion after upgrade
> I want to keep one stable release with pytest cmd "unoccupied" and keep
> pytest-3.
> 
> Bullseye will support Python2 interpreter so user can keep python-pytest
> package installed from buster.

Moreover, simply invoking "pytest-3" is not enough, one should be using:

for pyvers in $$(py3versions -vr 2>/dev/null) ; do \
python$$pyvers -m pytest ; \
done

otherwise, only the default version of Python3 is tested, and we really
want to test with all available versions (so we get results whenever
we're transitioning to a new Python 3 version).

Cheers,

Thomas Goirand (zigo)



Re: How to watch pypi.org

2020-11-01 Thread Thomas Goirand
On 10/31/20 1:10 PM, Jeremy Stanley wrote:
> On 2020-10-31 12:03:50 +0100 (+0100), Thomas Goirand wrote:
> [...]
>> On 10/31/20 3:07 AM, Jeremy Stanley wrote:
>>> I have to agree, though in the upstream projects with which I'm
>>> involved, those generated files are basically a lossy re-encoding of
>>> metadata from the Git repositories themselves: AUTHORS file
>>> generated from committer headers, ChangeLog files from commit
>>> subjects, version information from tag names, and so on. Some of
>>> this information may be referenced from copyright licenses, so it's
>>> important in those cases for package maintainers to generate it when
>>> making their source packages if not using the sdist tarballs
>>> published by the project.
>>
>> Unfortunately, the FTP masters do not agree with you. I've been told
>> that the OpenStack changelog is a way too big, and it's preferable to
>> not have it in the binary packages.
> 
> PBR started creating much smaller changelogs years ago, after you
> asked ftpmaster. I get that you see no value in changelog files, but
> it seems like it would be worth revisiting.

Probably it'd be worth revisiting indeed, especially by finding a way to
push the logs only in the relevant -doc packages. Maybe by using the
same kind of trick which I want to do with the release notes (see
below), ie storing these files in the debian folder.

However, if I am to put more efforts on stuff like that, my priority
would be first on getting the reno release notes published in the Debian
package. I've been thinking about this for a long time, and haven't
figured out yet what would be the best way, with a reasonable workflow.

>From the Debian perspective, I'd have to:
- generate the release notes from the last version in Debian Stable, up
to what's in Sid. For example, I would include all the OpenStack release
notes for Stein, Train, Ussuri and Victoria in all packages uploaded for
Debian Bullseye, as this would concern anyone upgrading from Buster.
- make it so that release notes would be generated from Git, and maybe
stored in a debian/release-notes folder, so that it wouldn't generate a
diff with the original upstream tag.

The drawback would be that, on each upload of a new Debian revision, the
debian.tar.xz will contain the release notes, which could be of
significant size (especially if they include static objects like CSS,
JS, and all what makes a theme).

If you have any other suggestion concerning how to handle these release
notes, please let me know.

Cheers,

Thomas Goirand (zigo)



Re: How to watch pypi.org

2020-10-31 Thread Thomas Goirand
On 10/30/20 3:18 PM, Fioddor Superconcentrado wrote:
> As I said I'm very new to this and all (python) packages I'm using
> lately use the usual python tools (pipy, setup.py, etc) and my first
> approach has been to stick as close as possible to the upstream
> procedures. But I might very likely be taking a wrong decision. What are
> the reasons to go for git instead of pypi? I see that it is 'more
> upstream' but it seems that everyone else is pointing to pypi as a
> distro-agnostic solution.

Pypi is often thought as a Python module source repository. It is *NOT*.
It is a repository for binaries to be consumed by pip. Therefore, the
tarballs may omit some artifacts, and add some more. For example, it's
preferable that the sources do not include the egg-info folder, or
pre-generated documentation. These must be built from source.

On 10/31/20 3:07 AM, Jeremy Stanley wrote:
> I have to agree, though in the upstream projects with which I'm
> involved, those generated files are basically a lossy re-encoding of
> metadata from the Git repositories themselves: AUTHORS file
> generated from committer headers, ChangeLog files from commit
> subjects, version information from tag names, and so on. Some of
> this information may be referenced from copyright licenses, so it's
> important in those cases for package maintainers to generate it when
> making their source packages if not using the sdist tarballs
> published by the project.

Unfortunately, the FTP masters do not agree with you. I've been told
that the OpenStack changelog is a way too big, and it's preferable to
not have it in the binary packages. Also, there's nothing in the Apache
license that mandates having an AUTHORS list as per what PBR builds. If
we are to care that much in OpenStack, then the license must be changed.

In Debian, quite the opposite, and like it or not (I personally don't
really feel the policy is right, but that's how it is), what Debian
cares is what's in the source code marked with "copyright (c) yeah,
copyright-holder", and this must already be in debian/copyright.

Cheers,

Thomas Goirand (zigo)



Re: Python 3.9 for bullseye

2020-10-26 Thread Thomas Goirand
On 10/18/20 12:13 PM, Matthias Klose wrote:
> Python 3.9 as a supported Python3 version is now in unstable, and all binNMUs
> are done (thanks to Graham for the work).   Bug reports should be all filed 
> for
> all known problems [1], and the current state of the 3.9 addition can be seen 
> at
> [2] (a few of the "bad" are false packages with b-d n python3-all-dev, but not
> building for 3.9, bug reports also filed).
> 
> The major outstanding issue is the pandas stack, all other problems are found 
> in
> leaf packages (leaf in the sense of that no other package for the 3.9 addition
> is blocked).
> 
> Please help fixing the remaining issues!
> 
> Matthias

Hi Matthias,

I don't know if that was on purpose, but you happen to upload Python 3.9
in Unstable the day OpenStack was released. I then rebuilt all of
OpenStack to upload from Experimental to Unstable, and to my surprise,
it went very well (note: all packages in the OpenStack team are running
unit tests at build time on all available Python versions). Only 2
issues happened multiple times:
- base64.{en,de}codestring removal (easy fix: s/string/bytes/)
- Threading.isAlive removal (easy fix too: s/isAlive/is_alive/)

This happened

This is on a set of 200+ packages which I manually rebuilt.

I do expect that there will be more packages with the same issue, so
it'd be nice to have all Python-using packages rebuilt. As Lucas
Nussbaum proposed such a service, should we ask him to do such a massive
rebuilt? Or maybe you have other plans?

Cheers,

Thomas Goirand (zigo)



Re: Newcomers project: DPMT/PAPT pristine-tar verification

2020-10-09 Thread Thomas Goirand
On 10/3/20 9:35 PM, Sandro Tosi wrote:
> attached the dd-list of the packages missing the pristine-tar branch
> (some may have been moved/removed, but these are actual repos in DPT)

For those:

   python-etcd3 (U)
   python-requests-mock (U)
   python-sphinxcontrib.apidoc (U)

they in fact live in the OpenStack team namespace, but I don't have
enough rights to delete the Git repositories. Can someone do it for me
please?

There's 2 more for which I'm listed, I'll see if I can fix.

Cheers,

Thomas Goirand (zigo)



Re: mercurial switch to python3 in debian unstable - July 16th, 2020

2020-07-18 Thread Thomas Goirand
On 7/16/20 9:48 PM, Sandro Tosi wrote:
>> If we dont hear otherwise, we plan to upload the python3 version of
>> mercurial in unstable on or around next Thursday, July 16th.
> 
> mercurial/5.4.1-2 has just been uploaded to unstable, switching it to
> use python3.
> 
> Regards,

Thanks a lot for your work, on this specifically, and on the Python 2
removal in general.

I guess a lot of things are unlocked now. I wonder how we can help
fixing what's remaining. Please do share your thoughts on that.

Cheers,

Thomas Goirand (zigo)



Re: The python command in Debian

2020-07-17 Thread Thomas Goirand
On 7/16/20 6:03 PM, Sandro Tosi wrote:
> it is my opinion that that's what we should do: not ship `python` at
> all and have users/packagers/developers use either python2 or python3
> as needed, and not to reintroduce `python` at a later time.

I agree.

It's trivial for anyone to manually "fix" (or break?!?) a system to have
/usr/bin/python anyway, so let's not ship Debian as already broken. :)

Thomas



Re: joining team / salsa access

2020-07-17 Thread Thomas Goirand
On 7/17/20 8:21 AM, jojo wrote:
> On 16.07.20 15:01, Jojo wrote:
> 
>> Hi,
>>
>> I'd like to join the debian python modules and application teams to be
>> able to push my gbp packing sources to salsa.debian.org.
>>
>> I've read and understood https://wiki.debian.org/Python
>>
>> Packages are:
>>
>> - python3-discogs-client (using pristine tar)
>>   source package name: python-discogs-client
>>   ITP: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=961714
>>
>> - python3-webdavclient(using pristine tar)
>>   source package name: python-discogs-client
>>   ITP:https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=961715
>>
>> - discodos (not using pristine tar)
>>   ITP: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=960698
>>
>> zigo was helping/mentoring the packaging and is about to do the final
>> reviews and ftp uploads.
>>
>>
> I forgot to mention that I already registered for an account on salsa:
> https://salsa.debian.org/jojo
> 
> all the best
> 
> Jojo

Unless someone opposes quickly, there's no need to wait 75 hours on this
case, as Jojo previously introduced himself in this list. He's been
doing good packaging work so far, and hopefully, his 3 packages will
soon make it to Debian.

Cheers,

Thomas Goirand (zigo)



Re: Request to join Python Modules Team

2020-07-16 Thread Thomas Goirand
On 7/15/20 12:02 PM, Ondrej Novy wrote:
> Hi,
> 
> út 14. 7. 2020 v 23:18 odesílatel Geert Stappers  <mailto:stapp...@debian.org>> napsal:
> 
> And please enlighten me (and us (through the mailinglist))
> what is holding back
>  https://lists.debian.org/debian-python/2020/07/msg00057.html
> 
> 
> huh, this is a bit rude.

I don't agree Geert is being rude. Besides that, I know Geert (from
Debconfs) to be a nice and polite person that isn't rude with others.

Please assume good faith.

Thomas Goirand (zigo)



Re: The python command in Debian

2020-07-14 Thread Thomas Goirand
On 7/9/20 9:31 PM, Ondrej Novy wrote:
> * keep "python" command pointing to python2.7 if I'm upgrading
> buster->bullseye with python2.7 installed. We are going to keep
> python2.7 interpreter for bullseye, so don't break old "python" command
> for third-parties apps/scripts/etc. (install python-is-python2 during
> buster->bullseye upgrade)

Please do break them. They are broken because they expect an interpreter
which we don't support anymore. So they MUST break. Leaving the distro
half-upgraded with the feeling that things are continuing to work is a
horrible design.

During the upgrade to bullseye, please do remove all things Python2,
including the interpreter, unless explicitly requested by the user (if
that's possible). If a user feels like he must install Python2, it must
be a manual decision (maybe after the upgrade and using /usr/local ?).
Otherwise, there's still Buster around for the next 4 years to come...

Cheers,

Thomas Goirand (zigo)



Re: pybuild vs os-pkg-tools [was: Maintaining all of the testing-cabal packages under the OpenStack team]

2020-07-08 Thread Thomas Goirand
On 7/7/20 8:20 AM, Ondrej Novy wrote:
> Hi,
> 
> po 6. 7. 2020 v 11:09 odesílatel Thomas Goirand  <mailto:z...@debian.org>> napsal:
> 
> This isn't about hating or loving pybuild. This is all about being able
> to control how this set of packages are build globally (the whole set of
> packages. 
> 
> 
> so you could wrap these global things around pybuild, right?

There would be no point doing that, as all what pybuild would be doing
(ie: setup.py install and unit testing), I'd be overriding it. The other
(IMO preferred) way would be to contribute to Pybuild to make it does
what's needed for these packages, and in a way so that we can also make
such global change for all OpenStack packages at once.

I've been thinking about it for a while, but my ideas aren't clear
enough to start something (yet).

Cheers,

Thomas Goirand (zigo)



Re: Timing of Python upstream and Debian releases

2020-07-08 Thread Thomas Goirand
On 7/6/20 9:04 PM, Matthias Klose wrote:
> Starting with Python 3.8, Python upstream changed to a time based yearly 
> release
> schedule, targeting the first release of a major Python version (3.x) for
> October of each year.  For the transition to 3.8:
> 
>  - we add 3.8 as supported in November
>  - made 3.8 the default in March
>  - dropped 3.7 in April
> 
> That's a little bit late looking at the Debian release schedule, so a little
> speedup would be needed if we want to target the current Python release for 
> the
> current Debian release.  For 3.8 Ubuntu started to prepare the switch a bit
> earlier, using the results for a better experience in Debian:
> 
>  - added 3.8 in mid October
>  - made 3.8 the default in mid December
> 
> Technically it would be possible to do the defaults change before the Debian
> freeze, however there's usually a tail of follow-up upstream releases required
> to support a new major Python version, unless you opt for actively backporting
> changes in various packages.  Having the confidence, that the switch is 
> feasible
> in another distro certainly helps doing the transition in Debian, however it
> adds a delay.  I don't think it's feasible to do the transition in 
> experimental
> first, or doing a large test rebuild, because it requires keeping the whole
> python stack in sync with testing/unstable.
> 
> So what I'm proposing here is to aim to support 3.9 as early as possible as a
> supported Python3 version, starting with the 3.9 upstream release, and fixing
> stuff on the go.  Then decide in November, if we can do the defaults change to
> 3.9, or if we drop 3.9 again, or ship with two supported Python3 versions.

This looks reasonable, thanks for opening the discussion and doing the
hard work. However, I wouldn't be for shipping 2 supported versions, and
would prefer if we could have only one if possible, as otherwise this
means running unit tests at build time against 2 python versions, which
then takes twice as much build time: that's annoying and useless
(because at the end, only one of these versions will be in use).

Cheers,

Thomas Goirand (zigo)



Re: pybuild vs os-pkg-tools [was: Maintaining all of the testing-cabal packages under the OpenStack team]

2020-07-06 Thread Thomas Goirand
On 6/29/20 8:34 AM, Ondrej Novy wrote:
> for my packages (i'm uploader): no, sorry.
> 
> Reasons:
> 1. I hate openstack-pkg-tools
> 2. I like pybuild
> 3. you hate pybuild and don't want to use it

I thought about this for a long time, thinking if I should leave it
unanswered or not. Finally, I have the opinion I can't let it blank.

You are so focus on nit-picking cosmetic fixes, that you are failing to
see the big picture. The same way Daniel did when he told me that
pybuild is "superior" and that I shouldn't waste my time working on my
own tooling. At least 2 others also asked, so maybe it's time to explain.

This isn't about hating or loving pybuild. This is all about being able
to control how this set of packages are build globally (the whole set of
packages. Let me explain why this is important using 3 examples, and
hopefully you'll get where I am at.

A few OpenStack releases ago, most packages migrated away from
testrepository to stestr. For good, because I believe stestr is better.
Anwyay, that's not the point. The point is, the only thing I had to do
to fix it, was to fix pkgos-dh_auto_test, and in there, check if a
.stestr.conf is present. If it is there, pkgos-dh_auto_test just use stestr.

The same way, some packages need to be installed, with a correct
egg-info and entrypoint, otherwise, unit tests are failing. So I could
change that in openstack-pkg-tools.

A 3rd example from last week: now, when unit tests are failing,
openstack-pkg-tools displays the output of "pip3 freeze", so I can do
report upstream more easily.

Each time, for all of these changes, I had to do a single unique change,
in a single place, without affecting other packages in the archive.

Hoping this explains well enough my choice, so that it makes sense to
everyone.

Cheers,

Thomas Goirand (zigo)



Re: Maintaining all of the testing-cabal packages under the OpenStack team

2020-06-30 Thread Thomas Goirand
On 6/30/20 12:41 AM, Jeremy Stanley wrote:
> On 2020-06-29 23:55:49 +0200 (+0200), Thomas Goirand wrote:
> [...]
>> nodepool from OpenStack,
> 
> Well, *formerly* from OpenStack, these days Nodepool is a component
> of the Zuul project gating system, which is developed by an
> independent project/community (still represented by the OSF):
> 
> https://zuul-ci.org/
> https://opendev.org/zuul/nodepool/
> 
> You could probably run a Nodepool launcher daemon stand-alone
> (without a Zuul scheduler), but it's going to expect to be able to
> service node requests queued in a running Apache Zookeeper instance
> and usually the easiest way to generate those is with Zuul's
> scheduler. You might be better off just trying to run Nodepool along
> with Zuul, maybe even set up a GitLab connection to Salsa:
> 
> https://zuul-ci.org/docs/zuul/reference/drivers/gitlab.html
> 
>> and use instances donated by generous cloud providers (that's not
>> hard to find, really, I'm convinced that all the providers that
>> are donating to the OpenStack are likely to also donate compute
>> time to Debian).
> [...]
> 
> They probably would, I've approached some of them in the past when
> it sounded like the Salsa admins were willing to entertain other
> backend storage options than GCS for GitLab CI/CD artifacts. One of
> those resource donors (VEXXHOST) also has a Managed Zuul offering of
> their own, which they might be willing to hook you up with instead
> if you decide packaging all of Zuul is daunting (it looks like both
> you and hashar from WMF started work on that at various times in
> https://bugs.debian.org/705844 but more recently there are some
> JavaScript deps for its Web dashboard which could get gnarly to
> unwind in a Debian context).

Hi Jeremy,

I gave up a few times, because the reverse dependencies for it were not
aligned with what was in use in Debian at the time, and gave up.

If there's some nasty NPM job behind, then I probably will just skip the
dashboard, and expect deployment to get the dashboard not from packages.
What is included in the dashboard? Things like https://zuul.openstack.org/ ?

Cheers,

Thomas Goirand (zigo)



Re: Maintaining all of the testing-cabal packages under the OpenStack team

2020-06-29 Thread Thomas Goirand
On 6/29/20 7:35 PM, Utkarsh Gupta wrote:
>> Running the script shows that 279 reverse (build?) dependencies are
>> affected by mock. This clearly isn't something one wants to run on a
>> personal computer, and even less a test which one wants to run sequentially.
> 
> Haha, right.
> What we (me and a couple others) do is run this build script on a
> server (via screen) and call it a night :P
> And we get the list of broken packages in the morning!
> 
> But of course, this is not a very "professional" way of doing it.

What we could do, is use nodepool from OpenStack, and use instances
donated by generous cloud providers (that's not hard to find, really,
I'm convinced that all the providers that are donating to the OpenStack
are likely to also donate compute time to Debian). And then we could
launch 150 builds at a time on 150 VMs. Then the time to wait is only
the time of the longest build.

>> Has any thought went into having some kind of runners running on a cloud
>> to run these tests, and maybe plug this into Salsa's CI to run it
>> automatically?
> 
> This seems to be a nice idea!
> I am not sure if someone had the time or energy to do this, but this
> is something we'd definitely love \o/

To get this to happen, we have no other way but using the power of some
kind of cloud / HTC.

>> I'd very much would love to set this up, at least as a first
>> experimentation on a bunch of package of the DPMT.
> 
> Me too!

I shall resume packaging nodepool then...

Cheers,

Thomas Goirand (zigo)



Re: Maintaining all of the testing-cabal packages under the OpenStack team

2020-06-29 Thread Thomas Goirand
On 6/29/20 2:33 PM, Utkarsh Gupta wrote:
> There exists such a thing which I use daily: ruby-team/meta[1].
> The meta/build script is (hopefully and exactly) what we need here!
> 
> It checks all the reverse(-build)-dependencies and lets you know what's
> going to break as soon as you dput.

Hi Utkarsh,

Nice! Thanks a lot for the pointer.

I very much agree with you that the debate has to be emptied from
emotions if possible. My goal has never been to point finger at anyone,
but try to fix a reoccurring situation which I would like to avoid.

Running the script shows that 279 reverse (build?) dependencies are
affected by mock. This clearly isn't something one wants to run on a
personal computer, and even less a test which one wants to run sequentially.

Has any thought went into having some kind of runners running on a cloud
to run these tests, and maybe plug this into Salsa's CI to run it
automatically?

I'd very much would love to set this up, at least as a first
experimentation on a bunch of package of the DPMT.

Your thoughts?

Cheers,

Thomas Goirand (zigo)



Re: Maintaining all of the testing-cabal packages under the OpenStack team

2020-06-29 Thread Thomas Goirand
On 6/29/20 12:58 PM, Scott Kitterman wrote:
> On June 29, 2020 10:12:49 AM UTC, Thomas Goirand  wrote:
>> On 6/29/20 8:34 AM, Ondrej Novy wrote:
>>> nope, this is not true. Using the newest debhelper compat level is
>>> recommended, see man page. There is no reason to __not__ upgrade
>>> debhelper compat level. I will always upgrade debhelper in my
>> packages
>>> to the newest debhelper as soon as possible. Please newer downgrade
>>> debhelper in my packages again without asking.
>>
>> I don't agree this is best practice when backports are to be expected.
> 
> I'm substantially less enthusiastic about bumping compat levels than
> Ondrej, but since debhelper 13 is available in buster-backports,> backporting 
> is unrelated to whether it's a good idea or not.

I'm not maintaining OpenStack through the official backports channel,
because OpenStack users need to have access to all versions of OpenStack
backported to the current Stable. These backports are available through
a debian.net channel (available using extrepo).

Therefore, the debhelper backport is not available in my build
environment unless I explicitly do some work to make this happen (and
Ondrej is aware of that). Just bumping the debhelper version (and
without a good reason to do so) just add some unnecessary work
maintaining the debhelper backport for me. By all means, let's bump to
version 12. But why version 13 if you don't need the added features?
This makes no sense to me.

Cheers,

Thomas Goirand (zigo)



Re: Maintaining all of the testing-cabal packages under the OpenStack team

2020-06-29 Thread Thomas Goirand
On 6/29/20 8:34 AM, Ondrej Novy wrote:
> Ondrej, you once cared for the OpenStack packages. Why are you now
> completely careless?
> 
> 
> because it's really hard to cooperate with you. I already tried to
> explain it to you but you didn't listen.

You're mixing 2 things: working on OpenStack package, and caring not to
break them. I'm just asking for the later.

On 6/29/20 8:34 AM, Ondrej Novy wrote:
> yep, that's how it works. We need to move forward and don't keep old,
> buggy and unmaintained packages in Debian, right?

If I'm getting this right, not only you break things (which is ok, if it
isn't on purpose), but now claim that this is the right thing to do. No,
this is not how Debian works, it never was, and hopefully, never will.

> More over, mock debhelper was upgraded to 13, for no apparent reason
> (yet another "cosmetic fix" that isn't helping?). I'd like to remind
> everyone that, increasing debhelper compat version to a number that
> isn't in stable, without a specific reason (like the need of a specific
> feature that wasn't there before) is just annoying for anyone
> maintaining backports. That's truth even for when debhelper itself is
> backported to oldstable (it's always nicer to be able to build a
> backport without requiring another backport at build time).
> 
> nope, this is not true. Using the newest debhelper compat level is
> recommended, see man page. There is no reason to __not__ upgrade
> debhelper compat level. I will always upgrade debhelper in my packages
> to the newest debhelper as soon as possible. Please newer downgrade
> debhelper in my packages again without asking.

I don't agree this is best practice when backports are to be expected.

Cheers,

Thomas Goirand (zigo)



Re: Maintaining all of the testing-cabal packages under the OpenStack team

2020-06-28 Thread Thomas Goirand
On 6/28/20 7:59 PM, Sandro Tosi wrote:
>> Is anyone from the team opposing to this?
> 
> Yes, i'm against your proposal.
> 
>> If so, please explain the
>> drawbacks if the OpenStack team takes over.
> 
> 1. you're personally attacking Ondrej, who is one of the very few
> members of this team doing team-wide work, and that should be enough
> to reject it
> 2. this is clearly an hostile take-over (even if you frame it as a
> proposal), and that should be enough to reject it
> 3. you propose to only update those packages every 6 months, i dont
> find it appropriate: OS is *just* another software we package for
> Debian; is it complex? sure, but it's not special, and it doesnt
> warrant any special treatment.
> 4. you clearly want to have sole and absolute control of the packages
> in the openstack-team, because what would happen if a os-team member
> will upgrade one of those packages (in good faith) and things will
> break? will they get another "well done! :( " email from you?
> 4.1. You wonder why Ondrey "stopped caring" about OS, if that's the
> case, i could see why
> 5. consolidating packages *into* the DPMT/PAPT gives a lot of
> benefits, f.e. people basically got "free" handling of the py2removal
> process; moving packages out is actually detrimental for the python
> ecosystem (at least that's my opinion).
> 
> Thomas, this is not the first time your temperament and aggressive
> behavior is causing some troubles, please reassess how you interact
> and work with other fellow contributors.

Sandro,

I'm sorry if the tone was inappropriate. It probably was. Though it's
not *that* harsh toward Ondrej. At least, it's really FAR from the
hostile behavior he had toward me last summer during debconf, after I
fixed 40 Django RC bugs (due to Django python2 removal), for which I was
thank with threatens.

What I'm painting of what happened is the reality. Let me explain. In
OpenStack, we have this repository:

https://github.com/openstack/requirements/

in this, you'll see the upper-constraints.txt file. This sets pinning
for the current release of OpenStack, which evolves at the same time as
the project. It's updated often during a cycle of 6 months before a
release, then it is frozen for the release. Right now, the stable/ussuri
branch matches what we have in Sid (so one should be looking at that).
Ondrej used to carefully check for this before doing any upload, as I
mentored him to do so. Now he apparently does not care anymore. Call it
personal attacks if you wish, I still don't think this is right.

When you write that:
> "OS is *just* another software we package for Debian; is it complex?
> sure, but it's not special, and it doesnt warrant any special
> treatment."

I don't agree with you here. Absolutely all of the other distributions
that include OpenStack are making sure that nothing breaks it by
careless uploads of not compatible releases of Python modules. Ubuntu
does it, Red Hat as well. Just in Debian, nobody cares but the
maintainers of OpenStack itself.

In fact, let me expand this further, because that's not the first time
I'm raising this issue: we do not threat Python libraries as candidate
for transitions enough, are countless uploads breaks the world of many.
One very good example would be Django, and in the past we had also
SQLAlchemy (though upstream got better for SQLA, so there's less
problems with that one). So yeah, OpenStack shouldn't have any special
treatment *IF* we care enough not breaking things when we update packages.

It'd be nice if we had a framework to be able to rebuild all reverse
build-dependency when we update a package. But currently, we don't have
such CI. If one volunteers to write it, probably we can find some
compute resources to make it happen. That's probably the way out, and
IMO we should really all think about it.

Now, please read what Jeremy wrote, and understand that these package
are really related to OpenStack. Given the fact that these packages are
tightly coupled with OpenStack, it does make sense.

Also, given how often Debian is released (every 2 years, these days?),
updating packages every 6 months doesn't seem that bad, especially if
you consider the set of packages that I'm talking about. They aren't
updated that often upstream.

Please take a step back and understand what's going on. What I would
like to happen, is making sure that things don't break, and currently,
this isn't the case with this set of packages. And this isn't the first
time. So I'm proposing to take measures to make this stop. If you feel
it's a hostile take over, then ok we shall find another way. But then
What is your proposal so that it doesn't happen anymore then?

Cheers,

Thomas Goirand (zigo)



Maintaining all of the testing-cabal packages under the OpenStack team

2020-06-28 Thread Thomas Goirand
Hi,

Under a single Github account, the below packages are maintained:
- mock
- subunit
- testtools
- fixtures
- funcsigs (deprecated, py2 backport)
- testresources
- traceback2
- testscenarios
- testrepository
- extras
- linecache2

Currently, these packages are maintained by a variety of DDs, and
there's no uniform maintenance of them.

The last upload of mock 4.0.2, by Ondrej, broke *a least*:
- nova (see: #963339)
- cloudkitty (see: #963069)
- congress (see: #963312)
- rally (see: #963381)

All of the 4 packages above were able to build in Bullseye (ie: mock
3.0.5) and FTBFS in Sid (with mock 4.0.2).

Well done! :(

Obviously, these couldn't be tested with Mock 4.x at the time of the
last OpenStack freeze, because Mock 4.x didn't exist. However, OpenStack
regularly increases the dependency versions and tries to stay current,
so no blame to be put on OpenStack upstream, just on mock package
maintenance here.

Ondrej, you once cared for the OpenStack packages. Why are you now
completely careless?

More over, mock debhelper was upgraded to 13, for no apparent reason
(yet another "cosmetic fix" that isn't helping?). I'd like to remind
everyone that, increasing debhelper compat version to a number that
isn't in stable, without a specific reason (like the need of a specific
feature that wasn't there before) is just annoying for anyone
maintaining backports. That's truth even for when debhelper itself is
backported to oldstable (it's always nicer to be able to build a
backport without requiring another backport at build time).

I don't want this to happen again. So I am hereby asking to take over
the maintenance of these packages which aren't in the OpenStack team.
They will be updated regularly, each 6 months, with the rest of
OpenStack, following the upstream global-requirement pace. I'm confident
it's going to work well for me and the OpenStack team, but as well for
the rest of Debian.

Is anyone from the team opposing to this? If so, please explain the
drawbacks if the OpenStack team takes over.

Cheers,

Thomas Goirand (zigo)



Re: DebCond20 @ Home -- Python Team Bof & Sprint ?

2020-06-26 Thread Thomas Goirand
On 6/25/20 3:13 AM, Louis-Philippe Véronneau wrote:
> Hello folks!
> 
> As some of you might have seen, DebConf20 @ Home will be happening end
> of August.
> 
> I was wondering if others would be interested in having a Python Team
> BoF to talk about ongoing work/issues (the Python 2 removals comes to my
> mind but I'm sure there are other discussion-worthy topics).
> 
> If there is interest, I'd be happy to submit something to the DebConf
> Content team.
> 
> Maybe it would also be good to set aside half a day during the conf
> (preferably after the BoF) for a sprint to try and fix things?

+1 to both proposal (ie: BoF + sprint)

Thomas



Re: Help fixing #959558 (case: FTBFS: AttributeError: 'tuple' object has no attribute 'lstrip' with sphinx 2.4)

2020-05-29 Thread Thomas Goirand
On 5/27/20 11:26 AM, Dmitry Shachnev wrote:
> Hi all!
> 
> On Tue, May 26, 2020 at 05:06:16PM -0400, Scott Talbert wrote:
>> On Tue, 26 May 2020, Thomas Goirand wrote:
>>> Hi there!
>>>
>>> Does any of you knows how to fix this bug?
>>> https://bugs.debian.org/959558
>>>
>>> Almost all of OpenStack can removed from Bullseye if not fixed in time,
>>> so I tried to fix, but couldn't.
>>
>> It looks like it's a bug in sphinx.  I tried sphinx 3.0.4 and it seems to
>> work fine.  2.4.4 still has the problem, however.
>>
>> Dmitry, do you plan to update to sphinx 3.0.x soon?
> 
> 3.x won't happen soon, but I will fix this bug in the next Sphinx 2.4 upload.
> 
> --
> Dmitry Shachnev

Hi Dmitry,

I stayed quite about this, though today, I'd like to thanks you a lot
for your prompt action fixing this bug! Much appreciated. :)

Cheers,

Thomas



Help fixing #959558 (case: FTBFS: AttributeError: 'tuple' object has no attribute 'lstrip' with sphinx 2.4)

2020-05-26 Thread Thomas Goirand
Hi there!

Does any of you knows how to fix this bug?
https://bugs.debian.org/959558

Almost all of OpenStack can removed from Bullseye if not fixed in time,
so I tried to fix, but couldn't.

Cheers,

Thomas Goirand (zigo)



Re: packaging DiscoDOS - a cli tool for vinyl DJs

2020-05-19 Thread Thomas Goirand
On 5/18/20 11:24 AM, jojo wrote:
> One private question: What do you use? I remember that back then when I
> found out that Debian stable is pretty oldish I was always using
> testing. My teamleader recently told me that he's actually using
> unstable these days and prefers it over testing. I think if one wants
> halfway modern software there is only two options: Use Ubuntu(ish) or
> use Debian unstable? Right?

I use Debian Stable, and whenever I need something new (which is very
rare), then I do my own backports.

> Thanks so much for all this hints

My pleasure.

> One question: Often I
> read the workflow is to first create a source package and then the
> binary package from it. Is that true for python tools as well?

Yes.

> I mean
> there is no "source/binary" in this sense.

There is. I am not sure I understand...

> And this does not seem to be
> a source package but a regular package I would install as user using
> apt: https://salsa.debian.org/python-team/applications/rename-flac.git

This git is using pristine-tar, just like everything else within the
Python team. The debian/master branch contains the upstream sources plus
the Debian folder. This really is a source package repository.

> I guess I knew the answer already ;-) IMHO it's a shame that Discogs is
> not responsive at all to pull-requests in the last ~1-2 years, I would
> rather prefer if I package the official discogs_client (really
> maintained by discogs.com themselves) but to complete my task, I will
> just package my fork because it just works and has two features that I
> just require.

Best is if you can package upstream code, and add your specific patches
in debian/patches, so that they are well identified.

> Anyway, tell me what you think: First package my fork to get things
> done, then ask discogs.com if they want to be in debian and I would do
> it if they would finally work on some of the pulls that are open for
> almost "years" already.

Hopefully that works. I can't tell though, you'll see!

>> You're talking about joining the list. But what about the Python APP
>> team? Do you intend to join it?
> 
> I think I still don't understand the difference between "being on the
> debian-python list" and joining the "python app packagning team". Please
> elaborate again! Most of all: What exactly would it mean if I "join the
> team".

Joining the list only means your receive messages from it. Joining the
team means you become a member of the Salsa Python APP group. We ask
people to first read our policy about it, and agree with it. Then you
can ask to join. If you're accepted, then you get write access to the
Git directly (and you can create new projects too).

Cheers,

Thomas Goirand (zigo)



Re: Bug#953970: Taking over DPMT (Was: python-boto: autopkgtest failure with Python 3.8 as default)

2020-05-16 Thread Thomas Goirand
On 5/16/20 11:02 AM, Andreas Tille wrote:
> On Mon, May 11, 2020 at 10:20:24PM -0400, Noah Meyerhans wrote:
>> Control: tags -1 + patch
>>
>>> I'll move this package to a cloud-team repository and prepare an upload
>>> to unstable on Monday if nobody beats me to it.
>>
>> https://salsa.debian.org/cloud-team/python-boto/-/merge_requests/1
> 
> Could somebody from the cloud-team please merge and upload?
> I'm not a member of this team and can not do anything here.
> 
> Thanks a lot
> 
> Andreas. 
> 

Merged, built and uploaded.

Cheers,

Thomas Goirand (zigo)



Re: packaging DiscoDOS - a cli tool for vinyl DJs

2020-05-16 Thread Thomas Goirand
On 5/16/20 3:36 AM, Paul Wise wrote:
> Would it be fair to say that your main objection is that Ubuntu has
> much higher popularity than Debian

This is what I regret, indeed. It's been like that for many years, and
the trend isn't reversing. We should ask ourselves why. From my point of
view, I see it as a problem of marketing more than OS content or
technical excellence.

> and so the Ubuntu policy to work
> upstream where possible leads people to come to Debian without
> necessarily caring about the Debian community or users but more about
> Ubuntu users?

We are lucky this happens.

> Personally, I think over the years Ubuntu's Debian involvement has
> been a net positive for Debian

Indeed.

> both in terms of packaging and other
> technical changes and in terms of attracting new contributors, often
> Ubuntu migrants end up contributing to Debian more than Ubuntu. I
> think the same goes for derivatives in general.

That's truth. I am very thankful for Canonical to contribute things like
Grub, Python, MySQL and many other things directly in Debian. It's very
nice that some full time employees can take care of stuff like that.
It's also nice that some components are explicitly worked on to be the
same in both distros. I also like the fact that they push contributors
to work on upstream Debian.

Never the less, I've seen multiple occurrences where some people vaguely
knew what Ubuntu was, but never heard of Debian. Saw others saying wrong
things, like Ubuntu was updating faster (which is wrong, as packages are
updated in Sid first). And many other things of that type. Isn't it
legitimate that I'm asking myself why? Shouldn't the Debian project try
to question its image?

Cheers,

Thomas Goirand (zigo)



Re: Disparaging people's motivation to contribute to Debian was: Re: packaging DiscoDOS - a cli tool for vinyl DJs

2020-05-16 Thread Thomas Goirand
On 5/15/20 10:42 PM, Scott Kitterman wrote:
> On Friday, May 15, 2020 4:36:52 PM EDT Thomas Goirand wrote:
>> On 5/15/20 7:09 PM, Scott Kitterman wrote:
>>> On Friday, May 15, 2020 12:55:48 PM EDT Thomas Goirand wrote:
>>>> On 5/15/20 5:43 PM, jojo wrote:
>>>>> Hi,
>>>>>
>>>>> I'd like to join the list because I think my software is a valuable
>>>>> addition to the debian universe, my ultimate goal would be to bring it
>>>>> into Ubuntu Studio because it is music-related.
>>>>
>>>> I really think it's a shame that people join Debian just because of
>>>> Ubuntu... :(
>>>
>>> Thomas,
>>>
>>> Ask yourself if you are modelling being a member of a welcoming community
>>> here?
>>>
>>> There are lots of examples of people who initially became interested in
>>> Debian via Ubuntu and are good Debian contributors and project members.
>>>
>>> You are free to think whatever you want, but I don't think this kind of
>>> sentiment has any place on Debian lists.
>>>
>>> Scott K
>>
>> This was kind of rhetorical, and it is my believe that if it is the way
>> it is, *we* are at fault, globally in Debian. I'm just not sure how to
>> fix that. I BTW don't agree with you, and IMO, this has some place on
>> the Debian lists. Having Debian (directly) appealing to everyone is very
>> important topic.
>>
>> Of course, Jojo is very much welcome, and I'm sorry that you take it
>> this way. I've pointed at many docs to help, so I very much believe he
>> knows I warmly welcome him.
>>
>> Cheers,
>>
>> Thomas Goirand (zigo)
> 
> If you don't actually think it's a shame he wants to participate in Debian, 
> it 
> might be better not to say so.

I should be more careful with my choice of words.

If I'm not mistaking, in English, "it's a shame" can have 2 meanings.
One is like "shame on you", which isn't what I meant. The other one is
"I regret that", which was what I originally wanted to say.

Does it make more sense now?

Cheers,

Thomas Goirand (zigo)



Re: packaging DiscoDOS - a cli tool for vinyl DJs

2020-05-15 Thread Thomas Goirand
On 5/15/20 7:09 PM, Scott Kitterman wrote:
> On Friday, May 15, 2020 12:55:48 PM EDT Thomas Goirand wrote:
>> On 5/15/20 5:43 PM, jojo wrote:
>>> Hi,
>>>
>>> I'd like to join the list because I think my software is a valuable
>>> addition to the debian universe, my ultimate goal would be to bring it
>>> into Ubuntu Studio because it is music-related.
>>
>> I really think it's a shame that people join Debian just because of
>> Ubuntu... :(
> 
> Thomas,
> 
> Ask yourself if you are modelling being a member of a welcoming community 
> here?
> 
> There are lots of examples of people who initially became interested in 
> Debian 
> via Ubuntu and are good Debian contributors and project members.
> 
> You are free to think whatever you want, but I don't think this kind of 
> sentiment has any place on Debian lists.
> 
> Scott K

This was kind of rhetorical, and it is my believe that if it is the way
it is, *we* are at fault, globally in Debian. I'm just not sure how to
fix that. I BTW don't agree with you, and IMO, this has some place on
the Debian lists. Having Debian (directly) appealing to everyone is very
important topic.

Of course, Jojo is very much welcome, and I'm sorry that you take it
this way. I've pointed at many docs to help, so I very much believe he
knows I warmly welcome him.

Cheers,

Thomas Goirand (zigo)



Re: packaging DiscoDOS - a cli tool for vinyl DJs

2020-05-15 Thread Thomas Goirand
On 5/15/20 5:43 PM, jojo wrote:
> Hi,
> 
> I'd like to join the list because I think my software is a valuable
> addition to the debian universe, my ultimate goal would be to bring it
> into Ubuntu Studio because it is music-related.

I really think it's a shame that people join Debian just because of
Ubuntu... :(

> I already filed a bug report against the wnpp pseudo package but I am
> not quite sure what would be the next step and which packaging guides it
> is best to follow to get started with packaging and finally uploading
> it.

IMO, the best thing to start with is the packaging tutorial:
apt-get install packaging-tutorial

It's nicely written. Then you should read the Debian Policy Manual.
Finally, search and read the python policy (in the wiki?) if your app is
Python based.

> Should my next step be following this tutorial on packaging?
> https://packaging.ubuntu.com/html/packaging-new-software.html

This guide talks about bzr. It's not in use anywhere these days, even
Ubuntu people don't use it anymore. It's also Python 2 only, and
therefore, we removed it from Debian.

IMO, you should install sbuild to start with:
https://wiki.debian.org/sbuild

and then go from the above. Note that I don't think using dh_make is a
good idea. It's IMO nicer to just take another Python app as example.
Look at the team's Git for that.

> Also some other questions arise as my tool has a dependency that I am
> pretty sure is not in debian already. the official discogs_client - a
> python sdk to access discogs.com rest api, and actually I forked and
> extended it. pull-request to official repo is pending:
> https://github.com/JOJ0/discogs_client

Well, if you need it for your app, then it must be packaged in Debian as
well if you intend to depend on it.

> Well enough already, let's discuss stuff when I am on the list :-)

You're talking about joining the list. But what about the Python APP
team? Do you intend to join it?

Thanks for your interest in Debian packaging and your intention to
package your app,
Cheers,

Thomas Goirand (zigo)



Re: Bug#937769: getting python-linecache2/python-traceback2 fixes into testing (FAO traceback2, funcsigs nipype and numba maintainers).

2020-04-22 Thread Thomas Goirand
On 4/22/20 6:23 AM, Valentin Vidić wrote:
> On Tue, Apr 21, 2020 at 11:20:16PM +0200, Thomas Goirand wrote:
>> You can remove all of the python-oslo* from the list. The versions in
>> Experimental, which are the next version of OpenStack, are fixed. In 2
>> weeks of time, I'll upload all what I staged in Experimental to Sid
>> (maybe 150 packages?) and that's going to fix it all.
> 
> Will the new OpenStack version also fix this issue?
> 
> #955116 python-murano-pkg-check: FTBFS with Sphinx 2.4: AttributeError:
> 'Sphinx' object has no attribute 'info'

Hopefully yes. As I understand, the issue is in oslo-sphinx, which is
deprecated. I checked, and the master branch of murano-pkg-check doesn't
use oslo-sphinx (and is therefore fixed). I'm waiting for it to be
released, hopefully this week or the next one.

Cheers,

Thomas Goirand (zigo)



Re: Bug#937769: getting python-linecache2/python-traceback2 fixes into testing (FAO traceback2, funcsigs nipype and numba maintainers).

2020-04-21 Thread Thomas Goirand
On 4/20/20 2:51 PM, peter green wrote:
> On 20/04/2020 08:57, Thomas Goirand wrote:
>>> Option 1: fix all four packages to be python 2 free.
>>>
>>> Option 2: Remove python2 stuff from traceback2, python-funcsigs and
>>> numba. Break the dependencies of nipype in sid.
>>>
>>> Option 3: Remove python2 stuff from traceback2, modify python-funcsigs
>>> so it still builds the python2 package but does not run tests with
>>> python 2.
>> Funcsigs is a backport of the PEP 362 function signature features from
>> Python 3.3's inspect module.
> Thanks for the info.
>> Python 2 has never been removed from this
>> package. Though instead, we shall remove this source package entirely
>> from Debian.
> # Broken Depends:
> nipype: python-nipype
> pytest: pypy-pytest
> python-logfury: python3-logfury
> python-oslo.utils: python3-oslo.utils
> 
> # Broken Build-Depends:
> beaker: python3-funcsigs
> kombu: python3-funcsigs
> nipype: python-funcsigs
> pagure: python3-funcsigs
> pytest: pypy-funcsigs
> python-oslo.log: python3-funcsigs
> python-oslo.utils: python3-funcsigs (>= 0.4)
> ripe-atlas-cousteau: python3-funcsigs

You can remove all of the python-oslo* from the list. The versions in
Experimental, which are the next version of OpenStack, are fixed. In 2
weeks of time, I'll upload all what I staged in Experimental to Sid
(maybe 150 packages?) and that's going to fix it all.

For the others, probably I should start filling bugs...

> If what you say is correct then it sounds like the python3-funcsigs
> revese depedencies could be dealt with fairly easily.
> 
> But that still leaves the question of what to do about the dependency of
> pytest on pypy-funcsigs ? should pypy modules be removed from pytest and
> it's reverse-dependencies in the same way that regular python2 modules
> were? how feasible is that? are pypy-* packages only useful with python2
> pypy or are they also useful with python3 pypy?

I really don't know about pypy. Probably the pypy-pytest should indeed
go away, as the initial plan was to switch to pypy3. Maybe tumbleweed
(Stefano Rivera) would be able to answer. I'm adding him as Cc.

>> Traceback2 *already* has Python 2 support removed in Sid. I uploaded
>> this on the 21st of march, pressured by its potential autoremoval.
> 
> Sorry it seems I got my package names mixed up when writing the list of
> options. I said traceback2 where I meant unittest2.

So, if I'm following correctly, what you seem to propose, is to remove
Python 2 from unittest2. If that's the case, then I agree with such a
plan. I just didn't dare to do it yet.

Though in fact, I already worked on that, but stopped, also because
unittest2 FTBFS when I try building it on my laptop. So I've pushed it
in its normal Git repo [1] under a py2-removal branch. If anyone has
some time available to look at it, that'd be nice (I currently don't...).

Cheers,

Thomas Goirand (zigo)

[1] https://salsa.debian.org/python-team/modules/unittest2/



Re: [felix+deb...@gueux.org: Bug#955862: RFS: sphinx-autoapi/1.2.1-1 [ITP]]

2020-04-21 Thread Thomas Goirand
On 4/21/20 11:12 AM, Félix Sipma wrote:
> dget -x
> https://mentors.debian.net/debian/pool/main/s/sphinx-autoapi/sphinx-autoapi_1.3.0-1.dsc
> && sbuild sphinx-autoapi_1.3.0-1.dsc

Hi Felix,

I believe I was downloading the previous version, 1.2.0-1. Just an
advice for lazy sponsors like myself, just put your .dsc URL on *EVERY*
email follow-up... :)

Uploaded.

Thanks for your patience,
Thanks for your contribution to Debian,
Please be patient with FTP masters work,
Cheers,

Thomas Goirand (zigo)



Re: [felix+deb...@gueux.org: Bug#955862: RFS: sphinx-autoapi/1.2.1-1 [ITP]]

2020-04-20 Thread Thomas Goirand
On 4/20/20 4:35 PM, Félix Sipma wrote:
> I use `dgit sbuild` to build the package, and the other tests are
> actually ran (I just tried again)...

Well, I'm just trying to sponsor what you've uploaded to mentors, and so
far, it didn't work (I just tried). Please try downloading the .dsc with
dget and try yourself.

Cheers,

Thomas Goirand (zigo)



Re: [felix+deb...@gueux.org: Bug#955862: RFS: sphinx-autoapi/1.2.1-1 [ITP]]

2020-04-20 Thread Thomas Goirand
On 4/20/20 2:45 PM, Félix Sipma wrote:
> On 2020-04-20 12:06+0200, Thomas Goirand wrote:
>> I'm not sure, the doc still accounts for half of the package. But let's
>> pretend it's ok.
> 
> We are talking about 80KB for the whole package. If it's not a blocker
> for you I prefer keeping it like that.

Fine, ok.

> 
>> Now, the package contains a test suite. It'd be nice to run it. If you
>> add, as dependency: python3-mock, python3-pytest, then pybuild runs the
>> tests automatically. It needs to be overridden though, to avoid running
>> integration tests which are failing (or these tests must be fixed).
>>
>> Can you do that? Then I'll happily upload...
> 
> I added the dependencies, and disabled the integration tests which
> requires golang and dotnet sphinxcontrib extensions. I don't have the
> energy to package these myself...

Of course! However, when I build the package:

The text files are in build/sphinx/text.
   dh_auto_test -O--buildsystem=pybuild
I: pybuild base:217: cd
/<>/.pybuild/cpython3_3.8_sphinx-autoapi/build; python3.8
-m unittest discover -v

--
Ran 0 tests in 0.000s

So, it's running zero tests now. I'm unsure what you did, but that's not
correct. When it tried myself, a lot of tests ran with success.

> Could you please also give me the upload rights for this package when it
> will enter sid?

I'm sorry, but considering the process and how many mistakes I had to
point at which were easy to spot, I wont take this responsibility. I
however will agree to review subsequent uploads.

Cheers,

Thomas Goirand (zigo)



Re: [felix+deb...@gueux.org: Bug#955862: RFS: sphinx-autoapi/1.2.1-1 [ITP]]

2020-04-20 Thread Thomas Goirand
On 4/20/20 11:13 AM, Félix Sipma wrote:
> On 2020-04-19 21:50+0200, Thomas Goirand wrote:
>> On 4/19/20 5:24 PM, Félix Sipma wrote:
>>> I hope I fixed the issues you found
>>
>> Not really... :(
> 
> Let's try again, then...
> 
>> Now the package is 2.7 MB, out of which the extracted library is only
>> 380 KB. Now, the extracted documentation is 3.5 MB and contains 3 MB of
>> fonts, like fontawesome and lato, which are already part of Debian.
>>
>> If everyone was doing like you, installing a small python app would
>> download gigabytes of packages! :)
>>
>> Please:
>> 1/ Push the docs into a separate package that you could call
>> python-sphinx-autoapi-doc.
>> 2/ Remove the embedded fonts fonts from that -doc package, and create
>> symlinks to the appropriate files in the fonts-font-awesome and
>> fonts-lato package.
>>
>> One little general advice now: use mc (or something similar) to go
>> within the resulting .deb files after you've built them, so you can
>> check its content. If you did that, you would have find the issues I'm
>> describing above.
> 
> For sure, getting a 2.7MB package for this small library is not good. I
> do display the list of the files after I build them, but I admit I
> usually don't look too closely to these, and rely on lintian warnings to
> see if fonts package are incorrectly embedded, if the resulting doc is
> too big and should be put to a separate package, etc. Maybe something
> changed recently on lintian (or in my config, but the warnings were not
> displayed on https://mentors.debian.net/package/sphinx-autoapi either)
> because I do not get the embedded fonts warnings I used to have...
> 
> The resulting package is much smaller, I don't think it justifies
> another -doc package. Do you agree?
> 
> Regards,

I'm not sure, the doc still accounts for half of the package. But let's
pretend it's ok.

Now, the package contains a test suite. It'd be nice to run it. If you
add, as dependency: python3-mock, python3-pytest, then pybuild runs the
tests automatically. It needs to be overridden though, to avoid running
integration tests which are failing (or these tests must be fixed).

Can you do that? Then I'll happily upload...

Cheers,

Thomas Goirand (zigo)



Re: [felix+deb...@gueux.org: Bug#955862: RFS: sphinx-autoapi/1.2.1-1 [ITP]]

2020-04-19 Thread Thomas Goirand
On 4/19/20 5:24 PM, Félix Sipma wrote:
> I hope I fixed the issues you found

Not really... :(

Now the package is 2.7 MB, out of which the extracted library is only
380 KB. Now, the extracted documentation is 3.5 MB and contains 3 MB of
fonts, like fontawesome and lato, which are already part of Debian.

If everyone was doing like you, installing a small python app would
download gigabytes of packages! :)

Please:
1/ Push the docs into a separate package that you could call
python-sphinx-autoapi-doc.
2/ Remove the embedded fonts fonts from that -doc package, and create
symlinks to the appropriate files in the fonts-font-awesome and
fonts-lato package.

One little general advice now: use mc (or something similar) to go
within the resulting .deb files after you've built them, so you can
check its content. If you did that, you would have find the issues I'm
describing above.

Cheers,

Thomas Goirand (zigo)



Re: [felix+deb...@gueux.org: Bug#955862: RFS: sphinx-autoapi/1.2.1-1 [ITP]]

2020-04-19 Thread Thomas Goirand
Hi Felix,

Thanks for working on this. Here's my comments. I'm sorry that there's a
lot to say on what you did... :/

On 4/19/20 11:53 AM, Félix Sipma wrote:
>   dget -x 
> https://mentors.debian.net/debian/pool/main/s/sphinx-autoapi/sphinx-autoapi_1.2.1-1.dsc

Looking at your debian/copyright, I'd strongly suggest releasing the
Debian packaging under the same license. For me that's a blocker for
sponsoring the package (it may be ok for others to sponsor).

You wrote in d/control as if the 2nd line of Description: was the
continuation of the short description. This is not the case. Please
write a proper short description (ie: the first line after
"Description:") and a long description (what goes after that first line)
as *not* the continuation of the first line. It's very much ok to repeat
the short description in the long one. This is also a blocker for me to
sponsor the package.

You're packaging the doc "as-is" without using Sphinx to build it. Any
reason why, or you just don't know how yet? :) I'd suggest looking at
other Python package building a -doc package to fix this. I'd also
suggest packaging the doc in a separate -doc package.

Also, please remove:

[import-orig]
merge-mode = replace

from debian/gbp.conf. This is typically something that goes into
~/.gbp.conf rather than on individual packages.

Can you explain in more details than in debian/rules why you're
overriding override_dh_auto_clean ? What does it try to download, apart
from the build dependencies? Can't you set $clean_source = 0; in your
~/.sbuildrc instead?

I hope this helps,
Cheers,

Thomas Goirand (zigo)



Re: py2removal: drop python-pytest by not running tests for python2 packages

2020-04-13 Thread Thomas Goirand
On 4/13/20 11:22 AM, Dmitry Shachnev wrote:
> On Sun, Apr 12, 2020 at 11:05:13PM -0400, Scott Kitterman wrote:
>> On Sunday, April 12, 2020 9:56:01 PM EDT Sandro Tosi wrote:
>>> Hello,
>>> python-pytest is blocking 18 packages from removal, most of them would
>>> be leaves once python-pytest is gone.
>>>
>>> so i was playing with the idea of tackling python-pytest removal by
>>> updating all its rdeps and not run unittests for the python2 binary
>>> (so dropping pytest and the other b-d* only used for tests).
>>>
>>> I know it's suboptimal (some python2 packages can still see updates
>>> until we're ready to drop them and running tests can still have
>>> value), but i think the cost/benefit ratio points towards removing
>>> python-pytest soon rather than wait.
>>>
>>> There are only 25 packages that would need updating, and most of them
>>> are in DPMT/PAPT.
>>
>> Go for it.
> 
> +1 from me too.

+1

Please publish a list here, so we can share the workload.

Cheers,

Thomas Goirand (zigo)



Re: Automatically removing "badges" pictures from README.rst files

2020-04-09 Thread Thomas Goirand
On 4/9/20 10:05 PM, PICCA Frederic-Emmanuel wrote:
> what about lintian brush ?
> 

What's that?

Thomas



Automatically removing "badges" pictures from README.rst files

2020-04-09 Thread Thomas Goirand
Hi team!

In many OpenStack packages, README.rst files are infected with badges.
For example, nova-client here:

https://opendev.org/openstack/python-novaclient/src/branch/master/README.rst

As I'm a good Debian citizen, I'm removing them from the resulting
sphinx doc, because as Lintian warns, it's a privacy breach, and also
because without internet access, the picture just not render.

The problem is that patching the README.rst files is very time
consuming. The number I had to rebase this type of patches is countless.

Therefore, I was wondering if anyone thought about an automated way to
remove these. Maybe we could have some options for dh_sphinxdoc?

Thoughts anyone?

Cheers,

Thomas Goirand (zigo)



Re: packaging manual for a beginner

2020-04-08 Thread Thomas Goirand
On 4/8/20 11:25 AM, Alex Mestiashvili wrote:
> I can imagine that for a newcomer it is kind of
> complicated to start packaging python software. There must be some easy
> py2deb stuff giving you almost ready debian package,(that's what stdeb
> is doing to some extent), but again the documentation is sparse and
> output of stdeb is sometimes scary :) I know, if you need it, just
> implement it, but may be I am missing some already available tools.

I wrote my own "pkgos-debpypi", though it's mainly aimed at OpenStack
packages (actually, not using pybuild as a result, which is kind of
controversial in itself). Piotr implemented something as well for
creating package to use pybuild, but I can't remember the name.

Cheers,

Thomas Goirand (zigo)



Re: packaging manual for a beginner

2020-04-08 Thread Thomas Goirand
On 4/7/20 8:13 PM, Alex Mestiashvili wrote:
> Hi Debian Python folks,
> 
> Is there a good entry point for a newbie who wants to package a python
> module? I am looking for a tool similar to dh-make-perl. In the past
> I've been using stdeb as far as I remember, but it is somehow painful
> compared to other dh-make-x helpers.
> A link with step-by-step instructions would be also helpful. Sorry but I
> am unable to find Debian documentation suitable for a newcomer.
> 
> Thank you,
> Alex
> 

Hi Alex,

I'd advise you to check the content of the packaging-tutorial package.
It contains nice PDFs under /usr/share/doc/packaging-tutorial/
(translated in multiple languages). This is a very good start, IMO.

When you're done with that one, you may start reading the Debian Policy
Manual, which is probably a lot harder to read.

I hope this helps,
Cheers,

Thomas Goirand (zigo)



Re: Reviving #debian-python-changes channel

2020-04-08 Thread Thomas Goirand
On 4/7/20 8:30 PM, Dmitry Shachnev wrote:
> Hi!
> 
> salsabot is dead for two months now, however we can replace it with KGB
> which is alive.
> 
> I am attaching a script that Lisandro Damián Nicanor Pérez Meyer kindly
> shared with me, he used it to revive the #debian-qt-notifications channel.
> 
> I have already replaced qt-kde-team/qt with python-team/modules.
> 
> Can some admin please run it? (You need to get an API token and put a
> SALSA_TOKEN=... line to ~/.devscripts first.)
> 
> python-team/applications is not affected because it seems to already use KGB.
> 
> --
> Dmitry Shachnev

Hi Dmitry,

Thanks a lot for sharing, I've used this script to reconfigure all of
the OpenStack team's repos so that it logs with KGB in
#debian-openstack-commits. You made me save a lot of time! :)

Cheers,

Thomas Goirand (zigo)



Re: Taking over DPMT (Was: python-boto: autopkgtest failure with Python 3.8 as default)

2020-04-04 Thread Thomas Goirand
On 3/30/20 11:44 AM, Andreas Tille wrote:
> I wonder whether we should take over python-boto into DPMT maintenance
> which would enable commits to Git way more easily. 

I'd very much be in the favor of this, especially considering the
package history.

Cheers,

Thomas Goirand (zigo)



When should I drop python2 support for python-linecache2 & python-traceback2 (therefore, unittest2, mock, sphinx, pytest... and the rest of the Python 2 world if I pull this string until end...)

2020-03-20 Thread Thomas Goirand
Hi,

python-fixture (the binary) is gone, therefore we have #952130 and
#952127. I've been reluctant to remove Py2 support from these, because
unittest2 needs it, and this would break a lot of packages (including,
indirectly, stuff like sphinx, pytest, etc.).

What is it that the team suggests at this point? Should I reintroduce
Py2 support in fixtures, or should we go ahead and do a missive Py2 RM
of what's left in the archive?

We're down to only 366 packages with Py2 in testing. We can either
postpone forever, or just go hardly on it (but there will be inevitable
collaterals...).

Your thoughts?

Cheers,

Thomas Goirand (zigo)



Re: Questions about including tests/ directory into package

2020-03-20 Thread Thomas Goirand
On 3/19/20 7:22 AM, Sao I Kuan wrote:
> Hi,
> 
> I'm newcomer to Debian packaging, and trying to add the autopkgtest
> test script into python-tinyalign[1].
> 
> [1] https://salsa.debian.org/med-team/python-tinyalign
> 
> And now I'm facing a (maybe simple) problem.
> 
> The upstream test files are located in tests/ directory, but seems
> this directory is excluded during packaging.
> I have no idea how to include this tests/ directory.
> 
> 1. Please let me know is there any variable for including the specific
> file/directory.
> 2. Why and when the tests/ directory excluded? I tried to find the
> logic in dh-python [2], but failed.
> 
> [2] https://salsa.debian.org/python-team/tools/dh-python
> 
> 3. Is this a correct approach for including the excluded test materials?
> 
> Always appreciates.
> 
> Sincerely,
> 
> Sao I Kuan
> saoik...@gmail.com

Hi,

Ah... how much I hate python and setuptools for being so liberal and let
every developer design its own thing... we end up with no standard, one
having to read setup.py, know all the internals of setuptools, and this
thread is one more collateral of that. :/

For this specific package, what's being packaged is what's found by
setup.py. In that file, you can see:

packages=find_packages("src"),

Setuptools will only find "tinyalign" under the "src" folder, as a
python module to package, which is why that's the only thing that is
going to be packaged.

IMO, the best approach to this problem is to convince upstream to move
their tests folder into the Python package folder. Best is even to make
upstream get rid of the "src" folder, rename that one "tinyalign" and
put the tests folder in it. In other words, make upstream do:

mv src/tinyalign .
rmdir src
mv tests tinyalign
sed -i 's/packages=.*/packages=["tinyalign"],/' setup.py

Cheers,

Thomas Goirand (zigo)



Re: python-babel

2020-02-11 Thread Thomas Goirand
On 2/11/20 4:45 PM, Sandro Tosi wrote:
>> In such case, could you provide me with the source package, rather than
>> just letting me try with Git? Maybe this is going to work then...
> 
> whatever turns out to be, please ensure the package is buildable from
> the git repo. and matches what's going to be uploaded. We have already
> enough packages with missing pristine-tar deltas, with orig tarballs
> that dont match what's in the upstream+pristine-tar output, let's not
> increase that number.
> 
> Thanks,

Sure, we're on the same page. The purpose is just to diagnose what's
going on, and fix it the Git repo!

Cheers,

Thomas



Re: How to find DD member to perform reviews and uploads

2020-02-11 Thread Thomas Goirand
On 2/11/20 12:12 PM, Ondrej Novy wrote:
> Hi,
> 
> út 11. 2. 2020 v 11:02 odesílatel Adam Cecile  <mailto:acec...@le-vert.net>> napsal:
> 
> Bringing the package into the archive would be doable, if someone get
> legal contacts at LSI and asked for official statement authorizing
> Debian to redistribute their binary. I did this in the past for another
> package but I cannot guarantee any success.
> 
> 
> be careful with this. See DFSG 8 :)

This is irrelevant. We're discussing uploading something into non-free,
and as you know, non-free is not part of Debian, and the DFSG doesn't
apply there.

Cheers,

Thomas Goirand (zigo)



Re: python-babel

2020-02-11 Thread Thomas Goirand
On 2/11/20 11:16 AM, Håvard Flaget Aasen wrote:
> 
>>
>> I tried to build python-babel to sponsor the upload from what's in the
>> Git in the DPMT, and it failed to build for me:
>>
>> # Generate the localedata folder data out of the xml files
>> # downloaded in debian/repack
>> scripts/import_cldr.py common
>> Processing root.xml (Language = root; Territory = 001)
>>
>> root: Unsupported number system "adlm" in 
>>
>> [... lots of lines like this one ...]
> 
> I'm also getting these lines. I tested with the current 2.6.0 which also
> gives the same messages, the build log for 2.6.0 in Ubuntu prints the
> same messages.
> 
> Now for the files actually being processed
> It is 719 in version 2.4.0
> 745 in version 2.6.0 and
> 764 in version 2.8.0
> 
> Not that it is supposed to be this way of course. Though I do believe
> the actual content in the package is correct.
> I can have a look at what we are actually importing in d/repack and see
> if anything has changed from version 2.4.0 to 2.8.0
> 
> 
>>
>> root: Unsupported number system "vaii" in > numberSystem="vaii">
>>
>> Traceback (most recent call last):
>>    File "scripts/import_cldr.py", line 968, in 
>>  main()
>>    File "scripts/import_cldr.py", line 193, in main
>>  dump_json=bool(options.dump_json)
>>    File "scripts/import_cldr.py", line 207, in process_data
>>  _process_local_datas(sup, srcdir, destdir, force=force,
>> dump_json=dump_json)
>>    File "scripts/import_cldr.py", line 435, in _process_local_datas
>>  write_datafile(data_filename, data, dump_json=dump_json)
>>    File "scripts/import_cldr.py", line 167, in write_datafile
>>  with open(path, 'wb') as outfile:
>> IOError: [Errno 2] No such file or directory:
>> '/<>/babel/locale-data/root.dat'
>>
> I can't replicate this. I tried with cowbuilder/pbuilder and sbuild and
> the build completes fine on my systems.
> 
> Håvard


In such case, could you provide me with the source package, rather than
just letting me try with Git? Maybe this is going to work then...

FYI, I'm just using plain sbuild with git-buildpackage.

Cheers,

Thomas Goirand (zigo)



Re: How to find DD member to perform reviews and uploads

2020-02-11 Thread Thomas Goirand
On 2/9/20 3:12 PM, Adam Cecile wrote:
> Hello Debian-Python,
> 
> 
> I have a couple of package ready on Salsa but my RFS sent here remained
> unanswered.
> 
> So I'm asking again for your help again ! Here is a list of package
> waiting for uploads:
> 
> 
> https://salsa.debian.org/python-team/modules/djangorestframework-api-key
> 
> https://salsa.debian.org/python-team/modules/aionotify
> 
> https://salsa.debian.org/python-team/modules/python-fastjsonschema
> 
> https://salsa.debian.org/python-team/modules/python-cassandra-driver
> 
> https://salsa.debian.org/python-team/modules/python-sunrise
> 
> 
> Thanks in advance,
> 
> 
> Regards, Adam.

Hi Adam,

I've built and uploaded python-cassandra-driver. Thanks for your
contribution to Debian. Do you still need sponsoring for the above?

Also, thanks a lot for your work on megacli packaging, it's very useful.
However, I haven't found the matching source packages. Do you also think
it'd be possible to push these into the non-free repository of Debian?
Do you even have some sources from upstream, or only binaries? I very
much need this for my work (we use Dell LSI hardware RAID a lot), and my
colleagues would love to have this through the official channel, rather
than from your unofficial repository. Your thoughts?

Cheers,

Thomas Goirand (zigo)



Re: python-babel

2020-02-11 Thread Thomas Goirand
On 1/10/20 8:50 PM, Håvard Flaget Aasen wrote:
> Hello,
> 
> The python-babel [1] package is currently broken, there isn't any bugs
> reported yet, but after the transit to
> python3.8 it failed the testsuite. New upstream version 2.7.0 and above
> fixes these issues.
> To update the package there is a 'repack' script which i believe is
> broken, I at least did most of the work
> manually. For that reason I haven't dared push it to the official repo
> yet, if anyone have time, it will be nice
> with a second opinion. It's still some work left, but it builds and
> tests fine. The version I updated is currently here [2]
> 
> The script might only require a minor fix but i noticed that the files
> being downloaded already exist
> within Debian. The package unicode-cldr-core [3].
> Is it possible to have this as a build-dependency, have a Files-Excluded
> in d/copyright file and
> drop the repack script altogether? If there still is a reason to change
> the files in the first place.
> If this makes it any easier/better of course.
> 
> I'll appreciate all kinds of feedback.
> 
> 
> Håvard
> 
> 
> [1] https://tracker.debian.org/pkg/python-babel
> [2] https://salsa.debian.org/haava-guest/python-babel
> [3] https://tracker.debian.org/pkg/unicode-cldr-core
> 

Hi,

I tried to build python-babel to sponsor the upload from what's in the
Git in the DPMT, and it failed to build for me:

# Generate the localedata folder data out of the xml files
# downloaded in debian/repack
scripts/import_cldr.py common
Processing root.xml (Language = root; Territory = 001)

root: Unsupported number system "adlm" in 

[... lots of lines like this one ...]

root: Unsupported number system "vaii" in 

Traceback (most recent call last):
  File "scripts/import_cldr.py", line 968, in 
main()
  File "scripts/import_cldr.py", line 193, in main
dump_json=bool(options.dump_json)
  File "scripts/import_cldr.py", line 207, in process_data
_process_local_datas(sup, srcdir, destdir, force=force,
dump_json=dump_json)
  File "scripts/import_cldr.py", line 435, in _process_local_datas
write_datafile(data_filename, data, dump_json=dump_json)
  File "scripts/import_cldr.py", line 167, in write_datafile
with open(path, 'wb') as outfile:
IOError: [Errno 2] No such file or directory:
'/<>/babel/locale-data/root.dat'

Can you fix this?

Cheers,

Thomas Goirand (zigo)



Removing python-xmlbuilder from Debian

2019-12-25 Thread Thomas Goirand
Hi Andreas,

Currently, python-xmlbuilder cannot easily be ported to Py3. When
running its unit tests, here's the result:

  File "/<>/xmlbuilder/test.py", line 135, in test_with6
eq_(str(self.xml), '11')
  File "/<>/xmlbuilder/__init__.py", line 208, in __str__
return hdr + str(self.__stack[0])
TypeError: __str__ returned non-string (type bytes)

Feel free to download the updated package from
http://salsa.debian.org/openstack-team/python/python-xmlbuilder to test
this yourself.

Currently, the only reverse dependency of this package is
python-pbcommand. So we have 2 choices:

1/ Fix python-xmlbuilder
2/ Get python-xmlbuilder and python-pbcommand removed from Debian.

Your thoughts?

Cheers,

Thomas Goirand (zigo)



The package python2.7 is RC?

2019-12-24 Thread Thomas Goirand
Hi,

I've seen that the package python2.7 is now of severity RC because of
the py2 removal process:
https://bugs.debian.org/937569

I wonder, is this a mistake done by the severity script? If so, are
there more mistakes?

Happy Chrismass,

Thomas Goirand (zigo)



Re: RFS: opentracing-python/2.2.0-2 [ITP] -- opentracing interface for Python

2019-11-24 Thread Thomas Goirand
On 11/24/19 10:42 PM, Fabrice BAUZAC-STEHLY wrote:
> Hello Thomas,
> 
> I have tried to fix all the issues you have mentioned in 2.2.0-2:
> 
> dget -x 
> https://mentors.debian.net/debian/pool/main/o/opentracing-python/opentracing-python_2.2.0-2.dsc
> 
> Could you please have a look and/or sponsor the package?
> 
> Thanks a lot!
> 
> Best regards

Well, remove the 2nd entry in debian/changelog: this package has never
been uploaded to Debian, so you are documenting something that never
happened in Debian.

Cheers,

Thomas Goirand (zigo)



Re: RFS: opentracing-python/2.2.0-1 [ITP] -- opentracing interface for Python

2019-11-24 Thread Thomas Goirand
On 11/20/19 10:53 PM, Fabrice BAUZAC-STEHLY wrote:
> Alternatively, one can download the package with dget using this command:
> 
>   dget -x 
> https://mentors.debian.net/debian/pool/main/o/opentracing-python/opentracing-python_2.2.0-1.dsc

Hi,

My review.

d/changelog:


No need to write "[ Fabrice BAUZAC ]". That's for when there's multiple
persons in the changelog.

d/compat:
=
Please get rid of this file, and in d/control write debhelper-compat
(=12) instead (and remove debhelper there).

d/control:
==
Not mandatory, but it'd be nice to run "wrap-and-sort -bastk" to
clean-up build-depends ordering.

d/copyright:

There's no "Files: *" entry there, and it looks like you're listing all
the files of upstream. Please don't do that.

The only blocker I see is about d/copyright, the other things are more
cosmetic changes that I'm suggesting.

Cheers,

Thomas Goirand (zigo)



Re: FYI: Python 3 migration of distributuion

2019-11-13 Thread Thomas Goirand
On 11/13/19 6:11 PM, Charles Cazabon wrote:
> Unfortunately, it's difficult to find the hours to devote to this task.  I
> don't know when, or even if, I could have a beta release ready.
> 
>> If you convert python 2 code for 2.7 style, then we can support both
>> python 2 and 3.
> 
> I'm not very interested in doing this, as it means not using a lot of 3.x
> features, or not using them in the most Pythonic way.  My thought was that
> this project would target at least Python 3.7; anyone who didn't want to run
> 3.7 could still run "legacy" getmail under Python 2.7.
> 
> Charles

Hi Charles,

Please allow me to comment on this.

Converting some Python 2.7 code to Python 3.x is not very hard for
anyone who's a little bit trained to it. Most of the time, we see the
same patterns over and over again. So it would be relatively easy for a
lot of us to help you transition the current code to Python 3.x, without
even the need to understand much about getmail itself. Add to this that
there's automated tools to do the most boring part of it (like sixer,
and someone mentioned another one which I can't remember).

I had a quick look into Getmail's code, and it looks very similar to
what I've seen in other programs. Lots of print statements that needs to
be converted, a bit of except to patch, a few itertools too. The code is
also of quite a manageable size (by that I mean: it's a lot smaller than
lots of other stuff I converted to Py3 in a few hours...).

Some of us are offering such a help, and quickly, if you accept the
patches, you'll be able to have Getmail in a good enough shape so it can
stay in Debian, Ubuntu, Fedora.

On the opposite side, if you attempt to do a huge rework of Getmail
(which is maybe needed, I don't know Getmail enough to be able to tell),
probably nobody else but you will be able to do it, and you wont get
much help. As you wrote it may take some time, and you aren't even able
to predict how much.

So, why don't you just take the offer, get the Python 3.x patches in,
and then *later* do your rework, maybe in a version 6 of Getmail? This
sounds a much more reasonable approach to me.

Note that I'm not at all involved in Getmail or I'm not even a user, I'm
just trying to convince you to do the right move here... on the
direction which I believe will serve your users best, for example have
Getmail stay on the next Ubuntu 20.04 LTS (Debian Bullseye is for in 2
years, so you may have more time for that one...).

Cheers,

Thomas Goirand (zigo)



Re: getmail: Python2 removal in sid/bullseye

2019-11-13 Thread Thomas Goirand
On 11/13/19 3:31 PM, Iustin Pop wrote:
> On 2019-11-13 15:06:54, Thomas Goirand wrote:
>> On 11/12/19 4:37 PM, Osamu Aoki wrote:
>>> The related binary packages are available in 2 binary names (depending on 
>>> release)
>>>  getmail4 (version=4,5) popcon installed ~2000
>>>  getmail  (version=3,5) popcon installed ~1000
>>>
>>> https://qa.debian.org/popcon-graph.php?packages=getmail%20getmail4_installed=on_legend=on_ticks=on_fmt=%25Y-%25m=1
>>>
>>> I think this qualifies for "py2keep".
>>
>> IMO, this qualifies for RM-RoM. getmail is an alternative to fetchmail,
>> which is still available in Debian (and with 4 times the number of
>> installed package in popcon...). So I see no reason to keep getmail
>> then. Maybe tell this to upstream, and they may think another time.
> 
> Uh, no. Functionality-wise, they're quite different. getmail is (AFAIK)
> the only tool that works for gmail with ASPs disabled (i.e. with OAUTH).

Oh ! Thanks for the insight. I didn't know.

> Heck, I'd be very willing to maintain Py3 patches myself, because I need
> this tool.

Then there's no issue anymore? :)

Thomas



Re: getmail: Python2 removal in sid/bullseye

2019-11-13 Thread Thomas Goirand
On 11/12/19 4:37 PM, Osamu Aoki wrote:
> The related binary packages are available in 2 binary names (depending on 
> release)
>  getmail4 (version=4,5) popcon installed ~2000
>  getmail  (version=3,5) popcon installed ~1000
> 
> https://qa.debian.org/popcon-graph.php?packages=getmail%20getmail4_installed=on_legend=on_ticks=on_fmt=%25Y-%25m=1
> 
> I think this qualifies for "py2keep".

IMO, this qualifies for RM-RoM. getmail is an alternative to fetchmail,
which is still available in Debian (and with 4 times the number of
installed package in popcon...). So I see no reason to keep getmail
then. Maybe tell this to upstream, and they may think another time.

Cheers,

Thomas Goirand (zigo)



Re: Policy About Maintainer and Uploaders Fields (was: PAPT: join request)

2019-11-12 Thread Thomas Goirand
On 11/12/19 2:38 AM, Sandro Tosi wrote:
> please also realize not everyone shares the
> same ideas as yours and you should try sometimes to respect those
> people decisions too.

With all due respect, the point that I'm trying to make is that this
policy is only there because what I believe is a minority (which you are
part of) supports it, and which everyone else doesn't like.

I do respect your view, and I'm sorry if voicing my opinion is (again)
going against what's been established for years, just like it did when
we moved from SVN to Git. Though that's how things evolve, hopefully for
the better. Also, it's ok if we don't agree... :)

Cheers,

Thomas Goirand (zigo)



Re: Policy About Maintainer and Uploaders Fields (was: PAPT: join request)

2019-11-11 Thread Thomas Goirand
On 11/11/19 9:17 PM, Scott Kitterman wrote:
> Personally, I've been judicious in putting myself as Maintainer in DPMT and 
> PAPT packages.  If we were to ditch the current policy, my immediate response 
> would be to remove DPMT/PAPT from uploaders and maintain them outside the 
> team.  It's about 1/4 of my DPMT/PAPT packages.
> 
> I suspect I'm not the only one.
> 
> Your proposal is not cost free for the teams.
> 
> Scott K

Hi Scott,

Exactly what difference would it make? Just that your package would
really appear as not team-maintained, which is already the current plain
reality (as the "unwritten policy" tells nobody else but you can touch
the package). So I don't see why anyone in the team would mind.

Cheers,

Thomas Goirand (zigo)



Re: Policy About Maintainer and Uploaders Fields (was: PAPT: join request)

2019-11-11 Thread Thomas Goirand
On 11/11/19 9:21 AM, Fabrice BAUZAC-STEHLY wrote:
> For the record, it looks like this policy comes from the package
> "developers-reference", section "Collaborative maintenance".

Absolutely not. The developers-reference doesn't tell what the Python
team policy is when the Uploaders field contains the team address. It
only tells that it is possible to do that, not what it implies in our team.

Cheers,

Thomas Goirand (zigo)



Re: Policy About Maintainer and Uploaders Fields (was: PAPT: join request)

2019-11-10 Thread Thomas Goirand
On 11/10/19 1:20 AM, Sandro Tosi wrote:
> is there any public trace of these "many voices"?

Just like when we discussed moving away from SVN to Git, we can't know
the exact number unless we have a kind of poll/vote (but we don't
actually *have* to start such poll... I'm just saying it's hard to know
otherwise).

However, I can account for maybe 5 people (it's hard to remember) who
told me (face to face) they hate this policy, and it felt like there was
a broad consensus that it was really against a team spirit, plus it made
little sense to have a package team maintained with strong ownership. I
wouldn't name the persons I have in mind publicly, because they haven't
granted me the permission to do so, and therefore, it wouldn't be nice
to do so.

All of this is just feelings I had during conversations with others, and
of course, cannot be accounted as just plain facts, so just take it as
it is: just a report of the feeling I had when talking with others, that
it looks like I'm far from the only one disliking this policy because of
the above mentioned reasons.

As Louis-Philippe wrote, it's very much ok to delay this conversation,
but sooner or later, it will come back on the table.

Thomas Goirand (zigo)



Re: Policy About Maintainer and Uploaders Fields (was: PAPT: join request)

2019-11-08 Thread Thomas Goirand
Hi intri!

I'm very glad to count you as a member of the team! Welcome. [1]
I'd be glad if more perl team members join, it's so much always a
pleasure to meet you at each debconf.

On 11/8/19 8:54 AM, intrigeri wrote:
>  - In https://wiki.debian.org/Teams/PythonModulesTeam/HowToJoin,
>the "Policy About Maintainer and Uploaders Fields" section
>mentions an "unwritten policy". Said policy seems to have been
>written since:
>
> https://salsa.debian.org/python-team/tools/python-modules/blob/master/policy.rst

It's probably time to revisit this policy. I've heard many voices
telling that a package should either be in the team, or just not, and I
very much agree with this. This middle-ground makes no sense.

Cheers,

Thomas Goirand (zigo)

[1] I have no admin rights to add you, but welcome anyways...



Re: Python module packages that don't bytecompile on installation?

2019-11-03 Thread Thomas Goirand
On 11/3/19 10:27 AM, Matthias Klose wrote:
> On 03.11.19 02:20, Paul Wise wrote:
>> On Sat, Nov 2, 2019 at 6:04 PM Matthias Klose wrote:
>>> Python2
>>> removal, or the introduction of Python 3.8.  3.8 also offers a
>>> central directory
>>> for bc files, so that's maybe another thing to look at, but not a
>>> priority now
>>> from my point of view.
>>
>> Agreed re Python 2 etc. Eventually switching to using a central
>> directory in /var would be nice, right now the .pyc files are dumped
>> cruftily into /usr subdirs.
> 
> yes, but that's something when 3.7 is removed.
> 
> Matthias

Maybe that's a newbie question, sorry for this, but...

What happens when we upgrade to a minor Python 3 version? Do we get
everything recompiled? Is there a userland command to rebuild everything?

Cheers,

Thomas Goirand (zigo)



Re: packages with the py2keep tag

2019-11-01 Thread Thomas Goirand
On 11/1/19 11:40 AM, Matthias Klose wrote:
> https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=py2keep;users=debian-python@lists.debian.org
> 
> 
> doesn't show show too many packages yet, which is good. Not sure if we
> should tag the dependencies of these packages with a different tag, e.g.
> py2keep-dep for python-setuptools.

I think that's a very good idea. A quick message in the relevant bugs
would also be good, as not everyone will understand the meaning of the tag.

> Im unsure if I understand the rationale for the mercurial py2keep tag.

I don't understand the rational for #936307 (ie: claws-mail) and many
others either.

Cheers,

Thomas Goirand (zigo)



Re: Raising severity to serious for some Python 2 leaf packages with no Python 3 support upstream

2019-10-15 Thread Thomas Goirand
On 10/15/19 7:08 PM, Sandro Tosi wrote:
> On Tue, Oct 15, 2019 at 12:55 PM Thomas Goirand  wrote:
>>
>> On 10/15/19 5:00 AM, Craig Small wrote:
>>>
>>>
>>> On Tue, 15 Oct. 2019, 1:04 pm Thomas Goirand, >> <mailto:z...@debian.org>> wrote:
>>>
>>> Please re-read the excellent contribution from Neil Williams
>>> in this thread, and explain again why we have a special case... :)
>>>
>>> I just did re-read it; especially about it's RC bugs not a total removal
>>> RM so these packages will sit in unstable and not move into testing.
>>>
>>> That works for me.
>>
>> Hi Craing,
>>
>> I'm not sure if that helps, but here's a patch... :)
> 
> did you test your patch?
> 
> +-print "v1 snmpset result: ", res, "\n"
> ++print9"v1 snmpset result: ", res, "\n")

I have not, because Craig said he would remove Py2, though I had it
written already, so I though I would send it anyways...

Thomas



Re: Raising severity to serious for some Python 2 leaf packages with no Python 3 support upstream

2019-10-15 Thread Thomas Goirand
On 10/15/19 5:00 AM, Craig Small wrote:
> 
> 
> On Tue, 15 Oct. 2019, 1:04 pm Thomas Goirand,  <mailto:z...@debian.org>> wrote:
> 
> Please re-read the excellent contribution from Neil Williams
> in this thread, and explain again why we have a special case... :)
> 
> I just did re-read it; especially about it's RC bugs not a total removal
> RM so these packages will sit in unstable and not move into testing.
> 
> That works for me.

Hi Craing,

I'm not sure if that helps, but here's a patch... :)

Thomas
diff -Nru net-snmp-5.7.3+dfsg/debian/changelog 
net-snmp-5.7.3+dfsg/debian/changelog
--- net-snmp-5.7.3+dfsg/debian/changelog2019-01-05 06:16:10.0 
+0100
+++ net-snmp-5.7.3+dfsg/debian/changelog2019-10-15 04:25:31.0 
+0200
@@ -1,3 +1,9 @@
+net-snmp (5.7.3+dfsg-5.1) UNRELEASED; urgency=medium
+
+  * Add Python 3 support.
+
+ -- Thomas Goirand   Tue, 15 Oct 2019 04:25:31 +0200
+
 net-snmp (5.7.3+dfsg-5) unstable; urgency=medium
 
   * Use debhelper macros for shlibs Closes: #912685
diff -Nru net-snmp-5.7.3+dfsg/debian/control net-snmp-5.7.3+dfsg/debian/control
--- net-snmp-5.7.3+dfsg/debian/control  2019-01-05 06:16:10.0 +0100
+++ net-snmp-5.7.3+dfsg/debian/control  2019-10-15 04:25:31.0 +0200
@@ -8,6 +8,7 @@
 Build-Depends: debhelper (>= 11),
  libtool, libwrap0-dev, libssl-dev, perl (>=5.8), libperl-dev,
  python-all (>= 2.6.6-3~), python-setuptools (>=0.6b3), python2.7-dev,
+ python3-all-dev, python3-setuptools,
  autoconf, automake, debianutils (>=1.13.1),
  dh-python,
  bash (>=2.05), findutils (>=4.1.20), procps,
@@ -160,6 +161,18 @@
  The Net-SNMP Python support files provide the Python functions for
  integration of SNMP into applications written in Python.
 
+Package: python3-netsnmp
+Section: python
+Architecture: any
+Depends: ${python3:Depends}, ${shlibs:Depends}, ${misc:Depends}
+Description: SNMP (Simple Network Management Protocol) Python 3 support
+ The Simple Network Management Protocol (SNMP) provides a framework
+ for the exchange of management information between agents (servers)
+ and clients.
+ .
+ The Net-SNMP Python support files provide the Python functions for
+ integration of SNMP into applications written in Python 3.
+
 Package: tkmib
 Architecture: all
 Depends: libsnmp-perl (>=${source:Version}), perl-tk, ${misc:Depends}
diff -Nru net-snmp-5.7.3+dfsg/debian/patches/fix-space-vs-tabs.patch 
net-snmp-5.7.3+dfsg/debian/patches/fix-space-vs-tabs.patch
--- net-snmp-5.7.3+dfsg/debian/patches/fix-space-vs-tabs.patch  1970-01-01 
01:00:00.0 +0100
+++ net-snmp-5.7.3+dfsg/debian/patches/fix-space-vs-tabs.patch  2019-10-15 
04:25:31.0 +0200
@@ -0,0 +1,16 @@
+Description: Fix space vs tabs
+Author: Thomas Goirand 
+Forwarded: no
+Last-Update: 2019-10-15
+
+--- net-snmp-5.7.3+dfsg.orig/python/setup.py
 net-snmp-5.7.3+dfsg/python/setup.py
+@@ -10,7 +10,7 @@ args = sys.argv[:]
+ for arg in args:
+ if arg.find('--basedir=') == 0:
+ basedir = arg.split('=')[1]
+-  sys.argv.remove(arg)
++sys.argv.remove(arg)
+ intree=1
+ 
+ if intree:
diff -Nru net-snmp-5.7.3+dfsg/debian/patches/py3-compat.patch 
net-snmp-5.7.3+dfsg/debian/patches/py3-compat.patch
--- net-snmp-5.7.3+dfsg/debian/patches/py3-compat.patch 1970-01-01 
01:00:00.0 +0100
+++ net-snmp-5.7.3+dfsg/debian/patches/py3-compat.patch 2019-10-15 
04:22:51.00000 +0200
@@ -0,0 +1,419 @@
+Description: Py3 compat
+Author: Thomas Goirand 
+Forwarded: no
+Last-Update: 2019-10-15
+
+--- net-snmp-5.7.3+dfsg.orig/python/netsnmp/client.py
 net-snmp-5.7.3+dfsg/python/netsnmp/client.py
+@@ -2,7 +2,7 @@ import client_intf
+ import string
+ import re
+ import types
+-from sys import stderr
++import sys
+ 
+ # control verbosity of error output
+ verbose = 1
+@@ -37,10 +37,10 @@ def _parse_session_args(kargs):
+ }
+ keys = kargs.keys()
+ for key in keys:
+-if sessArgs.has_key(key):
++if key in sessArgs:
+ sessArgs[key] = kargs[key]
+ else:
+-print >>stderr, "ERROR: unknown key", key
++print("ERROR: unknown key", key, file=sys.stderr)
+ return sessArgs
+ 
+ def STR(obj):
+--- net-snmp-5.7.3+dfsg.orig/python/netsnmp/tests/test.py
 net-snmp-5.7.3+dfsg/python/netsnmp/tests/test.py
+@@ -8,7 +8,7 @@ import time
+ 
+ class BasicTests(unittest.TestCase):
+ def testFuncs(self):
+-print ""
++print("")
+ var = netsnmp.Varbind('sysDescr.0')
+ var = netsnmp.Varbind('sysDescr','0')
+ var = netsnmp.Varbind(
+@@ -19,67 +19,67 @@ class BasicTests(unittest.TestCase):
+ 
+ var = netsnmp.Varbind('.1.3.6.1.2.1.1.1','0')
+ 
+-print "---v1 GET tests -\n"
++print("---v1 GET tests -\n")
+

Re: Raising severity to serious for some Python 2 leaf packages with no Python 3 support upstream

2019-10-14 Thread Thomas Goirand
On 10/14/19 3:54 PM, Sandro Tosi wrote:
>>> But in both cases, it's going to take a very long time. Do we really
>>> want to get stuck on these packages for like forever, or would it feel
>>> ok to raise the severity to serious, so that the package gets
>>> auto-removed and then we can work on removing Python 2 from its
>>> dependencies?
>>
>> for me it's perfectly fine to rise severity to serious of all leaf packages 
>> with py2remove bug as soon as possible. And let's do it "automatically" 
>> everytime any package gets leaf state.
> 
> i think it's a bit premature to raise severity to RC (we should also
> check with the release team): these bugs have been opened since just 2
> months and a half, and the development cycle for bullseye started not
> longer before. there's still a lot of time.

It's not as if it has been 10 years we know about this transition, right? :)

How many months do you want to add? Do you seriously think it's going to
significantly change something about an eventual situation where
upstream hasn't done the work? If upstream did the work, then what's
blocking us?

If we see a package where upstream has Py3 support, we don't need to
raise the severity, we can just do the work instead.

> (and removing a py2
> package from a leaf pkg takes 5 minutes, usually, so if everyone in
> this thread had done that instead of writing an email, we'd be down 5
> bugs already :D )

I probably have done hundreds of them already. And I do not agree that
it takes just 5 minutes. For a trivial leaf package, probably, but when
you're trying to fix a long chain of dependencies, it can sometimes be a
non-trivial work. Going from the leaf package and rewinding can
sometimes lead to mistakes.

> I think it's also extremely premature (and just a bad idea in general)
> to consider breakage of reverse dependencies but just dropping py2
> packages. the Release Team is extremely unhappy with this approach for
> dealing with the py2removal bugs, so let's not even consider that
> option.

Yes. Which is why we should raise severity of bugs to RC, and probably
even remove packages if we need to. Otherwise, this process will take
forever (ie: longer than a Debian release cycle).

Thomas Goirand (zigo)



Re: Raising severity to serious for some Python 2 leaf packages with no Python 3 support upstream

2019-10-14 Thread Thomas Goirand
On 10/14/19 11:05 PM, Craig Small wrote:
> Hi All,
>   Just be careful with the bugs severity on complicated packages. I
> totally get the python only packages that produce a single binary, go
> for it for those.
> 
> However consider the net-snmp python module. It's python 2 only and
> upstream isn't changing it. In fact I am pretty sure they don't support it.
> 
> So get rid of it right? Yes, except that I cannot get a new version of
> net-snmp into the archive due to a perl module bug.

Hi Craig,

Either we don't have enough details about net-snmp, or you're trying to
push for not-valid-yet-another-exception.

Could you explain exactly what this perl module bug is? How is this
related to a Python package? Why would we make an exception with
net-snmp? Please re-read the excellent contribution from Neil Williams
in this thread, and explain again why we have a special case... :)

Thomas



Re: Streamlining the use of Salsa CI on team packages

2019-10-14 Thread Thomas Goirand
On 9/16/19 10:03 PM, Hans-Christoph Steiner wrote:
>>> I know the Ruby team also decided to use debian/salsa-ci.yml instead of
>>> debian/gitlab-ci.yml [2]. I guess we should also do the same.
> This is still an open question:
> https://salsa.debian.org/salsa-ci-team/pipeline/issues/86
> 
> Debian has a bad habit of customizing things that do not need to be
> customizing.  That raises the barrier for contributors ever higher, in a
> "death by a thousand papercuts" kind of way.  I think we should stick to
> the standard file name for GitLab CI.

The issue is that, from a packaging standpoint, we cannot add a file if
it's not in the debian folder, because this makes change to the upstream
files. So, no choice...

Thomas Goirand (zigo)



Re: Python2 removal: package with low-popcon reverse dependencies

2019-10-13 Thread Thomas Goirand
On 10/11/19 9:26 PM, Christian Kastner wrote:
> On 11.10.19 19:47, Matthias Klose wrote:
>> On 11.10.19 18:27, Christian Kastner wrote:
>>> This would nevertheless be a case for the "py2keep", right?
>>
>> No.
>>
>> #933348 is another bug for removed packages (mopidy-scrobler). Do you
>> really want to keep that? 
> 
> Not at all, I'd actually prefer just dropping the Python2 version of
> python-cachetools. However, this breaks things, and it wasn't entirely
> clear to me much breakage is acceptable.

It's becoming increasingly clear to me that, at some point, we will need
to just ignore the breakage. Bust this needs to be discussed here first.
Maybe the better way to fix the situation is to increase the severity of
the py2 removal bug to serious, so we leave a chance to the maintainer
to take care of the package before the autorm. See the other thread I've
just started about this.

Cheers,

Thomas Goirand (zigo)



Raising severity to serious for some Python 2 leaf packages with no Python 3 support upstream

2019-10-13 Thread Thomas Goirand
Hi,

In some cases I've seen, particularly in the med or science team,
switching some packages to Python 3 requires a significant effort.

For example, today I looked into removing Python 2 from python-cogent.
Running sixer on all files lead to a huge log of problems to solve by
hand. There's no upstream support for Python 3 on that one.

For this kind of package, I see no way out except:
- Upstream works on Python 3 support
- Someone in Debian makes the effort

But in both cases, it's going to take a very long time. Do we really
want to get stuck on these packages for like forever, or would it feel
ok to raise the severity to serious, so that the package gets
auto-removed and then we can work on removing Python 2 from its
dependencies?

Cheers,

Thomas Goirand (zigo)



Re: Requesting a sponsor for my package

2019-10-13 Thread Thomas Goirand
On 10/14/19 12:36 AM, Gabe Livengood wrote:
> I would like to ask any Debian Developers on this mailing list to
> consider sponsoring my package
> (https://mentors.debian.net/package/css-html-js-minify). It is a
> streamlined minifier written in Python 3 that targets CSS, JS, and HTML.
> It is a single script, pretty easy to put together, but it has helped
> teach me the basics of packaging for Debian. I would appreciate it if
> someone could sponsor the package so that I can learn more about I may
> have done wrong or can improve on with this and future packages, and of
> course, get it into the repos. Thank you!
> 

Hi,

Your source package contains a .rpm and a .deb. You may want to fix that
to begin with...

Don't add anything to debian/changelog, just "Initial release." and
that's it.

Please remove the comments in debian/control.

You may want to remove debian/compat and replace, in debian/control,
debhelper by debhelper-compat

Your debian/patches/Initial-patch is just creating a new file, so you
may just get rid of it, and write the file in the debian folder.

At this point, I stopped reviewing the package. Please fix the above and
come back to the list with the things corrected.

Cheers,

Thomas Goirand (zigo)



Re: Packages depending on python-testtools are now RC: is bzr still a thing?

2019-09-16 Thread Thomas Goirand
On 9/16/19 12:31 AM, Simon McVittie wrote:
> On Sun, 15 Sep 2019 at 23:39:46 +0200, Thomas Goirand wrote:
>> reverse-depends takes sometimes forever in Sid for a reason I
>> can't figure out. And if I'm not mistaking, that's the only tool we have
>> that can check reverse dependencies in a meaningful way. Or is there a
>> better way? I've read others using a dak command, how?
> 
> Log in to the developer-accessible archive copy
> (mirror.ftp-master.debian.org, currently hosted on coccia.debian.org),
> and use "dak rm -R -n" (aka "dak rm --rdep-check --no-action"). Use
> "dak rm --help" for help.

Thanks, I'll do that from now on.

Cheers,

Thomas Goirand (zigo)



Re: Streamlining the use of Salsa CI on team packages

2019-09-15 Thread Thomas Goirand
On 9/15/19 4:10 AM, Louis-Philippe Véronneau wrote:
> On 19-09-14 17 h 35, Thomas Goirand wrote:
>> On 9/13/19 11:08 PM, Louis-Philippe Véronneau wrote:
>>> On 19-09-13 05 h 57, Thomas Goirand wrote:
>>>> On 9/5/19 7:40 AM, Louis-Philippe Véronneau wrote:
>>>>> Hello folks!
>>>>>
>>>>> I'd like to propose we start using Salsa CI for all the team packages. I
>>>>> think using a good CI for all our packages will help us find packaging
>>>>> bugs and fix errors before uploads :)
>>>>
>>>> I would agree *IF* and only *IF* we find better runners than the one
>>>> currently default in Salsa. The GCE runners are horribly slow (they are
>>>> the smallest to avoid cost). As a result, some tests may just fail
>>>> because of that, and it becomes just frustrating / annoying noise.
>>>
>>> I never experienced such timeouts, but I guess I don't work on very
>>> large packages or things that take more than a few minutes to build.
>>
>> The issue isn't build time. But when you have unit tests sensitive to
>> timing. See for example openvswitch:
>>
>> https://salsa.debian.org/openstack-team/third-party/openvswitch/pipelines/61713
> 
> Do you have similar issues running those CI tasks in a private runner?
> (I'm really curious since I haven't had problems and the Salsa runners
> don't seem slow compared to the private runners I run on my machines).

For this particular package, I even had issues with some buildd on some
slow architectures like older MIPS. Just, with the Salsa default
runners, it's a complete disaster where most of the tests fails, not
just a few, because the runner is too slow.

What this shows is that we should *not* just blindly add the CI to all
of the team's package. Potentially, this will be a disaster. You may add
the CI script here and there, but I am warning you: adding it to all
packages at once is a recipe for a big disaster.

> Maybe one solution to your problem would be to provide a fast/responsive
> shared runners to the Salsa Team and tag your CI pipelines to use that
> runner exclusively [1]?
> 
> [1] https://docs.gitlab.com/ee/ci/yaml/#tags

Yes, that's what I've been telling to begin with. We should try
providing other runners for the team if possible.

> [1]
> https://salsa.debian.org/salsa/salsa-terraform/blob/master/environments/prod/runner.tf

This tells "instance_type: g1-small", which doesn't match any name at:
https://cloud.google.com/compute/vm-instance-pricing

Am I right that this is n1-standard-1, which is 1 VCPU and 3.75 GB?

> It's possible to push to Salsa without triggering a CI run with "git
> push -o ci.skip" or by including "[ci-skip]" in the HEAD commit message.
> 
> IIUC, the problem isn't the overall amount of repositories using the CI,
> but adding a 1000 ones at the same time and overloading the runners.

Ah, nice, good to know.

>> 1/ Take a super big care when adding jobs.
> 
> I feel this is easily resolved by the "-o ci.skip" thing.

Good!

> I'm not 100% sure that's a good idea. The Salsa Team has pretty strict
> requirements about shared runners (they require root on those machines
> to make sure the .debs created by the runners can be trusted) and I'm
> happy they do.

I didn't know, and this makes me question the overall way it works, and
worries me a lot. ie we should be running on throwaway VMs, rather than
having a VM we should be able to trust. The way you describe things, I
wonder how easy it should be to get root on these VMs by running a
crafted CI job...

> I really wonder how common the issues you've experienced with the Salsa
> CI runners are. Has anyone here had similar problems?

Since we're talking about the smallest type of instance possible at
google, then other people may have experience the lack of RAM for sure.

> I'd be fine with 95% of our package using the same default pipeline and
> the last 5% using something else or disabling it and adding a few
> comments in d/gitlab-ci.yml explaining why.

The question is: how do you know who's the 5% that needs a better attention?

> FWIW, I've opened an issue on the Salsa Support issue tracker to see
> what the Salsa team thinks of this whole discussion [3]
> 
> [3]: https://salsa.debian.org/salsa/support/issues/170

Thanks a lot for doing this, taking the time to communicate with the
Salsa people, etc.

I'm all for more CI, so feel free to ignore my remarks and go ahead, my
intention was just bring your attention to things I've seen. If it works
well, then fantastic! :)

Cheers,

Thomas Goirand (zigo)



Re: Packages depending on python-testtools are now RC: is bzr still a thing?

2019-09-15 Thread Thomas Goirand
On 9/15/19 2:26 PM, Mattia Rizzolo wrote:
> Considering that this is bzr we are talking about, a package that is
> already entering the graveyard, I think it would be easiest to just
> disable the test suite and move on.
> 
> But I would be happier it Thomas at least checked the rdeps before
> dropping packages, at least evaluating if breaking things is alrightif
> he really likes to break packages :/

You mean check *better*. Because I do carefully check each time, as much
as I can, but in this occurrence, it looks like I didn't check well
enough. Mistakes unfortunately do happen when you work on a lot of
packages. Moreover, the current tooling we have at our disposal is kind
of broken. reverse-depends takes sometimes forever in Sid for a reason I
can't figure out. And if I'm not mistaking, that's the only tool we have
that can check reverse dependencies in a meaningful way. Or is there a
better way? I've read others using a dak command, how?

On 9/15/19 5:17 PM, Jelmer Vernooij wrote:
> It's not just bzr, it's also a bunch of plugins for bzr that we'd have
> to disable the testsuite for - as well as a bunch of other
non-bzr-related packages - python-subunit, python-fixtures,
python-testscenarios, python-daemon, python-fastimport.

I've already removed Python 2 support from subunit, python-fixtures, and
python-testscenarios. Now, as I wrote in the bug report, what worries me
aren't these, but the other packages that are depending on python-daemon:

Packages without Python 3 support upstream:
- bcfg2 doesn't seem to have Python 3 support upstream, neither the
Debian package.
- nss-pam-ldapd isn't Python 3 ready upstream. Note that the debian
maintainer the same person as upstream.
- There's been zero upstream work on this repository:
https://github.com/Yubico/python-pyhsm
so the package has no change to be Python 3 ready anytime soon.

Packages that simply need an upgrade from latest upstream release:
- lavapdu-daemon should be upgraded to latest upstream release to have
Python 3 support.
- mini-buildd in Experimental has been converted to Python 3, while the
version in Sid is still Python 2.
- I haven't been able to tell for lava-coordinator.

So, for bcfg2, nss-pam-ldapd, and python-pyhsm, I'm really not convinced
that waiting for longer will help. That's a general problem that we btw
need to address: how are we going to deal with this? There's going to be
a lot of it, and we need to find a way out if we really are going to get
Python 2 out.

For the other 3, I shouldn't be hard to address by the current maintainers.

I've raised the severity of #936189 #937165 #938069 #936819 #937049
#936818 to serious, and included them as Cc to this reply, in order to
warn the maintainers. I haven't done it for the BZR stuff, as obviously,
the package maintainer is aware now.

Again, sorry that it happened this way.

Cheers,

Thomas Goirand (zigo)



Packages depending on python-testtools are now RC: is bzr still a thing?

2019-09-14 Thread Thomas Goirand
Hi,

As I wrongly thought python-extras was used only by OpenStack stuff, I
removed Python 2 support for it. Then someone filed a bug against
python-testtools (ScottK, I believe) saying that it became RC.
Therefore, I went ahead and removed Python 2 support for testtools, but
now, this implies that a few packages which I didn't wish to impact are
also RC:

* bzr-builddeb
* bzr-email
* bzr-fastimport
* bzr-git
* bzr-stats
* bzr-upload
* loggerhead

So, basically, unfortunately, Bazaar has lost some of its build
dependencies.

So, I went ahead, and looked what I could do for Bazaar. Unfortunately,
when looking at:
https://launchpad.net/bzr

I can see no release since January 2016, no daily archive. The last
commit in the bzr repository of bzr is from 2017-03-17.

Then I went to see how much Python 3 effort would be needed, and I
quickly gave up. It'd be A LOT of work, but nobody seems doing ANY work
on bzr anymore.

So I wonder: is it time to remove bazaar from Debian? Or is there any
vague plan to make it work with Python 3? If we are to remove it from
Debian, then we'd better do it ASAP.

Your thoughts?

Cheers,

Thomas Goirand (zigo)



Re: Streamlining the use of Salsa CI on team packages

2019-09-14 Thread Thomas Goirand
On 9/13/19 11:08 PM, Louis-Philippe Véronneau wrote:
> On 19-09-13 05 h 57, Thomas Goirand wrote:
>> On 9/5/19 7:40 AM, Louis-Philippe Véronneau wrote:
>>> Hello folks!
>>>
>>> I'd like to propose we start using Salsa CI for all the team packages. I
>>> think using a good CI for all our packages will help us find packaging
>>> bugs and fix errors before uploads :)
>>
>> I would agree *IF* and only *IF* we find better runners than the one
>> currently default in Salsa. The GCE runners are horribly slow (they are
>> the smallest to avoid cost). As a result, some tests may just fail
>> because of that, and it becomes just frustrating / annoying noise.
> 
> I never experienced such timeouts, but I guess I don't work on very
> large packages or things that take more than a few minutes to build.

The issue isn't build time. But when you have unit tests sensitive to
timing. See for example openvswitch:

https://salsa.debian.org/openstack-team/third-party/openvswitch/pipelines/61713

Oh, in fact, don't ... Salsa doesn't keep artifacts / logs long enough,
so you will see nothing. I wonder why the Salsa admins decided on saving
them on Google infrastructure then! Or did I misunderstood the way it
works? Hard to tell, because we get almost zero communication about this
from the admins.

> If what you describe really is caused by the default runners not being
> fast enough, why couldn't we ask the Salsa team for more powerful ones?
> Have you talked to them about this?

Since when Salsa admins are receptive to critics and/or suggestions? The
last time we told them that using Google wasn't a good idea, they just
ignored it. Do you even know how the default runners were provisioned?
Who's paying? How much credit do we have? How much can we use them?

> It seems to me that spending money in QA like CI runners is very
> profitable for the project, as it saves everyone a lot of time dealing
> with unnecessary failure caused by lack of tests. It's not like Debian
> is a very poor organisation...

Indeed. It'd be even better if we could have our own cloud, but nobody
in power seem receptive to the idea.

Also, please consider what happened the last time someone added 1000+ CI
jobs (ie: Salsa went kind of down, and the person who did that got kind
of flamed by Salsa admins).

At this point in time, I really don't think Salsa is ready for what you
proposed, unless we at least:

1/ Take a super big care when adding jobs.
2/ We have our own CI runners.

As I wrote, I believe my company could provide a few of these runners
(pending my boss approval, who's currently on holidays). However, it'd
be nice if my company wasn't the only sponsor... Not only because of
financial issues, but at least for redundancy.

Your thoughts?

Cheers,

Thomas Goirand (zigo)



Re: Streamlining the use of Salsa CI on team packages

2019-09-13 Thread Thomas Goirand
On 9/5/19 7:40 AM, Louis-Philippe Véronneau wrote:
> Hello folks!
> 
> I'd like to propose we start using Salsa CI for all the team packages. I
> think using a good CI for all our packages will help us find packaging
> bugs and fix errors before uploads :)

I would agree *IF* and only *IF* we find better runners than the one
currently default in Salsa. The GCE runners are horribly slow (they are
the smallest to avoid cost). As a result, some tests may just fail
because of that, and it becomes just frustrating / annoying noise.

Would anyone but me (ie: my company) be able to provide decent runners?

Cheers,

Thomas Goirand (zigo)



Re: 2to3 adds '.' in front dir of "from dir import ..." statements (Was: [MoM] lefse migration to python 3])

2019-09-13 Thread Thomas Goirand
On 9/12/19 9:22 AM, Michael Kesper wrote:
> Hi all,
> 
> On 12.09.19 08:30, Thomas Goirand wrote:
>> I wont comment on the relative import ambiguity problem, as Ghislain
>> replied correctly. However, I do want to comment on 2to3.
>>
>> I generally recommend against using it, in the favor of other tools. 
> ...
>>
>> The advantage is that you'll get a source code that will work on both
>> Python 2 and 3. It's generally a way more easy to submit upstream, which
>> may not want to loose Python 2 compatibility.
> 
> We should stop caring about that.
> Python2 will be EOL'ed at January 1, 2020 [0].
> Python2 will vanish from next Debian release.
> Please convert into idiomatic Python3 code, that's the sane way going forward.

As much as WE are concerned, probably. However, there's still multiple
cases where you still care having compat with both versions of Python.
First, the case where upstream does care about Py2. You probably wont
succeed convincing them it is too late and they should stop caring.
There's also the case where, as a package maintainer, you see a Python
package that currently has only Py2 support. In some case, you need to
first introduce Python 3 compat without breaking Python 2 support, so
you that way get enough time to fix the reverse dependencies.

I am not buying into the argument that adding a dependency on six.py is
a problem. It generally isn't. If upstream cares about dependency-less
stuff, then probably you should ignore upstream and do the work in
Debian alone, until upstream fixes his code to do Python 3 only.

Cheers,

Thomas Goirand (zigo)



Re: 2to3 adds '.' in front dir of "from dir import ..." statements (Was: [MoM] lefse migration to python 3])

2019-09-12 Thread Thomas Goirand
On 9/10/19 7:50 AM, Andreas Tille wrote:
> Hi,
> 
> in the process of the Python3 migration the package lefse was converted
> using 2to3.

Hi Andreas,

I wont comment on the relative import ambiguity problem, as Ghislain
replied correctly. However, I do want to comment on 2to3.

I generally recommend against using it, in the favor of other tools. For
example, you can use sixer, which I maintain in Debian, and often used
(and abused) to do Python 3 conversions. There's also 2to6, which I
don't know much about, but I've read it works about the same way as
sixer (which was specifically written for OpenStack).

The advantage is that you'll get a source code that will work on both
Python 2 and 3. It's generally a way more easy to submit upstream, which
may not want to loose Python 2 compatibility.

Cheers,

Thomas Goirand (zigo)



Re: 2to3 adds '.' in front dir of "from dir import ..." statements (Was: [MoM] lefse migration to python 3])

2019-09-12 Thread Thomas Goirand
On 9/10/19 7:50 AM, Andreas Tille wrote:
> Hi,
> 
> in the process of the Python3 migration the package lefse was converted
> using 2to3.

Hi Andreas,

I wont comment on the relative import ambiguity problem, as Ghislain
replied correctly. However, I do want to comment on 2to3.

I generally recommend against using it, in the favor of other tools. For
example, you can use sixer, which I maintain in Debian, and often used
(and abused) to do Python 3 conversions. There's also 2to6, which I
don't know much about, but I've read it works about the same way as
sixer (which was specifically written for OpenStack).

The advantage is that you'll get a source code that will work on both
Python 2 and 3. It's generally a way more easy to submit upstream, which
may not want to loose Python 2 compatibility.

Cheers,

Thomas Goirand (zigo)



Removing py2 support from python-xattr

2019-08-24 Thread Thomas Goirand
Hi,

I came across python-xattr, which I know well because it's used in
Swift. I went to investigate it's reverse dependencies, and discovered
that it's py2 version is only used by calendarserver, which itself, is
py2 only upstream, and is full of py2-ism that would need to be fixed
(and the work is really non-trivial and huge).

Since we decided to work on leaf packages only, what to do in this case,
as it doesn't look like calendarserver will be fixed anytime soon? Shall
we wait? For how long? I've of course filed a bug against
calendarserver, but I'm quite sure this wont be enough to get things moving.

Please, let's generalize and see the whole picture. There will be A LOT
more cases like this one, and I don't think that waiting forever will
solve the situation. It is my opinion that we should set a kind of
policy (ie: wait for how long?) and then act...

Cheers,

Thomas Goirand (zigo)



Re: py2-rm: a few leaf packages to work on

2019-08-24 Thread Thomas Goirand
On 8/24/19 10:38 AM, Neil Williams wrote:
> How is that graph turned into a list of packages? It's too large to
> scan manually.

Well, I did it manually... and this is only a short list, as a
suggestion for a todo list, so nothing exhaustive... I very much would
welcome something automated.

BTW, working on this (as I've assigned myself to do a Python 2 support
removal for at least one package a day), I've seen that lots of the
packages are just bit-rotting stuff that has sometimes been uploaded
only once, and that nobody cares about anymore. We should have spotted
these earlier, IMO, and probably ping the maintainers.

I'm still waiting on Piotr (or someone else) to do the mail to -devel
and the mass-bug-filling ... Any news on this?

Thomas



Re: uscan, get-orig-source and multiple source packages

2019-08-17 Thread Thomas Goirand
On 8/17/19 5:28 PM, Guðjón Guðjónsson wrote:
> Hi list
> 
> I am upgrading my packages after the release of Buster and starting with
> eric.
> 
> The get-orig-source target has been removed from the rules file and I know 
> that
> is according to the current standard but I miss it :(
> In eric version 18.08 I could add the whole multiple source to the
> pristine-tar target:
> eric_18.08.orig.tar.gz.delta
> eric_18.08.orig.tar.gz.id
> eric_18.08.orig-transl-de.tar.gz.delta
> eric_18.08.orig-transl-de.tar.gz.id
> eric_18.08.orig-transl-en.tar.gz.delta
> eric_18.08.orig-transl-en.tar.gz.id
> eric_18.08.orig-transl-es.tar.gz.delta
> eric_18.08.orig-transl-es.tar.gz.id
> eric_18.08.orig-transl-ru.tar.gz.delta
> eric_18.08.orig-transl-ru.tar.gz.id
> 
> But I have no clue how to do that without the get-orig-source target.
> Can someone please point me to a well maintained multiple source package to
> take a look at.

In debian/watch, you can point to a script that does the download work.
Just do that...

Thomas



py2-rm: a few leaf packages to work on

2019-08-15 Thread Thomas Goirand
Hi there!

According to the daily graph I built here:
http://py2graph.infomaniak.ch/py2.7.deps.svg

we can work on Python 2 removal for the below packages. Note that I have
*not* checked for reverse dependencies, please do so before working on a
package. The list isn't exhaustive at all, and didn't check if a package
is just a remaining curft, though it's hopefully still helpful as a TODO
list.

Cheers,

Thomas Goirand (zigo)

- python-libssh2
- python-pyip
- python-hunspell
- python-gpiv
- python-pyflot
- python-pyethash
- pydf
- pycmail
- python-libpcap
- python-pycallgraph
- pyblosxom
- python-pybloomfiltermmap
- python-radix
- purity-ng
- pubtal
- pssh
- postnews
- podracer
- pmailq
- pius
- pidcat
- phenny
- petit
- python-pcp
- python-pypamtest
- python-pacparser
- python-ow
- os-autoinst
- python-optcomplete
- opensvc
- python-openscap
- python-pyopencolorio
- opencaster
- onetime
- oidua
- python-odil
- python-obexftp
- neurodebian
- python-netsnmp
- ncc
- python-ncap
- python-mwparserfromhell
- muse
- mountpy
- mlucas
- python-med
- mathomatic-primes
- python-marisa
- python-mailutils
- mailplate
- ludevit
- python-logbook
- python-llvmlite
- python-clang-6.0
- llvm-6.0-tools
- clang-format-6.0
- python-link-grammar
- lincredits
- python-libwfut-0.2
- virt-sandbox
- python-libuser
- python-solv
- python-semanage
- python-selinux
- python-seccomp
- python-pwquality
- python-libpfm4
- python-libmimic
- python-ktoblzcheck
- python-kolabformat
- python-kml
- python-iptcdata
- python-imobiledevice
- python-libiio
- python-ieee1284
- python-hdate
- python-gpod
- python-libfwsi
- python-libfwnt
- python-libfvde
- python-fte
- python-ftdi1
- python-libfsntfs
- python-fsapfs
- python-freenect
- python-fiu
- python-libewf
- python-libevtx
- python-libevt
- python-libesedb
- python-libemu
- python-dumbnet
- libdbusmenu-tools
- python-libcec
- python-cap-ng
- python-buffy
- python-libbtbb-pcapdump
- python-libbde
- python-ayatana-appindicator
- python-libavg
- python-ariapy
- python-ledger
- python-ldif3
- ldaptor-utils
- python-liblcm
- python-lfc
- python-dpm
- python-lazy-object-proxy
- latex-make
- python-lasso
- ladr4-apps
- koji-servers
- createrepo
- koji-common
- koji-client
- kiki
- python-kid
- kicad
- kicad-libraries
- keymapper
- python-keybinder
- key-mon
- keepnote
- kcachegrind-converters
- python-jsonpipe
- python-jpy
- jack-mixer
- iwyu
- clang-6.0
- itstool
- python-pyisomd5sum
- iptables-converter
- svgtoipe
- ipcheck
- python-ioprocess
- insighttoolkit4-python
- python-input-pad
- inosync
- python-indigo
- impressive
- mplayer
- python-imposm-parser
- python-imposm
- imgsizer
- ifupdown-multi
- ibus-braille
- python-smbus
- hugin-tools
- libhocr-python
- python-hivex
- python-libhfst
- hatop
- python-libhamlib2
- python-hachoir-wx
- python-hachoir-urwid
- python-hachoir-subfile
- python-hachoir-metadata
- libgyoto7
- libgwyddion2-0
- gwyddion-common
- gwyddion-plugins
- emma
- python-grpcio
- grokmirror
- grml2usb
- python-gribapi
- python-gv
- gr-iio
- python-gps
- python-gphoto2cffi
- gozerbot
- gourmet
- goldeneye
- barman
- python-gnucap
- globs
- gjots2
- gitso
- git-remote-hg
- git-remote-bzr
- git-notifier
- git-hub
- git-big-picture
- gimp-plugin-registry
- giella-core
- libghmm1
- python-gevent-websocket
- gettext-lint
- flawfinder
- flatpak-builder-tests
- python-flask-principal
- flashproxy-facilitator
- flashproxy-common
- flashproxy-client
- fishpolld
- fishpoke
- python-fibranet
- python-fast5
- expeyes-doc-common
- python-exactimage
- python-enki2
- python-elementtidy
- eclipse-titan
- drobo-utils
- drbdlinks
- doxypy
- ditrack
- discus
- python-dictdlib
- dehydrated-hook-ddns-tsig
- darcsweb
- crudini
- python-cracklib
- coz-profiler
- python-construct
- vim-conque
- colortest-python
- codeville
- python-pycodcif
- cmake-extras
- python-clearsilver
- claws-mail-tools
- python-circuits
- cfget
- python-workqueue
- cantor-backend-python2
- python-bunch
- btfs
- btest
- python-brlapi
- blockfinder
- icyclerepair
- python-beanstalkc
- python-bcolz
- bcfg2-server
- bcfg2
- bbqsql
- python-backup2swift
- agtl
- python-avogadro
- libavogadro1
- autotrash
- autorenamer
- autorandr
- automx
- autokey-gtk
- autokey-common
- python-audit
- python-aubio
- code-aster-run
- asciidoc-tests
- asciidoc-base
- xmlto
- arduino-mk
- ardentryst
- archivemail
- archipel-core
- archipel-agent-xmppserver
- archipel-agent-vmparking
- archipel-agent-vmcasting
- archipel-agent-virtualmachine-vnc
- archipel-agent-virtualmachine-snapshoting
- archipel-agent-virtualmachine-oomkiller
- archipel-agent-iphone-notification
- archipel-agent-hypervisor-platformrequest
- archipel-agent-hypervisor-network
- archipel-agent-hypervisor-health
- archipel-agent-hypervisor-geolocalization
- archipel-agent-action-scheduler
- apt-transport-s3
- android-logtags-tools
- python-adios
- velvet-tests
- libseed-gtk4-dev
- polipo
- postgresql-11-mimeo
- libtext-markup-perl
- liblapack-test
- konversation-data
- konversation

Re: Investigating the reverse dependencies of python-monotonic.

2019-08-14 Thread Thomas Goirand
On 8/13/19 9:58 PM, Simon Josefsson wrote:
> Once python3-m2crypto is in Debian, I will port oz to python3.
> 
> /Simon

Well, it's in unstable already...

Thomas Goirand (zigo)



Re: Investigating the reverse dependencies of python-monotonic.

2019-08-13 Thread Thomas Goirand
On 8/13/19 3:30 PM, peter green wrote:
> IMO python-monotonic should be reinstated until it's reverse
> dependencies are sorted out.

There's now only oz, googleapi and duplicity. The last 2 both have Py3
support upstream in a newer version. Let's fix these, as I fixed all the
rest already. If nobody does the work for googleapi and duplicity, ping
me and I'll do it.

While I do agree it's preferable to do things in order, without breaking
things, I very much prefer if we move forward, toward removing Py2,
rather than backward.

Thomas



Re: Investigating the reverse dependencies of python-monotonic.

2019-08-13 Thread Thomas Goirand
On 8/13/19 12:38 PM, peter green wrote:
> python-fasteners (has rdeps)
> python-oauth2client (via python-fasteners)

FYI, I removed Python 2 support from these 2 today! :)

Hopefully, Laszlo Boszormenyi (GCS) will do the work for googleapi, and
then we'll get another chain of Py2 removed. :)

Thomas



Re: Investigating the reverse dependencies of python-monotonic.

2019-08-13 Thread Thomas Goirand
On 8/13/19 12:38 PM, peter green wrote:
> Hi
> 
> I've been looking at various python 2 cruft packages (packages no longer
> built by the corresponding source packages) and why they can't be
> cleaned up, I have been looking in a derivative but many of my results
> are also relevant to debian proper. Where relevant I have been filing
> bugs against the reverse-dependencies of these cruft packages, so they
> can be fixed or ultimately removed.
> 
> Most such packages have been part of upstream projects that have dropped
> python 2 support, notablly django and openstack. In such cases it is
> pretty clear that the only reasonable way forward is for the reverse
> dependencies to be either removed or migrated to python 3.
> 
> One package that stood out from the rest was python-monotonic.
> python-monotonic is maintained by the Debian openstack team, but it
> doesn't seem to be in any way openstack specific, nor does upstream seem
> to have dropped python2 support. It seemed to have a fair few reverse
> dependencies.
> 
> python-humanfriendly (has rdeps)
> oz (rc bug filed https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=933509 )
> python-fasteners (has rdeps)
> python-futurist (has rdeps, cruft)
> python-octaviaclient (assumed to be openstack related)
> python-oslo.log (assumed to be openstack related)
> python-oslo.messaging (assumed to be openstack related)
> python-oslo.service (assumed to be openstack related)
> python-oslo.utils (assumed to be openstack related)
> python-tenacity (has rdeps, cruft)
> 
> There are also indirect reverse dependencies, (i'm not investigating
> reverse dependencies of packages that are clearly openstack specific here)
> 
> python-coloredlogs (via python-humanfriendly, no reverse dependencies)
> python-datalad (via python-fasteners, no reverse dependencies)
> duplicity (via python-fasteners)
> python-oauth2client (via python-fasteners)
> python-oslo.concurrency (via python-fastners, openstack related)
> python-taskflow (via python-fasteners, cruft)
> python-tooz (via python-fasteners, openstack related, cruft)
> python-googleapi (via python-oauth2client)
> python-pypowervm (via python-taskflow, openstack related, cruft)
> python-googleapi-samples (via python-googleapi)
> python-etcd3gw (no rdeps)
> python-gnocchiclient (openstack related, cruft)
> 
> If we ignore openstack stuff, python modules and an examples package the
> two main packages left seem to be oz and duplicity, oz seems to have
> very low popcon, but duplicity seems to have a popcon of around 3000 and
> growing.
> 
> So the main question seems to be can duplicity be reasonably migrated to
> python 3 and if not is it worth reinstating the python-monotonic binary
> package to save duplicity?

What happened is that I simply uploaded the latest version of OpenStack
from Experimental to Sid. This includes monotonic. It's been a long time
I maintain monotonic because it's used in OpenStack, then other packages
started using it. You may have noticed that it was in Experimental with
Python 2 removed for at least 6 months, just like for Django, giving the
sign that Py2 would soon go away. However, maybe bugs should have been
filled, sorry that I didn't do it in time.

Let's take a look at the situation.

As per Andrey's reply, we can fix duplicity.

>From your report above, if I remove all the OpenStack stuff (which at
this time, should all be only Python 3), the only affected packages
would be:

> python-humanfriendly (has rdeps)
> oz (rc bug filed #933509)
> python-coloredlogs (via python-humanfriendly, no reverse dependencies)
> python-datalad (via python-fasteners, no reverse dependencies)
> duplicity (via python-fasteners)
> python-oauth2client (via python-fasteners)
> python-googleapi-samples (via python-googleapi)

According to the setup.py of python-googleapi in the Github repo, latest
upstream version supports Python 3, so it should be doable to upload a
new version to fix this.

I will remove Python 2 suport from python-oauth2client and
python-fasteners tomorrow.

So, this leaves us only: humanfriendly, oz, python-coloredlogs,
python-datalad. I've uploaded humanfriendly and coloredlogs Py2 removal.
BTW, I believe python-coloredlogs was there as a build-depends of
cyvcf2, which has been fixed on the 1st of august by Andreas Tille.

By all means, let's not play the dance of re-introducting Python 2 when
we can move forward on the right direction.

Thanks for taking the time to investigate this, this is very useful, and
I have to admit that, even though I know how to do the work, I am a bit
lost into knowing from were to begin. The release tracker is not very
helpful in this regard.

Cheers,

Thomas Goirand (zigo)



Re: Removing python2 packages

2019-07-30 Thread Thomas Goirand
On 7/30/19 11:40 PM, Scott Talbert wrote:
> On Tue, 30 Jul 2019, Thomas Goirand wrote:
>> Do you mean, will python-foo be automatically removed from Sid/Testing,
>> after your upload? Normally yes, if nothing depends on it. And that's
>> probably harder to find out than one may think... :)
> 
> Yes, that's what I mean.  The reason I ask is because I've done such an
> upload (with python-xyz removed), but this package hasn't disappeared
> from the python2-rm transition tracker.  The specific package involved
> is concordance (python-libconcord was removed).
> 
> Scott

cc-ing the FTP master team.

My understanding is that there's issues with arch:all package in the
uncruft script of the FTP masters. It looks like a lot of python-*
packages are staying. I told about it to Luke at Debconf, though I'm not
sure if he had time to look into it.

Dear FTP master team,

As per above, this is an increasingly annoying issue that prevents us
from knowing what thing we should work on. Is the uncruft script really
broken? If yes, were should we look into, if we want to contribute a
fix? A pointer to the correct script in a git Salsa (if there's such a
thing), and advice on how to test would be helpful. Is this directly in
the Dak's sources?

Thomas Goirand (zigo)



Re: Removing python2 packages

2019-07-30 Thread Thomas Goirand
On 7/24/19 8:44 PM, Scott Talbert wrote:
> Assuming python 2 is removed from bullseye, upon an upgrade from buster
> to bullseye, I guess we are assuming python 2 would remain installed on
> upgraded systems?
> 
> Scott

We discussed this during Debconf. Possibly, we'll have to leave the
Python 2 interpreter in Bullseye to be able to upgrade, and also maybe
to build pypy3.

As written previously, after the upgrade this would be strongly advised:

apt-get --purge autoremove

When the time comes, we probably will need to write about it in the
Bullseye release notes. Though that's far from now...

Cheers,

Thomas Goirand (zigo)



Re: Re: dropping python2 [was Re: scientific python stack transitions]

2019-07-17 Thread Thomas Goirand
On 7/16/19 5:42 PM, Matthias Klose wrote:
> no, that's your interpretation, not mine.  If an important enough application
> still needs it, we should ship it.

Could you please clarify what you call "an important enough
application"? Important for who/what? Who's to decide, and on what criteria?

Also, if that application is important enough, why nobody worked on
porting it to Python 3?

Cheers,

Thomas Goirand (zigo)



  1   2   3   4   >