Re: Bug#936604: getmail: Python2 removal in sid/bullseye

2019-11-13 Thread Iustin Pop
On 2019-11-14 00:24:15, Osamu Aoki wrote:
> On Wed, Nov 13, 2019 at 03:31:04PM +0100, Iustin Pop wrote:
> > On 2019-11-13 15:06:54, Thomas Goirand wrote:
> > > On 11/12/19 4:37 PM, Osamu Aoki wrote:
> > > > The related binary packages are available in 2 binary names (depending 
> > > > on release)
> > > >  getmail4 (version=4,5) popcon installed ~2000
> > > >  getmail  (version=3,5) popcon installed ~1000
> > > >
> > > > https://qa.debian.org/popcon-graph.php?packages=getmail%20getmail4_installed=on_legend=on_ticks=on_fmt=%25Y-%25m=1
> > > >
> > > > I think this qualifies for "py2keep".
> > >
> > > IMO, this qualifies for RM-RoM. getmail is an alternative to fetchmail,
> > > which is still available in Debian (and with 4 times the number of
> > > installed package in popcon...). So I see no reason to keep getmail
> > > then. Maybe tell this to upstream, and they may think another time.
> >
> > Uh, no. Functionality-wise, they're quite different. getmail is (AFAIK)
> > the only tool that works for gmail with ASPs disabled (i.e. with OAUTH).
> >
> > Heck, I'd be very willing to maintain Py3 patches myself, because I need
> > this tool.
> 
> Please take over packaging from me then.  You are welcome.

I would gladly help with co-maintenance, but taking over packaging would
be my least preferred option.

Thanksfully, it seems the upstream is willing to move to Python 3, so I
think situation is pretty good, actually.

thank you!



Re: FYI: Python 3 migration of distributuion

2019-11-13 Thread Thomas Goirand
On 11/13/19 6:11 PM, Charles Cazabon wrote:
> Unfortunately, it's difficult to find the hours to devote to this task.  I
> don't know when, or even if, I could have a beta release ready.
> 
>> If you convert python 2 code for 2.7 style, then we can support both
>> python 2 and 3.
> 
> I'm not very interested in doing this, as it means not using a lot of 3.x
> features, or not using them in the most Pythonic way.  My thought was that
> this project would target at least Python 3.7; anyone who didn't want to run
> 3.7 could still run "legacy" getmail under Python 2.7.
> 
> Charles

Hi Charles,

Please allow me to comment on this.

Converting some Python 2.7 code to Python 3.x is not very hard for
anyone who's a little bit trained to it. Most of the time, we see the
same patterns over and over again. So it would be relatively easy for a
lot of us to help you transition the current code to Python 3.x, without
even the need to understand much about getmail itself. Add to this that
there's automated tools to do the most boring part of it (like sixer,
and someone mentioned another one which I can't remember).

I had a quick look into Getmail's code, and it looks very similar to
what I've seen in other programs. Lots of print statements that needs to
be converted, a bit of except to patch, a few itertools too. The code is
also of quite a manageable size (by that I mean: it's a lot smaller than
lots of other stuff I converted to Py3 in a few hours...).

Some of us are offering such a help, and quickly, if you accept the
patches, you'll be able to have Getmail in a good enough shape so it can
stay in Debian, Ubuntu, Fedora.

On the opposite side, if you attempt to do a huge rework of Getmail
(which is maybe needed, I don't know Getmail enough to be able to tell),
probably nobody else but you will be able to do it, and you wont get
much help. As you wrote it may take some time, and you aren't even able
to predict how much.

So, why don't you just take the offer, get the Python 3.x patches in,
and then *later* do your rework, maybe in a version 6 of Getmail? This
sounds a much more reasonable approach to me.

Note that I'm not at all involved in Getmail or I'm not even a user, I'm
just trying to convince you to do the right move here... on the
direction which I believe will serve your users best, for example have
Getmail stay on the next Ubuntu 20.04 LTS (Debian Bullseye is for in 2
years, so you may have more time for that one...).

Cheers,

Thomas Goirand (zigo)



Re: FYI: Python 3 migration of distributuion

2019-11-13 Thread Charles Cazabon
Osamu Aoki  wrote:
> 
> Currently, getmail is a candidate for removal from the upcoming Debian
> release if it is not updated to support python 3 by someone (not
> necessary by upstream).

Thanks for the update, Osamu.  I have actually been playing with a prototype
refactoring of getmail to not just support but require a recent Python 3.x
version.  Such a project would give me the opportunity to remove a lot of
historical cruft and backwards-compatibility code that getmail has accumulated
over 20+ years.

Unfortunately, it's difficult to find the hours to devote to this task.  I
don't know when, or even if, I could have a beta release ready.

> If you convert python 2 code for 2.7 style, then we can support both
> python 2 and 3.

I'm not very interested in doing this, as it means not using a lot of 3.x
features, or not using them in the most Pythonic way.  My thought was that
this project would target at least Python 3.7; anyone who didn't want to run
3.7 could still run "legacy" getmail under Python 2.7.

Charles
-- 
---
Charles Cazabon
GPL'ed software available at:   http://pyropus.ca/software/
---



Re: getmail: Python2 removal in sid/bullseye

2019-11-13 Thread Thomas Goirand
On 11/13/19 3:31 PM, Iustin Pop wrote:
> On 2019-11-13 15:06:54, Thomas Goirand wrote:
>> On 11/12/19 4:37 PM, Osamu Aoki wrote:
>>> The related binary packages are available in 2 binary names (depending on 
>>> release)
>>>  getmail4 (version=4,5) popcon installed ~2000
>>>  getmail  (version=3,5) popcon installed ~1000
>>>
>>> https://qa.debian.org/popcon-graph.php?packages=getmail%20getmail4_installed=on_legend=on_ticks=on_fmt=%25Y-%25m=1
>>>
>>> I think this qualifies for "py2keep".
>>
>> IMO, this qualifies for RM-RoM. getmail is an alternative to fetchmail,
>> which is still available in Debian (and with 4 times the number of
>> installed package in popcon...). So I see no reason to keep getmail
>> then. Maybe tell this to upstream, and they may think another time.
> 
> Uh, no. Functionality-wise, they're quite different. getmail is (AFAIK)
> the only tool that works for gmail with ASPs disabled (i.e. with OAUTH).

Oh ! Thanks for the insight. I didn't know.

> Heck, I'd be very willing to maintain Py3 patches myself, because I need
> this tool.

Then there's no issue anymore? :)

Thomas



Re: Bug#936604: getmail: Python2 removal in sid/bullseye

2019-11-13 Thread Osamu Aoki
On Wed, Nov 13, 2019 at 03:31:04PM +0100, Iustin Pop wrote:
> On 2019-11-13 15:06:54, Thomas Goirand wrote:
> > On 11/12/19 4:37 PM, Osamu Aoki wrote:
> > > The related binary packages are available in 2 binary names (depending on 
> > > release)
> > >  getmail4 (version=4,5) popcon installed ~2000
> > >  getmail  (version=3,5) popcon installed ~1000
> > >
> > > https://qa.debian.org/popcon-graph.php?packages=getmail%20getmail4_installed=on_legend=on_ticks=on_fmt=%25Y-%25m=1
> > >
> > > I think this qualifies for "py2keep".
> >
> > IMO, this qualifies for RM-RoM. getmail is an alternative to fetchmail,
> > which is still available in Debian (and with 4 times the number of
> > installed package in popcon...). So I see no reason to keep getmail
> > then. Maybe tell this to upstream, and they may think another time.
>
> Uh, no. Functionality-wise, they're quite different. getmail is (AFAIK)
> the only tool that works for gmail with ASPs disabled (i.e. with OAUTH).
>
> Heck, I'd be very willing to maintain Py3 patches myself, because I need
> this tool.

Please take over packaging from me then.  You are welcome.

Osamu



FYI: Python 3 migration of distributuion

2019-11-13 Thread Osamu Aoki
Hi,

This is just FYI on situation.

Currently, getmail is a candidate for removal from the upcoming Debian
release if it is not updated to support python 3 by someone (not
necessary by upstream).

Here is the background 

Debian and Ubuntu are pushing to remove python2 in next 2020 release.
  https://wiki.debian.org/Python/2Removal
  https://lists.ubuntu.com/archives/ubuntu-devel/2019-November/040842.html

You can see its progress for Debian at:
  http://sandrotosi.me/debian/py2removal/py2removal_progress.png
They converted packages to support python3 for almost the half of the
problematic python2 packages in 3 months.  With this pace, they think
they can remove python2.

Ubuntu also has list:
  https://people.canonical.com/~ubuntu-archive/transitions/html/python2-rm.html

Some migration activities were implemented by Debian maintainers
patching the upstream package without upstream help.  So something like
that may happen on getmail if some other Debian developers take actions.
(I, for one, will not do this considering getmail being a security
sensitive mail application and needs upstream support of python3 code.)

Please also note Fedora seems to be doing the similar:
  https://pagure.io/releng/issue/8221
  https://fedoraproject.org/wiki/Changes/F31_Mass_Python_2_Package_Removal

Please don't expect some major distros to be supporting python2 after 2020.

If you convert python 2 code for 2.7 style, then we can support both
python 2 and 3.  If some code needs to be made compatible with pre=2.7
version, may be making that section of code conditional to python
version may accommodate it without causing python 3 compatibility
problem.

If you update package to support python 3, I will upload it to Debian.

Regards,

Osamu



Re: getmail: Python2 removal in sid/bullseye

2019-11-13 Thread Iustin Pop
On 2019-11-13 15:06:54, Thomas Goirand wrote:
> On 11/12/19 4:37 PM, Osamu Aoki wrote:
> > The related binary packages are available in 2 binary names (depending on 
> > release)
> >  getmail4 (version=4,5) popcon installed ~2000
> >  getmail  (version=3,5) popcon installed ~1000
> > 
> > https://qa.debian.org/popcon-graph.php?packages=getmail%20getmail4_installed=on_legend=on_ticks=on_fmt=%25Y-%25m=1
> > 
> > I think this qualifies for "py2keep".
> 
> IMO, this qualifies for RM-RoM. getmail is an alternative to fetchmail,
> which is still available in Debian (and with 4 times the number of
> installed package in popcon...). So I see no reason to keep getmail
> then. Maybe tell this to upstream, and they may think another time.

Uh, no. Functionality-wise, they're quite different. getmail is (AFAIK)
the only tool that works for gmail with ASPs disabled (i.e. with OAUTH).

Heck, I'd be very willing to maintain Py3 patches myself, because I need
this tool.

regards,
iustin



Re: Bug#936604: getmail: Python2 removal in sid/bullseye

2019-11-13 Thread Osamu Aoki
Hi,

On Wed, Nov 13, 2019 at 09:27:08AM +0100, Ondrej Novy wrote:
> Hi,
>
> út 12. 11. 2019 v 16:37 odesílatel Osamu Aoki  napsal:
>
> Upstream is active and prides to keep python 2.5 compatibility code in
> it ... (Not just 2.7).  I (Osamu Aoki ) and dkg even
> made some effort to support both 2 and 3 but the idea was rejected by
> upstream in 2018.
>
>
> that's odd :/
>
>
> I think this qualifies for "py2keep".
>
>
> i don't think so. Imho is better to support Py3 in Debian with our patches.

dkj: Do you?  (Not me)

> If Debian and Fedora demonstrate its python3 removal, I am sure the
> upstream may change his thought.  If you have some progress indicator,
> that can be used to convince getmail upstream.  I or dkg need a solid
> fact to restart conversation with the upstream.
>
>
> yes, we have! :)
>
> http://sandrotosi.me/debian/py2removal/py2removal_progress.png
>
> Almost half of work is done in ~ 3 months.

This is interesting.  I will inform upstream about this progress.

But how much od current ~1800 packages fall into high popcon and build
dependency category.   Are these all high pop conpackages?

Osamu



Re: Bug#936604: getmail: Python2 removal in sid/bullseye

2019-11-13 Thread Osamu Aoki
Hi,

On Tue, Nov 12, 2019 at 04:57:17PM +0100, Mattia Rizzolo wrote:
> On Wed, Nov 13, 2019 at 12:37:27AM +0900, Osamu Aoki wrote:
> > Upstream is active and prides to keep python 2.5 compatibility code in
> > it ... (Not just 2.7).  I (Osamu Aoki ) and dkg even
> > made some effort to support both 2 and 3 but the idea was rejected by
> > upstream in 2018.  (Then we both lost motivation since upstream will not
> > include such code in near future)
> >   https://marc.info/?l=getmail=151542515929707=2
> >
> > The upstream is somehow convinced that python2 will be there for some
> > time (Year ~2020 and later).
> >   https://marc.info/?l=getmail=151542154628352=2
>
> uh. meh.
>
> I haven't looked at the code, but if you made the effort, how improbable
> would it be for you to just keep the patches for py3 support yourself in
> the packaging for the time being?

Neither of us got to compete patches to be accepted by the upstream or
fully functioning code for all versions upstream wanted to support, if I
recall correctly.

Besides, patches applied were extensive.  Considering security
implication, not accepted by upstream was the killer.  The upstream
updates this package when security concerns are raised.

Anyway, if Debian compiles a transition statistics of python3 migration,
we can point upstream to it.

Do we have stat of number of packages:
 A) Packages using python3
 B) Packages using python2 but python3 version is also available.
 C) Packages using python2 but python3 version is not available and
it is required by standard package building
 D) Packages using python2 but python3 version is not available and
it is not required by standard package building,
but it has high popcon over 1000.

If "C" portion is getting less than 1% and "D" portion is getting 1% of
"A" portion, I guess upstream may change mind.

If we know how Fedora has done, that may help convince upstream.

Regards,

Osamu



Re: getmail: Python2 removal in sid/bullseye

2019-11-13 Thread Thomas Goirand
On 11/12/19 4:37 PM, Osamu Aoki wrote:
> The related binary packages are available in 2 binary names (depending on 
> release)
>  getmail4 (version=4,5) popcon installed ~2000
>  getmail  (version=3,5) popcon installed ~1000
> 
> https://qa.debian.org/popcon-graph.php?packages=getmail%20getmail4_installed=on_legend=on_ticks=on_fmt=%25Y-%25m=1
> 
> I think this qualifies for "py2keep".

IMO, this qualifies for RM-RoM. getmail is an alternative to fetchmail,
which is still available in Debian (and with 4 times the number of
installed package in popcon...). So I see no reason to keep getmail
then. Maybe tell this to upstream, and they may think another time.

Cheers,

Thomas Goirand (zigo)



Re: reducing matplotlib2 build-depends.

2019-11-13 Thread Dmitry Shachnev
Hi Sandro!

On Tue, Nov 12, 2019 at 09:10:49PM -0500, Sandro Tosi wrote:
> well, i dont think it's a good reason to just stop testing complex
> software tho. some of those dependencies are required so that mpl can
> build extensions (in particular for the GUI backends) so if you remove
> them, the mpl build system is smart enough to disable those
> extensions, resulting in a graphic library without any GUI bindings to
> work with. that doesnt sound ideal.

That is a valid point. But please drop at least the PyQt4 GUI binding.

That one is clearly not needed when PyQt5 binding exists.

> I'm gonna go thru some of the dependencies and will see if i can
> remove some, in particular now that i've remored the -doc package from
> the git repo, but i'm not gonna get rid the whole unittest suite.
>
> thanks for your work on this, and to nudge me into looking deeper at mpl2
> deps

--
Dmitry Shachnev


signature.asc
Description: PGP signature


Re: getmail: Python2 removal in sid/bullseye

2019-11-13 Thread Ondrej Novy
Hi,

út 12. 11. 2019 v 16:37 odesílatel Osamu Aoki  napsal:

> Upstream is active and prides to keep python 2.5 compatibility code in
> it ... (Not just 2.7).  I (Osamu Aoki ) and dkg even
> made some effort to support both 2 and 3 but the idea was rejected by
> upstream in 2018.


that's odd :/


> I think this qualifies for "py2keep".
>

i don't think so. Imho is better to support Py3 in Debian with our patches.

If Debian and Fedora demonstrate its python3 removal, I am sure the
> upstream may change his thought.  If you have some progress indicator,
> that can be used to convince getmail upstream.  I or dkg need a solid
> fact to restart conversation with the upstream.
>

yes, we have! :)

http://sandrotosi.me/debian/py2removal/py2removal_progress.png

Almost half of work is done in ~ 3 months.

-- 
Best regards
 Ondřej Nový