Re: Raising severity to serious for some Python 2 leaf packages with no Python 3 support upstream

2019-10-13 Thread Scott Kitterman
On Sunday, October 13, 2019 10:52:17 PM EDT Thomas Goirand wrote:
> Hi,
> 
> In some cases I've seen, particularly in the med or science team,
> switching some packages to Python 3 requires a significant effort.
> 
> For example, today I looked into removing Python 2 from python-cogent.
> Running sixer on all files lead to a huge log of problems to solve by
> hand. There's no upstream support for Python 3 on that one.
> 
> For this kind of package, I see no way out except:
> - Upstream works on Python 3 support
> - Someone in Debian makes the effort
> 
> But in both cases, it's going to take a very long time. Do we really
> want to get stuck on these packages for like forever, or would it feel
> ok to raise the severity to serious, so that the package gets
> auto-removed and then we can work on removing Python 2 from its
> dependencies?

There are two python2 only packages that I maintain.  I don't intend to keep 
them in bullseye, so I filed "should not be in bullseye" RC bugs against them.  
They and their rdepends will be out of Testing shortly. 

If you are the maintainer of a package, I think that's something that doesn't 
need to wait.

In the case of things that aren't ported to python3 yet they are mostly dead 
upstream.  For the dead ones, I don't think anyone in Debian should port them 
unless they are willing to take over as upstream (I've done this in a few 
cases).  If there's no upstream, they ought to just be removed.  The sooner 
the better.

Scott K





Re: Raising severity to serious for some Python 2 leaf packages with no Python 3 support upstream

2019-10-13 Thread Nicholas D Steeves
Hi Thomas and Python Team,

Thomas Goirand  writes:

> For example, today I looked into removing Python 2 from python-cogent.
> Running sixer on all files lead to a huge log of problems to solve by
> hand. There's no upstream support for Python 3 on that one.
>
> For this kind of package, I see no way out except:
> - Upstream works on Python 3 support
> - Someone in Debian makes the effort
>
> But in both cases, it's going to take a very long time. Do we really
> want to get stuck on these packages for like forever, or would it feel
> ok to raise the severity to serious, so that the package gets
> auto-removed and then we can work on removing Python 2 from its
> dependencies?

It seems to me that a fair and conservative solution is to send an
announcement once a month that all Python 2 packages will have an RC bug
filed against them on 1 January 2020.  I work on the Calibre package,
and depend on it for my daily work.  I do not believe that its reverse
deps should be removed before 1 Jan 2020.

As far as the maximum number of announcements, how about: the first
announcement ASAP, the second one 1 Nov, then 1 Dec.  Lets CC this
announcement to devel, -python, -med, and any other teams with a
significant number of Python packages.

The issue is similar to global warming.  People will hide their head in
the sand singing "nah nah nah, it's not real, I don't have to deal with
it" until a tipping point occurs.

Honestly part of me wonders if RedHat/IBM is going to maintain Python 2,
like they did with Java.

  
https://developers.redhat.com/blog/2018/09/24/the-future-of-java-and-openjdk-updates-without-oracle-support

Barring that hypothetical possibility, I still think the right thing to
do is file RC bugs the 1 of January.  What happens then?  My guess is
upstreams start containerising their py2 software and someone will write
a helper script like py2-zombie-flatpack.


Cheers,
Nicholas


signature.asc
Description: PGP signature


Re: Python2 removal: package with low-popcon reverse dependencies

2019-10-13 Thread Thomas Goirand
On 10/11/19 9:26 PM, Christian Kastner wrote:
> On 11.10.19 19:47, Matthias Klose wrote:
>> On 11.10.19 18:27, Christian Kastner wrote:
>>> This would nevertheless be a case for the "py2keep", right?
>>
>> No.
>>
>> #933348 is another bug for removed packages (mopidy-scrobler). Do you
>> really want to keep that? 
> 
> Not at all, I'd actually prefer just dropping the Python2 version of
> python-cachetools. However, this breaks things, and it wasn't entirely
> clear to me much breakage is acceptable.

It's becoming increasingly clear to me that, at some point, we will need
to just ignore the breakage. Bust this needs to be discussed here first.
Maybe the better way to fix the situation is to increase the severity of
the py2 removal bug to serious, so we leave a chance to the maintainer
to take care of the package before the autorm. See the other thread I've
just started about this.

Cheers,

Thomas Goirand (zigo)



Raising severity to serious for some Python 2 leaf packages with no Python 3 support upstream

2019-10-13 Thread Thomas Goirand
Hi,

In some cases I've seen, particularly in the med or science team,
switching some packages to Python 3 requires a significant effort.

For example, today I looked into removing Python 2 from python-cogent.
Running sixer on all files lead to a huge log of problems to solve by
hand. There's no upstream support for Python 3 on that one.

For this kind of package, I see no way out except:
- Upstream works on Python 3 support
- Someone in Debian makes the effort

But in both cases, it's going to take a very long time. Do we really
want to get stuck on these packages for like forever, or would it feel
ok to raise the severity to serious, so that the package gets
auto-removed and then we can work on removing Python 2 from its
dependencies?

Cheers,

Thomas Goirand (zigo)



Re: Requesting a sponsor for my package

2019-10-13 Thread Thomas Goirand
On 10/14/19 12:36 AM, Gabe Livengood wrote:
> I would like to ask any Debian Developers on this mailing list to
> consider sponsoring my package
> (https://mentors.debian.net/package/css-html-js-minify). It is a
> streamlined minifier written in Python 3 that targets CSS, JS, and HTML.
> It is a single script, pretty easy to put together, but it has helped
> teach me the basics of packaging for Debian. I would appreciate it if
> someone could sponsor the package so that I can learn more about I may
> have done wrong or can improve on with this and future packages, and of
> course, get it into the repos. Thank you!
> 

Hi,

Your source package contains a .rpm and a .deb. You may want to fix that
to begin with...

Don't add anything to debian/changelog, just "Initial release." and
that's it.

Please remove the comments in debian/control.

You may want to remove debian/compat and replace, in debian/control,
debhelper by debhelper-compat

Your debian/patches/Initial-patch is just creating a new file, so you
may just get rid of it, and write the file in the debian folder.

At this point, I stopped reviewing the package. Please fix the above and
come back to the list with the things corrected.

Cheers,

Thomas Goirand (zigo)



Re: python-urllib3 1.25.6 uploaded to experimental (closes CVE-2019-11236) but fails build tests

2019-10-13 Thread Drew Parsons

Daniele wrote:
I hope to have the time to investigate also this: 
urllib3/contrib/pyopenssl.py
contains code to have SSL with SNI_-support for Python 2 and it depends 
on
pyOpenSSL, cryptography and idna. Maybe looking at them can give us 
more clues.


Also, could you see if using Python3 the connection to 
https://pub.orcid.org work?


It conditionally works.  Using curl, I found that TLSv1_0 or TLSv1_1 
will support a successful connection, but only if the maximum 
SSL_VERSION is constrained to TLSv1_0 or TLSv1_1 (e.g. curl -v --tlsv1.1 
--tls-max 1.1  https://pub.orcid.org). Without the max, the connection 
fails:

$ curl --tlsv1.1  https://pub.orcid.org
curl: (35) error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert 
handshake failure


The urllib3 failure was similar, but I do not know how to set tls-max 
with urllib3. I could only find the option with curl.  I could set up a 
custom HTTPAdapter as suggested at 
https://requests.readthedocs.io/en/master/user/advanced/#example-specific-ssl-version 
to set ssl_version=ssl.PROTOCOL_TLSv1_1 but the ssl module doesn't have 
the SSLVERSION_MAX_TLSv1_1 value that curl has. I could solve it with 
pycurl using c.setopt(pycurl.SSLVERSION, pycurl.SSLVERSION_TLSv1_1 | 
pycurl.SSLVERSION_MAX_TLSv1_1)


Evidently the orcid server only supports TLSv1.0 and TLSv1.1 and no 
higher (why haven't they activated TLSv1.3 yet?!), while curl and 
urllib3 without tls-max first test TLSv1.3 and then quit without 
cascading downwards once they receive the TLSv1.3 handshake failure.  
Which is rather odd behaviour when I think about it.  The whole point of 
supporting multiple protocol versions is to try the next available 
version if the first one doesn't work.




Th package build was successful on my system but gives build-time 
errors in

chroot (on buildd).  I'm not sure why that's failing.
I will look at them during this weekend, I already had a look at build 
log from

the phone, but it's better to look from a PC.


I had a closer look.  The failing tests were in python2 only, coming 
from the non-ascii (Gërman http://Königsgäßchen.de/straße and Japanese 
http://ヒ:キ@ヒ.abc.ニ/ヒ?キ#ワ) unicode url tests. So from one perspective we 
don't need to worry so much about them, we could just disable them (e.g. 
prepend @onlyPy3 to test_parse_url and test_url_vulnerabilities in 
test_util.py). We'll be dropping python2 any way in the near future.


On the other hand, given the nature of the vulnerabilities and the 
possible uses of urllib3, it's probably best not to leave python2 
untested, especially since they are known to pass on python2 anyway in 
the right conditions.  Probably there is some package that should be 
added to Build-Depends to enable python2 tests to pass, though I have no 
idea which that package might be.


Drew