Re: [Twisted-Python] [RFC] Drop support for Python 3.5 sometime after May 2021?

[Richard van der Hoff]

On 08/06/2020 08:04, Glyph wrote:
> 

I'm going to start here by saying: I agree with almost all of what you 
wrote, but at the end of the day, I don't get to determine our 
customers' policies. You can try to explain to them why their policies 
are misguided, but particularly when you're working with a large 
organisation, change can be very slow. So you end up working around the 
policy, whether you agree with it or not. In practical terms, that means 
that for now at least we need to support Python 3.5.


As Erik said, we certainly have no right to demand that Twisted continue 
to support 3.5: indeed, if dropping support will deliver value to the 
project, then I'd encourage you to go for it; and as you've already 
said, the whole thing is probably moot anyway given the timescales we're 
talking about.


I believe in this case its a general desire to keep track of what 
packages are running and where they've come from. They basically 
trust that packages from official Debian repositories are probably 
safe from being tampered with, whereas random tarballs of code from 
the web are not safe (unless they're signed by someone they trust or 
whatever).


I think this sounds like a misunderstanding of Debian's vetting 
process?  It's not like there's a ton of additional auditing that goes 
into packaging something.  There's definitely an authentication 
process for both Twisted and Python, although this attestation could 
be somewhat stronger and less centralized, PyPI does quite a bit of 
heavy lifting there.


I think it's less that they think that Debian does extra vetting, and 
more that, especially if you're managing whole fleets of servers, then 
if everything runs the same version, it's easier to keep track of what 
you need to upgrade when there's a "security" bug. And yes, there are 
plenty of counterarguments to this, but that's the reasoning.



 1. Many non-"security" bugs are in fact security bugs that nobody has
noticed you can exploit.
 2. Many "low-severity" or "un-exploiable" security bugs are in fact
exploitable
 3. "supported" distros rarely take care to backport many patches for
their software, and when they do, they often make undetected
errors (like debian's infamous ssh bug) which are analyzed by far
fewer security analysts than the upstream source code.

These are probably all true, but taken to their logical extreme, the 
conclusion seems to be "you should always run the bleeding edge of all 
software, to make sure you've got all the latest bug fixes". I don't 
think you're really arguing for that, so the point is: we end up 
nominating "stable" versions, and trying to make an assessment as to 
which bugfixes are worth backporting. That latter part is a subjective 
decision, and the question is who you trust to make it. You may not 
trust Debian to make that decision on your behalf (with perfectly valid 
reasons), but plenty of others do.
So I feel like the folks making the decision to stick with these old 
"supported" distros are only getting half the story - sure, it won't 
break, but are they /actually/ getting the security fixes that they 
think they are?  Debian's staff are stretched pretty thin as it is.


A counterpoint here is that the Python in oldstable has had several 
years of bugfixes, and of course it was the primary Debian-supported 
version of Python for a good couple of years. Again, I know that 
peoples' assessment of Debian's ability or competence varies, but I 
don't think it's *unreasonable* to assume that by this point the worst 
problems with that version of Python have been shaken out (and that if a 
significant new problem arises, a fix will be made).


In terms of twisted dropping support of 3.5, I guess the question is 
to what extent do you want applications to be hassle free to deploy 
on the more "enterprise" style environment?


One other confusion I have about these environments is why they want 
very-old Python but don't /also/ want very-old Twisted.


Well, again, it comes down to fleet management, and responsibility for 
"security". From the customer's point of view, they want to provide a 
python interpreter which runs our application. So the Python interpreter 
is their responsibility (hence: use Debian oldstable everywhere), 
whereas the stuff running on it is ours. Plus, since there's only one 
thing using Twisted in their network, it's inherently easier to maintain 
a single version.


(I also fear that there is a misguided belief that security 
vulnerabilities are "worse" if they are in the Python interpreter, 
because that runs native code, whereas Twisted can't possibly do 
anything that bad because "interpreted language". I mention this only 
for completeness, and fully realise it's nonsense.)


But supporting old Pythons, old service_identity modules, old 
OpenSSL's, etc, has been seeming more and more to me like a disservice 
to the community, because it facilitates the adoption of slow, 
insecure, dangerous 

Re: [Twisted-Python] [RFC] Drop support for Python 3.5 sometime after May 2021?

[Glyph]

> On May 28, 2020, at 8:31 AM, Erik Johnston  wrote:
> 
> 
> On 23/05/2020 06:39, Glyph wrote:
>> 
>>> On May 19, 2020, at 1:52 AM, Richard van der Hoff >> > wrote:
>>> 
>>> On 16/05/2020 06:56, Glyph wrote:
 
 
> On May 15, 2020, at 8:40 PM, Craig Rodrigues  > wrote:
> 
> Maybe it would be OK to do one more release of Twisted and announce that 
> as the last release supporting Python 3.5, before
> dropping support?  
 
 Yeah; whenever we drop a Python version we should always support at least 
 one more release, so that people have some notice before they lose access 
 to the next set of security updates.
 
 Any 3.5 users on this list who would want to postpone it longer than this?
 
>>> Sadly we have an important customer whose servers run debian oldstable, 
>>> which means we need to stay compatible with 3.5 until we can persuade them 
>>> to upgrade, and it's taken a couple of years to get them off python 2.7...
>>> 
>>> I'm not sure that should necessarily affect your plans, but I doubt we're 
>>> alone in this situation.
>>> 
>> 
>> I guess one thing I'm curious about is why your application would need to be 
>> installed along with the system Python on those OS versions?  It seems like 
>> a packaging strategy that ignored the fossilized versions that Debian 
>> packages with the system and just built its own Python would be more 
>> reliable and allow for upgrading at least most Python dependencies well 
>> beyond what the system would allow by policy.  Or, for that matter, why not 
>> just run in a Docker container?
>> 
>> Matrix is a pretty big user, and so in some sense I care about this specific 
>> case, but I also find the general question interesting, because I have 
>> difficulty reasoning about how long to support older versions of things in 
>> the modern application packaging environment where containers, virtualenvs, 
>> and associated tooling make it possible to effectively ignore the base 
>> environment. When & why do you have to pay attention to it?
>> 
>> -glyph
> 
> I believe in this case its a general desire to keep track of what packages 
> are running and where they've come from. They basically trust that packages 
> from official Debian repositories are probably safe from being tampered with, 
> whereas random tarballs of code from the web are not safe (unless they're 
> signed by someone they trust or whatever).
> 
I think this sounds like a misunderstanding of Debian's vetting process?  It's 
not like there's a ton of additional auditing that goes into packaging 
something.  There's definitely an authentication process for both Twisted and 
Python, although this attestation could be somewhat stronger and less 
centralized, PyPI does quite a bit of heavy lifting there.
> Now, I think it would be possible to get a newer version of Python on their 
> infrastructure if we needed, but I'm sure there would be hoops that would 
> need to be jumped through and justifications given, etc, which would 
> undoubtedly take some time. So really it just means extra faff for them and 
> us, especially since we're only a small part of their overall infrastructure.
> 
It's worth noting that Python 3.5 itself is out of security support (per 
https://www.python.org/dev/peps/pep-0478/ 
) and it's not like Debian employs a 
team of Python security-support engineers to continue maintaining this dead 
branch.  If their infrastructure were to try to comply with various security 
standards such as PCI-DSS, you might bump into the need to upgrade to a newer 
version with available future patch releases anyway.
> Then there is the fact that they're not unique. While oldstable is, well, 
> old, its still very much supported and so there's going to be a bunch of 
> "enterprise" (for want of a better term) customers who will still be using 
> it, and we'll need to go through the faff each and every time, which is quite 
> tedious. Come the Autumn when oldstable stops being supported (or at least, 
> goes into LTS mode),
> 
You say "Autumn", but...  oldstable is supported for the purposes of migrating 
off of it... it's not really even supported by Debian proper.  To quote from 
https://wiki.debian.org/DebianOldStable 
 :

>>> The security team tries to support a stable distribution for about one year 
>>> after the next stable distribution has been released, except when another 
>>> stable distribution is released within this year. It is not possible to 
>>> support three distributions; supporting two simultaneously is already 
>>> difficult enough.

According to https://wiki.debian.org/DebianBuster 
, Buster was released on 2019-07-06, 
which means we've now got less than one month of even this small amount of 
support that is offered; i.e. the expectation from