> On May 28, 2020, at 8:31 AM, Erik Johnston <e...@matrix.org> wrote:
>
>
> On 23/05/2020 06:39, Glyph wrote:
>>
>>> On May 19, 2020, at 1:52 AM, Richard van der Hoff <rich...@matrix.org
>>> <mailto:rich...@matrix.org>> wrote:
>>>
>>> On 16/05/2020 06:56, Glyph wrote:
>>>>
>>>>
>>>>> On May 15, 2020, at 8:40 PM, Craig Rodrigues <rodr...@crodrigues.org
>>>>> <mailto:rodr...@crodrigues.org>> wrote:
>>>>>
>>>>> Maybe it would be OK to do one more release of Twisted and announce that
>>>>> as the last release supporting Python 3.5, before
>>>>> dropping support?
>>>>
>>>> Yeah; whenever we drop a Python version we should always support at least
>>>> one more release, so that people have some notice before they lose access
>>>> to the next set of security updates.
>>>>
>>>> Any 3.5 users on this list who would want to postpone it longer than this?
>>>>
>>> Sadly we have an important customer whose servers run debian oldstable,
>>> which means we need to stay compatible with 3.5 until we can persuade them
>>> to upgrade, and it's taken a couple of years to get them off python 2.7...
>>>
>>> I'm not sure that should necessarily affect your plans, but I doubt we're
>>> alone in this situation.
>>>
>>
>> I guess one thing I'm curious about is why your application would need to be
>> installed along with the system Python on those OS versions? It seems like
>> a packaging strategy that ignored the fossilized versions that Debian
>> packages with the system and just built its own Python would be more
>> reliable and allow for upgrading at least most Python dependencies well
>> beyond what the system would allow by policy. Or, for that matter, why not
>> just run in a Docker container?
>>
>> Matrix is a pretty big user, and so in some sense I care about this specific
>> case, but I also find the general question interesting, because I have
>> difficulty reasoning about how long to support older versions of things in
>> the modern application packaging environment where containers, virtualenvs,
>> and associated tooling make it possible to effectively ignore the base
>> environment. When & why do you have to pay attention to it?
>>
>> -glyph
>
> I believe in this case its a general desire to keep track of what packages
> are running and where they've come from. They basically trust that packages
> from official Debian repositories are probably safe from being tampered with,
> whereas random tarballs of code from the web are not safe (unless they're
> signed by someone they trust or whatever).
>
I think this sounds like a misunderstanding of Debian's vetting process? It's
not like there's a ton of additional auditing that goes into packaging
something. There's definitely an authentication process for both Twisted and
Python, although this attestation could be somewhat stronger and less
centralized, PyPI does quite a bit of heavy lifting there.
> Now, I think it would be possible to get a newer version of Python on their
> infrastructure if we needed, but I'm sure there would be hoops that would
> need to be jumped through and justifications given, etc, which would
> undoubtedly take some time. So really it just means extra faff for them and
> us, especially since we're only a small part of their overall infrastructure.
>
It's worth noting that Python 3.5 itself is out of security support (per
https://www.python.org/dev/peps/pep-0478/
<https://www.python.org/dev/peps/pep-0478/>) and it's not like Debian employs a
team of Python security-support engineers to continue maintaining this dead
branch. If their infrastructure were to try to comply with various security
standards such as PCI-DSS, you might bump into the need to upgrade to a newer
version with available future patch releases anyway.
> Then there is the fact that they're not unique. While oldstable is, well,
> old, its still very much supported and so there's going to be a bunch of
> "enterprise" (for want of a better term) customers who will still be using
> it, and we'll need to go through the faff each and every time, which is quite
> tedious. Come the Autumn when oldstable stops being supported (or at least,
> goes into LTS mode),
>
You say "Autumn", but... oldstable is supported for the purposes of migrating
off of it... it's not really even supported by Debian proper. To quote from
https://wiki.debian.org/DebianOldStable
<https://wiki.debian.org/DebianOldStable> :
>>> The security team tries to support a stable distribution for about one year
>>> after the next stable distribution has been released, except when another
>>> stable distribution is released within this year. It is not possible to
>>> support three distributions; supporting two simultaneously is already
>>> difficult enough.
According to https://wiki.debian.org/DebianBuster
<https://wiki.debian.org/DebianBuster>, Buster was released on 2019-07-06,
which means we've now got less than one month of even this small amount of
support that is offered; i.e. the expectation from Debian's perspective is that
they'd be done migrating off of oldstable by that point. I guess
https://wiki.debian.org/LTS <https://wiki.debian.org/LTS> is a separate project?
> it might become easier to justify that it really is reasonable for us to
> require a newer version of Python (and that they should really really upgrade
> their debian version, and its their own fault that they haven't and have to
> deal with faff that comes from that).
>
> This would also be easier if debian had newer versions of python in
> backports, or someone ran a semi-official package repository which had them,
> but as far as I can tell no one does for debian.
>
> (Of course you can argue that all the above is a bit silly from a security
> perspective, especially when you start considering virtualenvs and the like,
> but I have some sympathy for their outlook even if it is a pain at times).
>
It's a bit of a pet peeve of mine for reasons which are mostly covered by this
blog post -https://alexgaynor.net/2015/mar/30/red-hat-open-source-community/
<https://alexgaynor.net/2015/mar/30/red-hat-open-source-community/>
But it bugs me not just because of the expectation of free labor (we can refuse
to support 3.5 if that is a point of contention, as you say) but because
"security support" is a bit of a misnomer, especially in Debian's case where
there's no commercial relationship and hence not much in the way of libability.
Much to my chagrin I can't find the paper I remember reading about this right
now, but, I believe these things are all true:
Many non-"security" bugs are in fact security bugs that nobody has noticed you
can exploit.
Many "low-severity" or "un-exploiable" security bugs are in fact exploitable
"supported" distros rarely take care to backport many patches for their
software, and when they do, they often make undetected errors (like debian's
infamous ssh bug) which are analyzed by far fewer security analysts than the
upstream source code.
So I feel like the folks making the decision to stick with these old
"supported" distros are only getting half the story - sure, it won't break, but
are they actually getting the security fixes that they think they are?
Debian's staff are stretched pretty thin as it is.
> In terms of twisted dropping support of 3.5, I guess the question is to what
> extent do you want applications to be hassle free to deploy on the more
> "enterprise" style environment?
>
One other confusion I have about these environments is why they want very-old
Python but don't also want very-old Twisted.
> Without having followed along with the thread my bias would be to keep
> support until stretch support is ended in the Autumn. Though if 3.5 support
> is holding things back a lot then I can completely understand dropping it
> sooner.
>
This is somewhat moot, since I'd be pleasantly surprised if we managed to get
another release out at all before September :). But if this is your time frame
then lagging your Twisted upgrade by one or two months doesn't seem like it
would be worth us supporting this version for much longer.
> (FWIW in Matrix/Synapse we take the view that we retain compat with
> old-but-supported dependencies, such as postgres, until/unless it
> starts being too costly in terms of maintenance and opportunity costs)
This is more or less the way I feel about Twisted, particularly around
supporting old and weird kernels, because sometimes you really just can't
control the environment you're being installed into and I don't want to make
Twisted a blocker for that. But supporting old Pythons, old service_identity
modules, old OpenSSL's, etc, has been seeming more and more to me like a
disservice to the community, because it facilitates the adoption of slow,
insecure, dangerous deployment practices so we're not doing anyone any favors.
Anyway thank you very much for providing this perspective. I'm curious about
how folks deploying ossified software like this think about its security
profile, but it's good to know that the practice is still going strong.
-glyph
_______________________________________________
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
