Re: [Twisted-Python] [RFC] Drop support for Python 3.5 sometime after May 2021?

[Tom Most]

I'm (unfortunately) still shipping some Twisted apps on Ubuntu 16.04's Python 
3.5. However, I am fine with dropping Python 3.5 support. I think it's 
certainly worth it for variable annotations and contextvars.

---Tom

On Fri, May 15, 2020, at 10:56 PM, Glyph wrote:
> 
> 
>> On May 15, 2020, at 8:40 PM, Craig Rodrigues  wrote:
>> 
>> Maybe it would be OK to do one more release of Twisted and announce that as 
>> the last release supporting Python 3.5, before
>> dropping support? 
> Yeah; whenever we drop a Python version we should always support at least one 
> more release, so that people have some notice before they lose access to the 
> next set of security updates.
> 
> Any 3.5 users on this list who would want to postpone it longer than this?
> 
> -glyph
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> 
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] unified filesystem API

[Chris Withers]

On 16/05/2020 07:42, Glyph wrote:



On May 15, 2020, at 11:30 PM, Chris Withers > wrote:


On 16/05/2020 06:55, Glyph wrote:
This does point out one of my secret hopes for SMB: that a file 
server's maintainers will care enough about file throughput that 
we'll finally get a centralized, official way of doing async file I/O 
that we can share with SFTP, FTP, and HTTP :).


I got excited by the title, but this all seems to be able giving 
non-filesystems a file-system like API.


What are you referring to as non-filesystems?


Ah, sorry, misread the above.

If I've missed something, please let me know, but I couldn't find an 
async interface for writing to actual file systems.


In Twisted? There isn't one, really, which is exactly the issue.  There 
are various operating system interfaces for this but none of them are great.


What's the asyncio spin on this? We server like uvicorn are all writing 
logs, are they too just punting on the blocking nature of writing to 
filesystems?


Chris

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] unified filesystem API

[Glyph]

> On May 15, 2020, at 11:30 PM, Chris Withers  wrote:
> 
> On 16/05/2020 06:55, Glyph wrote:
>> This does point out one of my secret hopes for SMB: that a file server's 
>> maintainers will care enough about file throughput that we'll finally get a 
>> centralized, official way of doing async file I/O that we can share with 
>> SFTP, FTP, and HTTP :).
> 
> I got excited by the title, but this all seems to be able giving 
> non-filesystems a file-system like API.

What are you referring to as non-filesystems?

> If I've missed something, please let me know, but I couldn't find an async 
> interface for writing to actual file systems.

In Twisted? There isn't one, really, which is exactly the issue.  There are 
various operating system interfaces for this but none of them are great.

> It feels like Twisted has just punted on them as "fast enough" to not need 
> async interfaces, but that can really not be the case for networked 
> filesystems (NFS, GPFS, etc) where the blocking time for reading or writing 
> can be seconds (or minutes if it's having a bad day!).

Indeed, I'm well aware!  That's what I'm talking about.  It's not quite worth 
the energy to do it just for your average HTTP server, but it sure would be 
nice if we did have it, since it works so much better in the edge cases you 
enumerated.  (Or even against a spinning rust platter under significant load.)

> What's the Twisted solution for these kinds of things? Defer all file IO into 
> a thread?

That is the only really portable solution, but there are sometimes 
platform-specific ones which can do better in some circumstances.  However, if 
we had a unified API for accessing the filesystem that presented a non-blocking 
interface without *talking* about threads, we'd be able to take user code 
written against that API and instead make it do something smarter than just 
using a thread - or even just use an intelligent number of threads (one per 
platter, for example, which you can figure out by looking at various files in 
/etc, on some platforms, sometimes).

-glyph

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] unified filesystem API

[Chris Withers]

On 16/05/2020 06:55, Glyph wrote:

This does point out one of my secret hopes for SMB: that a file server's 
maintainers will care enough about file throughput that we'll finally get a 
centralized, official way of doing async file I/O that we can share with SFTP, 
FTP, and HTTP :).


I got excited by the title, but this all seems to be able giving 
non-filesystems a file-system like API.


If I've missed something, please let me know, but I couldn't find an 
async interface for writing to actual file systems. It feels like 
Twisted has just punted on them as "fast enough" to not need async 
interfaces, but that can really not be the case for networked 
filesystems (NFS, GPFS, etc) where the blocking time for reading or 
writing can be seconds (or minutes if it's having a bad day!).


What's the Twisted solution for these kinds of things? Defer all file IO 
into a thread?


Chris

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python