Re: [Distutils] Heads up: https://bootstrap.pypa.io/ez_setup.py is broken

[Jim Fulton]

On Mon, Apr 25, 2016 at 2:45 PM, Jim Fulton  wrote:
> Due to recent changes in PyPI URLs.  This breaks buildout's
> bootstrap.py as well as other tools.

I created an PR against the setuptools repo that should fix this.

Jim

-- 
Jim Fulton
http://jimfulton.info
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Heads up: https://bootstrap.pypa.io/ez_setup.py is broken

[Jim Fulton]

Due to recent changes in PyPI URLs.  This breaks buildout's
bootstrap.py as well as other tools.

Jim

-- 
Jim Fulton
http://jimfulton.info
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Current Warehouse Problems

[Ionel Cristian Mărieș]

On Mon, Apr 25, 2016 at 5:46 PM, Donald Stufft  wrote:

>
> Primarily it did that so that it could dynamically reject file uploads
> without having to buffer them entirely in memory or on disk. Without
> something asynchronous to the Python process then we end up having a hard
> limit set in whatever the WSGI server is that buffers files to disk or
> rejects file uploads over a certain size.
>

​Sounds like a webserver feature (being able to stream the request​ body)
rather than a constraint that flows from the types of syscalls it use.
Doesn't gunicorn (prefork) allow streaming the request body?



Thanks,
-- Ionel Cristian Mărieș, http://blog.ionelmc.ro
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Current Warehouse Problems

[Donald Stufft]

> On Apr 25, 2016, at 11:06 AM, Ionel Cristian Mărieș  
> wrote:
> 
> 
> On Mon, Apr 25, 2016 at 5:46 PM, Donald Stufft  > wrote:
> 
> Primarily it did that so that it could dynamically reject file uploads 
> without having to buffer them entirely in memory or on disk. Without 
> something asynchronous to the Python process then we end up having a hard 
> limit set in whatever the WSGI server is that buffers files to disk or 
> rejects file uploads over a certain size.
> 
> ​Sounds like a webserver feature (being able to stream the request​ body) 
> rather than a constraint that flows from the types of syscalls it use. 
> Doesn't gunicorn (prefork) allow streaming the request body?
> 


It does by default yes, but the problem is if you stream the request body 
instead of buffer it, then you need either a lot of threads or you need 
asynchronous IO to prevent a slowloris attack.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Current Warehouse Problems

[Donald Stufft]

> On Apr 25, 2016, at 10:37 AM, Ionel Cristian Mărieș  
> wrote:
> 
> Probably none of my damn business but why does Warehouse use gevent/eventlet? 
> Does it really need them? Are there hosting constraints that severely limit 
> available RAM (so threads/processes aren't an option)?
> 
> 

Primarily it did that so that it could dynamically reject file uploads without 
having to buffer them entirely in memory or on disk. Without something 
asynchronous to the Python process then we end up having a hard limit set in 
whatever the WSGI server is that buffers files to disk or rejects file uploads 
over a certain size.

Other than that, it doesn’t need it.


-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Current Warehouse Problems

[Ionel Cristian Mărieș]

Probably none of my damn business but why does Warehouse use
gevent/eventlet? Does it really need them? Are there hosting constraints
that severely limit available RAM (so threads/processes aren't an option)?


Thanks,
-- Ionel Cristian Mărieș, http://blog.ionelmc.ro

On Sun, Apr 24, 2016 at 6:00 PM, Donald Stufft  wrote:

>
> > On Apr 24, 2016, at 10:49 AM, Phil Thompson 
> wrote:
> >
> > I am currently unable to upload to warehouse and get...
> >
> > ConnectionError: ('Connection aborted.', BrokenPipeError(32, 'Broken
> pipe'))
> >
> > ...from twine.
> >
> > Is this a known problem?
> >
> > Thanks,
> > Phil
> > ___
> > Distutils-SIG maillist  -  Distutils-SIG@python.org
> > https://mail.python.org/mailman/listinfo/distutils-sig
>
>
> Yea I’m working on fixing it. We’ve had some problems with gevent/eventlet
> either taking progressively more time on every single request (gevent) or
> blocking forever on uploads to S3 (eventlet). This PR reverts an earlier
> change to attempt to work around it
> https://github.com/pypa/warehouse/pull/1110 and once that lands I’ll have
> another PR switching away from eventlet.
>
> -
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
> DCFA
>
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig