[issue24291] Many servers (wsgiref, http.server, etc) can truncate large output blobs

[Martin Panter]

Martin Panter  added the comment:

Closing because I understand it is too late to do anything for 3.5 now.

--
resolution:  -> out of date
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24291] Many servers (wsgiref, http.server, etc) can truncate large output blobs

[Martin Panter]

Martin Panter added the comment:

The bug should no longer be a problem in 3.6, as long as my change for Issue 
26721 survives.

Here is an updated patch against 3.5.

--
versions:  -Python 3.6
Added file: http://bugs.python.org/file43840/wfile-partial.v5.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24291] Many servers (wsgiref, http.server, etc) can truncate large output blobs

[Martin Panter]

Martin Panter added the comment:

Here is an updated patch with the remaining fixes for other servers.

I considered changing the behaviour of shutil.copyfileobj() to handle partial 
writes better. But I decided against it, because: (a) nobody has complained 
yet, and (b) it could potentially break stuff, e.g. if fsrc.read() returned a 
strange kind of object.

--
Added file: http://bugs.python.org/file43233/wfile-partial.v4.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24291] Many servers (wsgiref, http.server, etc) can truncate large output blobs

[Roundup Robot]

Roundup Robot added the comment:

New changeset a57703119f40 by Martin Panter in branch '3.5':
Issue #24291: Avoid WSGIRequestHandler doing partial writes
https://hg.python.org/cpython/rev/a57703119f40

New changeset d8f021944e0b by Martin Panter in branch 'default':
Issue #24291: Merge wsgi partial write fix from 3.5
https://hg.python.org/cpython/rev/d8f021944e0b

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24291] Many servers (wsgiref, http.server, etc) can truncate large output blobs

[Martin Panter]

Martin Panter added the comment:

In wsgiref-only.v2.patch I added deprecated support for continuing when a 
partial write is detected. I don’t think it needs documenting though.

--
Added file: http://bugs.python.org/file43143/wsgiref-only.v2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24291] Many servers (wsgiref, http.server, etc) can truncate large output blobs

[Martin Panter]

Martin Panter added the comment:

I prefer your first solution because it seems to fit in better with how things 
were intended.

I can add in handling of partial writes with a deprecation warning when I get a 
chance. I guess the documentation would be something like “Since 3.5.2, partial 
writes are also handled, but this is deprecated.”

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24291] Many servers (wsgiref, http.server, etc) can truncate large output blobs

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

It looks to me that there is two opposite ways to resolve this issue.

1. Document the requirement for write() and raise an exception if write() 
returned not the size of full data. But I think it would be more user-friendly 
first emit a warning and continue calling write() until full data is written.

2. Don't document any requirement, but just silently handle partial writing.

wsgiref-only.patch contains parts from both ways. It documents new requirement, 
but doesn't deprecate partial writing. What way is chosen?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24291] Many servers (wsgiref, http.server, etc) can truncate large output blobs

[Martin Panter]

Martin Panter added the comment:

Here is a patch for just the wsgiref module, which I plan to commit in time for 
3.5.2. I also updated the test case to avoid hanging due to a (theoretical) 
race with how signal handling works.

--
Added file: http://bugs.python.org/file43066/wsgiref-only.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24291] Many servers (wsgiref, http.server, etc) can truncate large output blobs

[Martin Panter]

Martin Panter added the comment:

Thanks for your comments Serhiy. The main change in v3 is that I added a 
socketserver._write_exactly() function instead of the 
_BaseHTTPRequestHandler__write_exactly() method.

--
Added file: http://bugs.python.org/file42870/wfile-partial.v3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24291] Many servers (wsgiref, http.server, etc) can truncate large output blobs

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

SimpleHTTPRequestHandler uses shutil.copyfileobj(), and shutil.copyfileobj() 
expects that write() writes all data.

--
components: +IO
nosy: +benjamin.peterson, pitrou, stutzbach, tarek

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24291] Many servers (wsgiref, http.server, etc) can truncate large output blobs

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24291] Many servers (wsgiref, http.server, etc) can truncate large output blobs

[Martin Panter]

Martin Panter added the comment:

Since there are three different people reporting the problem with wsgiref, but 
no other reports for other server modules, I might commit the change to 
Lib/wsgiref/simple_server.py soon, and leave the other changes until they can 
get a wider review.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24291] Many servers (wsgiref, http.server, etc) can truncate large output blobs

[Martin Panter]

Martin Panter added the comment:

Widening the scope of the title to attract reviewers :)

--
title: wsgiref.handlers.SimpleHandler truncates large output blobs -> Many 
servers (wsgiref, http.server, etc) can truncate large output blobs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com