[issue18073] pickle.Unpickler may read too many bytes, causing hangs with blocking input stream
Serhiy Storchaka added the comment: If issue17897 fixes this bug (I'm not absolutely sure), perhaps it should be backported to 3.3. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18073 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18073] pickle.Unpickler may read too many bytes, causing hangs with blocking input stream
Changes by Alexandre Vassalotti alexan...@peadrop.com: -- assignee: - alexandre.vassalotti resolution: - fixed stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18073 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18073] pickle.Unpickler may read too many bytes, causing hangs with blocking input stream
Serhiy Storchaka added the comment: Seems as the patch for issue17897 fixes this case. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18073 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18073] pickle.Unpickler may read too many bytes, causing hangs with blocking input stream
Dylan Radcliffe added the comment: What I see in gdb is that that read is occurring due to the unpickling thread reading peeking ahead in the stream due to buffered io being used. #0 0x77bcbcbd in read () from /lib/x86_64-linux-gnu/libpthread.so.0 #1 0x0061f521 in fileio_readinto (self=0x76491528, args=0x766dc190) at ./Modules/_io/fileio.c:541 #2 0x004b96f0 in PyCFunction_Call (func=0x7601d598, arg=0x766dc190, kw=0x0) at Objects/methodobject.c:90 #3 0x0045a0c2 in PyObject_Call (func=0x7601d598, arg=0x766dc190, kw=0x0) at Objects/abstract.c:2105 #4 0x0045ad3e in PyObject_CallMethodObjArgs (callable=0x7601d598, name=0x77f3ccf0) at Objects/abstract.c:2341 #5 0x00626e65 in _bufferedreader_raw_read (self=0x76777e58, start=0xa89210 .\002X\367\017, len=4096) at ./Modules/_io/bufferedio.c:1438 #6 0x006270ed in _bufferedreader_fill_buffer (self=0x76777e59) at ./Modules/_io/bufferedio.c:1470 #7 0x00628474 in _bufferedreader_peek_unlocked (self=0x76777e58) at ./Modules/_io/bufferedio.c:1714 #8 0x00624b1d in buffered_peek (self=0x76777e58, args=0x766d4948) at ./Modules/_io/bufferedio.c:880 As a possible workaround if you disable buffering when opening your pipe for input it seems to prevent the problem (though I am not sure about performance implications). 22c22 load = Unpickler(os.fdopen(r, 'rb')).load --- load = Unpickler(os.fdopen(r, 'rb', 0)).load -- nosy: +dylanr ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18073 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18073] pickle.Unpickler may read too many bytes, causing hangs with blocking input stream
Antoine Pitrou added the comment: For the record, this is one of the things that the framing proposal in pickle protocol 4 (pep 3154 - http://www.python.org/dev/peps/pep-3154/#framing) is able to address. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18073 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18073] pickle.Unpickler may read too many bytes, causing hangs with blocking input stream
New submission from Julien Muchembled: I use pickle over sockets to make 2 processes communicate and I experience hangs when the resulting pickle message is exactly 4097 bytes. Attached file is a small script reproducing the issue, trying a few values around 4096. It runs on both Python 2.7 3.3: - Python 2.7.3: ok (script exits) - Python 3.3.2: last iteration hangs With strace, you can easily see that after reading the 4097 bytes: [pid 17100] read(3, unfinished ... [pid 17100] ... read resumed \200\2X\367\17\0\0 ..., 4096) = 4096 [pid 17100] read(3, ., 4096) = 1 it reads again without reason: [pid 17100] read(3, No issue with the Python implementation, that you can try with little change: - Python 2.7.3: ok (from pickle import Unpickler) - Python 3.3.2: ok (from pickle import _Unpickler as Unpickler) -- components: Extension Modules files: test_case.py messages: 190149 nosy: jm priority: normal severity: normal status: open title: pickle.Unpickler may read too many bytes, causing hangs with blocking input stream type: behavior versions: Python 3.3 Added file: http://bugs.python.org/file30388/test_case.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18073 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18073] pickle.Unpickler may read too many bytes, causing hangs with blocking input stream
Changes by Serhiy Storchaka storch...@gmail.com: -- nosy: +alexandre.vassalotti, pitrou, serhiy.storchaka versions: +Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18073 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
