[issue24550] maxint on 64 bit platforms breaks os.read

[Martin Panter]

Martin Panter added the comment:

Sorry in Python 3.5 the change was actually Issue 21932 (not Arg Clinic). But 
Victor said that change shouldn’t go into Python 2.

BTW Issue 21199 is about Python 2’s file.read() method, not os.read().

--
resolution: duplicate -> wont fix
superseder:  -> os.read() must use Py_ssize_t for the size parameter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24550] maxint on 64 bit platforms breaks os.read

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

There is more specific issue for 32-bit reads in 2.7 (with patch): issue21199.

I think this issue can be closed as a duplicate of two other issues.

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24550] maxint on 64 bit platforms breaks os.read

[Martin Panter]

Martin Panter added the comment:

Yes 3.5 should do 64-bit reads (if you have enough memory) thanks to revision 
0c57aba6b1a3 (Argument Clinic conversion):

>>> os.read(0, 2**32)
Traceback (most recent call last):
  File "", line 1, in 
MemoryError
>>> os.read(0, 2**63)
Traceback (most recent call last):
  File "", line 1, in 
OverflowError: Python int too large to convert to C ssize_t

I understand 3.4 is only open to security fixes, not changes like this. But 2.7 
is affected:

>>> os.read(0, 2**32)
Traceback (most recent call last):
  File "", line 1, in 
OverflowError: signed integer is greater than maximum

I guess this could be considered a bug in 2.7 (not an enhancement), but I’m not 
really sure.

--
nosy: +martin.panter
versions: +Python 2.7 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24550] maxint on 64 bit platforms breaks os.read

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The 32-bit size limit in multiprocessing is issue17560. AFAIK read() and 
write() now support 64-bit size on 64-bit systems.

--
nosy: +haypo, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24550] maxint on 64 bit platforms breaks os.read

[Brian Mingus]

New submission from Brian Mingus:

The lower range for this bug may be anything greater than 32 bit maxint. Other 
modules such as multiprocessing are limited passing objects of size 32 bit 
maxint, even on 64 bit systems, likely due to this issue. I have demonstrated 
this by modifying multiprocessing/connection.py to use longs in send and recv, 
which it surfaces the following error (note that read is os.read).

Traceback (most recent call last):
  File /usr/lib/python3.4/multiprocessing/process.py, line 254, in _bootstrap
self.run()
  File /usr/lib/python3.4/multiprocessing/process.py, line 93, in run
self._target(*self._args, **self._kwargs)
  File /usr/lib/python3.4/multiprocessing/pool.py, line 103, in worker
initializer(*initargs)
  File 
/usr/local/lib/python3.4/dist-packages/gensim-0.11.1_1-py3.4-linux-x86_64.egg/gensim/models/ldamulticore.py,
 line 266, in worker_e_step
chunk_no, chunk, worker_lda = input_queue.get()
  File /usr/lib/python3.4/multiprocessing/queues.py, line 96, in get
res = self._recv_bytes()
  File /usr/lib/python3.4/multiprocessing/connection.py, line 216, in 
recv_bytes
buf = self._recv_bytes(maxlength)
  File /usr/lib/python3.4/multiprocessing/connection.py, line 420, in 
_recv_bytes
return self._recv(size)
  File /usr/lib/python3.4/multiprocessing/connection.py, line 383, in _recv
chunk = read(handle, remaining)
OverflowError: signed integer is greater than maximum


Which can be traced back to this cpython code:

https://github.com/python/cpython/blob/3.4/Modules/posixmodule.c#L8048-L8065

--
components: IO
messages: 246063
nosy: Brian Mingus
priority: normal
severity: normal
status: open
title: maxint on 64 bit platforms breaks os.read
type: enhancement
versions: Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24550
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com