[issue20954] Bug in subprocess._args_from_interpreter_flags causes MemoryError

[Roundup Robot]

Roundup Robot added the comment:

New changeset d42f6ce08362 by Gregory P. Smith in branch '2.7':
Fixes issue #20954: _args_from_interpreter_flags used by multiprocessing
https://hg.python.org/cpython/rev/d42f6ce08362

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20954] Bug in subprocess._args_from_interpreter_flags causes MemoryError

[Gregory P. Smith]

Gregory P. Smith added the comment:

_args_from_interpreter_flags really deserves its own unittest using a stubbed 
out sys.flags.  if someone wants to go ahead with the test provided in the 
existing patch here or write one of those, feel free.

As is, i've committed a patch logically equivalent (but less hacky) to the one 
from #19284.  Ironically _that_ patch wasn't the right thing to do in python 
_3_ but that's a separate issue i'll address outside of this bug.  It is 
effective for Python 2 where hash randomization must be explicitly enabled.

--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20954] Bug in subprocess._args_from_interpreter_flags causes MemoryError

[Gregory P. Smith]

Changes by Gregory P. Smith :


--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20954] Bug in subprocess._args_from_interpreter_flags causes MemoryError

[STINNER Victor]

STINNER Victor added the comment:

The fix for Lib/subprocess.py looks good to me, but the test doesn't fail 
without the fix, so the test doesn't look good to me:

$ ./python -m test.regrtest -v test_multiprocessing_subprocess

At least, the test doesn't ensure that the bug is fixed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20954] Bug in subprocess._args_from_interpreter_flags causes MemoryError

[R. David Murray]

Changes by R. David Murray :


--
stage:  -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20954] Bug in subprocess._args_from_interpreter_flags causes MemoryError

[Martin Dengler]

Martin Dengler added the comment:

FWIW, I've been using a this patch for 2.7.5 in a production setting for a 
month now and have had no problems.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20954
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20954] Bug in subprocess._args_from_interpreter_flags causes MemoryError

[Martin Dengler]

Changes by Martin Dengler mar...@martindengler.com:


--
nosy: +mdengler

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20954
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20954] Bug in subprocess._args_from_interpreter_flags causes MemoryError

[Martin Dengler]

Martin Dengler added the comment:

Just got hit with this in 2.7.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20954
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20954] Bug in subprocess._args_from_interpreter_flags causes MemoryError

[Marc Schlaich]

Changes by Marc Schlaich marc.schla...@googlemail.com:


--
nosy: +ncoghlan

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20954
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20954] Bug in subprocess._args_from_interpreter_flags causes MemoryError

[Marc Schlaich]

Marc Schlaich added the comment:

This was fixed in #19284 for Python 3.4 (without having possible consequences 
in mind). I have updated my patch accordingly.

Maybe it's worth to port my test case to Python 3.4.

Removed Python 3.3 as it isn't in bugfix maintenance anymore.

--
versions:  -Python 3.3
Added file: http://bugs.python.org/file34480/Issue20954.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20954
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20954] Bug in subprocess._args_from_interpreter_flags causes MemoryError

[Marc Schlaich]

New submission from Marc Schlaich:

Steps to reproduce:

- clone pytest-cov at 
https://bitbucket.org/schlamar/pytest-cov/commits/ac14225a67d715b6649f8158e05d2389b78620d2
- remove `@pytest.mark.skipif` from `test_multiprocessing_subprocess` in 
test_pytest_cov.py
- run: `tox --develop -e py27 -- -x`

Result: it fails with MemoryError in `subprocess._args_from_interpreter_flags` 
because sys.flags.hash_randomization has a value of 2147483647 and it tries to 
build a string with such a length. Find more details in attached log. Patch is 
following.

--
components: Library (Lib), Windows
files: output.log
messages: 213844
nosy: schlamar
priority: normal
severity: normal
status: open
title: Bug in subprocess._args_from_interpreter_flags causes MemoryError
type: crash
versions: Python 2.7
Added file: http://bugs.python.org/file34452/output.log

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20954
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20954] Bug in subprocess._args_from_interpreter_flags causes MemoryError

[Marc Schlaich]

Marc Schlaich added the comment:

Patch added.

--
keywords: +patch
Added file: http://bugs.python.org/file34453/Issue20954.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20954
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20954] Bug in subprocess._args_from_interpreter_flags causes MemoryError

[Marc Schlaich]

Marc Schlaich added the comment:

This comes from http://bugs.python.org/issue12098. Python 3.3 is affected, too.

Reproduction can be minimized by running the following script:


import multiprocessing


def main():
p = multiprocessing.Process(target=lambda: None)
p.start()
p.join()


if __name__ == '__main__':
main()

as

$ PYTHONHASHSEED=2147483647 python test.py
Traceback (most recent call last):
  File test.py, line 12, in module
main()
  File test.py, line 7, in main
p.start()
  File c:\Python27\lib\multiprocessing\process.py, line 130, in start
self._popen = Popen(self)
  File c:\Python27\lib\multiprocessing\forking.py, line 258, in __init__
cmd = get_command_line() + [rhandle]
  File c:\Python27\lib\multiprocessing\forking.py, line 364, in 
get_command_line
opts = util._args_from_interpreter_flags()
  File c:\Python27\lib\subprocess.py, line 508, in 
_args_from_interpreter_flags
args.append('-' + opt * v)
MemoryError

--
versions: +Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20954
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20954] Bug in subprocess._args_from_interpreter_flags causes MemoryError

[Marc Schlaich]

Marc Schlaich added the comment:

Added TestCase.

--
Added file: http://bugs.python.org/file34459/20954_test.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20954
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20954] Bug in subprocess._args_from_interpreter_flags causes MemoryError

[Marc Schlaich]

Marc Schlaich added the comment:

BTW, patches are for 2.7 branch.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20954
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20954] Bug in subprocess._args_from_interpreter_flags causes MemoryError

[Marc Schlaich]

Marc Schlaich added the comment:

Merged test case and fix in a single commit/patch.

--
Added file: http://bugs.python.org/file34460/Issue20954.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20954
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20954] Bug in subprocess._args_from_interpreter_flags causes MemoryError

[STINNER Victor]

Changes by STINNER Victor victor.stin...@gmail.com:


--
nosy: +haypo

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20954
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20954] Bug in subprocess._args_from_interpreter_flags causes MemoryError

[STINNER Victor]

Changes by STINNER Victor victor.stin...@gmail.com:


--
nosy: +sbt

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20954
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com