[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Tal Einat]

Tal Einat  added the comment:

Please do, Ananthakrishnan!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Ananthakrishnan]

Ananthakrishnan  added the comment:

I want to do a PR,if this is still needeed.

--
nosy: +Ananthakrishnan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Tal Einat]

Tal Einat  added the comment:

Ivan, can you supply a PR or would you like someone else to do so?

--
nosy: +taleinat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

LGTM.

> What should I write instead of _?

(ValueError, OSError)

> And will the next call be effective (do anything), if we have already set the 
> limit with the testing call?

This doesn't matter. We test that it doesn't crash when parse arguments.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Ivan Zakharyaschev]

Ivan Zakharyaschev  added the comment:

And will the next call be effective (do anything), if we have already set the 
limit with the testing call?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Ivan Zakharyaschev]

Ivan Zakharyaschev  added the comment:

Thanks! I also thought about this simplest way. What about this:

diff --git a/Python/Lib/test/test_resource.py b/Python/Lib/test/test_resource.py
index de29d3b..bec4440 100644
--- a/Python/Lib/test/test_resource.py
+++ b/Python/Lib/test/test_resource.py
@@ -102,16 +102,21 @@ class ResourceTest(unittest.TestCase):
 
 # Issue 6083: Reference counting bug
 def test_setrusage_refcount(self):
+howmany = 100
 try:
 limits = resource.getrlimit(resource.RLIMIT_CPU)
 except AttributeError:
 self.skipTest('RLIMIT_CPU not available')
+try:
+resource.setrlimit(resource.RLIMIT_CPU, (howmany, howmany))
+except _:
+self.skipTest('Setting RLIMIT_CPU not possible')
 class BadSequence:
 def __len__(self):
 return 2
 def __getitem__(self, key):
 if key in (0, 1):
-return len(tuple(range(100)))
+return len(tuple(range(howmany)))
 raise IndexError
 
 resource.setrlimit(resource.RLIMIT_CPU, BadSequence())

What should I write instead of _?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

The simplest way is to try passing the limit as a tuple

resource.setrlimit(resource.RLIMIT_CPU, (100, 100))

and skip the test if it failed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Ivan Zakharyaschev]

Ivan Zakharyaschev  added the comment:

>>> import resource
>>> resource.getrlimit(resource.RLIMIT_CPU) 
(7200, 7260)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

What does resource.getrlimit(resource.RLIMIT_CPU) return?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Ivan Zakharyaschev]

Ivan Zakharyaschev  added the comment:

> New changeset a4c85f9b8f58 by Serhiy Storchaka in branch '2.7':
Issue #6083: Fix multiple segmentation faults occured when PyArg_ParseTuple
http://hg.python.org/cpython/rev/a4c85f9b8f58

This test has a problem: though it tests not the ability to set a CPU hard 
limit, it fails if the hard limit is limited. Perhaps, ignore any exception 
there? Could you please help me re-write it correctly, so that I can run it on 
gyle--ALT's builder host--successfully):

# Issue 6083: Reference counting bug
def test_setrusage_refcount(self):
try:
limits = resource.getrlimit(resource.RLIMIT_CPU)
except AttributeError:
self.skipTest('RLIMIT_CPU not available')
class BadSequence:
def __len__(self):
return 2
def __getitem__(self, key):
if key in (0, 1):
return len(tuple(range(100)))
raise IndexError

resource.setrlimit(resource.RLIMIT_CPU, BadSequence())

The failure:

[builder@team ~]$ python /usr/lib64/python2.7/test/test_resource.py
test_args (__main__.ResourceTest) ... ok
test_fsize_enforced (__main__.ResourceTest) ... ok
test_fsize_ismax (__main__.ResourceTest) ... ok
test_fsize_toobig (__main__.ResourceTest) ... ok
test_getrusage (__main__.ResourceTest) ... ok
test_setrusage_refcount (__main__.ResourceTest) ... ERROR

==
ERROR: test_setrusage_refcount (__main__.ResourceTest)
--
Traceback (most recent call last):
  File "/usr/lib64/python2.7/test/test_resource.py", line 117, in 
test_setrusage_refcount
resource.setrlimit(resource.RLIMIT_CPU, BadSequence())
ValueError: not allowed to raise maximum limit

--
Ran 6 tests in 0.085s

FAILED (errors=1)
Traceback (most recent call last):
  File "/usr/lib64/python2.7/test/test_resource.py", line 123, in 
test_main()
  File "/usr/lib64/python2.7/test/test_resource.py", line 120, in test_main
test_support.run_unittest(ResourceTest)
  File "/usr/lib64/python2.7/test/support/__init__.py", line 1577, in 
run_unittest
_run_suite(suite)
  File "/usr/lib64/python2.7/test/support/__init__.py", line 1542, in _run_suite
raise TestFailed(err)
test.support.TestFailed: Traceback (most recent call last):
  File "/usr/lib64/python2.7/test/test_resource.py", line 117, in 
test_setrusage_refcount
resource.setrlimit(resource.RLIMIT_CPU, BadSequence())
ValueError: not allowed to raise maximum limit

[builder@team ~]$

--
nosy: +imz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Martin Panter]

Changes by Martin Panter :


--
dependencies: +resource.prlimit(int, int, str) crashs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Stefan Krah]

Changes by Stefan Krah :


--
nosy:  -skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
priority: normal -> high

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Accepting an arbitrary sequence when "(...)" is used in the format string was 
introduced in changeset 0ef1071cb7fe.

--
versions:  -Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Oh, I shouldn't close this until this dangerous feature will be deprecated.

--
assignee: serhiy.storchaka -> 
resolution: fixed -> 
stage: committed/rejected -> needs patch
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

FreeBSD 6.4 and Windows test failures was fixed in changesets 8fb98fb758e8 and 
ec70abe8c886.

--
assignee:  -> serhiy.storchaka
resolution:  -> fixed
stage: needs patch -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

There are 6 different ways to get a function (see comment around 
PyCFuncPtr_new() in Modules/_ctypes/_ctypes.c). The other tests just use other 
ways.

I'm more carefully read ctype code and found my mistake. Need to import 
"my_strchr", and not "strchr".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Stefan Krah]

Stefan Krah added the comment:

The FreeBSD 6.4 bot is failing, too. Note that the other functions
in test_returnfuncptrs.py do this in order to get strchr():

dll = CDLL(_ctypes_test.__file__)
get_strchr = dll.get_strchr
get_strchr.restype = CFUNCTYPE(c_char_p, c_char_p, c_char)
strchr = get_strchr()

--
nosy: +skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
assignee: serhiy.storchaka -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I do not have possibility and desires blind-repair a test on alien platform, so 
just temporarily disable a new test in Lib/ctypes/test/test_returnfuncptrs.py 
on Windows. If someone has a desire to fix it fell free to do this.

I do not close this issue because committed patch only fix existing crashes in 
Python. There should be plenty of such bugs in third-party code. We have to 
deprecate this unsafe feature or reject any sequences except tuple as Alexander 
proposed.

--
stage: patch review -> needs patch
versions:  -Python 3.1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Roundup Robot]

Roundup Robot added the comment:

New changeset a4c85f9b8f58 by Serhiy Storchaka in branch '2.7':
Issue #6083: Fix multiple segmentation faults occured when PyArg_ParseTuple
http://hg.python.org/cpython/rev/a4c85f9b8f58

New changeset 4bac47eb444c by Serhiy Storchaka in branch '3.2':
Issue #6083: Fix multiple segmentation faults occured when PyArg_ParseTuple
http://hg.python.org/cpython/rev/4bac47eb444c

New changeset e0ee10f27e5f by Serhiy Storchaka in branch '3.3':
Issue #6083: Fix multiple segmentation faults occured when PyArg_ParseTuple
http://hg.python.org/cpython/rev/e0ee10f27e5f

New changeset 3e3a7d825736 by Serhiy Storchaka in branch 'default':
Issue #6083: Fix multiple segmentation faults occured when PyArg_ParseTuple
http://hg.python.org/cpython/rev/3e3a7d825736

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Gregory P. Smith]

Gregory P. Smith added the comment:

Serhiy's patch looks good to me.

--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
assignee:  -> serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
versions: +Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Here is a patch which get rid of all three PyArg_ParseTuple usage with parsing 
nested sequences. Thanks Evgeny for reproducers.

--
nosy: +serhiy.storchaka
Added file: http://bugs.python.org/file27567/PyArg_ParseTuple_refcount.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

Attached patch passes the regrtest and makes test-functools.py raise an 
exception rather than crash.  The proposed change will make functions like 
partial.__setstate__ require tuple argument even though currently it would 
accept any container.  This is not an issue with __setstate__ because it should 
only be called with arguments produced by __reduce__ and in the case of 
partial, __reduce__ produces state as a tuple.  Other functions may need to be 
modified if they need to continue to accept arbitrary sequences.

--
stage: needs patch -> patch review
Added file: http://bugs.python.org/file20400/issue6083.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

Let me summarize the issue: the PyArg_ParseTuple format code 'O' returns a 
borrowed reference.  However, when the 'O' code appears inside parenthesis, 
there may not be an object to hold the reference to borrow from.  This is what 
happens in the test-functools.py crasher:  partial.__setstate__() takes a 
4-tuple argument that is unpacked using a "()" format.  The test case 
passes an instance instead of a tuple that supports the sequence methods, but 
does not hold the reference to the "items" that its []-operator returns.  This 
is not a problem at the top level because args argument to PyArg_ParseTuple is 
always a real tuple.

I think that rather than deprecating the use of 'O' format inside parentheses, 
"(..O..)" unpacking should reject to unpack arguments other than tuples or 
maybe lists.

--
nosy: +belopolsky
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Eugene Kapun]

Changes by Eugene Kapun :


Added file: http://bugs.python.org/file19618/test-functools.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Eugene Kapun]

Changes by Eugene Kapun :


Added file: http://bugs.python.org/file19617/test-ctypes.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6083] Reference counting bug in PyArg_ParseTuple and PyArg_ParseTupleAndKeywords

[Eugene Kapun]

Eugene Kapun  added the comment:

Actually, this can't be fixed without modifying C API methods PyArg_ParseTuple 
and PyArg_ParseTupleAndKeywords, because it's possible to make an object 
deallocated before PyArg_ParseTuple returns, so Py_INCREF immediately after 
parsing would be already too late.

Here are my test cases:
test-resource.py - in Modules/resource.c, and python-bug-01.patch won't work 
against it.
test-ctypes.py - in Modules/_ctypes/_ctypes.c.
test-functools.py - in Modules/_functoolsmodule.c (py3k only).

--
components: +Interpreter Core -Extension Modules
nosy: +abacabadabacaba
title: Reference counting bug in setrlimit -> Reference counting bug in 
PyArg_ParseTuple and PyArg_ParseTupleAndKeywords
Added file: http://bugs.python.org/file19616/test-resource.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com