[issue30695] add a nomemory_allocator to the _testcapi module

[Xavier de Gaye]

Change by Xavier de Gaye :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[Xavier de Gaye]

Xavier de Gaye  added the comment:


New changeset aaf6a3dbbdb9754f98d480b468adfcae0f66e3a2 by xdegaye (Miss 
Islington (bot)) in branch '3.6':
[3.6] bpo-30695: Add set_nomemory(start, stop) to _testcapi (GH-2406) (#4083)
https://github.com/python/cpython/commit/aaf6a3dbbdb9754f98d480b468adfcae0f66e3a2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +4053
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[Xavier de Gaye]

Xavier de Gaye  added the comment:

Test cases in issues #30697 and #30817, back ported to 3.6, need 
_testcapi.set_nomemory().

--
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[Xavier de Gaye]

Xavier de Gaye added the comment:


New changeset 85f643023fed3d4e2fb8e399f9ad57f3a65ef237 by xdegaye in branch 
'master':
bpo-30695: Add set_nomemory(start, stop) to _testcapi (GH-2406)
https://github.com/python/cpython/commit/85f643023fed3d4e2fb8e399f9ad57f3a65ef237


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[Xavier de Gaye]

Xavier de Gaye added the comment:

> Oh, I'm curious about that one :-)

This is issue 30817.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[STINNER Victor]

STINNER Victor added the comment:

2017-06-26 15:34 GMT+02:00 Xavier de Gaye :
> python: Objects/call.c:89: _PyObject_FastCallDict: Assertion 
> `!PyErr_Occurred()' failed.
> Aborted (core dumped)
>
> I will create a new issue when the current issue is closed.

Oh, I'm curious about that one :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[Xavier de Gaye]

Xavier de Gaye added the comment:

> But is it useful to test N memory allocation failures in a row?

I think it is useful. See my previous post.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[Xavier de Gaye]

Xavier de Gaye added the comment:

For the record, while working on the test case of PR 2406, I found by chance 
that the following script:

# Script start.
import _testcapi
class C(): pass
_testcapi.set_nomemory(0, 5)
C()
# Script end.

fails with:

python: Objects/call.c:89: _PyObject_FastCallDict: Assertion 
`!PyErr_Occurred()' failed.
Aborted (core dumped)

I will create a new issue when the current issue is closed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[STINNER Victor]

STINNER Victor added the comment:

I prefer your new PR. But is it useful to test N memory allocation failures in 
a row? The API and the code would be simpler if we would only test a single 
failure. What do you think?

I know that pyfailmalloc is different, but I'm not sure that pyfailmalloc 
design is right neither :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[Xavier de Gaye]

Changes by Xavier de Gaye :


--
pull_requests: +2454

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[Xavier de Gaye]

Changes by Xavier de Gaye :


--
pull_requests:  -2415

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[Xavier de Gaye]

Changes by Xavier de Gaye :


--
pull_requests: +2415

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[Xavier de Gaye]

Changes by Xavier de Gaye :


--
pull_requests:  -2406

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[Xavier de Gaye]

Changes by Xavier de Gaye :


--
pull_requests: +2406

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[Xavier de Gaye]

Xavier de Gaye added the comment:

Good point. Not freeing the memory made the implementation of 
set_nomemory_allocator() simpler, no need to call PyMem_GetAllocator(). Forget 
about this point, sorry :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[STINNER Victor]

STINNER Victor added the comment:

> I am not sure, failmalloc.enable(range) accepts only range > 1, hook_malloc() 
> fails only once instead of permanently and hook_free() does free memory.

All these things can change :-)

> Also what happens then after this call is not very close to real life as 
> memory is never freed.

Why not freeing memory?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[Xavier de Gaye]

Xavier de Gaye added the comment:

I am not sure, failmalloc.enable(range) accepts only range > 1, hook_malloc() 
fails only once instead of permanently and hook_free() does free memory.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[STINNER Victor]

STINNER Victor added the comment:

Your code can be reproduced with pyfailmalloc using N=0 :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[Xavier de Gaye]

Xavier de Gaye added the comment:

The chain of events following a call to set_nomemory_allocator() is 
deterministic, this is another difference with pyfailmalloc. Also what happens 
then after this call is not very close to real life as memory is never freed.

Having the possibility to trigger memory exhaustion in N allocations instead of 
rightaway would be very useful. Maybe this could be done by including 
pyfailmalloc in _testcapi with some changes to enable multiple behaviors, among 
which one of them would be deterministic.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[STINNER Victor]

STINNER Victor added the comment:

The main difference between your change and pyfailmalloc is the ability in 
pyfailmalloc to only fail in N allocations. If you want to test various code 
paths, you need to allow N allocations to reach the deep code that you want to 
test.

My code is something like 100 lines of C code.
https://bitbucket.org/haypo/pyfailmalloc/src/cdaf3609a30b88243d04e79d6074f3b28d9b64e3/failmalloc.c?at=default&fileviewer=file-view-default

IMHO it's small enough to fit directly into _testcapi.

The question is more what do you want to do with it? It was proposed to 
"include" pyfailmalloc directly in the Python test suite. Maybe add an option 
to enable it?

I ran the Python test suite with pyfailmalloc in gdb using this script:
https://bitbucket.org/haypo/pyfailmalloc/src/cdaf3609a30b88243d04e79d6074f3b28d9b64e3/debug_cpython.gdb?at=default&fileviewer=file-view-default

./python -m test -F -r -v -x test_tracemalloc

I waited for the next *crash* and ignored all expected tests failing with 
MemoryError. --forever (-F) is nice to run random tests until you get a crash.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[Xavier de Gaye]

Xavier de Gaye added the comment:

> I think the allocators must just return NULL, without setting the error.
You are right, thanks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[Xavier de Gaye]

Xavier de Gaye added the comment:

Yes, actually I thought first about re-opening issue 19817 instead of opening 
this new issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[STINNER Victor]

STINNER Victor added the comment:

Nice. Do you know my https://pypi.python.org/pypi/pyfailmalloc project? It
helped me to identify and fix dozens of code which didn't handle properly
memory allocation failures. Seach for "pyfailmalloc" in the bug tracker
(closed issues) ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I think the allocators must just return NULL, without setting the error.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[Xavier de Gaye]

New submission from Xavier de Gaye:

Add the set_nomemory_allocator() function to _testcapi that sets a no memory 
allocator.

--
components: Tests
files: nomemory_allocator.patch
keywords: patch
messages: 296266
nosy: haypo, xdegaye
priority: normal
severity: normal
status: open
title: add a nomemory_allocator to the _testcapi module
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file46958/nomemory_allocator.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com