[issue22896] Don't use PyObject_As*Buffer() functions

Martin Panter added the comment: Please see Issue 25498 for a crash possibly caused by the memoryview hack in CDataType_from_buffer(). -- ___ Python tracker

[issue22896] Don't use PyObject_As*Buffer() functions

Roundup Robot added the comment: New changeset 1da9630e9b7f by Serhiy Storchaka in branch '3.4': Issue #22896: Avoid to use PyObject_AsCharBuffer(), PyObject_AsReadBuffer() https://hg.python.org/cpython/rev/1da9630e9b7f New changeset 2e684ce772de by Serhiy Storchaka in branch 'default': Issue

[issue22896] Don't use PyObject_As*Buffer() functions

Serhiy Storchaka added the comment: In any case we need a hack in 3.4. Let open new issue for adding PyMemoryView_FromObjectEx() or like. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22896

[issue22896] Don't use PyObject_As*Buffer() functions

Stefan Krah added the comment: Thanks. No, I don't think there's an official way to accomplish that, but let's create one. How about a new function that takes the buffer request flags: PyMemoryView_FromObjectEx(exporter, PyBUF_SIMPLE|PyBUF_WRITABLE) If we can spare a new format code,

[issue22896] Don't use PyObject_As*Buffer() functions

Serhiy Storchaka added the comment: Thanks Antoine and Stefan for your reviews. -- resolution: - fixed stage: patch review - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22896

[issue22896] Don't use PyObject_As*Buffer() functions

Stefan Krah added the comment: Nice patch. I've found one issue (see Rietveld). I'm not sure about 3.4 (the patch contains minor refactorings), but otherwise I'd say go ahead with it. -- ___ Python tracker rep...@bugs.python.org

[issue22896] Don't use PyObject_As*Buffer() functions

Serhiy Storchaka added the comment: Could you please look at the patch Stefan? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22896 ___ ___

[issue22896] Don't use PyObject_As*Buffer() functions

Stefan Krah added the comment: [Slow internet connection, can't use Rietveld.] CDataType_from_buffer(): I'm not that familiar with ctypes. What is the high level goal here? Allocate a chunk of memory, wrap it in a memoryview and have the memoryview release that memory when its refcount is 0?

[issue22896] Don't use PyObject_As*Buffer() functions

Serhiy Storchaka added the comment: from_buffer() uses a memory buffer of other object. It keeps a reference to the object to prevent deallocation of memory when there will be no more external references. But this doesn't prevent from reallocating of memory of living object (e.g. for resizing

[issue22896] Don't use PyObject_As*Buffer() functions

Martin Panter added the comment: _CData.from_buffer() is meant to take a writable buffer, and create a “ctypes” object that shares the same memory. So it should not release the buffer until that “ctypes” object is no longer needed. However I don’t know the insides of memoryview() objects that

[issue22896] Don't use PyObject_As*Buffer() functions

Serhiy Storchaka added the comment: Oh, Martin expressed the same thing better. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22896 ___ ___

[issue22896] Don't use PyObject_As*Buffer() functions

Serhiy Storchaka added the comment: Here is simplified patch. _PyBuffer_Converter() and simple_buffer converter are gone. Fixed few leaks found by Martin. Fixed potential crash in ctypes. Fixed minor bugs and cleaned up ctypes tests for from_buffer(). --

[issue22896] Don't use PyObject_As*Buffer() functions

Changes by Serhiy Storchaka storch...@gmail.com: Added file: http://bugs.python.org/file37514/buffers_3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22896 ___

[issue22896] Don't use PyObject_As*Buffer() functions

Serhiy Storchaka added the comment: Updated patch addresses Antoine's comments. I still hesitate about C-contiguousity. Looks as all buffers created in the stdlib are C-contiguous, so we can't test non-contiguous buffers. Shouldn't PyObject_AsCharBuffer (or even PyObject_AsReadBuffer

[issue22896] Don't use PyObject_As*Buffer() functions

Antoine Pitrou added the comment: Shouldn't PyObject_AsCharBuffer (or even PyObject_AsReadBuffer and_PyBuffer_Converter) accept only C-contiguous buffers? PyBUF_SIMPLE enforces contiguity. See https://www.python.org/dev/peps/pep-3118/#access-flags and

[issue22896] Don't use PyObject_As*Buffer() functions

Serhiy Storchaka added the comment: Ah, there is a way to create non-contiguous buffers. b = bytes(range(16)) m = memoryview(b) m[::2].c_contiguous False PyBUF_SIMPLE enforces contiguity. Then contiguousity check in getbuffer() in Python/getargs.c is redundant. And in most cases the use

[issue22896] Don't use PyObject_As*Buffer() functions

Stefan Krah added the comment: Yes, a PyBUF_SIMPLE request implies c-contiguous, so it's ok. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22896 ___

[issue22896] Don't use PyObject_As*Buffer() functions

Changes by Martin Panter vadmium...@gmail.com: -- nosy: +vadmium ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22896 ___ ___ Python-bugs-list

[issue22896] Don't use PyObject_As*Buffer() functions

Changes by Serhiy Storchaka storch...@gmail.com: -- keywords: +needs review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22896 ___ ___

[issue22896] Don't use PyObject_As*Buffer() functions

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22896 ___

[issue22896] Don't use PyObject_As*Buffer() functions

Changes by Alexander Belopolsky alexander.belopol...@gmail.com: -- nosy: +belopolsky ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22896 ___ ___

[issue22896] Don't use PyObject_As*Buffer() functions

Serhiy Storchaka added the comment: Here is a patch which replaces deprecated functions with PyObject_GetBuffer() and like. It also introduce _PyBuffer_Converter for using in PyArgs_Parse* and clinic converter simple_buffer_converter which unlike to Py_buffer_converter (y*) does not not force

[issue22896] Don't use PyObject_As*Buffer() functions

New submission from Serhiy Storchaka: PyObject_AsCharBuffer(), PyObject_AsReadBuffer() and PyObject_AsWriteBuffer() release the buffer right after acquiring and return a pointer to released buffer. This is not safe and could cause issues sooner or later. These functions shouldn't be used in