Hi,
My pull request has been merged into numpy. numpy now uses
PyMem_RawMalloc() rather than PyMem_Malloc() since it uses the memory
allocator without holding the GIL:
https://github.com/numpy/numpy/pull/7404
It was proposed to modify numpy to hold the GIL. Maybe it will be done later.
It means
Ping? Is someone still opposed to my change #26249 "Change
PyMem_Malloc to use pymalloc allocator"? If no, I think that I will
push my change.
My change only changes two lines, so it can be easily reverted before
CPython 3.6 if we detect major issues in third-party extensions. And
maybe it's bette
So what do you think? Is it worth to change PyMem_Malloc() allocator
to pymalloc for a small speedup?
Should we do something else before doing that?
Or do you expect that too many applications use PyMem_Malloc() without
holding the GIL and will not run try to run their application with
PYTHONMALL
2016-02-12 14:31 GMT+01:00 M.-A. Lemburg :
>>> If your program has bugs, you can use a debug build of Python 3.5 to
>>> detect misusage of the API.
>
> Yes, but people don't necessarily do this, e.g. I have
> for a very long time ignored debug builds completely
> and when I started to try them, I f
2016-03-09 18:54 GMT+01:00 Brett Cannon :
>>> https://docs.python.org/dev/c-api/memory.html#c.PyMem_SetupDebugHooks
>>
>> The main advantage of this variable is that you don't have to
>> recompile Python in debug mode to benefit of these checks.
>
> I just wanted to say this all sounds awesome! Tha
On Wed, 9 Mar 2016 at 06:57 Victor Stinner wrote:
> 2016-02-08 15:18 GMT+01:00 Victor Stinner :
> >> Perhaps if you add some guards somewhere :-)
> >
> > We have runtime checks but only implemented in debug mode for efficiency.
> >
> > By the way, I proposed once to add an environment variable to
2016-02-08 15:18 GMT+01:00 Victor Stinner :
>> Perhaps if you add some guards somewhere :-)
>
> We have runtime checks but only implemented in debug mode for efficiency.
>
> By the way, I proposed once to add an environment variable to allow to
> enable these checks without having to recompile Pyth
Hi,
2016-02-12 14:31 GMT+01:00 M.-A. Lemburg :
> Sorry, your email must gotten lost in my inbox.
no problemo
> Yes, but those are part of the stdlib. You'd need to check
> a few C extensions which are not tested as part of the stdlib,
> e.g. numpy, scipy, lxml, pillow, etc. (esp. ones which imp
On 12.02.2016 12:18, Victor Stinner wrote:
> ping?
Sorry, your email must gotten lost in my inbox.
> 2016-02-08 15:18 GMT+01:00 Victor Stinner :
>> 2016-02-04 15:05 GMT+01:00 M.-A. Lemburg :
>>> Sometimes, yes, but we also do allocations for e.g.
>>> parsing values in Python argument tuples (e.g.
ping?
2016-02-08 15:18 GMT+01:00 Victor Stinner :
> 2016-02-04 15:05 GMT+01:00 M.-A. Lemburg :
>> Sometimes, yes, but we also do allocations for e.g.
>> parsing values in Python argument tuples (e.g. using
>> "es" or "et"):
>>
>> https://docs.python.org/3.6/c-api/arg.html
>>
>> We do document to u
2016-02-07 9:22 GMT+01:00 Stefan Behnel :
> Note that the PyObject_Malloc() functions have never been documented.
Yeah, there is an old bug to track this:
http://bugs.python.org/issue20064
> And, for example, the "what's new in 2.5" document says:
>
> """
> Python’s API has many different functio
2016-02-04 15:05 GMT+01:00 M.-A. Lemburg :
> Sometimes, yes, but we also do allocations for e.g.
> parsing values in Python argument tuples (e.g. using
> "es" or "et"):
>
> https://docs.python.org/3.6/c-api/arg.html
>
> We do document to use PyMem_Free() on those; not sure whether
> everyone does t
M.-A. Lemburg schrieb am 04.02.2016 um 13:54:
> On 04.02.2016 13:29, Victor Stinner wrote:
>> But, why not PyObject_Malloc() & PObject_Free() were not used in the
>> first place?
>
> Good question. I guess developers simply thought of PyObject_Malloc()
> being for PyObjects, not arbitrary memory b
On 04.02.2016 14:25, Victor Stinner wrote:
> Thanks for your feedback, you are asking good questions :-)
>
> 2016-02-04 13:54 GMT+01:00 M.-A. Lemburg :
>>> There are 536 calls to the functions PyMem_Malloc(), PyMem_Realloc()
>>> and PyMem_Free().
>>>
>>> I would prefer to modify a single place hav
Thanks for your feedback, you are asking good questions :-)
2016-02-04 13:54 GMT+01:00 M.-A. Lemburg :
>> There are 536 calls to the functions PyMem_Malloc(), PyMem_Realloc()
>> and PyMem_Free().
>>
>> I would prefer to modify a single place having to replace 536 calls :-/
>
> You have a point the
On 04.02.2016 13:29, Victor Stinner wrote:
> Hi,
>
> 2016-02-04 11:17 GMT+01:00 M.-A. Lemburg :
>>> Do you see any drawback of using pymalloc for PyMem_Malloc()?
>>
>> Yes: You cannot free memory allocated using pymalloc with the
>> standard C lib free().
>
> That's not completly new.
>
> If Pyt
Hi,
2016-02-04 11:17 GMT+01:00 M.-A. Lemburg :
>> Do you see any drawback of using pymalloc for PyMem_Malloc()?
>
> Yes: You cannot free memory allocated using pymalloc with the
> standard C lib free().
That's not completly new.
If Python is compiled in debug mode, you get a fatal error with a h
On 03.02.2016 22:03, Victor Stinner wrote:
> Hi,
>
> There is an old discussion about the performance of PyMem_Malloc()
> memory allocator. CPython is stressing a lot memory allocators. Last
> time I made statistics, it was for the PEP 454:
> "For example, the Python test suites calls malloc() , r
> There is an old discussion about the performance of PyMem_Malloc() memory
> allocator.
Oops, I forgot to mention that my patch is a follow-up of a previous
patch showing nice speedup on dict:
http://bugs.python.org/issue23601
(but I said it in my issue ;-))
Well, see http://bugs.python.org/iss
Hi,
There is an old discussion about the performance of PyMem_Malloc()
memory allocator. CPython is stressing a lot memory allocators. Last
time I made statistics, it was for the PEP 454:
"For example, the Python test suites calls malloc() , realloc() or
free() 270,000 times per second in average.
20 matches
Mail list logo