On 14/12/2021 19.19, Eric Snow wrote:
A while back I concluded that neither approach would work for us. The
approach I had taken would have significant cache performance
penalties in a per-interpreter GIL world. The approach that modifies
Py_INCREF() has a significant performance penalty due to
One thing to consider: ideally, inmortal objects should not participate in
the GC. There is nothing inheritly wrong if they do but we would need to
update the GC (and therefore add more branching in possible hot paths) to
deal with these as the algorithm requires the refcount to be exact to
correct
On Wed, 15 Dec 2021 10:42:17 +0100
Christian Heimes wrote:
> On 14/12/2021 19.19, Eric Snow wrote:
> > A while back I concluded that neither approach would work for us. The
> > approach I had taken would have significant cache performance
> > penalties in a per-interpreter GIL world. The approac
On Wed, Dec 15, 2021 at 2:21 AM Antoine Pitrou wrote:
>
> On Wed, 15 Dec 2021 10:42:17 +0100
> Christian Heimes wrote:
> > On 14/12/2021 19.19, Eric Snow wrote:
> > > A while back I concluded that neither approach would work for us. The
> > > approach I had taken would have significant cache per
Hi Brett,
IMO the PEP 630 is a good summary and a practical guide explaining how
to port existing C extensions to newer C API which are compatible with
subinterpreters, unloading a C extension and be able to load a C
extension more than once (in the same interpreter):
https://www.python.org/dev/pe
On Tue, Dec 14, 2021 at 7:27 PM Eric Snow wrote:
> We have some options:
>
> * live with the full penalty
> * make other changes to reduce the penalty to a more acceptable
> threshold than 5%
> * eliminate the penalty (e.g. claw back 5% elsewhere)
The last time I saw a benchmark on immortal objec
Hi,
On Tue, Dec 14, 2021 at 6:13 PM Eric Snow wrote:
> I'm still hoping to land a per-interpreter GIL for 3.11. There is
> still a decent amount of work to be done but little of it will require
> solving any big problems:
>
> * pull remaining static globals into _PyRuntimeState and PyInterpreter
+1 for consolidated documentation about per-interpreter GIL.
On Tue, Dec 14, 2021 at 9:12 AM Eric Snow
wrote:
> Hi all,
>
> I'm still hoping to land a per-interpreter GIL for 3.11. There is
> still a decent amount of work to be done but little of it will require
> solving any big problems:
>
>
Hi,
Le 15/12/2021 à 02:42, Gregory P. Smith a écrit :
On Tue, Dec 14, 2021 at 9:06 AM Yann Droneaud
wrote:
Should I open a bug for this issue ?
Makes sense, it is a pretty small change to make to the announcement
format. Filed. https://bugs.python.org/issue46077
Thanks, you transcrib
Hi,
Have I missed something?
Having the SHA-2 256 in the release email makes it a bit more immutable: my
mailbox is not
reachabable by the same people that can replace the release archives on the
server.
Let's say it's adding a second factor of trust.
Regards.
--
Yann Droneaud
OPTEYA
Hi Eric,
Did you try to take into account the envisioned project for adding a
"complete" GC and removing the GIL?
Regards
Antoine.
On Tue, 14 Dec 2021 10:12:07 -0700
Eric Snow wrote:
> Hi all,
>
> I'm still hoping to land a per-interpreter GIL for 3.11. There is
> still a decent amount of
On Wed, 15 Dec 2021 14:13:03 +0100
Antoine Pitrou wrote:
> Hi Eric,
>
> Did you try to take into account the envisioned project for adding a
> "complete" GC and removing the GIL?
Sorry, I was misremembering the details. Sam Gross' proposal
(posted here on 07/10/2021) doesn't switch to a "comple
It might be worth (re)reviewing Sam Gross's nogil effort to see how he
approached this:
https://github.com/colesbury/nogil#readme
He goes into plenty of detail in his design document about how he deals
with immortal objects. From that document:
Some objects, such as interned strings, small inte
On Wed, Dec 15, 2021 at 2:50 AM Pablo Galindo Salgado
wrote:
> One thing to consider: ideally, inmortal objects should not participate in
> the GC. There is nothing inheritly wrong if they do but we would need to
> update the GC (and therefore add more branching in possible hot paths) to
> deal
On Wed, Dec 15, 2021 at 2:42 AM Christian Heimes wrote:
> Would it be possible to write the Py_INCREF() and Py_DECREF() macros in
> a way that does not depend on branching? For example we could use the
> highest bit of the ref count as an immutable indicator and do something like
As Antoine point
On Tue, Dec 14, 2021 at 11:19 AM Eric Snow wrote:
> The idea of objects that never get deallocated isn't new and has been
> explored here several times. Not that long ago I tried it out by
> setting the refcount really high. That worked. Around the same time
> Eddie Elizondo at Facebook did som
All singletons do, AFAIK. And most static types that I can think of also
do, even the empty tuple.
On Wed, 15 Dec 2021 at 16:49, Eric Snow wrote:
> On Wed, Dec 15, 2021 at 2:50 AM Pablo Galindo Salgado
> wrote:
> > One thing to consider: ideally, inmortal objects should not participate
> in the
On Wed, Dec 15, 2021 at 4:03 AM Victor Stinner wrote:
> The last time I saw a benchmark on immortal object, it was clearly 10%
> slower overall on the pyperformance benchmark suite. That's a major
> slowdown.
Yes, I plan on benchmarking the change as soon as we can run
pyperformance on main.
> >
On Wed, Dec 15, 2021 at 8:16 AM Skip Montanaro wrote:
> It might be worth (re)reviewing Sam Gross's nogil effort to see how he
> approached this:
Yeah, there is good info in there.
-eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscr
On Wed, Dec 15, 2021 at 6:16 AM Antoine Pitrou wrote:
> Did you try to take into account the envisioned project for adding a
> "complete" GC and removing the GIL?
Yeah. I was going to start a separate thread about per-interpreter
GIL vs. no-gil, but figured I was already pushing my luck with 3
s
On Tue, Dec 14, 2021 at 11:19 AM Eric Snow wrote:
> There is one solution that would help both of the above in a nice way:
> "immortal" objects.
FYI, here are some observations that came up during some discussions
with the "faster-cpython" team today:
* immortal objects should probably only be i
On Thu, Dec 16, 2021 at 6:03 AM Eric Snow wrote:
> * using the ref count isn't the only viable approach; another would be
> checking the pointer itself
>+ put the object in a specific section of static data and compare
> the pointer against the bounds
>+ this avoids loading the actual obje
On Thu, Dec 16, 2021 at 7:03 AM Eric Snow wrote:
>
> On Wed, Dec 15, 2021 at 12:18 PM Chris Angelico wrote:
> > Sorry if this is a dumb question, but would it be possible to solve
> > that last point with an immortal arena [1] from which immortal objects
> > could be allocated? None/True/False co
On Wed, Dec 15, 2021 at 12:18 PM Chris Angelico wrote:
> Sorry if this is a dumb question, but would it be possible to solve
> that last point with an immortal arena [1] from which immortal objects
> could be allocated? None/True/False could be allocated there, but so
> could anything that is more
On Wed, Dec 15, 2021 at 6:04 AM Antoine Pitrou wrote:
> On Wed, 15 Dec 2021 14:13:03 +0100
> Antoine Pitrou wrote:
>
> > Did you try to take into account the envisioned project for adding a
> > "complete" GC and removing the GIL?
>
> Sorry, I was misremembering the details. Sam Gross' proposal
On Tue, Dec 14, 2021 at 10:12 AM Eric Snow wrote:
> * it is fully backward compatible and the C-API is essentially unaffected
Hmm, this is a little misleading. It will definitely be backward
incompatible for extension modules that don't work under multiple
subinterpreters (or rely on the GIL to
On Wed, Dec 15, 2021 at 3:00 PM Guido van Rossum wrote:
> who cares if the refcount for None is 5000 or 1610612736? As long as the
> refcount of *mortal* objects is the same as it was before, this shouldn't
> be a problem.
>
indeed:
$ python -c "import sys; print(sys.getrefcount(None))"
4110
>> It does seem a bit silly to actually be tracking that refcount :-)
Not that silly. It can easily help in C extensions to detect wrong DECREF
calls:
>> import ctypes
>> non = ctypes.c_long.from_address(id(None))
>> non.value = 10
>>
Fatal Python error: none_dealloc: deallocating None
Python run
On 2021-12-15 2:57 p.m., Guido van Rossum wrote:
But as long as the imbalance is less than 0x_2000_, the refcount
will remain in the inclusive range [ 0x_4000_ , 0x_7FFF_ ] and
we can test for immortality by testing a single bit:
if (o->ob_refcnt & 0x_4000_)
Could we have a
On Wed, Dec 15, 2021 at 3:07 AM Victor Stinner wrote:
> I wrote https://bugs.python.org/issue39511 and
> https://github.com/python/cpython/pull/18301 to have per-interpreter
> None, True and False singletons. My change is backward compatible on
> the C API: you can still use "Py_None" in your C co
(I just realized that we started discussing details of immortal objects in
the wrong thread -- this is Eric's overview thread, there's a separate
thread on immortal objects. But alla, I'll respond here below.)
On Wed, Dec 15, 2021 at 5:05 PM Neil Schemenauer wrote:
> On 2021-12-15 2:57 p.m., Gui
On Wed, Dec 15, 2021 at 2:57 PM Guido van Rossum wrote:
>
> I don't know how long that would take, but I suspect that a program that
> just increments the refcount relentlessly would have to run for hours
> before hitting this range. On a 64-bit machine the same approach would
> require years to
> In Python 3.11, Python still implements around 100 types as "static
> types" which are not compatible with subinterpreters, like
> &PyLong_Type and &PyUnicode_Type. I opened
> https://bugs.python.org/issue40601 about these static types, but it
> seems like changing it may break the C API *and* th
Immortal objects shouldn't be reclaimed by garbage collection, but they still
count as potential external roots for non-cyclic liveness.
-jJ
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
How common is it to reload a module in production code?
It seems like "object created at the module level" (excluding __main__) is at
least as good of an heuristic for immortality as "string that meets the
syntactic requirements for an identifier". Perhaps also anything created as
part of clas
On Wed, Dec 15, 2021 at 6:57 PM Jim J. Jewett wrote:
> Immortal objects shouldn't be reclaimed by garbage collection, but they
> still count as potential external roots for non-cyclic liveness.
>
So everything referenced by an immortal object should also be made immortal
-- even its type. Hence
On Wed, Dec 15, 2021 at 6:21 PM Guido van Rossum wrote:
> On Wed, Dec 15, 2021 at 2:57 PM Guido van Rossum wrote:
>
>>
>> I don't know how long that would take, but I suspect that a program that
>> just increments the refcount relentlessly would have to run for hours
>> before hitting this range
On Wed, Dec 15, 2021 at 02:57:46PM -0800, Guido van Rossum wrote:
> Another potential issue is that there may be some applications that take
> refcounts at face value (perhaps obtained using sys.getrefcount()). These
> would find that immortal objects have a very large refcount, which might
> surp
38 matches
Mail list logo
