On 12. 03. 22 2:45, Eric Snow wrote:
responses inline
I'll snip some discussion for a reason I'll get to later, and get right
to the third alternative:
[...]
"Special-casing immortal objects in tp_dealloc() for the relevant types
(but not int, due to frequency?)" sounds promising.
The
responses inline
-eric
On Wed, Mar 9, 2022 at 8:23 AM Petr Viktorin wrote:
> "periodically reset the refcount for immortal objects (only enable this
> if a stable ABI extension is imported?)" -- that sounds quite expensive,
> both at runtime and maintenance-wise.
Are you talking just about
On 10. 03. 22 3:35, Jim J. Jewett wrote:
"periodically reset the refcount for immortal objects (only enable this
if a stable ABI extension is imported?)" -- that sounds quite expensive,
both at runtime and maintenance-wise.
As I understand it, the plan is to represent an immortal object by
> "periodically reset the refcount for immortal objects (only enable this
> if a stable ABI extension is imported?)" -- that sounds quite expensive,
> both at runtime and maintenance-wise.
As I understand it, the plan is to represent an immortal object by setting two
high-order bits to 1. The
On 09. 03. 22 4:58, Eric Snow wrote:
On Mon, Feb 28, 2022 at 6:01 PM Eric Snow wrote:
The updated PEP text is included below. The largest changes involve
either the focus of the PEP (internal mechanism to mark objects
immortal) or the possible ways that things can break on older 32-bit
stable
On Mon, Feb 28, 2022 at 6:01 PM Eric Snow wrote:
> The updated PEP text is included below. The largest changes involve
> either the focus of the PEP (internal mechanism to mark objects
> immortal) or the possible ways that things can break on older 32-bit
> stable ABI extensions. All other
On Wed, Feb 23, 2022 at 4:21 PM Antonio Cuni wrote:
> When refcheck=True (the default), numpy raises an error if you try to resize
> an array inplace whose refcnt > 2 (although I don't understand why > 2 and
> not > 1, and the docs aren't very clear about this).
>
> That said, relying on the
On Wed, Feb 23, 2022 at 9:16 AM Petr Viktorin wrote:
>>> But tp_dict is also public C-API. How will that be handled?
>>> Perhaps naively, I thought static types' dicts could be treated as
>>> (deeply) immutable, and shared?
>>
>> They are immutable from Python code but not from C (due to
Responses inline below.
-eric
On Tue, Feb 22, 2022 at 7:22 PM Inada Naoki wrote:
> > For a recent example, see
> > https://mail.python.org/archives/list/python-dev@python.org/message/B77BQQFDSTPY4KA4HMHYXJEV3MOU7W3X/.
>
> It is not proven example, but just a hope at the moment. So option is
>
On Thu, 2022-02-24 at 00:21 +0100, Antonio Cuni wrote:
> On Mon, Feb 21, 2022 at 5:18 PM Petr Viktorin
> wrote:
>
> Should we care about hacks/optimizations that rely on having the only
> > reference (or all references), e.g. mutating a tuple if it has
> > refcount
> > 1? Immortal objects
On Mon, Feb 21, 2022 at 5:18 PM Petr Viktorin wrote:
Should we care about hacks/optimizations that rely on having the only
> reference (or all references), e.g. mutating a tuple if it has refcount
> 1? Immortal objects shouldn't break them (the special case simply won't
> apply), but this
On Wed, Feb 23, 2022 at 8:19 AM Petr Viktorin wrote:
> On 23. 02. 22 2:46, Eric Snow wrote:
>
>
[SNIP]
>
> > So it seems like the bar should be pretty low for this one (assuming
> > we get the performance penalty low enough). If it were some massive
> > or broadly impactful (or even clearly
On 23. 02. 22 2:46, Eric Snow wrote:
Thanks for the responses. I've replied inline below.
Same here :)
Immortal Global Objects
---
All objects that we expect to be shared globally (between interpreters)
will be made immortal. That includes the following:
* singletons
On Tue, Feb 22, 2022, 20:26 Larry Hastings wrote:
> Are these optimizations specifically for the PR, or are these
> optimizations we could apply without taking the immortal objects? Kind of
> like how Sam tried to offset the nogil slowdown by adding optimizations
> that we went ahead and added
On 2/22/22 6:00 PM, Eric Snow wrote:
On Sat, Feb 19, 2022 at 12:46 AM Eric Snow wrote:
Performance
---
A naive implementation shows `a 4% slowdown`_.
Several promising mitigation strategies will be pursued in the effort
to bring it closer to performance-neutral. See the `mitigation`_
On Wed, Feb 23, 2022 at 1:46 AM Eddie Elizondo via Python-Dev
wrote:
>
>
> That article is five years old so it doesn't reflect the current state of the
> system! We have continuous profiling and monitoring of Copy on Writes and
> after introducing the techniques described in this PEP, we have
On Wed, Feb 23, 2022 at 10:12 AM Eric Snow wrote:
>
> Thanks for the feedback. I've responded inline below.
>
> -eric
>
> On Sat, Feb 19, 2022 at 8:50 PM Inada Naoki wrote:
> > I hope per-interpreter GIL success at some point, and I know this is
> > needed for per-interpreter GIL.
> >
> > But I
On Sat, Feb 19, 2022 at 12:46 AM Eric Snow wrote:
> Performance
> ---
>
> A naive implementation shows `a 4% slowdown`_.
> Several promising mitigation strategies will be pursued in the effort
> to bring it closer to performance-neutral. See the `mitigation`_
> section below.
FYI, Eddie
On Mon, Feb 21, 2022 at 4:56 PM Terry Reedy wrote:
> We could say that the only refcounts with any meaning are 0, 1, and > 1.
Yeah, that should work.
-eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to
On Mon, Feb 21, 2022 at 10:56 AM wrote:
> For what it's worth Cython does this for string concatenation to concatenate
> in place if possible (this optimization was copied from CPython). It could be
> disabled relatively easily if it became a problem (it's already CPython only
> and version
Thanks for the responses. I've replied inline below.
-eric
On Mon, Feb 21, 2022 at 9:11 AM Petr Viktorin wrote:
>
> On 19. 02. 22 8:46, Eric Snow wrote:
> > Thanks to all those that provided feedback. I've worked to
> > substantially update the PEP in response. The text is included below.
>
Thanks for the feedback. I've responded inline below.
-eric
On Sat, Feb 19, 2022 at 8:50 PM Inada Naoki wrote:
> I hope per-interpreter GIL success at some point, and I know this is
> needed for per-interpreter GIL.
>
> But I am worrying about per-interpreter GIL may be too complex to
>
Hey Inada, thanks for the feedback
> Generally speaking, fork is a legacy API. It is too difficult to know which
> library is fork-safe, even for stdlibs.
Yes, this is something that Instagram has to go into great lengths to make sure
that we get the entire execution into a state where it's
On 2/21/2022 11:11 AM, Petr Viktorin wrote:
On 19. 02. 22 8:46, Eric Snow wrote:
As part of this proposal, we must make sure that users can clearly
understand on which parts of the refcount behavior they can rely and
which are considered implementation details. Specifically, they should
use
On Tue, 22 Feb 2022 at 03:00, Larry Hastings wrote:
>
>
> On 2/21/22 22:06, Chris Angelico wrote:
>
> On Mon, 21 Feb 2022 at 16:47, Larry Hastings wrote:
>
> While I don't think it's fine to play devil's advocate, given the choice
> between "this will help a common production use-case"
Petr Viktorin wrote:
> Should we care about hacks/optimizations that rely on having the only
> reference (or all references), e.g. mutating a tuple if it has refcount
> 1? Immortal objects shouldn't break them (the special case simply won't
> apply), but this wording would make them illegal.
>
On 19. 02. 22 8:46, Eric Snow wrote:
Thanks to all those that provided feedback. I've worked to
substantially update the PEP in response. The text is included below.
Further feedback is appreciated.
Thank you! This version is much clearer. I like the PEP more and more!
I've sent a PR with a
On 2/21/22 22:06, Chris Angelico wrote:
On Mon, 21 Feb 2022 at 16:47, Larry Hastings wrote:
While I don't think it's fine to play devil's advocate, given the choice between "this will
help a common production use-case" (pre-fork servers) and "this could hurt a hypothetical
production use
On 2/21/22 21:44, Larry Hastings wrote:
While I don't think it's fine to play devil's advocate,"
Oh! Please ignore the word "don't" in the above sentence. I /do/ think
it's fine to play devil's advocate.
Sheesh,
//arry/
___
Python-Dev
On Mon, 21 Feb 2022 at 16:47, Larry Hastings wrote:
>
>
> While I don't think it's fine to play devil's advocate, given the choice
> between "this will help a common production use-case" (pre-fork servers) and
> "this could hurt a hypothetical production use case" (long-running
> applications
While I don't think it's fine to play devil's advocate, given the choice
between "this will help a common production use-case" (pre-fork servers)
and "this could hurt a hypothetical production use case" (long-running
applications that reload modules enough times this could waste a
On Sat, 19 Feb 2022 12:05:22 -0500
Larry Hastings wrote:
> On 2/19/22 04:41, Antoine Pitrou wrote:
> > On Fri, 18 Feb 2022 14:56:10 -0700
> > Eric Snow wrote:
> >> On Wed, Feb 16, 2022 at 11:06 AM Larry Hastings
> >> wrote:
> >>> He suggested(*) all the constants unmarshalled as part of
Hi,
I hope per-interpreter GIL success at some point, and I know this is
needed for per-interpreter GIL.
But I am worrying about per-interpreter GIL may be too complex to
implement and maintain for core developers and extension writers.
As you know, immortal don't mean sharable between
On 2/19/22 04:41, Antoine Pitrou wrote:
On Fri, 18 Feb 2022 14:56:10 -0700
Eric Snow wrote:
On Wed, Feb 16, 2022 at 11:06 AM Larry Hastings wrote:
He suggested(*) all the constants unmarshalled as part of loading a module should be
"immortal", and if we could rejigger how we allocated
On Fri, 18 Feb 2022 14:56:10 -0700
Eric Snow wrote:
> On Wed, Feb 16, 2022 at 11:06 AM Larry Hastings wrote:
> > I experimented with this at the EuroPython sprints in Berlin years ago. I
> > was sitting next to MvL, who had an interesting observation about it.
>
> Classic MvL! :)
>
> > He
On Wed, Feb 16, 2022 at 11:06 AM Larry Hastings wrote:
> I experimented with this at the EuroPython sprints in Berlin years ago. I
> was sitting next to MvL, who had an interesting observation about it.
Classic MvL! :)
> He suggested(*) all the constants unmarshalled as part of loading a
On Wed, Feb 16, 2022 at 8:45 PM Inada Naoki wrote:
> Is there any common tool that utilize CoW by mmap?
> If you know, please its link to the PEP.
> If there is no common tool, most Python users can get benefit from this.
Sorry, I'm not aware of any, but I also haven't researched the topic
much.
Again, thanks for the reply. It's helpful. My further responses are
inline below.
-eric
On Thu, Feb 17, 2022 at 3:42 AM Petr Viktorin wrote:
> > Agreed. However, what behavior do users expect and what guarantees do
> > we make? Do we indicate how to interpret the refcount value they
> >
On Wed, Feb 16, 2022 at 10:43 PM Jim J. Jewett wrote:
> I suggest being a little more explicit (even blatant) that the particular
> details of:
> [snip]
> are not only Cpython-specific, but are also private implementation details
> that are expected to change in subsequent versions.
Excellent
On 17. 02. 22 2:13, Eric Snow wrote:
Thanks for the feedback. My responses are inline below.
-eric
On Wed, Feb 16, 2022 at 6:36 AM Petr Viktorin wrote:
Thank you very much for writing this down! It's very helpful to see a
concrete proposal, and the current state of this idea.
I like the
I suggest being a little more explicit (even blatant) that the particular
details of:
(1) which subset of functionally immortal objects are marked as immortal
(2) how to mark something as immortal
(3) how to recognize something as immortal
(4) which memory-management activities are skipped
On Thu, Feb 17, 2022 at 7:01 AM Eric Snow wrote:
>
> > > Also note that "fork" isn't the only operating system mechanism
> > > that uses copy-on-write semantics.
> >
> > Could you elaborate? mmap, maybe?
> > [snip[
> > So if you know how to get benefit from CoW without fork, I want to know it.
>
On Wed, Feb 16, 2022 at 2:41 PM Terry Reedy wrote:
> > * the naive implementation shows a 4% slowdown
>
> Without understanding all the benefits, this seems a bit too much for
> me. 2% would be much better.
Yeah, we consider 4% to be too much. 2% would be great.
Performance-neutral would be
On Wed, Feb 16, 2022 at 12:14 PM Kevin Modzelewski wrote:
> fwiw Pyston has immortal objects, though with a slightly different goal and
> thus design [1]. I'm not necessarily advocating for our design (it makes most
> sense if there is a JIT involved), but just writing to report our experience
Thanks for the feedback. My responses are inline below.
-eric
On Wed, Feb 16, 2022 at 6:36 AM Petr Viktorin wrote:
> Thank you very much for writing this down! It's very helpful to see a
> concrete proposal, and the current state of this idea.
> I like the change,
That's good to hear. :)
>
On Wed, Feb 16, 2022 at 12:37 AM Inada Naoki wrote:
> +1 for overall idea.
Great!
> > Also note that "fork" isn't the only operating system mechanism
> > that uses copy-on-write semantics.
>
> Could you elaborate? mmap, maybe?
> [snip[
> So if you know how to get benefit from CoW without fork,
On 2/15/2022 7:10 PM, Eric Snow wrote:
* the naive implementation shows a 4% slowdown
Without understanding all the benefits, this seems a bit too much for
me. 2% would be much better.
* we have a number of strategies that should reduce that penalty
I would like to see that before
Thanks!
On Wed, Feb 16, 2022 at 11:19 AM Kevin Modzelewski wrote:
> Importantly, our system allows for the reference count of immortal objects
> to change, as long as it doesn't go below half of the original very-high
> value. So extension code with no concept of immortality will still update
>
fwiw Pyston has immortal objects, though with a slightly different goal and
thus design [1]. I'm not necessarily advocating for our design (it makes
most sense if there is a JIT involved), but just writing to report our
experience of making a change like this and the compatibility effects.
I experimented with this at the EuroPython sprints in Berlin years ago.
I was sitting next to MvL, who had an interesting observation about it.
He suggested(*) all the constants unmarshalled as part of loading a
module should be "immortal", and if we could rejigger how we allocated
them to
On 16. 02. 22 1:10, Eric Snow wrote:
Eddie and I would appreciate your feedback on this proposal to support
treating some objects as "immortal". The fundamental characteristic
of the approach is that we would provide stronger guarantees about
immutability for some objects.
A few things to
+1 for overall idea.
Some comments:
>
> Also note that "fork" isn't the only operating system mechanism
> that uses copy-on-write semantics.
>
Could you elaborate? mmap, maybe?
Generally speaking, fork is very difficult to use in safe.
My company's web apps load applications and libraries
52 matches
Mail list logo
