[Python-ideas] Permanent code objects (less memory, quicker load, less Unix Copy On Write)

Hi All Summary: Shared objects in Unix are a major influence. This proposal can be seen as a first step towards packaging pure Python modules as Unix shared objects. First, there's a high level overview. Then some technical stuff in Appendices. An object is transient if it can be garbage collect

[Python-ideas] Re: "return if "

I love the do_stuff if cond syntax in Ruby and in perl. It's very natural to real, much more to follow than if cond: do_stuff But still I don't think that it is enough to demand a language change. Something near this is to have a default of none for A if B else None So we can ommit the else Non

[Python-ideas] Re: "return if "

To read* Em qui, 18 de jun de 2020 09:30, Daniel. escreveu: > I love the do_stuff if cond syntax in Ruby and in perl. It's very natural > to real, much more to follow than if cond: do_stuff > > But still I don't think that it is enough to demand a language change. > > Something near this is to h

[Python-ideas] Re: EVENTFD(2) support

> On 16 Jun 2020, at 19:57, doods...@gmail.com wrote: > > I've looked over the PyPi version of `eventfd`, and I feel that it is trying > to be more than just a thin wrapper. The attempt to make it cross platform > has given it more fields than it requires, as has the attempt to wrap it as a

[Python-ideas] Re: Permanent code objects (less memory, quicker load, less Unix Copy On Write)

> On 18 Jun 2020, at 10:36, Jonathan Fine wrote: > > Hi All > > Summary: Shared objects in Unix are a major influence. This proposal can be > seen as a first step towards packaging pure Python modules as Unix shared > objects. > > First, there's a high level overview. Then some technical st

[Python-ideas] Re: "return if "

> On 18 Jun 2020, at 13:30, Daniel. wrote: > > I love the do_stuff if cond syntax in Ruby and in perl. It's very natural to > real, much more to follow than if cond: do_stuff I on the other hand hate that syntax and find it harder to read. Why put the code out of sequence? if read_t

[Python-ideas] Re: "return if "

This is very personal preference I don't feel it pythonic but... Ruby make return keywork optional so is very idiomatic (in Ruby) to have multiple onliner early return with this kind of guards at beginning of methods and a "main" case below For me this is just eye preference, we have eyes trained

[Python-ideas] Re: approximate equality operator ("PEP 485 follow-up")

On Sun, Jun 14, 2020 at 02:39:49PM +0200, Sebastian M. Ernst wrote: > Hi all, > > after just having typed tons of `math.isclose` (see PEP 485 [1]) and > `numpy.isclose` calls (while basically using their default tolerances > most of the time), I was wondering whether it makes sense to add a > matc

[Python-ideas] Re: "return if "

On Thu, Jun 18, 2020 at 10:00 AM Daniel. wrote: > def foo(): > if a: > return > if B: > return > if C: > return > > do_stuff() > > > Vs > > def foo(): >return if a >return if B >return if C >do_stuff > > The second is much more compact > is compact good ?

[Python-ideas] Re: Permanent code objects (less memory, quicker load, less Unix Copy On Write)

On Thu, Jun 18, 2020 at 9:34 AM Barry Scott wrote: > To make the code avoid COW you would need to be able to make sure that all > code memory blocks are not mixed in with PyObject memory blocks. > > Then the ref count dance will have trigger COW for the code. > indeed. cPython already has its ow

[Python-ideas] Re: Permanent code objects (less memory, quicker load, less Unix Copy On Write)

Hi Barry Thank you for your interest in my proposal. Let me try to answer your question. You wrote: To make the code avoid COW you would need to be able to make sure that all > code memory blocks are not mixed in with PyObject memory blocks. > > Then the ref count dance will have trigger COW fo

[Python-ideas] Re: Permanent code objects (less memory, quicker load, less Unix Copy On Write)

> On 18 Jun 2020, at 18:37, Christopher Barker wrote: > > On Thu, Jun 18, 2020 at 9:34 AM Barry Scott > wrote: > To make the code avoid COW you would need to be able to make sure that all > code memory blocks are not mixed in with PyObject memory blocks. > > Th

[Python-ideas] Re: Permanent code objects (less memory, quicker load, less Unix Copy On Write)

> On 18 Jun 2020, at 18:42, Jonathan Fine wrote: > > Hi Barry > > Thank you for your interest in my proposal. Let me try to answer your > question. You wrote: > > To make the code avoid COW you would need to be able to make sure that all > code memory blocks are not mixed in with PyObject m

[Python-ideas] Re: Permanent code objects (less memory, quicker load, less Unix Copy On Write)

Hi Barry You wrote: > Did my last reply cover a possible implementation of this? > e.g. The code is nowhere near the ref-count that triggers COW. > Could say, do you think it's possible to extend Python so that it can use permanent code objects, when they are made available? For the moment, tha

[Python-ideas] Re: Permanent code objects (less memory, quicker load, less Unix Copy On Write)

> On 18 Jun 2020, at 19:00, Jonathan Fine wrote: > > Hi Barry > > You wrote: > Did my last reply cover a possible implementation of this? > e.g. The code is nowhere near the ref-count that triggers COW. > > Could say, do you think it's possible to extend Python so that it can use > permanent

[Python-ideas] Re: Permanent code objects (less memory, quicker load, less Unix Copy On Write)

Hi Barry You wrote: We need to define terms here. What do you mean by permanent? > Good question. I think I answered it in my original post: An object is transient if it can be garbage collected. An object is > permanent if it will never be garbage collected. You also wrote: > Being able t

[Python-ideas] Re: Permanent code objects (less memory, quicker load, less Unix Copy On Write)

> On 18 Jun 2020, at 19:30, Jonathan Fine wrote: > > Hi Barry > > You wrote: > > We need to define terms here. What do you mean by permanent? > > Good question. I think I answered it in my original post: > > An object is transient if it can be garbage collected. An object is permanent > i

[Python-ideas] Re: Permanent code objects (less memory, quicker load, less Unix Copy On Write)

Hello, I think you forgot the all-important parts: 1) How does it work technically? 2) What performance gain on which benchmark? Regards Antoine. On Thu, 18 Jun 2020 10:36:11 +0100 Jonathan Fine wrote: > Hi All > > Summary: Shared objects in Unix are a major influence. This proposal can b

[Python-ideas] Re: Permanent code objects (less memory, quicker load, less Unix Copy On Write)

Hi Antoine Thank you for your interest. You wrote: I think you forgot the all-important parts: > 1) How does it work technically? > 2) What performance gain on which benchmark? In my original post I wrote: It might be helpful, after checking the analysis and before coding, to do > some simple

[Python-ideas] Re: "return if "

it is not worth changing Python over. I agreed here, is doesn't make that difference Em qui, 18 de jun de 2020 14:25, Christopher Barker escreveu: > On Thu, Jun 18, 2020 at 10:00 AM Daniel. wrote: > >> def foo(): >> if a: >> return >> if B: >> return >> if C: >> return

[Python-ideas] Re: Permanent code objects (less memory, quicker load, less Unix Copy On Write)

On Thu, Jun 18, 2020 at 06:49:13PM +0100, Barry Scott wrote: > The key part of the idea is that the memory holding the ref count is > not adjacent to the memory holding the objects state. Further that > rarely modified state should be kept away from usually modified state. Isn't that going to p

[Python-ideas] Re: Permanent code objects (less memory, quicker load, less Unix Copy On Write)

On Thu, Jun 18, 2020 at 5:36 AM Jonathan Fine wrote: > Python allows the user to replace fn.__code__ by a different code object. > This is a rarely done dirty trick. > A dirty trick to you maybe, but occasionally useful. For example, it can be used to implement goto: https://github.com/snoack/py

[Python-ideas] Re: Permanent code objects (less memory, quicker load, less Unix Copy On Write)

One thought that I had is the fact that this whole proposal seems to be based on code blocks never needing to be collected? given the program: def fun1(v):     return v def fun2(v)     return v+1 fun1 = fun2 The function code block that was originally bound to the name fun1 should now be

[Python-ideas] Re: Permanent code objects (less memory, quicker load, less Unix Copy On Write)

On 19/06/20 9:28 am, Steven D'Aprano wrote: I know very little about how this works except a vague rule of thumb that in the 21st century memory locality is king. If you want code to be fast, keep it close together, not spread out. Python objects are already scattered all over memory, and a fun