Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-25 Thread Jonathan Fine
Hi All INTRODUCTION This is the third and concluding post, where I describe a scheme for multi-core reference counting garbage collection. The first two posts are https://mail.python.org/pipermail/python-ideas/2018-July/052054.html

Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-21 Thread Chris Barker - NOAA Federal via Python-ideas
> You wrote: > > I'd argue that the ref counts are not interesting at all, only a > > side effect of one possible solution to the object life time problem. > > I'm happy for you to regard multi-core reference counting (MCRC) as a toy > problem, which won't become part of useful software. Perhaps

Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-21 Thread Steven D'Aprano
On Sat, Jul 21, 2018 at 11:05:43AM +0100, Daniel Moisset wrote: [snip interesting and informative discussion, thank you] > @Steven D'Aprano: you mentioned soemthign about race conditions but I don't > think this algorithm has any (the article you linked just said that doing > refcounting in the

Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-21 Thread Jonathan Fine
Hi Barry We've met before. Nice to meet you again, this time electronically. You suggested that is a different problem that needs a solution. To help maintain focus, I won't respond to that now. You wrote: > I'd argue that the ref counts are not interesting at all, only a > side effect of one

Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-21 Thread Barry Scott
> On 21 Jul 2018, at 08:54, Jonathan Fine wrote: > > Hi Steve > > Thank you for your message. I think my response below allows us to go move > forward. > > WHAT'S THE PROBLEM > You asked: > > What problem are you trying to solve? > > Its okay if there is no immediate problem, that you're

Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-21 Thread Jonathan Fine
Hi Steve Thank you for your message. I think my response below allows us to go move forward. WHAT'S THE PROBLEM You asked: > What problem are you trying to solve? > Its okay if there is no immediate problem, that you're just exploring > alternative garbage collection strategies. Or if you're

Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-21 Thread Jonathan Fine
Hi Chris Thank you for your message about two processes together sorting a deck of cards. My example was in response to a comment from Steve D'Aprano. He understood in the way it was intended, which was the algorithm is in execution non-deterministic, but that the outcome is deterministic.

Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-20 Thread Steven D'Aprano
On Fri, Jul 20, 2018 at 09:44:39PM +0100, Jonathan Fine wrote: > Hi Steve > > You wrote: > > My understanding is that reference counting is both deterministic and > > immediate. Shifting the reference counting into another thread so that > > it becomes non-deterministic and potentially delayed

Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-20 Thread Chris Angelico
On Sat, Jul 21, 2018 at 6:44 AM, Jonathan Fine wrote: > Hi Steve > > You wrote: >> My understanding is that reference counting is both deterministic and >> immediate. Shifting the reference counting into another thread so that >> it becomes non-deterministic and potentially delayed doesn't sound

Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-20 Thread Jonathan Fine
Hi Steve You wrote: > My understanding is that reference counting is both deterministic and > immediate. Shifting the reference counting into another thread so that > it becomes non-deterministic and potentially delayed doesn't sound like > an advantage to my naive understanding. The choice is

Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-20 Thread Steven D'Aprano
On Fri, Jul 20, 2018 at 06:37:49PM +0100, Jonathan Fine wrote: > Hi Steve > > You wrote > > Sorry Jonathan, can you please explain: > > > > - what problem you are solving with this? > > > > - and what's wrong with the existing garbage collector? > > The problem I'm trying to solve doing

Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-20 Thread Jonathan Fine
Hi Steve You wrote > Sorry Jonathan, can you please explain: > > - what problem you are solving with this? > > - and what's wrong with the existing garbage collector? The problem I'm trying to solve doing multi-core reference counting garbage collection without a GIL. As far as I know, we're not

Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-20 Thread Steven D'Aprano
On Thu, Jul 19, 2018 at 07:33:13AM +0100, Jonathan Fine wrote: > Hi > > Based on other people's work (including in particular talks by Larry > Hastings) and my own thinking, I've come up with a scheme for multi-core > reference count garbage collection. Sorry Jonathan, can you please explain: -

Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-20 Thread Jonathan Fine
In an earlier post, I defined BUFFERED multi-core reference count garbage collection. The basic idea is that each worker process store in buffers the need to do an INCR and DECR on the reference count for a piece of memory with a given ID. There is then a garbage collection process that does the

Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-19 Thread MRAB
On 2018-07-19 11:53, Jonathan Fine wrote: Hi Stephan Thank you for the extract from the GC Handbook, which I think I may have seen before. Yes, it is GOOD that it's an already known idea. Searching for "buffered reference counting" I found

Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-19 Thread Jonathan Fine
Hi Stephan Thank you for the extract from the GC Handbook, which I think I may have seen before. Yes, it is GOOD that it's an already known idea. Searching for "buffered reference counting" I found https://mail.python.org/pipermail/python-dev/2016-October/146696.html in which Larry Hastings says

Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-19 Thread Stephan Houben
Hi Jonathan, 2018-07-19 8:33 GMT+02:00 Jonathan Fine : > I call any such scheme BUFFERED multi-core reference count garbage > collection. The worker processes know nothing about how garbage collection > is managed. Instead, they pass over to the GC process sufficient > information to allow it to

[Python-ideas] Multi-core reference count garbage collection

2018-07-19 Thread Jonathan Fine
Hi Based on other people's work (including in particular talks by Larry Hastings) and my own thinking, I've come up with a scheme for multi-core reference count garbage collection. I think it works, and much or all of it comes from others. It might even be in the literature. Of course, if it's