Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-20 Thread Steven D'Aprano
On Fri, Jul 20, 2018 at 09:44:39PM +0100, Jonathan Fine wrote: > Hi Steve > > You wrote: > > My understanding is that reference counting is both deterministic and > > immediate. Shifting the reference counting into another thread so that > > it becomes non-deterministic and potentially delayed

[Python-ideas] if we were to ever shorten `A if A else B` (was: PEP 505: None-aware operators)

2018-07-20 Thread Brett Cannon
On Thu, 19 Jul 2018 at 14:29 Steve Dower wrote: > [SNIP] > > * "We could have 'else', we don't need '??'" > > This is the "a else 'default'" rather than "a ?? 'default'" proposal, > which I do like the look of, but I think it will simultaneously mess > with operator precedence and also force me

Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-20 Thread Chris Angelico
On Sat, Jul 21, 2018 at 6:44 AM, Jonathan Fine wrote: > Hi Steve > > You wrote: >> My understanding is that reference counting is both deterministic and >> immediate. Shifting the reference counting into another thread so that >> it becomes non-deterministic and potentially delayed doesn't sound

Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-20 Thread Jonathan Fine
Hi Steve You wrote: > My understanding is that reference counting is both deterministic and > immediate. Shifting the reference counting into another thread so that > it becomes non-deterministic and potentially delayed doesn't sound like > an advantage to my naive understanding. The choice is

Re: [Python-ideas] PEP 505: None-aware operators

2018-07-20 Thread Steve Dower
On 20Jul2018 1119, Brendan Barnwell wrote: In this situation I lean toward "explicit is better than implicit" --- if you want to compare against None, you should do so explicitly --- and "special cases aren't special enough to break the rules" --- that is, None is not special enough to warrant

Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-20 Thread Steven D'Aprano
On Fri, Jul 20, 2018 at 06:37:49PM +0100, Jonathan Fine wrote: > Hi Steve > > You wrote > > Sorry Jonathan, can you please explain: > > > > - what problem you are solving with this? > > > > - and what's wrong with the existing garbage collector? > > The problem I'm trying to solve doing

Re: [Python-ideas] PEP 505: None-aware operators

2018-07-20 Thread Brendan Barnwell
On 2018-07-20 10:52, Steven D'Aprano wrote: The problem is that `A else B` looks like it ought to be the same as "else B" in if...else statements and the ternary if operator. That is, "if the condition is false", and in this case there is nothing that even hints that the condition is "A is None"

Re: [Python-ideas] Idea: Deferred Default Arguments?

2018-07-20 Thread Steven D'Aprano
On Fri, Jul 20, 2018 at 05:54:44PM +0200, Peter O'Connor wrote: > On Fri, Jul 20, 2018 at 5:41 PM, Steven D'Aprano > wrote: > > > > > > What makes you think that a built-in deferred feature won't have exactly > > the same issues? Do you have an implementation that doesn't need to do > >

Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-20 Thread Jonathan Fine
Hi Steve You wrote > Sorry Jonathan, can you please explain: > > - what problem you are solving with this? > > - and what's wrong with the existing garbage collector? The problem I'm trying to solve doing multi-core reference counting garbage collection without a GIL. As far as I know, we're not

Re: [Python-ideas] PEP 505: None-aware operators

2018-07-20 Thread Steven D'Aprano
On Fri, Jul 20, 2018 at 12:03:47PM +1200, Greg Ewing wrote: > Rhodri James wrote: > >If anyone can think of a good word for "if it isn't None, otherwise", > >I'd be all for it :-) > > I don't think there's any single Engish word that captures > all of that, so we'd have to invent one. > > Some

Re: [Python-ideas] PEP 505: None-aware operators

2018-07-20 Thread Steven D'Aprano
On Thu, Jul 19, 2018 at 10:33:21AM +0200, Antoine Pitrou wrote: > def insort_right(a, x, lo=0, hi=None): > # ... > hi = hi else len(a) I read that as "hi, if it is truthy, else len(a)". The advantage of ?? as None-aware operator is that it cannot possibly be misread as

Re: [Python-ideas] PEP 505: None-aware operators

2018-07-20 Thread Steven D'Aprano
On Thu, Jul 19, 2018 at 08:57:50PM -0400, Michael Selik wrote: > Try/except also looks decent. > > try: > x = foo['bar'][0] > except TypeError: > x = 'default' Consider the case that foo['bar'] is supposed to return either a collection (something that can be indexed) or

Re: [Python-ideas] PEP 505: None-aware operators

2018-07-20 Thread Mark E. Haase
On Thu, Jul 19, 2018 at 2:36 PM Brendan Barnwell wrote: > People keep saying that this null-coalescing behavior is so common > and > useful, etc., but that hasn't been my experience at all. In my > experience, the desire to shortcut this kind of logic is more often a > sign of

Re: [Python-ideas] PEP 505: None-aware operators

2018-07-20 Thread Steve Dower
Just for fun, I decided to go through some recently written code by some genuine Python experts (without their permission...) to see what changes would be worth taking. So I went to the sources of our github bots. Honestly, I only found three places that were worth changing (though I'm now

Re: [Python-ideas] PEP 505: None-aware operators

2018-07-20 Thread Steven D'Aprano
On Thu, Jul 19, 2018 at 12:57:01PM +0100, Jonathan Fine wrote: > > Possibly this is exactly the wrong time to propose the next big syntax > > change, since we currently have nobody to declare on it, but since we're > > likely to argue for a while anyway it probably can't hurt [...] > > Perhaps

Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-20 Thread Steven D'Aprano
On Thu, Jul 19, 2018 at 07:33:13AM +0100, Jonathan Fine wrote: > Hi > > Based on other people's work (including in particular talks by Larry > Hastings) and my own thinking, I've come up with a scheme for multi-core > reference count garbage collection. Sorry Jonathan, can you please explain: -

Re: [Python-ideas] The future of Python parallelism. The GIL. Subinterpreters. Actors.

2018-07-20 Thread Nick Coghlan
On 18 July 2018 at 05:35, Eric Snow wrote: > In order to make all this work the missing piece is a mechanism by > which the decref (#3) happens under the original interpreter. At the > moment Emily Morehouse and I are pursuing an approach that extends the > existing ceval "pending call"

Re: [Python-ideas] Idea: Deferred Default Arguments?

2018-07-20 Thread Peter O'Connor
On Fri, Jul 20, 2018 at 5:41 PM, Steven D'Aprano wrote: > > > What makes you think that a built-in deferred feature won't have exactly > the same issues? Do you have an implementation that doesn't need to do > intraspection? I don't know about these low level things, but I assume it'd be

Re: [Python-ideas] Idea: Deferred Default Arguments?

2018-07-20 Thread Nick Coghlan
On 20 July 2018 at 22:45, Steven D'Aprano wrote: > Perhaps you mean duplicate, or repeat, or copy. But surely they're not > redefined -- then they would have different values. Being able to > redefine the defaults in a wrapper function is a feature. > > Putting aside the terminology, I think this

Re: [Python-ideas] Idea: Deferred Default Arguments?

2018-07-20 Thread Steven D'Aprano
On Fri, Jul 20, 2018 at 04:43:56PM +0200, Peter O'Connor wrote: > I still think it would be nice to have this as a built-in python feature, > for a few reasons: > - When using non-differable functions (say in other codebases), we have to > do a bunch of "func = deferrable_args(func)" at the top

Re: [Python-ideas] Idea: Deferred Default Arguments?

2018-07-20 Thread Jonathan Fine
Hi Peter You wrote: On Fri, Jul 20, 2018 at 3:43 PM, Peter O'Connor wrote: > Ah, right, the fix_it(fcn) is a nice idea. It might also be a good idea, if > we're making an external library anyway, to have a "deferred" object to > avoid overloading "None" (which may mean something else than

Re: [Python-ideas] Idea: Deferred Default Arguments?

2018-07-20 Thread Peter O'Connor
Ah, right, the fix_it(fcn) is a nice idea. It might also be a good idea, if we're making an external library anyway, to have a "deferred" object to avoid overloading "None" (which may mean something else than "differ argument"). I implemented the decorator here

Re: [Python-ideas] Idea: Deferred Default Arguments?

2018-07-20 Thread Brice Parent
It might be stupid, but how about solving this problem using the following: from . import other_func, SomeClass def my_func(a=other_func.defaults.a, b=other_func.defaults.b, c=SomeClass.some_method.defaults.c):     ... or def my_func(a=None, b=None, c=None):  # or use some sentinel value

Re: [Python-ideas] PEP 505: None-aware operators: operators ?= and ?? and OR

2018-07-20 Thread Rhodri James
On 20/07/18 14:06, Michel Desmoulin wrote: Proposal a __very__ dumbed down "try" expression variable = try Expression as not sentinel [else default] I like this approach. I think it should play nice with the ternary if, though I

Re: [Python-ideas] PEP 505: None-aware operators: operators ?= and ?? and OR

2018-07-20 Thread Grégory Lielens
I like it much better than the operator version, if I understand correctly, it's like a watcher for special value, that monitor the stack. Danger is that it monitor the whole stack, so that final result is dependent on how much you decompose your algorithm into nested fiction calls: all return

Re: [Python-ideas] Idea: Deferred Default Arguments?

2018-07-20 Thread Jonathan Fine
Hi Peter You make the very good point, that > subfunction_1 may be written by someone totally different from the author of > main_function, and may even be in a different codebase. For the author of > subfunction_1, it makes no sense to use the "None" approach instead of > python's normal

Re: [Python-ideas] PEP 505: None-aware operators: operators ?= and ?? and OR

2018-07-20 Thread Michel Desmoulin
Proposal a __very__ dumbed down "try" expression variable = try Expression as not sentinel [else default] The semantic is: 1 - try the expression. If any call (from a normal method, or a __dunder__ one) returns the sentinel, we

Re: [Python-ideas] Idea: Deferred Default Arguments?

2018-07-20 Thread Steven D'Aprano
On Fri, Jul 20, 2018 at 11:03:12AM +0200, Peter O'Connor wrote: > Often when programming I run into a situation where it would be nice to > have "deferred defaults". Here is an example of what I mean: > > def subfunction_1(a=2, b=3, c=4): > return a+b*c > > def

Re: [Python-ideas] PEP 505: None-aware operators

2018-07-20 Thread Judah Levy
One of the big problems with most of the proposed spellings is that question marks to a learner will feel like truthiness tests and not None tests. I would propose adding a new default statement for the most useful part of this PEP, the ?= assignment currently: if x is None: x = blah with

Re: [Python-ideas] Idea: Deferred Default Arguments?

2018-07-20 Thread Peter O'Connor
On Fri, Jul 20, 2018 at 1:30 PM, Jonathan Fine wrote: > > > I sort of think we've now got a reasonable answer for Peter's problem. > What do you think, Peter? And Brice, are you happy with my > interpretation of your deferred keyword? I think the problem with the "None" approach (second

Re: [Python-ideas] PEP 505: None-aware operators

2018-07-20 Thread Chris Angelico
On Fri, Jul 20, 2018 at 10:24 PM, Paul Moore wrote: > On 20 July 2018 at 13:16, Chris Angelico wrote: >> On Fri, Jul 20, 2018 at 10:14 PM, Rhodri James wrote: >>> I go with SF fandom's traditional :-) definition: "somebody did it once." >>> If it's been done more than once, it's an honoured

Re: [Python-ideas] PEP 505: None-aware operators

2018-07-20 Thread Paul Moore
On 20 July 2018 at 13:16, Chris Angelico wrote: > On Fri, Jul 20, 2018 at 10:14 PM, Rhodri James wrote: >> I go with SF fandom's traditional :-) definition: "somebody did it once." >> If it's been done more than once, it's an honoured tradition. > > But if Shakespeare did it, it's just the way

Re: [Python-ideas] PEP 505: None-aware operators

2018-07-20 Thread Chris Angelico
On Fri, Jul 20, 2018 at 10:14 PM, Rhodri James wrote: > I go with SF fandom's traditional :-) definition: "somebody did it once." > If it's been done more than once, it's an honoured tradition. But if Shakespeare did it, it's just the way the language is. I think Fortran is the programming

Re: [Python-ideas] PEP 505: None-aware operators

2018-07-20 Thread Rhodri James
On 20/07/18 01:30, Greg Ewing wrote: Rhodri James wrote: On 19/07/18 07:06, Greg Ewing wrote: There's no such tradition for the new operators being proposed. There is, actually, it's just not a long one.  C# has had null-aware operators for a while, for example. THere's a precedent, yes,

Re: [Python-ideas] Idea: Deferred Default Arguments?

2018-07-20 Thread Jonathan Fine
Excellent contributions. I'm going to try to (partially) consolidate what we've got. REVIEW === I'll start by reviewing the situation regarding default arguments. There are two basic patterns for default arguments. The first is --- def fn(a=EXP): # body of function --- The second is ---

Re: [Python-ideas] Idea: Deferred Default Arguments?

2018-07-20 Thread Grégory Lielens
A crazy idea, didn't think much about it yet: def subfunc(c,a=0,b=1): Blabla def function(c,d=3, from args(subfunc) import a,b): Blabla return anotherfunc(a+b+c+d,subfunc(c,a,b)) ___ Python-ideas mailing list Python-ideas@python.org

Re: [Python-ideas] Multi-core reference count garbage collection

2018-07-20 Thread Jonathan Fine
In an earlier post, I defined BUFFERED multi-core reference count garbage collection. The basic idea is that each worker process store in buffers the need to do an INCR and DECR on the reference count for a piece of memory with a given ID. There is then a garbage collection process that does the

Re: [Python-ideas] Performance improvements via static typing

2018-07-20 Thread Stefan Behnel
Michael Hall schrieb am 19.07.2018 um 15:51: > While I am aware of projects like Cython and mypy, it seems to make sense > for CPython to allow optional enforcement of type hints, with compiler > optimizations related to it to be used. While this would not receive the > same level of performance

Re: [Python-ideas] Idea: Deferred Default Arguments?

2018-07-20 Thread Grégory Lielens
Excellent point! I am not fully convinced by the syntax yet, but having proposed something is already very valuable and I do not have a better proposal. As I had to defer defaults countless times, and each times grumped about it, I hope something will come out of this...

Re: [Python-ideas] Idea: Deferred Default Arguments?

2018-07-20 Thread Chris Angelico
On Fri, Jul 20, 2018 at 7:03 PM, Peter O'Connor wrote: > Often when programming I run into a situation where it would be nice to have > "deferred defaults". Here is an example of what I mean: > > def subfunction_1(a=2, b=3, c=4): > return a+b*c > > def subfunction_2(d=5, e=6,

Re: [Python-ideas] PEP 505: None-aware operators

2018-07-20 Thread Grégory Lielens
On Friday, July 20, 2018 at 11:12:26 AM UTC+2, Chris Angelico wrote: > > On Fri, Jul 20, 2018 at 5:10 PM, Grégory Lielens > > wrote: > > Excuse me, that wasn't my words you quoted. Watch your citations please :) > > ChrisA > Yes, sorry, it was Brendan Barnwell. And I can not edit my previous

Re: [Python-ideas] Idea: Deferred Default Arguments?

2018-07-20 Thread Jonathan Fine
Hi Peter Interesting problem. We can already get something like your proposed solution by using None instead of deferred. You have to start with def subfunction_1(a=None, b=None, c=None): if a is None: a = 2 # similarly for b and c. return a+b*c You will loose the

Re: [Python-ideas] PEP 505: None-aware operators

2018-07-20 Thread Chris Angelico
On Fri, Jul 20, 2018 at 5:10 PM, Grégory Lielens wrote: > > >> On 2018-07-19 02:11, Chris Angelico wrote: > > >> >> -snip- >> As far as I can see, these null-coalescing operators would break >> that >> model. The PEP doesn't seem to provide for a "real" magic method >> allowing users to

Re: [Python-ideas] Adding Python interpreter info to "pip install"

2018-07-20 Thread Chris Angelico
On Fri, Jul 20, 2018 at 5:56 PM, Thomas Jollans wrote: > On 20/07/18 05:10, Al Sweigart wrote: >> Sorry, I meant "pip list", rather than "pip info". >> >> I thought about the fact that "pip --version" provides this info, but 1) >> it provides the location of pip, not the python interpreter it

[Python-ideas] Idea: Deferred Default Arguments?

2018-07-20 Thread Peter O'Connor
Often when programming I run into a situation where it would be nice to have "deferred defaults". Here is an example of what I mean: def subfunction_1(a=2, b=3, c=4): return a+b*c def subfunction_2(d=5, e=6, f=7): return d*e+f def main_function(a=2, b=3, c=4, d=5,

Re: [Python-ideas] Adding Python interpreter info to "pip install"

2018-07-20 Thread Thomas Jollans
On 20/07/18 05:10, Al Sweigart wrote: > Sorry, I meant "pip list", rather than "pip info". > > I thought about the fact that "pip --version" provides this info, but 1) > it provides the location of pip, not the python interpreter it installs > packages for and 2) it would be an additional step

Re: [Python-ideas] Adding Python interpreter info to "pip install"

2018-07-20 Thread Terry Reedy
On 7/20/2018 12:21 AM, Nathaniel Smith wrote: On Thu, Jul 19, 2018 at 5:45 PM, Al Sweigart wrote: The goal of this idea is to make it easier to find out when someone has installed packages for the wrong python installation. I'm coming across quite a few StackOverflow posts and emails where

Re: [Python-ideas] PEP 505: None-aware operators

2018-07-20 Thread Grégory Lielens
On 2018-07-19 02:11, Chris Angelico wrote: > -snip- > As far as I can see, these null-coalescing operators would break > that > model. The PEP doesn't seem to provide for a "real" magic method > allowing users to override the actual behavior of the method. (You can > only

Re: [Python-ideas] PEP 505: None-aware operators: operators ?= and ?? and OR

2018-07-20 Thread Aliaksei Yaletski
Hi I'm new to that list, hope, that my idea will be helpfull. hi = hi OR len(a) > --- > where (A OR B) returns A if A is not None, otherwise it returns B. > Probably, we can introduce syntactic sugar for "if not equal" comparison? Not only for "is not None" check, but for comparison with any