On Fri, Oct 29, 2021 at 10:22:43PM +1100, Chris Angelico wrote:
> > Pardon me if this has already been discussed, but wouldn't it be better
> > to leave defaults and kwdefaults alone, and add a new pair of attributes
> > for late bound defaults? `__late_defaults__` and `__late_kwdefaults__`.
>
> The trouble with that is that positional arguments could have any
> combination of early and late defaults.
That's not a problem.
> If the late ones are in a
> separate attribute, there'd need to be some sort of synchronization
> between them.
It's not like they are mutable attributes that are constantly changing
after the function is defined. (The values *inside* __defaults__ may or
may not be mutable, but that's neither here nor there.)
> (It would work for kwdefaults, but they only apply to
> kwonly args - positional-or-keyword args go into __defaults__.)
>
> > Otherwise its a backwards-incompatable change to the internals of the
> > function object, and one which is not (so far as I can tell) necessary.
> >
> > Obviously you need a way to indicate that a value in __defaults__ should
> > be skipped. Here's just a sketch. Given:
> >
> > def func(a='alpha', b='beta', @c=expression, d=None)
> >
> > where only c is late bound, you could have:
> >
> > __defaults__ = ('alpha', 'beta', None, None)
> > __late_defaults__ = (None, None, <code for expression>, None)
> >
> > The None values in __defaults__ mean to look in the __late_defaults__
> > tuple. If the appropriate value there is also None, return it, otherwise
> > the parameter is late-bound. Evaluate it and return the result.
>
> Except that that's still backward-incompatible, since None is a very
> common value.
How is it backwards incompatible? Any tool that looks at __defaults__
finds *exactly* what was there before: a tuple of default values, not a
tuple of tuples (desc, value) or (value,) as in your implementation.
For functions that don't have any late-bound defaults, just set the
`__late_defaults__` attribute to None and nothing changes. If the
function defaults would be `(None, 1, 2, 3, 4)` today, they will remain
`(None, 1, 2, 3, 4)` tomorrow, and the interpretation will be exactly
the same.
Only in functions that actually use late defaults, and set the
`__late_defaults__` attribute to a non-None value, will see any
difference. And even then, the difference only applies to early-bound
defaults that match the sentinel.
Suppose some introspection tool that knows nothing of late-defaults
inspects the function. Here's the function again:
def func(a='alpha', b='beta', @c=expression, d=None)
For parameters a and b, nothing has changed as far as the tool is
concerned. It will look in __defaults__ and see the strings 'alpha' and
'beta'. For parameter d, it will look at the value in `__defaults__[3]`,
and see None, and *correctly* report that the default was None, so
again, nothing has changed.
It is only for parameter c that the tool will get it wrong. But then,
what else could it do? It knows nothing about late-bound defaults. It's
either going to fail, or lie. There is no other option.
With your implementation, it will always lie. Always. Every single time,
no exceptions: it will report that the default value is a tuple
(desc, value), which is wrong.
With mine, it will be correct nearly always, especially if we use
NotImplemented as the sentinel instead of None. And the cases that it
gets wrong will only be the ones that use late-binding. It will never
get an early-bound default wrong.
There is nothing better that we can do with an introspection tool that
doesn't know about late defaults, except break it by removing
`__defaults__` altogether.
> So this form of synchronization wouldn't work; in fact,
> *by definition*, any object can be in __defaults__, so there's no
> possible sentinel that can indicate that late defaults should be
> checked.
I just gave you two.
> The only way would be to first look in late defaults, and
> only then look in defaults...
Other way around. I expect that early bound defaults will continue to be
the most common, by far, so we prefer to look there first.
1. Look in the early defaults. If the value found is not the sentinel,
use it as the default. This part is effectively that same as the status
quo.
2. If it is the sentinel, look in the late defaults.
3. If the value you find in the late defaults is the same sentinel, then
use it as the default.
4. If it is a code object (function?) then evaluate it, and use whatever
it returns as the default.
5. If it is something else, you can treat it as an error.
Similar steps for the keyword-only defaults.
> which is basically the same as I have,
> only all in a single attribute.
Right. And by combining them into a single attribute, you break
backwards compatibility. I think unnecessarily, at the cost of more
complexity.
I gave a step by step strategy for using a sentinel that I am confident
that would work. There's a little bit of cost involved, but I think
that's unavoidable, and in this case not excessive.
> > That means that param=None will be a little bit more costly to fill at
> > function call time than it is now, but not by much. And non-None
> > defaults won't have any significant extra cost (just one check to see if
> > they are None).
> >
> > And if you really want to keep arg=None as fast as possible, we could
> > use some other sentinel like NotImplemented that is much less common.
>
> Having "a bit less" backward incompatibility isn't really a solution;
> if we need to have any, why have all the complexity?
I don't think my suggestion is any more complex than yours. I think it
is less complex. For functions that don't use any late-bound defaults,
they will be essentially unchanged except that they will have a new pair
of attributes, `__late_defaults__` and `__late_kwdefaults__`, both of
which will be None.
Adding new dunders doesn't count as breaking backwards compatibility.
They are reserved for the interpreter's use.
In your case the interpreter has to check the length of each tuple in
the defaults, and decide whether it is an early or late bound default
according to the length. (I forget whether the one-item tuple is the
early or late bound version.)
Remember that __defaults__ is writable. What happens if somebody sticks
a non-tuple into the __defaults__? Or a tuple with more than two items?
func.__defaults__ = ((desc, value), (descr, value), 999, (1, 2, 3))
So under your scheme, the interpreter cannot trust that the defaults are
tuples that can be interpreted as (desc, value).
> > > So far unimplemented is the description of the argument default. My
> > > plan is for early-bound defaults to have None there (as they currently
> > > do), but late-bound ones get the source code.
> >
> > That's not what I see currently in 3.10:
> >
> > >>> def func(a=1, b=2, c="hello"):
> > ... pass
> > ...
> > >>> func.__defaults__
> > (1, 2, 'hello')
> >
> >
> > What am I missing?
>
> Currently, you don't get a description, you get a value.
Right, but you said that the early bound defaults **currently** have a
None there. They don't. The current status quo of early bound defaults
is that they are set to the actual default value, not a tuple with None
in it. Obviously you know that. So that's why I'm asking, what have
I misunderstood?
> Anyway, if someone wants to make changes to the implementation, or
> even do up their own from scratch, I would welcome it. It's much
> easier to poke holes in an implementation than to actually write one
> that is perfect.
I've suggested an implementation that, I think, will be less complex and
backwards compatible. I don't know if it will be faster. I expect in the
common case of early binding, it will be, but what do I know about C?
I am confident that for the common case of functions that only use early
binding, the runtime cost might be as little as one check per function
call.
if func.__late_defaults__ is None:
# legacy behaviour with no extra runtime cost
In the worst case that we test every default value, for the common case
of early-bound defaults, it's just a fast identity comparison against
NotImplemented.
> And fundamentally, there WILL be behavioural changes
> here, so I'm not hugely bothered by the fact that the inspect module
> needs to change for this.
It's not just the inspect module. __defaults__ is public[1]. Anyone and
everyone can read it and write it. Your implementation is breaking
backwards compatibility, and I believe you don't need to.
[1] Whether it is *officially* public or not, it is de facto public.
--
Steve
_______________________________________________
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/LFLXZRGW3G35WIDVKUCFOXUH2IB4BVEC/
Code of Conduct: http://python.org/psf/codeofconduct/
