date:20200826

>
> ONE way might be to have the new method call the existing dunders. But
> that's not my preference-- I would like to find another way. I do have
> another idea, and I presented it in a previous response. Here it is again:
>
> def __key_or_index_translator__(self, pos1, pos2, *, x) ->
> Tuple[Tuple[Any], Dict[str, Any]]:
> """I only allow subscripting like this:
>
> >>> obj[1, x=2]
> """
> return (pos1, pos2), dict(x=x)
>
> The returned two-tuple, `t`, in turn gets unpacked like for each of the
> existing item dunders:
>
> obj.__getitem__(*t[0], **t[1])
> obj.__setitem__(value, *t[0], **t[1])
> obj.__delitem__(*t[0], **t[1])
>

 Cripes, I screwed up the __key_or_index_translator__ docstring. See
correction below, sorry.

---

def __key_or_index_translator__(self, pos1, pos2, *, x) ->
Tuple[Tuple[Any], Dict[str, Any]]:
"""I only allow subscripting like this:

>>> obj[1, 2, x=2]
"""
return (pos1, pos2), dict(x=x)

---
Ricky.

"I've never met a Kentucky man who wasn't either thinking about going home
or actually going home." - Happy Chandler
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NFRYYCDWALWZE5OP3EGAIRG2ECYDI65W/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

On Wed, Aug 26, 2020 at 10:34 PM Steven D'Aprano 
wrote:

> On Wed, Aug 26, 2020 at 12:32:56PM -0400, Ricky Teachey wrote:
> > It creates a language supported way for the creator of the class to
> decide
> > how to interpret the contents inside a subscript operation.
>
> We already have that: `__getitem__`.
>

Actually no, we have *THREE* dunders: get, set, and del -item. And
they accept a single argument containing a single object representing a
single key or index. And if I had to make a prediction, then probably
pretty soon they'll accept kwargs, too.

And these are great as-is when the way you want to use them is in total
agreement with the way they were envisioned to be used: for single argument
keys/indexes.

But any use case that deviates from that vision requires effort spread
across three dunder methods to set it up right. And it requires writing
code that breaks up the positional arguments in just the right way; code
that you pretty much never have to write when using the function paradigm
(rather than the item dunder paradigm), because the language does it all
for us:

def f(pos1, pos2, *args, kwd1, kwd2=None, **kwargs): ...

I don't have to write code in the function body of `f` breaking the
arguments up the way I intend to use them, and errors are raised when
required arguments are missing, etc etc. This is a set of fantastic
language features.

But the positional part of it is missing from subscripting, from
__getitem__, __setitem__, and __delitem__ and you have to break up your
positionals in three different dunder methods.

If you are a person who wants to use the subscripting operator from a more
function-like paradigm, this is kind of a lot of work and extra code to
maintain-- code that simply would not be needed at all if there was a
single dunder responsible for accepting the key/index "stuff" and then
sending it off-- somehow-- into the item dunders to be used.

How is it sent? I don't know yet. Thinking about it.

> > It greatly
> > alleviates (though not perfectly-- see the end of this message) the
> > incongruity between how the indexing operator behaves and function calls
> > behave. As explained by Jonathan Fine, it adds flexibility
>
> What flexibility is added that `__getitem__` doesn't already provide?
>

I explained above: a single point of subscript argument acceptance that
provides all the language features of function signatures.

> and smoothes out
> > the differences the different paradigms of subscript operations:
> sequences,
> > and mappings.
>
> Won't sequences and mappings still call the same dunder method?
>

Yes, but again, all the existing power the language has for calling
functions and passing them to the function signature can be brought to the
table for all three subscript operators using a single new dunder. This
smoothes things out, because if you don't want to use the default python
treatment of positional subscript arguments as a single key or index
object, it makes it much easier to do things another way.

> And it opens up opportunities for other paradigms to be
> > created, the most prominent example of which is type-hint creation
> > shortcuts, like:
> >
> > Vector = Dict[i=float, j=float]
>
> That's not a paradigm. That's just an application of the feature.
> *Type-hints* was a major change to the Python's language paradigm. This
> is just a (potential) nice syntax that allows a concise but readable
> type-hint.
>
> https://en.wikipedia.org/wiki/Paradigm

I'm not going to respond to this other than to say that I think I was clear
and providing a link to a dictionary entry for "paradigm" sort of feels
like picking on my choice of words rather than actually responding to me
with candor. If so, that doesn't seem like a very kind conversation choice.
If I am correct, I'd like to ask that in the future, please choose to be
more kind when you respond to me? Thank you.

On the other hand if I am wrong and it wasn't clear what I am saying, or if
you were just being humorous, I very much apologize for being
oversensitive. :)

You're suggesting that if subscript keywords were permitted, rather than
> having the keywords passed directly to `__getitem__` where they are
> wanted, the interpreter should pass them to another method and then that
> method could pass them to `__getitem__`.
>

The only thing Jonathan and I are proposing at this time is a __dunder__
that gets passed the contents of the subscript operator and does the
handling of the contents they get passed along:

1.  New __dunder__ that gets the contents
2. ???
3. Item dunders called the way the user wants them called.

The details of 2-- how that function "passes on" that information to the
item dunders-- I am currently unsure about.

ONE way might be to have the new method call the existing dunders. But
that's not my preference-- I would like to find another way. I do have
another idea, and I presented it in a previous response. Here it is again:

def __key_or_index_translator__(self, pos1, pos2,

[Python-ideas] Re: Changing item dunder method signatures to utilize positional arguments (open thread)

On Thu, Aug 27, 2020 at 03:28:07AM +1200, Greg Ewing wrote:
> On 27/08/20 12:53 am, Steven D'Aprano wrote:
> 
> >Presumably the below method is provided by `object`. If not, what
> >provides it?
> >
> >> def __getindex__(self, *args, **kwds):
> >> if kwds:
> >> raise TypeError("Object does not support keyword indexes")
> >> if not args:
> >> raise TypeError("Object does not accept empty indexes")
> 
> It's not literally a method, I just wrote it like that to
> illustrate the semantics. It would be done by the interpreter
> as part of the process of translating indexing operations into
> dunder calls.

Okay, so similar to my suggestion that this would be better implemented 
in the byte-code rather than as a method of object.

> >What is your reasoning behind prohibiting keywords, when we're right in
> >the middle of a discussion over PEP 474 which aims to allow keywords?
> 
> We're falling back to __getitem__ here, which doesn't currently allow
> keywords, and would stay that way. The point of this proposal is to
> not change __getitem__. If you want to get keywords, you provide
> __getindex__.

Point of order: **the getitem dunder** already allows keywords, and 
always has, and always will. It's just a method.

It's the **subscript (pseudo-)operator** which doesn't support keywords. 
This is a syntax limitation, not a limitation of the dunder method. If 
the interpreter supports the syntax, it's neither here nor there to the 
interpreter whether it calls `__getitem__` or `__getindex__` or 
`__my_hovercraft_is_full_of_eels__` for that matter.

So if you want to accept keywords, you just add keywords to your 
existing dunder method. If you don't want them, don't add them. We don't 
need a new dunder just for the sake of keywords.

> >This is going to slow down the most common cases of subscripting: the
> >interpreter has to follow the entire MRO to find `__getindex__` in
> >object, which then dispatches to the `__getitem__` method.
> 
> No, it would be done by checking type slots, no MRO search involved.

Okay, I didn't think of type slots.

But type slots are expensive in other ways. Every new type slot 
increases the size of objects, and I've seen proposals for new dunders 
knocked back for that reason, so presumably the people care about the 
C-level care about the increase in memory and complexity from adding new 
type slots.

Looking here:

https://docs.python.org/3/c-api/typeobj.html

I see that `__setitem__` and `__delitem__` are handled by the same type 
slot, so presumably the triplet get-, set- and del- index type slots 
would be the same. Still, that means adding two new type slots to 
both sequence and mapping objects.

(I assume that it's not *all* objects that carry these slots. If it is 
all objects, that makes the cost of this proposal correspondingly 
higher.)

So if I understand it correctly, we have some choices when it comes to 
sequence/mapping types:

1. the existing `__*item__` methods keep their slots, and new `__*index__` 
   slots are created, which makes both the item and index dunders fast 
   but increases the size of every object which uses any of those 
   methods;

2. the existing item slots stay as they are, there are no new index 
   slots, which keeps objects the same size but the new index protocol 
   will be slow;

3. the existing item slots are repurposed for index, which keeps objects 
   the same size, and the new protocol fast, but makes calling item 
   dunders slow;

4. and just for completion because of course this is not going to 
   happen, we could remove the existing item slots and not add index 
   slots so that both protocols are equally slow;

5. alternatively, we could leave the existing C-level sequence and 
   mapping objects alone, and create *four* brand new C-level objects:

   - a sequence object that supports only the new index protocol;
   - a sequence object that supports both index and item protocols;
   - and likewise two new mapping objects.

Do I understand this correctly? Have I missed any options?

Assuming I do, 4 is never going to happen and each of the others have 
some fairly large disadvantages and costs in speed, memory, and 
complexity. Without a correspondingly large advantage to this new 
`__*index__` protocol, I don't see this going anywhere.

> >In your earlier statement, you said that it would be possible for
> >subscripting to mean something different depending on whether the
> >comma-separated subscripts had parentheses around them or not:
> >
> > obj[(2, 3)]
> > obj[2, 3]
> >
> >How does that happen?
> 
> If the object has a __getindex__ method, it gets whatever is between
> the [] the same way as a normal function call, so comma-separated
> expressions become separate positional arguments.

The compiler doesn't know whether the object has the `__getindex__` 
method at compile time, so any process that relies on that knowledge 
isn't going to work. There can only be one

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

2020-08-26 Thread David Mertz

On Wed, Aug 26, 2020 at 4:01 PM Christopher Barker 
wrote:

> Yes, it's slightly funny that square brackets convert to `index` rather
>> than `*index`, but that ship sailed very long ago, and it's no big deal.
>> There's no problem that needs solving and no need for code churn.
>>
> TL;DR -- let's watch the strong language here -- these proposals do
> attempt to address a real problem -- probably not worth the downsides, but
> this "doesn't solve any problem", "no one asked for it", etc, is really
> dismissive.
>

I have read hundreds of comments in this thread.  As I stated, I have yet
to see *anyone* identify a "problem that needs solving" with all the
convoluted new objects and weird support methods.  There is absolutely
nothing whatsoever that cannot be easily done with the obvious and
straightforward signature:

.__getitem__(self, index, **kws)

Not one person has even vaguely suggested a single use case that cannot be
addressed that way, nor even a single case where a different approach would
even be slightly easier to work with.

... there was an odd response that a custom class might want to specify its
special keywords.  Which of course they can1  I was just giving the
shorthand version.  Some particular class can allow and define whichever
keywords it wants, just like with every other function or method.

---

Yes, the parser will have to do something special with the stuff in square
brackets.  As it has always done something special for stuff in square
brackets.  Obviously, if we call:

mything[1, 2:3, four=4, five=5]

That needs to get translated, at the bytecode level, into the equivalent of:

mything.__getitem__((1, slice(2,3)), four=4, five=5)

But the class MyThing is free to handle its keyword arguments `four` and
`five` how it likes.  They might have defaults or not (i.e. **kws).  I
think dict and list should, for now, raise an exception if they see any
**kws (but ignoring them is not completely absurd either).

The only question that seems slightly reasonable to see as open is "What is
`index` if only keywords are subscripted?"  Some sort of sentinel is
needed, but conceivably None or an empty tuple wouldn't be backward
compatible since those *can* be subscripts legally now.

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/K4BRZIV6KIQIHLMU2SV765XGG73OGA3H/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

On Wed, Aug 26, 2020 at 12:32:56PM -0400, Ricky Teachey wrote:
> On Wed, Aug 26, 2020 at 11:19 AM Steven D'Aprano 
> wrote:
> 
> > On Wed, Aug 26, 2020 at 10:10:34AM -0400, Ricky Teachey wrote:
> >
> > > But I have to say that I think this latest is a fantastic idea, and when
> > > Jonathan presented it to me it was very surprising that I had not seen it
> > > presented by anyone else yet. I think it solves a ton of problems,
> >
> > Such as?
> >
> 
> It creates a language supported way for the creator of the class to decide
> how to interpret the contents inside a subscript operation.

We already have that: `__getitem__`.

> This is a
> problem because disagreement over this matter is a large part of the reason
> PEP 472-- the spirit of which I support-- has been held up.

As I see it, the disagreement comes about from two main areas:

- People who want to bake into the language semantics which solve a tiny 
niche problem ("what if I want to treat keyword:argument pairs as 
keys?"), making the common use-cases much harder to solve.

- And people trying to over-engineer some complicated solution to 
something that isn't actually a problem: the supposed inconsistency 
between subscripts and function calls.

The disagreement isn't from people arguing about how your class should 
interpret the keywords. That isn't holding up the PEP.

Interpreting the keywords is easy: interpret them however you want. 
Python doesn't tell you what the keywords mean, it just hands them to 
your method.

> It greatly
> alleviates (though not perfectly-- see the end of this message) the
> incongruity between how the indexing operator behaves and function calls
> behave. As explained by Jonathan Fine, it adds flexibility 

What flexibility is added that `__getitem__` doesn't already provide?

> and smoothes out
> the differences the different paradigms of subscript operations: sequences,
> and mappings.

Won't sequences and mappings still call the same dunder method?

> And it opens up opportunities for other paradigms to be
> created, the most prominent example of which is type-hint creation
> shortcuts, like:
> 
> Vector = Dict[i=float, j=float]

That's not a paradigm. That's just an application of the feature. 
*Type-hints* was a major change to the Python's language paradigm. This 
is just a (potential) nice syntax that allows a concise but readable 
type-hint.

https://en.wikipedia.org/wiki/Paradigm

[...]
> > > For example: if this proposal were already in place and PEP 472 were to
> > > continue to be held up because of terrorists like me ;) *, one could have
> > > written this translation function and PEP-472-ify their classes already:
> > >
> > > def yay_kwargs(self, *args, **kwargs):
> > > return self.__getitem__(args, **kwargs)
> >
> > You're calling the `__getitem__` dunder with arbitrary keyword
> > arguments. Are you the same Ricky Teachey who just suggested that we
> > should be free to break code that uses `__getitem__` methods that don't
> > obey the intent that they have only a single parameter and no keywords?
> >
> 
> I am talking hypothetically-- if this proposal were already in place (which
> *includes* passing kwargs to the *new dunder* rather than passing them to
> the existing item dunders by default), you could write code like yay_kwargs
> today, even if the default way the language behaves did not change.

You're suggesting that if subscript keywords were permitted, rather than 
having the keywords passed directly to `__getitem__` where they are 
wanted, the interpreter should pass them to another method and then that 
method could pass them to `__getitem__`.

Or we could just pass them directly into `__getitem__`, as requested.

How does the interpreter know to call yay_kwargs rather than some other 
method?

> In that universe, if I wrote this:
> 
> class C:
> def __getitem__(self, key):
> print(key)
> 
> ...and tried to do this:
> 
> >>> C()[a=1]
> SomeError
>
> ...*in that universe*, without PEP 472, the language will STILL, by
> default, give an error as it does today (though it probably would no longer
> be a SyntaxError).

That is a remarkably unenlightening example.

There's no sign of what your new method is or what it does. There's an 
error, but you don't know what it is except it "probably" won't be a 
syntax error, so you don't even know if this error is detected at 
compile time or runtime.

You say PEP 472 is not accepted, but maybe it is accepted since keyword 
subscripts are possibly not a syntax error, or maybe they are still a 
syntax error. Who knows? Not me, that's for sure.

I feel like I'm trying to nail jelly to a wall, trying to understand 
your proposal. Every time I ask a question, it seems to twist and mutate 
and become something else.

And how does this relate to Jonathan's idea of "signature dependent 
semantics" which you were singing the praises of, but seems to have 
nothing to do with what you are describing now.

[...]
> We could make the

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

On Wed, Aug 26, 2020 at 11:03:20PM +0100, Stefano Borini wrote:
> On Wed, 26 Aug 2020 at 17:50, David Mertz  wrote:
> >
> > In my mind, *anything* other than the straightforward and obvious 
> > signature `__getitem__(self, index, **kws)` is a pointless 
> > distraction.
> 
> It isn't.
> and the reason why it isn't is that it makes it much harder for the
> implementing code to decide how to proceed, for the following reasons:
> 1. you will have to check if the keywords you receive are actually in
> the acceptable set

I assumed -- possibly this was wishful thinking on my part :-) -- that 
David didn't mean *literally* only as a collection of arbitrary keywords 
`**kws` but was using that as an abbreviation for "whatever named 
keyword parameters are written in the method signature".

E.g. I can write:

def __getitem__(self, index, *, spam, eggs=None):

to have one mandatory keyword "spam" and one optional keyword "eggs" and 
it will all Just Work. I presume that David would be good with that too, 
but if he's not, he really needs to explain why not.

Before someone else points this out, I'll mention that *technically* 
someone could use index as a keyword argument too, but so what if they 
do? However if you really want to prevent it, you can make it 
positional-only:

def __getitem__(self, index, /, *, spam, eggs=None):

but honestly that's only necessary if you also collect arbitrary 
keywords in a `**kwargs` and want to allow "index" as one of them. 
Otherwise it's overkill.

[...]
> To me, most of the discussion is being derailed in deciding how this
> should look like on the __getitem__ end and how to implement it, but
> we haven't even decided what it should look like from the outside

Is that even a question?

obj[index, keyword=value]

where index is any comma-separated list of expressions, including 
slices, keyword is an identifier, and value is any expression including 
slices. Are there even any other options being considered?

A few points:

- The index is optional, but it will be a runtime TypeError if the 
method isn't defined with the corresponding parameter taking a default 
value.

- You can have multiple keyword=value pairs, separated by commas. They 
must all follow the index part. Duplicate keywords is a runtime error, 
just as they are are for function calls.

- An empty subscript remains a syntax error, even if the method 
signature would allow it.

> (although it's probably in the massive number of emails I am trying to
> aggregate in the new PEP).

As PEP author, you don't have to include every little tiny detail of 
every rejected idea. It should be a summary, and it is okay to skip over 
brief ideas that went nowhere and just point back to the thread.

One of the weaknesses, in my opinion, of the existing PEP is that it 
tries to exhaustively cover every possible permutation of options (and 
in doing so, managed to skip the most useful and practical option!).

A PEP can be, and most of the time should be, an opinionated persuasive 
essay which aims to persuade the readers that your solution is the right 
solution. The PEP has to be fair to objections, but it doesn't have to 
be neutral on the issue. It shouldn't be neutral unless it is being 
written merely to document why the issue is being rejected.

So my advise is: choose the model you want, and write the PEP to 
persuade that is the best model, why the others fail to meet the 
requirements. I really hope the model you choose is the one I describe 
above, in which case I will support it, but if you hate that model and 
want something else that's your right as the PEP author.

(Although the earlier co-authors may no longer wish to be associated 
with the PEP if you change it radically.)

-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UGPQWV5C6QCRUYPLAL453FOS2NWT5YFG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

On Wed, Aug 26, 2020 at 9:39 PM Greg Ewing 
wrote:

> If we use a new set of dunders, the signature of the one for
> setting could be
>
>  def __setindex__(self, value, *args, **kwds)
>
> which is not completely unsurprising, but plays better with
> an arbitrary number of positional indexes.
>
> --
> Greg
>

On Wed, Aug 26, 2020 at 9:34 PM Greg Ewing 
wrote:

> On 27/08/20 4:32 am, Ricky Teachey wrote:
>
> > class Q:
> >  def __subscript__(self, method_name, *args, **kwargs):
> >   return getattr(self, method_name)(*args, **kwargs)
> >  def __getitem__(self, *args, **kwargs): ...
> >  # Note that I have made the RHS value the first argument in
> __setitem__
> >  def __setitem__(self, value, *args, **kwargs): ...
> >  def __delitem__(self, *args, **kwargs): ...
>
> This seems like a very convoluted way of going about it as
> opposed to just adding a new full set of dunders. And I don't
> see how it actually provides any more flexibility.
>
> --
> Greg

That version of the idea I gave isn't great, I don't disagree. It wasn't
really intended to be the proposed dunder function just the quickest
way I could think of to write it up to attempt to illustrate to Steve in
the bit below:

> With the proposal, the language would support any function desired to turn
> > the "stuff" inside a subscripting operation into the item dunder calls.
>
> I'm sorry, I don't understand that sentence.
>

I think the argument for providing a SINGLE new dunder, rather than three,
could be that you don't have two sets of dunders that do the same thing,
just with different signatures. The purpose of a new single dunder should
simply be to translate the contents of the subscript operator to the
existing dunders.

The thing I am struggling with is how would it RETURN that translation...?
In my slapdash implementation I just CALLED them, but that seems less than
ideal.

I supposed it could do it by returning a tuple and a dict, like this:

def __key_or_index_translator__(self, pos1, *, x) -> Tuple[Tuple[Any],
Dict[str, Any]]:
"""I only allow subscripting like this:

>>> obj[1, x=2]
"""
return (pos1,), dict(x=x)

The return value then in turn gets unpacked for each of the existing item
dunders:

obj.__getitem__(*(pos1,), **dict(x=x))
obj.__setitem__(value, *(pos1,), **dict(x=x))
obj.__delitem__(*(pos1,), **dict(x=x))

Now:

class C:
__key_or_index_translator__ = __key_or_index_translator__
def __getitem__(self, *args, **kwargs): ...
def __setitem__(self, value, *args, **kwargs): ...
def __delitem__(self, *args, **kwargs): ...

>>> C()[1, 2, x=1, y=2]
TypeError

---
Ricky.

"I've never met a Kentucky man who wasn't either thinking about going home
or actually going home." - Happy Chandler
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UQOOOJRHE4LEMIZF7UQNTEVZO5H45MSD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access


On 27/08/20 10:53 am, Todd wrote:
On Wed, Aug 26, 2020, 18:04 Stefano Borini > wrote:


On Wed, 26 Aug 2020 at 17:50, David Mertz mailto:me...@gnosis.cx>> wrote:
you want this

acquired_data[time, detector]

to be allowed to be written as

acquired_data[time=time, detector=detector]

Again, implicit on your argument here is the assumption that all keyword 
indices necessarily map into positional indices.


No, David is saying that it should be *possible* (and not too difficult)
to make that happen. Positional-only arguments can be used by classes
that want to keep positional and keyword args completely separate.

--
Greg
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/RLUZM4XDYLTPUK3A7V25EPDPYKUIRYXA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access


On 27/08/20 8:00 am, Christopher Barker wrote:

Translating to:

thing.__setitem__(self, (ind1, ind2), value, kwd1=v1, kw2=v2)

which is pretty darn weird


If we use a new set of dunders, the signature of the one for
setting could be

def __setindex__(self, value, *args, **kwds)

which is not completely unsurprising, but plays better with
an arbitrary number of positional indexes.

--
Greg
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/JPLQHKYUCCYYJAXQ3PEUU3KBUWSL243R/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access


On 27/08/20 4:32 am, Ricky Teachey wrote:


class Q:
     def __subscript__(self, method_name, *args, **kwargs):
          return getattr(self, method_name)(*args, **kwargs)
     def __getitem__(self, *args, **kwargs): ...
     # Note that I have made the RHS value the first argument in __setitem__
     def __setitem__(self, value, *args, **kwargs): ...
     def __delitem__(self, *args, **kwargs): ...


This seems like a very convoluted way of going about it as
opposed to just adding a new full set of dunders. And I don't
see how it actually provides any more flexibility.

--
Greg
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZWLNKSZ7IU5TGIPZY7CDZBPVQXYPCGT7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

2020-08-26 Thread Todd

On Wed, Aug 26, 2020 at 7:11 PM Stefano Borini 
wrote:

> On Wed, 26 Aug 2020 at 23:56, Todd  wrote:
> > Again, implicit on your argument here is the assumption that all keyword
> indices necessarily map into positional indices.  This may be the case with
> the use-case you had in mind.  But for other use-cases brought up so far
> that assumption is false.  Your approach would make those use cases
> extremely difficult if not impossible.
>
> Please remind me of one. I'm literally swamped.

xarray, which is the primary python package for numpy arrays with labelled
dimensions.  It supports adding and indexing by additional dimensions that
don't correspond directly to the dimensions of the underlying numpy array,
and those have no position to match up to.  They are called "non-dimension
coordinates".

Other people have wanted to allow parameters to be added when indexing,
arguments in the index that change how the indexing behaves.  These don't
correspond to any dimension, either.

In any case, this leads to a different question: should we deprecate
> anonymous axes, and if not, what is the intrinsic meaning and
> difference between anonymous axes and named axes?

Anonymous axes are axes that someone hasn't spent the time adding names
to.  Although they are unsafe, there is an absolutely immense amount of
code built around them.  And it takes a lot of additional work for simple
cases.  So I think deprecated them at this point would be completely
unworkable.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NJM5HO5ROVPO3ES73PV633UNUCJD22ZI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Changing item dunder method signatures to utilize positional arguments (open thread)

On Wed, Aug 26, 2020 at 9:00 PM Greg Ewing 
wrote:

> On 27/08/20 3:51 am, Ricky Teachey wrote:
> > On Wed, Aug 26, 2020 at 11:30 AM Greg Ewing  > > wrote:
> >
> > No, it would be done by checking type slots, no MRO search involved.
> >
> > Can you elaborate on this for my understanding?
>
> For frequently-used special methods such as __getitem__, the type
> object contains direct pointers to C functions (so-called "type slots").
> ...
> The overhead for this would be just one extra C pointer check, which
> you would be hard-pressed to measure.
>

Wow that's great news.


> > the index operator recognizes this as a tuple:
> >
> > 1,
> >
> > But function calls do not.
>
> We could probably cope with that by generating different bytecode
> when there is a single argument with a trailing comma, so that a
> runtime decision can be made as to whether to tupleify it.
>
> However, I'm not sure whether it's necessary to go that far. The
> important thing isn't to make the indexing syntax exactly match
> function call syntax, it's to pass multiple indexes as positional
> arguments to __getindex__. So I'd be fine with having to write
> a[(1,)] to get a one-element tuple in both the old and new cases.
>
> It might actually be better that way, because having trailing
> commas mean different things depending on the type of object
> being indexed could be quite confusing.
>
> --
> Greg
>

I didn't know that was possible. It's far beyond my knowledge.

The idea of a new dunder is starting to sound like a much more realistic
possibility than I had hoped.

---
Ricky.

"I've never met a Kentucky man who wasn't either thinking about going home
or actually going home." - Happy Chandler
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XNQENI43SIACJNRY522ILERSNSNITF2J/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

On Wed, Aug 26, 2020 at 11:06:26AM -0400, Ricky Teachey wrote:

> Well I did give a couple examples: language supported opt-in for
> pep-472-ification, and pep-472 opt-out. But Ok, more are needed.

That's one example, not a couple: the ability to choose whether to 
opt-in or opt-out of keyword subscripts.

Except you can't really. Either the interpreter allows keyword syntax in 
subscripts, or it doesn't. You can't opt-in if the interpreter doesn't 
support it, and if the interpreter does support it, you can't prevent it 
from being supported.

Of course you can choose whether to use keywords in your own classes, 
but we can already do that the same way we opt out of any other feature: 
just don't use it!

(I expect that the built-ins list, tuple, dict, str etc will all do 
exactly that.)

-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/Y3FA7XFB5GODINBAC6UVWGQKXYHO3OJ5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Changing item dunder method signatures to utilize positional arguments (open thread)

On 27/08/20 3:51 am, Ricky Teachey wrote:
On Wed, Aug 26, 2020 at 11:30 AM Greg Ewing > wrote:

No, it would be done by checking type slots, no MRO search involved.

Can you elaborate on this for my understanding?

For frequently-used special methods such as __getitem__, the type
object contains direct pointers to C functions (so-called "type slots").
These are set up when the type object is created (and updated if the
MRO is modified). For classes written in Python, they are filled with
wrappers that invoke the user's Python code.

So the implementation of the bytecodes that perform indexing would
first look for a value in the slot for __getindex__; if it finds one,
it calls it like a normal function. Otherwise it does the same as now
with the slot for __getitem__.

The overhead for this would be just one extra C pointer check, which
you would be hard-pressed to measure.

the index operator recognizes this as a tuple:

But function calls do not.

We could probably cope with that by generating different bytecode
when there is a single argument with a trailing comma, so that a
runtime decision can be made as to whether to tupleify it.

However, I'm not sure whether it's necessary to go that far. The
important thing isn't to make the indexing syntax exactly match
function call syntax, it's to pass multiple indexes as positional
arguments to __getindex__. So I'd be fine with having to write
a[(1,)] to get a one-element tuple in both the old and new cases.

It might actually be better that way, because having trailing
commas mean different things depending on the type of object
being indexed could be quite confusing.

--
Greg
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/XWE73VLLGYWYLNFMRKZXIBILBOLAI6Z3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

2020-08-26 Thread Chris Angelico

On Thu, Aug 27, 2020 at 8:20 AM Stefano Borini  wrote:
>
> Personally I think Jonathan and I (and possibly a couple others)
> should form a separate subgroup and come up with a sensible and
> logical set of options in a proto-PEP. The topic has been discussed
> and we have plenty of ideas and opinions, and if we want to achieve
> something coherent we need to take it aside and dig deeper the various
> options until the whole proposal (or set of proposals) has been
> considered fully. There's little sense in going back and forth in the
> mailing list.
>
> One thing I want to understand though, and it's clear that this is a
> potential dealbreaker: is there any chance that the steering council
> will actually accept such a feature once it's been fully fleshed out,
> considering that it's undeniable that there are use cases spanning
> through multiple fields?
> Because if not (whatever the reason might be) regardless of the
> possible options to implement it, it's clear that there's no point in
> exploring it further.
>

Another way to word that question is: Do you have a sponsor for your
PEP? Every PEP needs a core dev (or closely connected person) to
sponsor it, otherwise it won't go forward. Do you have any core devs
who are on side enough to put their name down as sponsor? If not, it's
not even going to get as far as the Steering Council.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3LQCUFFBGLA5DJGTPLTSO4UAN7SQ2HHN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

2020-08-26 Thread Stefano Borini

On Wed, 26 Aug 2020 at 23:56, Todd  wrote:
> Again, implicit on your argument here is the assumption that all keyword 
> indices necessarily map into positional indices.  This may be the case with 
> the use-case you had in mind.  But for other use-cases brought up so far that 
> assumption is false.  Your approach would make those use cases extremely 
> difficult if not impossible.

Please remind me of one. I'm literally swamped.

In any case, this leads to a different question: should we deprecate
anonymous axes, and if not, what is the intrinsic meaning and
difference between anonymous axes and named axes?



--
Kind regards,

Stefano Borini
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UK7XFPZUDJTZGW4W5FM5ERDPKCJUK4WD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - slices in keyword indices, d[x=1:3]

2020-08-26 Thread tcphone93

My suggestion to solve this would be to use a similar rule to the walrus 
operator; only allow slice literals within either `()` brackets or `[]` square 
brackets; thus `{a: b: c: d, x: y}` becomes illegal and would need to be 
`{(a:b): (c:d), x: y}`
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5YOUH4F56IKGUPRAKDO46BUVVXVOW7Z4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

2020-08-26 Thread Todd

On Wed, Aug 26, 2020, 18:04 Stefano Borini  wrote:

> On Wed, 26 Aug 2020 at 17:50, David Mertz  wrote:
> >
> > In my mind, *anything* other than the straightforward and obvious
> signature `__getitem__(self, index, **kws)` is a pointless distraction.
>
> It isn't.
> and the reason why it isn't is that it makes it much harder for the
> implementing code to decide how to proceed, for the following reasons:
> 1. you will have to check if the keywords you receive are actually in
> the acceptable set
> 2. you will have to disambiguate an argument that is supposed to be
> passed _either_ as position or with keyword. Always remember this use
> case: a matrix of acquired data, where the row is the time and the
> column is the detector.
> I've seen countless times situations where people were creating the
> matrix with the wrong orientation, putting the detector along the rows
> and the time along the column. so you want this
>
> acquired_data[time, detector]
>
> to be allowed to be written as
>
> acquired_data[time=time, detector=detector]
>
> so that it's unambiguous and you can even mess up the order, but the
> right thing will be done, without having to remember which one is
> along the row and which one is along the column.
>
> If you use the interface you propose, now the __getitem__ code has to
> resolve the ambiguity of intermediate cases, and do the appropriate
> mapping from keyword to positional. Which is annoying and you are
> basically doing manually what you would get for free in any standard
> function call.
>
> Can you do it with that implementation you propose? sure, why not,
> it's code after all. Is it easy and/or practical? not so sure. Is
> there a better way? I don't know. That's what we are here for.
>
> To me, most of the discussion is being derailed in deciding how this
> should look like on the __getitem__ end and how to implement it, but
> we haven't even decided what it should look like from the outside
> (although it's probably in the massive number of emails I am trying to
> aggregate in the new PEP).
>

Again, implicit on your argument here is the assumption that all keyword
indices necessarily map into positional indices.  This may be the case with
the use-case you had in mind.  But for other use-cases brought up so far
that assumption is false.  Your approach would make those use cases
extremely difficult if not impossible.

>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WZZRIT7XOBTIN4PHVN24MNTESX73Q4UW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

2020-08-26 Thread Stefano Borini

Personally I think Jonathan and I (and possibly a couple others)
should form a separate subgroup and come up with a sensible and
logical set of options in a proto-PEP. The topic has been discussed
and we have plenty of ideas and opinions, and if we want to achieve
something coherent we need to take it aside and dig deeper the various
options until the whole proposal (or set of proposals) has been
considered fully. There's little sense in going back and forth in the
mailing list.

One thing I want to understand though, and it's clear that this is a
potential dealbreaker: is there any chance that the steering council
will actually accept such a feature once it's been fully fleshed out,
considering that it's undeniable that there are use cases spanning
through multiple fields?
Because if not (whatever the reason might be) regardless of the
possible options to implement it, it's clear that there's no point in
exploring it further.

On Wed, 26 Aug 2020 at 12:58, Chris Angelico  wrote:
>
> On Wed, Aug 26, 2020 at 9:44 PM Jonathan Fine  wrote:
> >
> > PROPOSAL
> > I think it will help solve our problem, to give Z = type(z) a new dunder 
> > attribute that either is used as the internal_get_function, or is used 
> > inside a revised system-wide internal_get_function.
> >
> > That way, depending on the new dunder attribute on Z = type(z), sometimes
> > >>> z[1, 2, a=3, b=4]
> > >>> z.__getitem__(1, 2, a=3, b=4)
> > are equivalent. And sometimes
> > >>> z[1, 2, a=3, b=4]
> > is equivalent to
> > >>> key = K(1, 2, a=3, b=4)
> > >>> z.__getitem__(key)
> > all depending on the new dunder attribute on Z = type(z).
> >
>
> -1. We have already had way WAY too much debate about the alternative
> ways to handle kwargs in subscripts, and quite frankly, I don't think
> this is adding anything new to the discussion. Can we just stop
> bikeshedding this already and let this matter settle down? You still
> aren't showing any advantage over just allowing the current dunder to
> receive kwargs.
>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/UCYMGB4BKZK7HHUNEBA6OIMCGIZFRXMA/
> Code of Conduct: http://python.org/psf/codeofconduct/

-- 
Kind regards,

Stefano Borini
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4637KB2T45S7BTWZU4RNXXBZTWPV24ZD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

2020-08-26 Thread Stefano Borini

On Wed, 26 Aug 2020 at 17:50, David Mertz wrote:
>
> In my mind, *anything* other than the straightforward and obvious signature
> `__getitem__(self, index, **kws)` is a pointless distraction.

It isn't.
and the reason why it isn't is that it makes it much harder for the
implementing code to decide how to proceed, for the following reasons:
1. you will have to check if the keywords you receive are actually in
the acceptable set
2. you will have to disambiguate an argument that is supposed to be
passed _either_ as position or with keyword. Always remember this use
case: a matrix of acquired data, where the row is the time and the
column is the detector.
I've seen countless times situations where people were creating the
matrix with the wrong orientation, putting the detector along the rows
and the time along the column. so you want this

acquired_data[time, detector]

to be allowed to be written as

acquired_data[time=time, detector=detector]

so that it's unambiguous and you can even mess up the order, but the
right thing will be done, without having to remember which one is
along the row and which one is along the column.

If you use the interface you propose, now the __getitem__ code has to
resolve the ambiguity of intermediate cases, and do the appropriate
mapping from keyword to positional. Which is annoying and you are
basically doing manually what you would get for free in any standard
function call.

Can you do it with that implementation you propose? sure, why not,
it's code after all. Is it easy and/or practical? not so sure. Is
there a better way? I don't know. That's what we are here for.

To me, most of the discussion is being derailed in deciding how this
should look like on the __getitem__ end and how to implement it, but
we haven't even decided what it should look like from the outside
(although it's probably in the massive number of emails I am trying to
aggregate in the new PEP).

--
Kind regards,

Stefano Borini
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/QQBD4LNE46C7NIKSH5VWT7MBTJYTMHNB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: What about having a .get(index, default) method for arrays like we have for dicts?

Oh thanks, I had no idea that existed.

I wanted to experiment with Mapping and unions but it's not working. I
couldn't find any existing issues about this. Should I be opening an issue?
If not then there's no need to explain what I'm doing wrong.

```
$ cat test.py
print({} | {})
$ ./python test.py
{}
$ mypy --python-version 3.9 test.py
test.py:1: error: Unsupported left operand type for | ("Dict[,
]")
Found 1 error in 1 file (checked 1 source file)
$ mypy --version
mypy 0.782
```

On Wed, Aug 26, 2020 at 11:18 PM Guido van Rossum  wrote:

> Use typing.AbstractSet for the ABC (there was an unfortunate naming
> conflict -- we have dict and Mapping, list and Sequence, but set and Set...)
>
> On Wed, Aug 26, 2020 at 2:16 PM Alex Hall  wrote:
>
>> On Wed, Aug 26, 2020 at 10:10 PM Christopher Barker 
>> wrote:
>>
>>> Honestly, I haven't delved into type checking much, so maybe I'm making
>>> this up, but what if you write a function that is type hinted to take a
>>> Mapping, and uses | or |=, and all the test code uses a built in dict, and
>>> all seems to be well.
>>>
>>
>> This made me curious so I did an experiment, but with Set because I
>> couldn't be bothered to worry about Python versions. This code:
>>
>> ```
>>
>> import typing
>> import collections
>> import builtins
>>
>> def foo(t: typing.Set, c: collections.Set, b: builtins.set):
>> print(t + t)
>> print(t | t)
>> print(t.union(t))
>>
>> print(c + c)
>> print(c | c)
>> print(c.union(c))
>>
>> print(b + b)
>> print(b | b)
>> print(b.union(b))
>>
>> ```
>>
>> complains about `+` (as it should) and `c.union` in both PyCharm and
>> mypy, and nothing else. So typing.Set is based on the builtin and not the
>> ABC. If you want to support any subclass of collections.Set, your type hint
>> needs to be collections.Set. Using typing.Set will fail to show warnings
>> when you use a builtin only method, and will show warnings when you try to
>> assign an ABC set (e.g. I get a warning with `foo(t=> collections.abc.Set>)`. The problem then is that `collections.abc.Set[int]`
>> doesn't work, i.e. you need to use typing.Set if you want generics.
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CBLLORSVOSCWK2STUAIOVJAPR3L4LL4R/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: What about having a .get(index, default) method for arrays like we have for dicts?

Use typing.AbstractSet for the ABC (there was an unfortunate naming
conflict -- we have dict and Mapping, list and Sequence, but set and Set...)

On Wed, Aug 26, 2020 at 2:16 PM Alex Hall  wrote:

> On Wed, Aug 26, 2020 at 10:10 PM Christopher Barker 
> wrote:
>
>> Honestly, I haven't delved into type checking much, so maybe I'm making
>> this up, but what if you write a function that is type hinted to take a
>> Mapping, and uses | or |=, and all the test code uses a built in dict, and
>> all seems to be well.
>>
>
> This made me curious so I did an experiment, but with Set because I
> couldn't be bothered to worry about Python versions. This code:
>
> ```
>
> import typing
> import collections
> import builtins
>
> def foo(t: typing.Set, c: collections.Set, b: builtins.set):
> print(t + t)
> print(t | t)
> print(t.union(t))
>
> print(c + c)
> print(c | c)
> print(c.union(c))
>
> print(b + b)
> print(b | b)
> print(b.union(b))
>
> ```
>
> complains about `+` (as it should) and `c.union` in both PyCharm and mypy,
> and nothing else. So typing.Set is based on the builtin and not the ABC. If
> you want to support any subclass of collections.Set, your type hint needs
> to be collections.Set. Using typing.Set will fail to show warnings when you
> use a builtin only method, and will show warnings when you try to assign an
> ABC set (e.g. I get a warning with `foo(t= collections.abc.Set>)`. The problem then is that `collections.abc.Set[int]`
> doesn't work, i.e. you need to use typing.Set if you want generics.
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HNB7GNDRX5MFRHRIFHJIR2A6U6SH3QK5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: What about having a .get(index, default) method for arrays like we have for dicts?

On Wed, Aug 26, 2020 at 10:10 PM Christopher Barker 
wrote:

> Honestly, I haven't delved into type checking much, so maybe I'm making
> this up, but what if you write a function that is type hinted to take a
> Mapping, and uses | or |=, and all the test code uses a built in dict, and
> all seems to be well.
>

This made me curious so I did an experiment, but with Set because I
couldn't be bothered to worry about Python versions. This code:

```

import typing
import collections
import builtins

def foo(t: typing.Set, c: collections.Set, b: builtins.set):
print(t + t)
print(t | t)
print(t.union(t))

print(c + c)
print(c | c)
print(c.union(c))

print(b + b)
print(b | b)
print(b.union(b))

```

complains about `+` (as it should) and `c.union` in both PyCharm and mypy,
and nothing else. So typing.Set is based on the builtin and not the ABC. If
you want to support any subclass of collections.Set, your type hint needs
to be collections.Set. Using typing.Set will fail to show warnings when you
use a builtin only method, and will show warnings when you try to assign an
ABC set (e.g. I get a warning with `foo(t=)`. The problem then is that `collections.abc.Set[int]`
doesn't work, i.e. you need to use typing.Set if you want generics.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4LJ6RMIJXE4WFRTCAK4B4HUQL4ATOETY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: still more use for pathlib.Path ....

2020-08-26 Thread Matthias Bussonnier

Many of the tempfile modules' classes and utils could maybe return
Path instead of str. I'm not sure how disruptive that would be, but I
would definitely welcome something that avoids having to wrap
everything in Path.

I think they might need to return a "StrPath" for compatibility, but
IMHO only for some transition period. It is relatively easy to call
str(p) whether p is a string or a path if you actually want a string.

On Wed, 26 Aug 2020 at 13:23, Christopher Barker  wrote:
>
> On Thu, Aug 20, 2020 at 9:08 PM Ethan Furman  wrote:
>>
>> > But for a while is was painful to use, 'cause there was som much code
>> > that still used strings for paths. That was made a lot better when we
>> > introduced the __fspath__ protocol, and then updated the standard
>> > library to use it (everywhere?).
>>
>> Unfortunately not [1].  The __fspath__ protocol is supported in
>> locations where a path is expected as an argument (e.g. os.path.join());
>> anywhere else it is not supported except by accident (meaning it could
>> easily become actually unsupported in the future).
>
>
> I guess this is why some folks though making PATH a subclass of str might be 
> a good idea in the first place.
>
> So a new idea (very poorly thought out) -- make a StrPath that is both a 
> subclass of Path and a str -- so it could be used for things like __file__, 
> and code that expects a string will still work, and code that expects a Path 
> object will work as well. Since both Path and str are immutable, str methods 
> would return strings, and Path methods would return Path objects.
>
> I haven't looked for name clashes yet, but does this seem possible?
>
> -CHB
>
>
> --
> Christopher Barker, PhD
>
> Python Language Consulting
>   - Teaching
>   - Scientific Software Development
>   - Desktop GUI and Web Development
>   - wxPython, numpy, scipy, Cython
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/WB35LZWNKAUWPM4WQ7GBLJIHXGBGKM6C/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HDZXTFK6T6KJ2X6YO6QMEXMQLVHQ4RJX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: What about having a .get(index, default) method for arrays like we have for dicts?

I’m sorry, I just can’t teach you how static type checking works. Read the
mypy docs, install it, and experiment.

On Wed, Aug 26, 2020 at 13:10 Christopher Barker 
wrote:

> On Wed, Aug 26, 2020 at 12:38 PM Alex Hall  wrote:
>
>>
>> This is why we have ABCs in the first place (and static checking). So
>> that you can't add it to Sequence is already a smell. If it's in the
>> Mapping protocol, why isn't it in the Sequence protocol?
>>
>> This is all just a general feature of ABCs. They were written once and
>> frozen in place and no new methods can be added to them because it would
>> break compatibility. It's not specific to this proposal. | and |= operators
>> were added to dict but not to Mapping/MutableMapping for the same reason,
>> even though the same operators exist with a similar meaning in the
>> Set/MutableSet protocols. So there's a clear precedent for this.
>>
>
> and the Set ABC doesn't have the set methods, like .union, .intersection
> ... (though I still don't get why that is ...)
>
>>
>> It's a little unfortunate that we can't put this in the Sequence ABC, but
>> it's not a big deal. The vast majority of the time I'm interfacing with my
>> own code and so I know that I'm working with a list, not some mysterious
>> Sequence. I don't think people are going to change their public facing
>> function signatures from `def foo(x: Sequence)` to `(x: list)` just so that
>> they can use `x.get`. Even if they do, it suggests that they think `.get`
>> is really useful, and callers can just convert their argument to a list
>> first. Similarly adding the union operator to dict is fine because people
>> usually use dicts, not generic Mappings.
>>
>
> This is something to be thinking about, however -- having the builtins
> extend the ABCs hasn't been a big deal, what with dynamic typing and all --
> but as folks start using type hints, and static type checking more, it
> could be an issue.
>
> Honestly, I haven't delved into type checking much, so maybe I'm making
> this up, but what if you write a function that is type hinted to take a
> Mapping, and uses | or |=, and all the test code uses a built in dict, and
> all seems to be well.
>
> Then this code starts being more widely used, and someone runs it with a
> different Mapping object, and it passes the static type checking pass, and
> then, oops, a run-time failure.
>
> It makes me wonder of the utility of the ABCs is the builtins are
> extending them.
>
> Maybe we need a set of (at least for testing) builtins that are fully ABC
> conformant ?
>
> -CHB
>
>
>
>
>
>
>
>
>
>
>>
>> Besides, list already defines a bunch of stuff not found in Sequence or
>> MutableSequence: copy, sort, `<`, `+`, and `*`. What's one more?
>>
>>
>>> There's also the slippery-slope argument brought up earlier -- should it
>>> be added to tuple? To range?
>>>
>>
>> I think Steven D'Aprano asked this. I don't see it as an argument against
>> or a concern about a slippery slope, just a question. I think it'd be nice
>> to have it on lots of sequences, but it's OK if it's not. Neither scenario
>> strikes me as problematic.
>>
>> The union operator was also added to defaultdict, OrderedDict, and
>> ChainMap. Was this a problem?
>>
>>
>>> What should it do for a negative index?
>>>
>>
>> I think most people expect that if `lst[i]` returns a value then
>> `lst.get(i)` should return the same value, therefore it should return a
>> default only when `lst[i]` fails. Sometimes you specifically want
>> `lst.get(-1)` - there's a couple examples of that in [my earlier post](
>> https://mail.python.org/archives/list/python-ideas@python.org/message/7W74OCYU5WTYFNTKW7PHONUCD3U2S3OO/)
>> and Daniel recently presented a case when he wanted that.
>>
>> This isn't the ideal behaviour for all use cases, but only as much as
>> normal indexing isn't the ideal behaviour for all use cases. Sometimes I've
>> had bugs because I forgot to account for negative indices, but I'm still
>> very grateful to you that negative indices work. We can't make something
>> equally convenient for all use cases, but that doesn't diminish the value
>> of this method.
>>
>> You said yourself that the generic alternative for all sequences is to
>> "check for IndexError", a sign that doing the equivalent of that is more
>> intuitive.
>>
>>
>>> I also think it negatively affects readability. Today when I see
>>> `foo.get(bar)` I assume that (unless the code is full of tricks) foo is a
>>> dict or Mapping
>>>
>>
>> That's already a poor assumption to make. `get` is an extremely generic
>> method name that has lots of meanings in different classes.
>>
>>
>>> But if list also grows a .get() method I have to assume in my head that
>>> foo is a list and then bar an int, and I basically have to read the rest of
>>> the code twice, once assuming foo is a Mapping, and once assuming it's a
>>> list.
>>>
>>
>> The purpose of `foo.get(bar)` is to get a value, not to tell you that foo
>> is a dict. If the rest of the code isn't able to

[Python-ideas] Re: still more use for pathlib.Path ....

On Thu, Aug 20, 2020 at 9:08 PM Ethan Furman  wrote:

> > But for a while is was painful to use, 'cause there was som much code
> > that still used strings for paths. That was made a lot better when we
> > introduced the __fspath__ protocol, and then updated the standard
> > library to use it (everywhere?).
>
> Unfortunately not [1].  The __fspath__ protocol is supported in
> locations where a path is expected as an argument (e.g. os.path.join());
> anywhere else it is not supported except by accident (meaning it could
> easily become actually unsupported in the future).
>

I guess this is why some folks though making PATH a subclass of str might
be a good idea in the first place.

So a new idea (very poorly thought out) -- make a StrPath that is both a
subclass of Path and a str -- so it could be used for things like __file__,
and code that expects a string will still work, and code that expects a
Path object will work as well. Since both Path and str are immutable, str
methods would return strings, and Path methods would return Path objects.

I haven't looked for name clashes yet, but does this seem possible?

-CHB

-- 
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WB35LZWNKAUWPM4WQ7GBLJIHXGBGKM6C/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: FEATURE REQUEST: Make `logging` Module more Pythonic

On Tue, Aug 25, 2020 at 12:58 AM Alex Hall  wrote:

>
> I haven't tried it myself but https://github.com/Delgan/loguru looks
> promising.
>

Thanks -- that DOES look nice, and is on PyPi and maintained -- I'll give a
try!

-CHB


-- 
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7BJPOFQVL4YCVZ4R2OLWMH4SX4TJZZUS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: What about having a .get(index, default) method for arrays like we have for dicts?

On Wed, Aug 26, 2020 at 12:38 PM Alex Hall  wrote:

>
> This is why we have ABCs in the first place (and static checking). So that
> you can't add it to Sequence is already a smell. If it's in the Mapping
> protocol, why isn't it in the Sequence protocol?
>
> This is all just a general feature of ABCs. They were written once and
> frozen in place and no new methods can be added to them because it would
> break compatibility. It's not specific to this proposal. | and |= operators
> were added to dict but not to Mapping/MutableMapping for the same reason,
> even though the same operators exist with a similar meaning in the
> Set/MutableSet protocols. So there's a clear precedent for this.
>

and the Set ABC doesn't have the set methods, like .union, .intersection
... (though I still don't get why that is ...)

>
> It's a little unfortunate that we can't put this in the Sequence ABC, but
> it's not a big deal. The vast majority of the time I'm interfacing with my
> own code and so I know that I'm working with a list, not some mysterious
> Sequence. I don't think people are going to change their public facing
> function signatures from `def foo(x: Sequence)` to `(x: list)` just so that
> they can use `x.get`. Even if they do, it suggests that they think `.get`
> is really useful, and callers can just convert their argument to a list
> first. Similarly adding the union operator to dict is fine because people
> usually use dicts, not generic Mappings.
>

This is something to be thinking about, however -- having the builtins
extend the ABCs hasn't been a big deal, what with dynamic typing and all --
but as folks start using type hints, and static type checking more, it
could be an issue.

Honestly, I haven't delved into type checking much, so maybe I'm making
this up, but what if you write a function that is type hinted to take a
Mapping, and uses | or |=, and all the test code uses a built in dict, and
all seems to be well.

Then this code starts being more widely used, and someone runs it with a
different Mapping object, and it passes the static type checking pass, and
then, oops, a run-time failure.

It makes me wonder of the utility of the ABCs is the builtins are extending
them.

Maybe we need a set of (at least for testing) builtins that are fully ABC
conformant ?

-CHB










>
> Besides, list already defines a bunch of stuff not found in Sequence or
> MutableSequence: copy, sort, `<`, `+`, and `*`. What's one more?
>
>
>> There's also the slippery-slope argument brought up earlier -- should it
>> be added to tuple? To range?
>>
>
> I think Steven D'Aprano asked this. I don't see it as an argument against
> or a concern about a slippery slope, just a question. I think it'd be nice
> to have it on lots of sequences, but it's OK if it's not. Neither scenario
> strikes me as problematic.
>
> The union operator was also added to defaultdict, OrderedDict, and
> ChainMap. Was this a problem?
>
>
>> What should it do for a negative index?
>>
>
> I think most people expect that if `lst[i]` returns a value then
> `lst.get(i)` should return the same value, therefore it should return a
> default only when `lst[i]` fails. Sometimes you specifically want
> `lst.get(-1)` - there's a couple examples of that in [my earlier post](
> https://mail.python.org/archives/list/python-ideas@python.org/message/7W74OCYU5WTYFNTKW7PHONUCD3U2S3OO/)
> and Daniel recently presented a case when he wanted that.
>
> This isn't the ideal behaviour for all use cases, but only as much as
> normal indexing isn't the ideal behaviour for all use cases. Sometimes I've
> had bugs because I forgot to account for negative indices, but I'm still
> very grateful to you that negative indices work. We can't make something
> equally convenient for all use cases, but that doesn't diminish the value
> of this method.
>
> You said yourself that the generic alternative for all sequences is to
> "check for IndexError", a sign that doing the equivalent of that is more
> intuitive.
>
>
>> I also think it negatively affects readability. Today when I see
>> `foo.get(bar)` I assume that (unless the code is full of tricks) foo is a
>> dict or Mapping
>>
>
> That's already a poor assumption to make. `get` is an extremely generic
> method name that has lots of meanings in different classes.
>
>
>> But if list also grows a .get() method I have to assume in my head that
>> foo is a list and then bar an int, and I basically have to read the rest of
>> the code twice, once assuming foo is a Mapping, and once assuming it's a
>> list.
>>
>
> The purpose of `foo.get(bar)` is to get a value, not to tell you that foo
> is a dict. If the rest of the code isn't able to tell you whether `foo` is
> a dict or a list, then either the code is poorly written (and .get is not
> to blame) or it doesn't matter because you're just getting individual
> values. `foo[bar]` doesn't differentiate between lists and dicts and this
> is generally considered a good thing, not a problem.
>
>
>>

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

On Wed, Aug 26, 2020 at 9:46 AM David Mertz  wrote:

> In my mind, *anything* other than the straightforward and obvious
> signature `__getitem__(self, index, **kws)` is a pointless distraction.
>
> ...

> Yes, it's slightly funny that square brackets convert to `index` rather
> than `*index`, but that ship sailed very long ago, and it's no big deal.
> There's no problem that needs solving and no need for code churn.
>

I disagree here -- sure, for full backwards compatibility and preservation
of performance, we'll probably have to live with it. But the fact that the
square brackets don't create a tuple makes it fundamentally odd and
confusing to work with -- not too big a deal when [] only accepted a single
expression, and therefor passed a single value on to the dunders, but it
gets very odd when you have something that looks a lot like a function
call, but is different. And it gets worse for __setitem__, which I guess
will be:

thing[ind1, ind2, kwd1=v1, kw2=v2] = value

Translating to:

thing.__setitem__(self, (ind1, ind2), value, kwd1=v1, kw2=v2)

which is pretty darn weird -- particularly if you try to write the handler
this way:

def __setitem__(self, *args, **kwargs):

so: args would always be a 2-tuple, something like: ((ind1, ind2), value)

At least **kwargs would be "normal", yes?

On the plus side, this weirdness is only really exposed to folks writing
classes with complex custom indexing behavior, and would still look like
the same semantics as a function call to the users of that class.

well, almost -- it would not support [*args] -- would it support [**kwargs]
?

And this kind of thing is very much a write once, use a lot situation.

TL;DR -- let's watch the strong language here -- these proposals do attempt
to address a real problem -- probably not worth the downsides, but this
"doesn't solve any problem", "no one asked for it", etc, is really
dismissive.

-CHB

>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/IXGZNYZEKHBNE6E52WKTENNGOIKXOVV6/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

-- 
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YSQMO4ZLYI3OUGYL254ZIX6MK2CQ3DCU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: What about having a .get(index, default) method for arrays like we have for dicts?

On Wed, Aug 26, 2020 at 7:56 PM Guido van Rossum  wrote:

> On Wed, Aug 26, 2020 at 10:45 AM Alex Hall  wrote:
>
>> On Wed, Aug 26, 2020 at 7:30 PM Guido van Rossum 
>> wrote:
>>
>>> But for your convenience you are proposing a problematic change (see
>>> posts by others).
>>>
>>
>> I think we've established it's problematic to add it to the Sequence ABC,
>> but in terms of only adding it to specific sequences like lists, I don't
>> know what other posts you're referring to. Why is it problematic?
>>
>
> It would break duck typing -- if you check for IndexError you can accept
> pretty much any sequence. But if you use .get() you can only work with
> lists.
>
> This is why we have ABCs in the first place (and static checking). So that
> you can't add it to Sequence is already a smell. If it's in the Mapping
> protocol, why isn't it in the Sequence protocol?
>

This is all just a general feature of ABCs. They were written once and
frozen in place and no new methods can be added to them because it would
break compatibility. It's not specific to this proposal. | and |= operators
were added to dict but not to Mapping/MutableMapping for the same reason,
even though the same operators exist with a similar meaning in the
Set/MutableSet protocols. So there's a clear precedent for this.

It's a little unfortunate that we can't put this in the Sequence ABC, but
it's not a big deal. The vast majority of the time I'm interfacing with my
own code and so I know that I'm working with a list, not some mysterious
Sequence. I don't think people are going to change their public facing
function signatures from `def foo(x: Sequence)` to `(x: list)` just so that
they can use `x.get`. Even if they do, it suggests that they think `.get`
is really useful, and callers can just convert their argument to a list
first. Similarly adding the union operator to dict is fine because people
usually use dicts, not generic Mappings.

Besides, list already defines a bunch of stuff not found in Sequence or
MutableSequence: copy, sort, `<`, `+`, and `*`. What's one more?

> There's also the slippery-slope argument brought up earlier -- should it
> be added to tuple? To range?
>

I think Steven D'Aprano asked this. I don't see it as an argument against
or a concern about a slippery slope, just a question. I think it'd be nice
to have it on lots of sequences, but it's OK if it's not. Neither scenario
strikes me as problematic.

The union operator was also added to defaultdict, OrderedDict, and
ChainMap. Was this a problem?

> What should it do for a negative index?
>

I think most people expect that if `lst[i]` returns a value then
`lst.get(i)` should return the same value, therefore it should return a
default only when `lst[i]` fails. Sometimes you specifically want
`lst.get(-1)` - there's a couple examples of that in [my earlier post](
https://mail.python.org/archives/list/python-ideas@python.org/message/7W74OCYU5WTYFNTKW7PHONUCD3U2S3OO/)
and Daniel recently presented a case when he wanted that.

This isn't the ideal behaviour for all use cases, but only as much as
normal indexing isn't the ideal behaviour for all use cases. Sometimes I've
had bugs because I forgot to account for negative indices, but I'm still
very grateful to you that negative indices work. We can't make something
equally convenient for all use cases, but that doesn't diminish the value
of this method.

You said yourself that the generic alternative for all sequences is to
"check for IndexError", a sign that doing the equivalent of that is more
intuitive.

> I also think it negatively affects readability. Today when I see
> `foo.get(bar)` I assume that (unless the code is full of tricks) foo is a
> dict or Mapping
>

That's already a poor assumption to make. `get` is an extremely generic
method name that has lots of meanings in different classes.

> But if list also grows a .get() method I have to assume in my head that
> foo is a list and then bar an int, and I basically have to read the rest of
> the code twice, once assuming foo is a Mapping, and once assuming it's a
> list.
>

The purpose of `foo.get(bar)` is to get a value, not to tell you that foo
is a dict. If the rest of the code isn't able to tell you whether `foo` is
a dict or a list, then either the code is poorly written (and .get is not
to blame) or it doesn't matter because you're just getting individual
values. `foo[bar]` doesn't differentiate between lists and dicts and this
is generally considered a good thing, not a problem.

> The generalization I am talking about here is the assumption that because
> list and dict both support x[y], and dict also has x.get(y) which returns a
> default instead of raising, then it would be a good idea to add the latter
> also to list.
>

I'm not assuming that list should have .get because dict has it. I'm
claiming list should have it because it would often be useful, based on
[empirical evidence](

[Python-ideas] Re: Type Implications in Python 4 (or 3.9)

On Wed, Aug 26, 2020 at 11:57 AM Guido van Rossum  wrote:

> IIUC Cython (even if they were to adopt annotations) is not meant to be a
> strict subset of Python -- almost no Cython module is valid Python.
>

correct. rather, it is a strict superset -- any valid Python should be
compilable with Cython. But Cython type annotations are not valid Python.

However, Cython has a "pure python" mode that IS valid Python:

https://cython.readthedocs.io/en/latest/src/tutorial/pure.html

It used to rely almost entirely on decorators and special function (e.g.
cython.declare()) to add the Cython annotations -- these could simply do
nothing when run through the Python interpreter. But apparently it now is
using Type Hints as well.

Or you can put the annotations in a separate file.

But it does not support full Cython functionality. But if what you want is
faster Python, it may well get you there.

I hope the mypyc folks are at least keeping an eye on it -- the projects
seem to have overlapping goals.

In fact, when type hinting was standardized, it was explicitly said that
>> that was not the goal.
>>
>
> Right. And mypyc is experimental. But type hints *are* useful for
> compilers (see also TorchScript).
>

Indeed. I suppose it's not the syntax that's the issue, but what types you
use. If you use, e.g. Sequence, won't get much gain, but if you use a
specific type, like List, then it can compile down to more efficient code.

-CHB

-- 
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FQY2QL4YNE6RVKFWA3QAHRRK7VZMJ3F2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Type Implications in Python 4 (or 3.9)

On Wed, Aug 26, 2020 at 11:20 AM Christopher Barker 
wrote:

>
>
> On Wed, Aug 26, 2020 at 9:02 AM Guido van Rossum
>
>> Mypy includes an experimental compiler from typed Python to C that
>> actually does what the OP is looking for.
>>
>> https://github.com/python/mypy/tree/master/mypyc
>>
>
> There is also Cython, which is almost exactly what the OP is looking for.
>

And certainly more mature.

(I’m curious about how mypyc is different than Cython (other than syntax)
> and I’ve lost track, but I thought Cython was looking into using standard
> type hint notation.
>

IIUC Cython (even if they were to adopt annotations) is not meant to be a
strict subset of Python -- almost no Cython module is valid Python. OTOH
mypyc has this as a goal -- you write annotated Python, and you test it
using the Python interpreter, and then when you need more speed, mypyc can
translate it to run faster. But you can still execute the mypyc input with
the regular interpreter, and it will have the same semantics. (There are
situations where mypyc doesn't handle certain dynamic constructs, but there
are no cases where code can only be run by mypyc.)

> But there’s a trick: type hints can be (and often are) duck typed. That
> is, they might specify a Sequence of Numbers, which doesn’t provide enough
> information to compile down to fully performant code.
>
> In fact, when type hinting was standardized, it was explicitly said that
> that was not the goal.
>

Right. And mypyc is experimental. But type hints *are* useful for compilers
(see also TorchScript).

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VN2COA6SFVEQ7BHVNZ5ENBSJAEQ5IGKI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Type Implications in Python 4 (or 3.9)

On Wed, Aug 26, 2020 at 9:02 AM Guido van Rossum

> Mypy includes an experimental compiler from typed Python to C that
> actually does what the OP is looking for.
>
> https://github.com/python/mypy/tree/master/mypyc
>

There is also Cython, which is almost exactly what the OP is looking for.

(I’m curious about how mypyc is different than Cython (other than syntax)
and I’ve lost track, but I thought Cython was looking into using standard
type hint notation.

But there’s a trick: type hints can be (and often are) duck typed. That is,
they might specify a Sequence of Numbers, which doesn’t provide enough
information to compile down to fully performant code.

In fact, when type hinting was standardized, it was explicitly said that
that was not the goal.

-CHB




> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
>
>
> ___
>
> Python-ideas mailing list -- python-ideas@python.org
>
> To unsubscribe send an email to python-ideas-le...@python.org
>
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/3FS5C32FVB6YNN5TL5WTZETSCGLEHDTI/
>
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> --
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2VQYOMAK5U6VBXTQHJA7KGATPPCCUX76/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: What about having a .get(index, default) method for arrays like we have for dicts?

On Wed, Aug 26, 2020 at 10:45 AM Alex Hall  wrote:

> On Wed, Aug 26, 2020 at 7:30 PM Guido van Rossum  wrote:
>
>> But for your convenience you are proposing a problematic change (see
>> posts by others).
>>
>
> I think we've established it's problematic to add it to the Sequence ABC,
> but in terms of only adding it to specific sequences like lists, I don't
> know what other posts you're referring to. Why is it problematic?
>

It would break duck typing -- if you check for IndexError you can accept
pretty much any sequence. But if you use .get() you can only work with
lists.

This is why we have ABCs in the first place (and static checking). So that
you can't add it to Sequence is already a smell. If it's in the Mapping
protocol, why isn't it in the Sequence protocol?

There's also the slippery-slope argument brought up earlier -- should it be
added to tuple? To range?

What should it do for a negative index?

I also think it negatively affects readability. Today when I see
`foo.get(bar)` I assume that (unless the code is full of tricks) foo is a
dict or Mapping, and bar a key. But if list also grows a .get() method I
have to assume in my head that foo is a list and then bar an int, and I
basically have to read the rest of the code twice, once assuming foo is a
Mapping, and once assuming it's a list.

> Maybe I meant overgeneralization instead.
>>
>
> Again, the goal isn't to generalize. I'm not trying to make lists look
> like dicts. It's not even possible to get close - `if i in x: print(x[i])`
> does very different things for lists and dicts.
>

The generalization I am talking about here is the assumption that because
list and dict both support x[y], and dict also has x.get(y) which returns a
default instead of raising, then it would be a good idea to add the latter
also to list. And I judge it an overgeneralization because there's more to
it than meets the eye.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6XNUGAALBEO2MIR3TI3CMYUQAAVBZXUX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: What about having a .get(index, default) method for arrays like we have for dicts?

2020-08-26 Thread David Mertz

I think this method would be useful for list itself (maybe tuple, but I
care less about that). It certainly should not be part of the Sequence
protocol.

I have often programmed a pattern where I .append() elements to a list in a
loop, and *expect* at least N things to make it in.  In such case, getting
a sentinel answer to mylist.get(N-1) would be convenient.

Obviously, I've lived without that fine. But it's a little bit shorter and
more expressive than:

x = mylist[N-1] if len(mylist) >= N else None

On Wed, Aug 26, 2020, 1:30 PM Guido van Rossum  wrote:

> But for your convenience you are proposing a problematic change (see posts
> by others). Maybe I meant overgeneralization instead.
>
> On Wed, Aug 26, 2020 at 10:20 AM Alex Hall  wrote:
>
>> On Wed, Aug 26, 2020 at 4:32 PM Guido van Rossum 
>> wrote:
>>
>>> FWIW I don’t think we should add a .get() method for Sequence or for
>>> list. It reeks of hyper-correctness.
>>>
>>
>> What do you mean by this? I want the method because it's useful and
>> convenient, not because it's 'correct'. I don't think that lists are
>> 'wrong' right now.
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/D7JMHGT4VE3ZIXPVHD4FQMIDLJ6H2H4V/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/B3KJKBAAO5T5PBK6JPEMBPKIBP67QC5F/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: What about having a .get(index, default) method for arrays like we have for dicts?

On Wed, Aug 26, 2020 at 7:30 PM Guido van Rossum  wrote:

> But for your convenience you are proposing a problematic change (see posts
> by others).
>

I think we've established it's problematic to add it to the Sequence ABC,
but in terms of only adding it to specific sequences like lists, I don't
know what other posts you're referring to. Why is it problematic?

> Maybe I meant overgeneralization instead.
>

Again, the goal isn't to generalize. I'm not trying to make lists look like
dicts. It's not even possible to get close - `if i in x: print(x[i])` does
very different things for lists and dicts.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YDSC5T7MLM2VBJCLCFP2V5OPS34LD73B/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: What about having a .get(index, default) method for arrays like we have for dicts?

But for your convenience you are proposing a problematic change (see posts
by others). Maybe I meant overgeneralization instead.

On Wed, Aug 26, 2020 at 10:20 AM Alex Hall  wrote:

> On Wed, Aug 26, 2020 at 4:32 PM Guido van Rossum  wrote:
>
>> FWIW I don’t think we should add a .get() method for Sequence or for
>> list. It reeks of hyper-correctness.
>>
>
> What do you mean by this? I want the method because it's useful and
> convenient, not because it's 'correct'. I don't think that lists are
> 'wrong' right now.
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/D7JMHGT4VE3ZIXPVHD4FQMIDLJ6H2H4V/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: What about having a .get(index, default) method for arrays like we have for dicts?

On Wed, Aug 26, 2020 at 4:32 PM Guido van Rossum  wrote:

> FWIW I don’t think we should add a .get() method for Sequence or for list.
> It reeks of hyper-correctness.
>

What do you mean by this? I want the method because it's useful and
convenient, not because it's 'correct'. I don't think that lists are
'wrong' right now.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/I3YEXPXW3KMZZ75UH5AJSIASZFMDQH2F/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

2020-08-26 Thread Antoine Pitrou

On Wed, 26 Aug 2020 12:46:18 -0400
David Mertz  wrote:
> In my mind, *anything* other than the straightforward and obvious signature
> `__getitem__(self, index, **kws)` is a pointless distraction.

Probably.  However, one must define how it's exposed in C.

Regards

Antoine.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BK7UCYHIN7VRREVMJNQR35GT2AIKF3PN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

2020-08-26 Thread David Mertz

In my mind, *anything* other than the straightforward and obvious signature
`__getitem__(self, index, **kws)` is a pointless distraction.

We don't need new custom objects to hold keywords. We don't need funny
conditional logic about one versus multiple index objects. We don't need
some other method that sometimes takes priority.

Yes, it's slightly funny that square brackets convert to `index` rather
than `*index`, but that ship sailed very long ago, and it's no big deal.
There's no problem that needs solving and no need for code churn.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IXGZNYZEKHBNE6E52WKTENNGOIKXOVV6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

On Wed, Aug 26, 2020 at 11:19 AM Steven D'Aprano 
wrote:

> On Wed, Aug 26, 2020 at 10:10:34AM -0400, Ricky Teachey wrote:
>
> > But I have to say that I think this latest is a fantastic idea, and when
> > Jonathan presented it to me it was very surprising that I had not seen it
> > presented by anyone else yet. I think it solves a ton of problems,
>
> Such as?
>

It creates a language supported way for the creator of the class to decide
how to interpret the contents inside a subscript operation. This is a
problem because disagreement over this matter is a large part of the reason
PEP 472-- the spirit of which I support-- has been held up. It greatly
alleviates (though not perfectly-- see the end of this message) the
incongruity between how the indexing operator behaves and function calls
behave. As explained by Jonathan Fine, it adds flexibility and smoothes out
the differences the different paradigms of subscript operations: sequences,
and mappings. And it opens up opportunities for other paradigms to be
created, the most prominent example of which is type-hint creation
shortcuts, like:

Vector = Dict[i=float, j=float]

> adds a huge amount of flexibility and functionality,
>
> Such as?
>
> With the proposal, the language would support any function desired to turn
> > the "stuff" inside a subscripting operation into the item dunder calls.
>
> I'm sorry, I don't understand that sentence.
>

I'll provide examples.

> > For example: if this proposal were already in place and PEP 472 were to
> > continue to be held up because of terrorists like me ;) *, one could have
> > written this translation function and PEP-472-ify their classes already:
> >
> > def yay_kwargs(self, *args, **kwargs):
> > return self.__getitem__(args, **kwargs)
>
> You're calling the `__getitem__` dunder with arbitrary keyword
> arguments. Are you the same Ricky Teachey who just suggested that we
> should be free to break code that uses `__getitem__` methods that don't
> obey the intent that they have only a single parameter and no keywords?
>

I am talking hypothetically-- if this proposal were already in place (which
*includes* passing kwargs to the *new dunder* rather than passing them to
the existing item dunders by default), you could write code like yay_kwargs
today, even if the default way the language behaves did not change.

In that universe, if I wrote this:

class C:
def __getitem__(self, key):
print(key)

...and tried to do this:

>>> C()[a=1]
SomeError

...*in that universe*, without PEP 472, the language will STILL, by
default, give an error as it does today (though it probably would no longer
be a SyntaxError).

PEP 472 is a proposal to change *the current, default key or index
translation function* to pass **kwargs. *This *proposal is to allow for an
intervening function that controls HOW they are passed.

If PEP 472 is held up, then `obj[1, 2, axis='north']` is a SyntaxError,
> so how does this method yay_kwargs make it legal?
>
>
>
> --
> Steve
>

Because the proposal is that if there is a dunder present containing the
class attribute function, the contents of the [ ] operator get passed to
that function for translation into the key.

We could make the dunder to accept the target dunder method name as a
parameter. This way there is only a single new dunder, rather than 3.

The single new dunder might look like this:

class Q:
def __subscript__(self, method_name, *args, **kwargs):
 return getattr(self,  method_name)(*args, **kwargs)
def __getitem__(self, *args, **kwargs): ...
#  Note that I have made the RHS value the first argument in __setitem__
def __setitem__(self, value, *args, **kwargs): ...
def __delitem__(self, *args, **kwargs): ...

Above I am calling the appropriate dunder method directly inside of
__subscript__. Again, there are other ways to do it and it does not have to
be this way.

If it is done that way, the  __subscript__ dunder gets passed which item
dunder method is being called (__getitem__, __setitem__, or __delitem__),
and the arguments. Examples:

No  CODECALLS
1.q[1] q.__subscript__("__getitem__", 1)
2.q[1,]q.__subscript__("__getitem__", 1)
3.q[(1,)]  q.__subscript__("__getitem__", 1)
4.q[(1,),] q.__subscript__("__getitem__", (1,))
5.q[1] = 2   q.__subscript__("__setitem__", 2, 1)
6.q[1,] = 2  q.__subscript__("__setitem__", 2, 1)
7.q[(1,)] = 2q.__subscript__("__setitem__", 2, 1)
8.q[(1,),] = 2   q.__subscript__("__setitem__", 2, (1,))

And so on for the __delitem__ calls.

NOTE: #3 and #7 are very unfortunate, but we cannot change this without
breaking backwards compatibility.

---
Ricky.

"I've never met a Kentucky man who wasn't either thinking about going home
or actually going home." - Happy Chandler
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an

[Python-ideas] Re: What about having a .get(index, default) method for arrays like we have for dicts?

Yes. That's what I meant by "the mixins are part of the contract", sorry.

On Wed, Aug 26, 2020 at 8:45 AM Greg Ewing 
wrote:

> On 27/08/20 2:32 am, Guido van Rossum wrote:
> > The mixins are part of the contract. You are free to override them,
> > their implementation is just a helpful,default.
>
> That doesn't really answer my question. To put it another way:
> If someone wanted to implement an object that obeys the mapping
> protocol, but without inheriting from the Mapping ABC, would they
> be obliged to provide a get() method?
>
> --
> Greg
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/JJVNPCQP4N3X72GPKTVOFFSMZEZ5JSYW/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/RFINQ4D6DROUKYDTPTCLCKHCJFCGKWVM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Type Implications in Python 4 (or 3.9)

On Wed, Aug 26, 2020 at 8:41 AM Ronald Oussoren via Python-ideas <
python-ideas@python.org> wrote:

>
> On 26 Aug 2020, at 17:02, krishnans2...@gmail.com wrote:
>
> One suggestion I had for the next Python release is to add
> type-implication support. Many developers have learned Python, but do not
> want to use it since it is slow. An awesome way to fix this is to have
> optional type-implications. For example, if you know for sure that variable
> x is an int, you can make Python just a bit smaller by somehow specifying
> to the interpreter that the variable is an integer. Something like 'x::int
> = 5'. By having optional type implications, you can still do everything you
> do with normal Python, except you can speed it up a little bit by telling
> the Interpreter that this variable is starting off with this datatype.
>
>
> Python as type annotations, but those are primarily used for type checking
> using tools such as mypy and are not used by the interpreter itself.
>

Mypy includes an experimental compiler from typed Python to C that actually
does what the OP is looking for.

https://github.com/python/mypy/tree/master/mypyc

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3FS5C32FVB6YNN5TL5WTZETSCGLEHDTI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Changing item dunder method signatures to utilize positional arguments (open thread)

On Wed, Aug 26, 2020 at 11:30 AM Greg Ewing 
wrote:

> On 27/08/20 12:53 am, Steven D'Aprano wrote:
>
> > This is going to slow down the most common cases of subscripting: the
> > interpreter has to follow the entire MRO to find `__getindex__` in
> > object, which then dispatches to the `__getitem__` method.
>
> No, it would be done by checking type slots, no MRO search involved.
>

Can you elaborate on this for my understanding?

> > In your earlier statement, you said that it would be possible for
> > subscripting to mean something different depending on whether the
> > comma-separated subscripts had parentheses around them or not:
> >
> >  obj[(2, 3)]
> >  obj[2, 3]
> >
> > How does that happen?
>
> If the object has a __getindex__ method, it gets whatever is between
> the [] the same way as a normal function call, so comma-separated
> expressions become separate positional arguments.
>
> --
> Greg
>

I am interested in this proposal, but I am pretty sure that last goal isn't
entirely possible, it is only mostly possible.

I believe that the way python handles the comma-separated expression
inside the [ ] operator is just due to operator precedence, as Steve
pointed out previously. So without a major change we cannot write code
where this test suite fully passes (three part test):

# TEST SUITE 1
assert q[1,] == f(1,)  # part 1
assert q[1] == f(1)  # part 2
assert q[1,] == f((1,))  # part 3

The only way around this would be for the function, f, to "know" about the
presence of hanging commas.

Similarly, without such "runtime comma detection" we cannot make it
possible for this test suite to fully pass:

# TEST SUITE 2
assert q[1,2] == f(1,2)  # part 1
assert q[(1,2)] == f((1,2))  # part 2
assert q[(1,2),] == f((1,2),)  # part 3

Can we?

---
Ricky.

"I've never met a Kentucky man who wasn't either thinking about going home
or actually going home." - Happy Chandler
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DASYPSTG2GIO3DENXO4R5QAKP6264L7Y/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: What about having a .get(index, default) method for arrays like we have for dicts?


On 27/08/20 2:32 am, Guido van Rossum wrote:
The mixins are part of the contract. You are free to override them, 
their implementation is just a helpful,default.


That doesn't really answer my question. To put it another way:
If someone wanted to implement an object that obeys the mapping
protocol, but without inheriting from the Mapping ABC, would they
be obliged to provide a get() method?

--
Greg
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/JJVNPCQP4N3X72GPKTVOFFSMZEZ5JSYW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Type Implications in Python 4 (or 3.9)

2020-08-26 Thread Ronald Oussoren via Python-ideas



> On 26 Aug 2020, at 17:02, krishnans2...@gmail.com wrote:
> 
> One suggestion I had for the next Python release is to add type-implication 
> support. Many developers have learned Python, but do not want to use it since 
> it is slow. An awesome way to fix this is to have optional type-implications. 
> For example, if you know for sure that variable x is an int, you can make 
> Python just a bit smaller by somehow specifying to the interpreter that the 
> variable is an integer. Something like 'x::int = 5'. By having optional type 
> implications, you can still do everything you do with normal Python, except 
> you can speed it up a little bit by telling the Interpreter that this 
> variable is starting off with this datatype.

Python as type annotations, but those are primarily used for type checking 
using tools such as mypy and are not used by the interpreter itself.  

PyPy is an implementation of Python that includes an advanced JIT compiler, and 
their FAQ mentions that having type annotations won’t help for performance, 
even if they’d look at them. See: 
https://doc.pypy.org/en/latest/faq.html#would-type-annotations-help-pypy-s-performance
 
.

Ronald

—

Twitter / micro.blog: @ronaldoussoren
Blog: https://blog.ronaldoussoren.net/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3G24YXW5HQ7YTCSGNEBZFE7FNID34EKV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Type Implications in Python 4 (or 3.9)

2020-08-26 Thread Calvin Spealman

Do you mean the type annotations that Python has had for about ten years?

And the typing library that uses them to specify type hints to compilers
and other tooling?
https://docs.python.org/3/library/typing.html

On Wed, Aug 26, 2020 at 11:07 AM  wrote:

> One suggestion I had for the next Python release is to add
> type-implication support. Many developers have learned Python, but do not
> want to use it since it is slow. An awesome way to fix this is to have
> optional type-implications. For example, if you know for sure that variable
> x is an int, you can make Python just a bit smaller by somehow specifying
> to the interpreter that the variable is an integer. Something like 'x::int
> = 5'. By having optional type implications, you can still do everything you
> do with normal Python, except you can speed it up a little bit by telling
> the Interpreter that this variable is starting off with this datatype.
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/ZGTZNKOSTNTEA3NLJWERIGC3IHG7WU5B/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>

-- 

CALVIN SPEALMAN

SENIOR QUALITY ENGINEER

cspea...@redhat.com  M: +1.336.210.5107
[image: https://red.ht/sig] 
TRIED. TESTED. TRUSTED. 
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/V63JTFW5RLUDJ7DAAD5B7R7MV7KECIQO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Changing item dunder method signatures to utilize positional arguments (open thread)


On 27/08/20 12:53 am, Steven D'Aprano wrote:


Presumably the below method is provided by `object`. If not, what
provides it?


 def __getindex__(self, *args, **kwds):
 if kwds:
 raise TypeError("Object does not support keyword indexes")
 if not args:
 raise TypeError("Object does not accept empty indexes")


It's not literally a method, I just wrote it like that to
illustrate the semantics. It would be done by the interpreter
as part of the process of translating indexing operations into
dunder calls.


What is your reasoning behind prohibiting keywords, when we're right in
the middle of a discussion over PEP 474 which aims to allow keywords?


We're falling back to __getitem__ here, which doesn't currently allow
keywords, and would stay that way. The point of this proposal is to
not change __getitem__. If you want to get keywords, you provide
__getindex__.


 if len(args) == 1:
 args = args[0]
 return self.__getitem__(args)


This is going to slow down the most common cases of subscripting: the
interpreter has to follow the entire MRO to find `__getindex__` in
object, which then dispatches to the `__getitem__` method.


No, it would be done by checking type slots, no MRO search involved.


In your earlier statement, you said that it would be possible for
subscripting to mean something different depending on whether the
comma-separated subscripts had parentheses around them or not:

 obj[(2, 3)]
 obj[2, 3]

How does that happen?


If the object has a __getindex__ method, it gets whatever is between
the [] the same way as a normal function call, so comma-separated
expressions become separate positional arguments.

--
Greg
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/A6PUWAGM3FAIWPDCMVZLFUZYZJZ2SK4L/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: What about having a .get(index, default) method for arrays like we have for dicts?

On Wed, Aug 26, 2020 at 5:00 PM Stephen J. Turnbull <
turnbull.stephen...@u.tsukuba.ac.jp> wrote:

> If the "keyword arguments in __getitem__" feature is added, .get() is
> purely redundant.  (Of course this thread would then become "make
> 'default' a standard keyword argument for mutable collections.")
>

Is that something people want to do? Do people want to be able to write
`my_dict[key, default=0]` instead of `my_dict.get(key, 0)`? What
about `my_dict[key, default=None]` instead of `my_dict.get(key)`?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5LW366TDX3SKJ2W6D6KCCJZBKSKZRU2W/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

On Wed, Aug 26, 2020 at 10:10:34AM -0400, Ricky Teachey wrote:

> But I have to say that I think this latest is a fantastic idea, and when
> Jonathan presented it to me it was very surprising that I had not seen it
> presented by anyone else yet. I think it solves a ton of problems,

Such as?


> adds a huge amount of flexibility and functionality,

Such as?


> With the proposal, the language would support any function desired to turn
> the "stuff" inside a subscripting operation into the item dunder calls.

I'm sorry, I don't understand that sentence.


> For example: if this proposal were already in place and PEP 472 were to
> continue to be held up because of terrorists like me ;) *, one could have
> written this translation function and PEP-472-ify their classes already:
> 
> def yay_kwargs(self, *args, **kwargs):
> return self.__getitem__(args, **kwargs)

You're calling the `__getitem__` dunder with arbitrary keyword 
arguments. Are you the same Ricky Teachey who just suggested that we 
should be free to break code that uses `__getitem__` methods that don't 
obey the intent that they have only a single parameter and no keywords?

If PEP 472 is held up, then `obj[1, 2, axis='north']` is a SyntaxError, 
so how does this method yay_kwargs make it legal?



-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DYJX34W3NLRIEN2PO4WV4QEGX5CBODWH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

On Wed, Aug 26, 2020 at 10:40 AM Paul Moore  wrote:

> On Wed, 26 Aug 2020 at 15:12, Ricky Teachey  wrote:
>
> > Objection 1: Slowing Things Down
> >
> > The INTENDED EFFECT of the changes to internals will be as Jonathan Fine
> described: every time a subscript operation occurs, this new dunder
> attribute gets investigated on the class, and if it is not present then the
> default key translation function is executed.
> >
> > If things were implemented in exactly that way, obviously it would slow
> everything down a lot. Every subscripting operation gets slowed down
> everywhere and that's probably not an option.
> >
> > However, for actual implementation, there's no reason to do that.
> Instead we could wrap the existing item dunder methods automatically at
> class creation time only when the new dunder attribute is present. If it is
> not present, nothing happens. In other words, what has been described as
> the "default key or index translation function" already exists. Let's not
> change that at all for classes that do not choose to use it.
>
> That would mean the effect was to disallow runtime monkeypatching -
> the new dunder is *only* effective if added at class creation time,
> not if it's added later. You may not care about this, but it is a very
> different behaviour than any other dunder method Python supports - so
> quite apart from the problems people would have learning and
> remembering that this is a special case, you have to document *in your
> proposal* that you intend to allow this. And other people *do* care
> about disallowing dynamic features like monkeypatching.
>
> I do understand that Python's extreme dynamism is both disconcerting
> and frustrating when it makes proposals like this harder to implement.
> But conversely, that same dynamism is what makes tools like pytest
> possible, so we all benefit from it, even if we don't directly go
> around monkeypatching classes at runtime.
>

I agree with you. Let's preserve the dynamism. I abandon that
implementation idea.


> If you think you can implement this proposal without blocking Python's
> dynamic features, and without introducing a performance impact, I'd
> say go for it - provide an example implementation, and that would
> clearly show people concerned about performance that it's not an
> issue. But personally, I'm not convinced that's possible without
> adding constraints that *aren't* currently included in the proposal.
>

Yup. That is going to be a hard one. Hopefully others smarter than me can
help think about how we could do it. But I for sure see the problem.

> Objection 3: Doesn't add anything useful/helpful
> >
> > This objection seems obviously false.
>
> Hardly. What are the use cases? It's "obviously false" to state that
> the proposal doesn't add anything at all, true. But the question is
> whether the addition is *useful* or *helpful*. And not just to one
> individual who thinks "this would be cool", but to *enough* people,
> whose code would be improved, to justify the cost of adding the
> feature. You yourself conceded that the feature adds complexity, this
> is where you get to explain why that cost is justified. "It's
> obviously helpful" isn't much of a justification.
>
> Paul
>

Well I did give a couple examples: language supported opt-in for
pep-472-ification, and pep-472 opt-out. But Ok, more are needed.

---
Ricky.

"I've never met a Kentucky man who wasn't either thinking about going home
or actually going home." - Happy Chandler
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BXTZRCFMFFDLXAW6O2SQACWCHHMYQ6E4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Type Implications in Python 4 (or 3.9)

2020-08-26 Thread krishnans2006

One suggestion I had for the next Python release is to add type-implication 
support. Many developers have learned Python, but do not want to use it since 
it is slow. An awesome way to fix this is to have optional type-implications. 
For example, if you know for sure that variable x is an int, you can make 
Python just a bit smaller by somehow specifying to the interpreter that the 
variable is an integer. Something like 'x::int = 5'. By having optional type 
implications, you can still do everything you do with normal Python, except you 
can speed it up a little bit by telling the Interpreter that this variable is 
starting off with this datatype.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZGTZNKOSTNTEA3NLJWERIGC3IHG7WU5B/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: What about having a .get(index, default) method for arrays like we have for dicts?

2020-08-26 Thread Stephen J. Turnbull

Alex Hall writes:

 > I can't see any reason not to add it to range, strings, bytes,
 > deque, array, etc. beyond implementation effort. I also think it's
 > fine to leave them out at first.

If the "keyword arguments in __getitem__" feature is added, .get() is
purely redundant.  (Of course this thread would then become "make
'default' a standard keyword argument for mutable collections.")

Since .get() doesn't seem to be terribly useful, and there is quite
a bit of support for keyword arguments in __getitem__, I think this
can be tabled pending resolution of that proposal.

Steve

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2ADC2KPDKGFASEMKKR5DSCKKRHNM6SSO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

2020-08-26 Thread Paul Moore

On Wed, 26 Aug 2020 at 15:12, Ricky Teachey  wrote:

> Objection 1: Slowing Things Down
>
> The INTENDED EFFECT of the changes to internals will be as Jonathan Fine 
> described: every time a subscript operation occurs, this new dunder attribute 
> gets investigated on the class, and if it is not present then the default key 
> translation function is executed.
>
> If things were implemented in exactly that way, obviously it would slow 
> everything down a lot. Every subscripting operation gets slowed down 
> everywhere and that's probably not an option.
>
> However, for actual implementation, there's no reason to do that. Instead we 
> could wrap the existing item dunder methods automatically at class creation 
> time only when the new dunder attribute is present. If it is not present, 
> nothing happens. In other words, what has been described as the "default key 
> or index translation function" already exists. Let's not change that at all 
> for classes that do not choose to use it.

That would mean the effect was to disallow runtime monkeypatching -
the new dunder is *only* effective if added at class creation time,
not if it's added later. You may not care about this, but it is a very
different behaviour than any other dunder method Python supports - so
quite apart from the problems people would have learning and
remembering that this is a special case, you have to document *in your
proposal* that you intend to allow this. And other people *do* care
about disallowing dynamic features like monkeypatching.

I do understand that Python's extreme dynamism is both disconcerting
and frustrating when it makes proposals like this harder to implement.
But conversely, that same dynamism is what makes tools like pytest
possible, so we all benefit from it, even if we don't directly go
around monkeypatching classes at runtime.

If you think you can implement this proposal without blocking Python's
dynamic features, and without introducing a performance impact, I'd
say go for it - provide an example implementation, and that would
clearly show people concerned about performance that it's not an
issue. But personally, I'm not convinced that's possible without
adding constraints that *aren't* currently included in the proposal.

> Objection 3: Doesn't add anything useful/helpful
>
> This objection seems obviously false.

Hardly. What are the use cases? It's "obviously false" to state that
the proposal doesn't add anything at all, true. But the question is
whether the addition is *useful* or *helpful*. And not just to one
individual who thinks "this would be cool", but to *enough* people,
whose code would be improved, to justify the cost of adding the
feature. You yourself conceded that the feature adds complexity, this
is where you get to explain why that cost is justified. "It's
obviously helpful" isn't much of a justification.

Paul
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/X342S76IP2AGIK3DPKSK6BI2XJE5KV74/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

On Wed, Aug 26, 2020 at 10:10 AM Ricky Teachey  wrote:

> *Objection 1: Slowing Things Down*
> ...for actual implementation... we could wrap the existing item dunder
> methods automatically at class creation time only when the new dunder
> attribute is present. If it is not present, nothing happens. In other
> words, what has been described as the "default key or index
> translation function" already exists. Let's not change that at all for
> classes that do not choose to use it.
>

After reading Steve's response to me in the other thread, where he says
this:

On Wed, Aug 26, 2020 at 9:48 AM Steven D'Aprano  wrote:

> On Tue, Aug 25, 2020 at 10:51:42PM -0400, Ricky Teachey wrote:
>
> > Well it would not have to inspect the signature on every subscript call,
>
> Python is a very dynamic language and objects can change their own
> methods and even their class at any time, so, yes, it will have to
> inspect the signature on every call. Otherwise you are changing
> the language execution model.
>
> Demonstration:
>
>
> py> class Dynamic:
> ... def __getitem__(self, arg):
> ... print("original")
> ... type(self).__getitem__ = lambda *args:
> print("replaced")
> ...
> py>
> py> x = Dynamic()
> py> x[0]
> original
> py> x[0]
> replaced
>

...I am less optimistic that this can be implemented without slowing things
down. And wrapping the methods at class creation time may not be a good
idea after all.

I am a little crestfallen now, because I still think the core of the idea
is a wonderful idea.

Is there another clever way this could be implemented, to avoid the
slowdown of existing code?

---
Ricky.

"I've never met a Kentucky man who wasn't either thinking about going home
or actually going home." - Happy Chandler
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6XIRDYIQAJMQJOZQX4DUTNHLWPKSIS6V/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: What about having a .get(index, default) method for arrays like we have for dicts?

The mixins are part of the contract. You are free to override them, their
implementation is just a helpful,default.

FWIW I don’t think we should add a .get() method for Sequence or for list.
It reeks of hyper-correctness.

On Wed, Aug 26, 2020 at 01:33 Alex Hall  wrote:

> On Wed, Aug 26, 2020 at 1:47 AM Greg Ewing 
> wrote:
>
>> As a point of interest, is get() considered an official part of the
>>
>>
>> mapping protocol, or just nice-to-have?
>>
>>
>>
>>
>>
>> The docs don't seem to be very clear about that. There used to be
>>
>>
>> some tables listing the methods making up the core sequence and
>>
>>
>> mapping protocols, but I can't find them now. Have they been removed?
>>
>>
>> Are the ABCs now the definitions of the protocols?
>>
>
>  I would also like to see an official answer to this. Are the mixin
> methods of the ABCs considered part of their contract? Is it documented
> somewhere? I think it should be.
>
> On the one hand it sort of feels like the contract is just the abstract
> methods that users are supposed to implement themselves, and the mixin
> methods are just a convenient benefit of subclassing. It's hard to say
> where that feeling comes from, but I think the name "mixin method" is part
> of it. I'm glad to see I'm not the only one that gets this feeling.
>
> On the other hand, if I have a variable of type Mapping, I'm usually going
> to assume it has a .get method that behaves like dict. Most tools will
> assume that too. If it doesn't, I'll be quite surprised and probably
> annoyed.
>
>
>
>
> ___
>
> Python-ideas mailing list -- python-ideas@python.org
>
> To unsubscribe send an email to python-ideas-le...@python.org
>
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/UCJXMBLFIFLTB7ITAW6UQHW6JBMZKHDP/
>
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> --
--Guido (mobile)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/GQNROHDAWECOXNBOGRJEUPE7UEY5UJ4T/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Changing item dunder method signatures to utilize positional arguments (open thread)

On Wed, Aug 26, 2020 at 9:48 AM Steven D'Aprano 
wrote:

> Demonstration:
>
>
> py> class Dynamic:
> ... def __getitem__(self, arg):
> ... print("original")
> ... type(self).__getitem__ = lambda *args:
> print("replaced")
> ...
> py>
> py> x = Dynamic()
> py> x[0]
> original
> py> x[0]
> replaced
>

I see what you're saying there. So any solution involving `type` of
`object` wrapping a child class method at class creation time should
probably not be considered as a cpython implementation option because of
the dynamic nature of classes. Understood.

>  > Sure, it might break some code because somebody, somewhere, has a
> > __getitem__ method written like this:
> >
> > def __getitem__(self, a, b=None, c=None): ...
> [...]
> > But are you really saying there is a very important responsibility not to
> > break that person's code?
>
> Yes.
>

Alright, you've made progress with me on this one. I'm still not totally
convinced such code isn't asking to be broken, but you're right: just
dismissing that as a nonissue isn't correct. It has to be weighed.

> > > Maybe we wouldn't have designed subscripting this way back in Python 1
> > > if we know what we know now, but it works well enough, and we have
> heard
> > > from numpy developers like Stephan Hoyer that this is not a problem
> that
> > > needs fixing. Can we please stop trying to "fix" positional subscripts?
> > >
> >
> > I'm not done trying, sorry. I think the incongruity is a problem.
>
> A problem for whom?
>
> A problem in what way?
>
> PEP 472 goes into detail about the sorts of things people find really
> hard to do with subscripting because of the lack of keywords. Where is
> the discussion by people about the things that are really hard to do
> because of the comma handling in subscripts?
>

Well I will point to Greg Ewin's message from a while ago that I quoted at
the start of this thread:

On Tue, Aug 4, 2020, 2:57 AM Greg Ewing  wrote:

On 4/08/20 1:16 pm, Steven D'Aprano wrote:
> Why would we want to even consider a new approach to handling keyword
> arguments which applies only to three dunder methods, `__getitem__`,
> `__setitem__` and `__delitem__`, instead of handling keyword arguments
> in the same way that every other method handles them?

These methods are already kind of screwy in that they don't
handle *positional* arguments in the usual way -- packing them
into a tuple instead of passing them as individual arguments.

I think this is messing up everyone's intuition on how indexing
should be extended to incorporate keyword args, or even whether
this should be done at all.

-- 
Greg

If we simply add kwarg passing to item dunders, the incongruity between the
subscripting operator and regular function calls will really hinder
people's ability to understand the two. I think most people in these
conversations seem to now agree that this isn't a good enough reason to
exercise the do nothing option, but I think it is a very good reason to
explore other ways of doing it to alleviate that incongruity.

---
Ricky.

"I've never met a Kentucky man who wasn't either thinking about going home
or actually going home." - Happy Chandler
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VR6TP5QBLUI7BD6WR4AFGLD7UNGFLX66/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

On Wed, Aug 26, 2020 at 7:58 AM Chris Angelico  wrote:

> -1. We have already had way WAY too much debate about the alternative
> ways to handle kwargs in subscripts, and quite frankly, I don't think
> this is adding anything new to the discussion. Can we just stop
> bikeshedding this already and let this matter settle down? You still
> aren't showing any advantage over just allowing the current dunder to
> receive kwargs.
>
> ChrisA
>

On Wed, Aug 26, 2020, 8:02 AM Steven D'Aprano  wrote:

> Please don't hijack an existing PEP for an unrelated issue.
>
> PEP 472 is an existing PEP for "Support for indexing with keyword
> arguments".
>
> https://www.python.org/dev/peps/pep-0472/
>
> If you want to write a competing PEP, see PEP 1 for the steps you must
> follow:
>
> https://www.python.org/dev/peps/pep-0001

I think Jonathan included the name of pep 472 in the subject line just
because it's related to the previous conversations that have been going
on, not to hijack the PEP for a different purpose.

I understand that Steve has said he is very grumpy about all of this
hijacking, derailing, bikeshedding, or whatever other metaphors are
appropriate for the alternative ideas that are being proposed for future
functionality that he cares about and wants to use. Perhaps this applies to
Chris and others as well.

But I have to say that I think this latest is a fantastic idea, and when
Jonathan presented it to me it was very surprising that I had not seen it
presented by anyone else yet. I think it solves a ton of problems, adds a
huge amount of flexibility and functionality, and as such has the potential
to be very powerful.

There are at least three substantive objections I see to this:

1. It will slow down subscripting
2. It adds complexity
3. It actually doesn't add anything helpful/useful

*Objection 1: Slowing Things Down*

The INTENDED EFFECT of the changes to internals will be as Jonathan Fine
described: every time a subscript operation occurs, this new dunder
attribute gets investigated on the class, and if it is not present then the
default key translation function is executed.

If things were implemented in exactly that way, obviously it would slow
everything down a lot. Every subscripting operation gets slowed down
everywhere and that's probably not an option.

However, for actual implementation, there's no reason to do that. Instead
we could wrap the existing item dunder methods automatically at class
creation time only when the new dunder attribute is present. If it is not
present, nothing happens. In other words, what has been described as the
"default key or index translation function" already exists. Let's not
change that at all for classes that do not choose to use it.

This would have the same effect as the proposal, but it would avoid any
slowdown when a class has not provided the new attribute.

Also note that this can coexist alongside Steve's and Chris's preferred
solution, which is to just add kwarg passing to the item dunders. That
change would constitute and adjustment to the "default key or index
translation function", which is sort of like this (from Jonathan's first
message):

def internal_get_function(self, *argv, **kwargs):
if kwargs:
raise SyntaxError
if len(argv) == 1:
key = argv[0]
else:
key = argv
type(self).__getitem__(self, key)

*Objection 2: Adds complexity*

This is hard to argue with. It certainly does add complexity. The question
is whether the complexity is worth the added benefits.

But in the interest of arguing that the additional complexity is not THAT
onerous, I will point out that in order to add flexibility, complexity is
nearly always necessitated.

*Objection 3: Doesn't add anything useful/helpful*

This objection seems obviously false.

With the proposal, the language would support any function desired to turn
the "stuff" inside a subscripting operation into the item dunder calls.

For example: if this proposal were already in place and PEP 472 were to
continue to be held up because of terrorists like me ;) *, one could have
written this translation function and PEP-472-ify their classes already:

def yay_kwargs(self, *args, **kwargs):
return self.__getitem__(args, **kwargs)

But another person, who doesn't like PEP 472, could do something else:

def boo_kwargs(self, *args, **kwargs):
if kwargs:
raise SyntaxError("I prefer to live in the past.")
return self.__getitem__(args)

* I'm trying to be light, I'm really not offended by Steve's or Chris's
comments, and I really am earnestly trying to offer something helpful here,
not hold things up for people. I care about adding kwargs to the subscript
operator, too.

>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at

[Python-ideas] Re: Changing item dunder method signatures to utilize positional arguments (open thread)

On Tue, Aug 25, 2020 at 10:51:42PM -0400, Ricky Teachey wrote:

> > The only way it could tell that would be to inspect *at runtime* the
> > `__setitem__` method. And it would have to do this on every subscript
> > call. Introspection is likely to be slow, possibly very slow. This would
> > make subscripting slow.
> >
> 
> Well it would not have to inspect the signature on every subscript call,

Python is a very dynamic language and objects can change their own 
methods and even their class at any time, so, yes, it will have to 
inspect the signature on every call. Otherwise you are changing 
the language execution model.

Demonstration:

py> class Dynamic:
... def __getitem__(self, arg):
... print("original")
... type(self).__getitem__ = lambda *args: print("replaced")
... 
py> 
py> x = Dynamic()
py> x[0]
original
py> x[0]
replaced

[...]
> People have actually been asking for ways to make subscripting operator act
> more like a function call, so that's not true. 

Yes, people have asked for keyword arguments. This proposal doesn't get 
us any closer to the feature wanted.

> And it could be useful.

It doesn't give subscripting any additional functionality that doesn't 
already exist. It's a purely cosmetic change.

> And
> it does help address the problem of incongruity (though not perfectly)
> between the way a function call handles args and kwargs, and the way the
> subscript operator does it. And it is totally backwards compatible except
> for the case
[...]

So not actually totally backwards compatible.

Please stop calling things "totally backwards compatible" if they aren't 
totally backwards compatible. If you change the behaviour of existing 
legal code, it's not backwards compatible.

> > Maybe we wouldn't have designed subscripting this way back in Python 1
> > if we know what we know now, but it works well enough, and we have heard
> > from numpy developers like Stephan Hoyer that this is not a problem that
> > needs fixing. Can we please stop trying to "fix" positional subscripts?
> >
> 
> I'm not done trying, sorry. I think the incongruity is a problem.

A problem for whom?

A problem in what way?

PEP 472 goes into detail about the sorts of things people find really 
hard to do with subscripting because of the lack of keywords. Where is 
the discussion by people about the things that are really hard to do 
because of the comma handling in subscripts?

[...]
> Remember: the signature dependent semantics proposal is to maintain
> backwards compatibility, 100%, for any code that has been following the
> very clear intent of the item dunder methods.

The Python language doesn't follow the rule "Do what we intend you to 
do", it follows the rule "this is how the language works, do whatever 
the language allows".

PEP 5 doesn't mention the word "intent" and doesn't say "its okay to 
break people's code if they are using Python in ways we don't like".

You don't get to say that breaking legal code that works today is okay 
because it doesn't follow "the intent". It's still a breaking change and 
you have to convince the Steering Council that breaking other people's 
code for the sake of fixing an incongruity is worthwhile.

Imagine it was code *you* relied on that broke, and you no longer had 
the source code for it, it was a library compiled away in a .pyc file 
and the company that distributed it has gone broke and the source code 
lost, and replacing that library is going to cost your business a year 
of time and a hundred thousand dollars in development costs.

But that's okay, because an incongruity that makes no difference to 
anyone's code has been fixed.

Now maybe you can convince the Steering Council that this is a clear win 
for the broader community. We do break backwards compatibility, 
sometimes, if the risks are small enough and the benefits large enough. 
But be honest about what you are doing.

> Sure, it might break some code because somebody, somewhere, has a
> __getitem__ method written like this:
> 
> def __getitem__(self, a, b=None, c=None): ...
[...]
> But are you really saying there is a very important responsibility not to
> break that person's code? 

Yes.

-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/B3ZY3BS6DHB23HNVERKXYVGXB2TSM56P/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Changing item dunder method signatures to utilize positional arguments (open thread)

On Wed, Aug 26, 2020 at 03:06:25PM +1200, Greg Ewing wrote:
> On 26/08/20 1:59 pm, Steven D'Aprano wrote:
> >Most existing uses of subscripts already don't fit that key:value
> >mapping idea, starting with lists and tuples.
> 
> Not sure what you mean by that.

Lists and tuples aren't key:value mappings.

> >Given `obj[spam]`, how does the interpreter know whether to call
> >`__getitem__` or `__getindex__`? What if the class defines both?
> 
> If it has a __getindex__ method, it calls that using normal function
> parameter passing rules. Otherwise it uses a fallback something like

Presumably the below method is provided by `object`. If not, what 
provides it?

> def __getindex__(self, *args, **kwds):
> if kwds:
> raise TypeError("Object does not support keyword indexes")
> if not args:
> raise TypeError("Object does not accept empty indexes")

Empty subscripts will remain a syntax error, so I don't think this case 
is possible.

What is your reasoning behind prohibiting keywords, when we're right in 
the middle of a discussion over PEP 474 which aims to allow keywords?

> if len(args) == 1:
> args = args[0]
> return self.__getitem__(args)

This is going to slow down the most common cases of subscripting: the 
interpreter has to follow the entire MRO to find `__getindex__` in 
object, which then dispatches to the `__getitem__` method.

It would make more sense to put the logic in the byte-code, rather than 
`object`:

if the object supports __getindex__ call it
otherwise call __getitem__

which is closer to how other operators work, and avoids giving object a 
spurious dunder that it doesn't use. However it still has to check the 
MRO, so I don't know if that actually gains us much performance.

The fundamental issue with this proposal is that, as far as I can see, 
it solves no problems and offers no new functionality. It just adds 
complexity.

As Stephan Hoyer has said, the existing handling of commas in subscripts 
is not a problem for Numpy. I doubt it is a problem for Pandas either. 
Is it a problem for anyone?

> >Right now, both sets of syntax mean the same thing and call the same
> >method, so you are introducing a backwards incompatible change that will
> >break code.
> 
> No existing object will have a __getindex__ method[1], so it won't
> change any existing behaviour.

In your earlier statement, you said that it would be possible for 
subscripting to mean something different depending on whether the 
comma-separated subscripts had parentheses around them or not:

obj[(2, 3)]
obj[2, 3]

How does that happen? I thought I understood what you meant by 
that, but I obviously didn't. Can you explain how they would be 
different?

I understood that you meant that the intepreter would choose which 
dunder to call according to whether or not there are parentheses around 
the items, but now I'm not sure what you meant.

-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4XHPAWCXT3INKOZHSW5JLUKE3A2M5Q4X/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

Please don't hijack an existing PEP for an unrelated issue.

PEP 472 is an existing PEP for "Support for indexing with keyword 
arguments".

https://www.python.org/dev/peps/pep-0472/

If you want to write a competing PEP, see PEP 1 for the steps you must 
follow:

https://www.python.org/dev/peps/pep-0001




-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/S2ZLDZN3KHUNMMFG6R4UNQWF5PYV7DHY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

2020-08-26 Thread Chris Angelico

On Wed, Aug 26, 2020 at 9:44 PM Jonathan Fine  wrote:
>
> PROPOSAL
> I think it will help solve our problem, to give Z = type(z) a new dunder 
> attribute that either is used as the internal_get_function, or is used inside 
> a revised system-wide internal_get_function.
>
> That way, depending on the new dunder attribute on Z = type(z), sometimes
> >>> z[1, 2, a=3, b=4]
> >>> z.__getitem__(1, 2, a=3, b=4)
> are equivalent. And sometimes
> >>> z[1, 2, a=3, b=4]
> is equivalent to
> >>> key = K(1, 2, a=3, b=4)
> >>> z.__getitem__(key)
> all depending on the new dunder attribute on Z = type(z).
>

-1. We have already had way WAY too much debate about the alternative
ways to handle kwargs in subscripts, and quite frankly, I don't think
this is adding anything new to the discussion. Can we just stop
bikeshedding this already and let this matter settle down? You still
aren't showing any advantage over just allowing the current dunder to
receive kwargs.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UCYMGB4BKZK7HHUNEBA6OIMCGIZFRXMA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] PEP 472 - new dunder attribute, to influence item access

2020-08-26 Thread Jonathan Fine

SUMMARY
Sequences and mappings are both similar and different. Let's introduce and
use a new dunder attribute, to give item access appropriate behaviour. This
proposal is based on an earlier thread - see Acknowledgments for URL.

INTRODUCTION
In Python, there are two sorts of builtin objects that have item access.
They are mappings and sequences. They have different abstract base classes.
Mappings and sequences have fundamental similarities, and also fundamental
differences.

To see an example of this, let's consider
>>> x = dict() # Mapping
>>> y = [None] # Sequence

Consider now
>>> x[0] = 'hi'
>>> x[0] == 'hi'
True
>>> 0 in x, 'hi' in x
(True, False)

Compare it with
>>> y[0] = 'hi' # Similar
>>> y[0] == 'hi'
True
>>> 0 in y, 'hi' in y # Different
(False, True)

THE PROBLEM
Not taking into account the fundamental differences between mappings and
sequences can, I think, cause of difficulty when considering the semantics
of
>>> z[1, 2, a=3, b=4]

If z is a sequence (or multi-dimensional array) then many of us would like
to think of item access as a function call. In other words, ideally,
>>> z[1, 2, a=3, b=4]
>>> z.__getitem__(1, 2, a=3, b=4)
are to be equivalent.

But if z is a mapping, then perhaps ideally we'd like an object
>>> key = K(1, 2, a=3, b=4)
such that
>>> z[1, 2, a=3, b=4]
>>> z.__getitem__(key)
are equivalent.

PRESENT BEHAVIOUR
At present
>>> z[1, 2, a=3, b=4]
roughly speaking calls
>>> internal_get_function(z, 1, 2, a=3, b=4)
where we have something like
def internal_get_function(self, *argv, **kwargs):
if kwargs:
raise SyntaxError
if len(argv) == 1:
key = argv[0]
else:
key = argv
type(self).__getitem__(self, key)

PROPOSAL
I think it will help solve our problem, to give Z = type(z) a new dunder
attribute that either is used as the internal_get_function, or is used
inside a revised system-wide internal_get_function.

That way, depending on the new dunder attribute on Z = type(z), sometimes
>>> z[1, 2, a=3, b=4]
>>> z.__getitem__(1, 2, a=3, b=4)
are equivalent. And sometimes
>>> z[1, 2, a=3, b=4]
is equivalent to
>>> key = K(1, 2, a=3, b=4)
>>> z.__getitem__(key)
all depending on the new dunder attribute on Z = type(z).

I hope this contribution helps.

ACKNOWLEDGEMENTS
I've had much valuable help from Ricky Teachey in preparing this message.
I've also been influenced by his and others contributions to an earlier
thread, which he started 3 weeks ago.
https://mail.python.org/archives/list/python-ideas@python.org/thread/FFXXO5NNUTMDKDAWIQS7JCHPA27Y7637/#FFXXO5NNUTMDKDAWIQS7JCHPA27Y7637

-- 
Jonathan
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IN5W3H2JLFNUCVRNC7I56ZOWACAG5OFW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: What about having a .get(index, default) method for arrays like we have for dicts?

On Wed, Aug 26, 2020 at 1:41 AM Steven D'Aprano  wrote:

> On Tue, Aug 25, 2020 at 10:22:20AM -0700, Christopher Barker wrote:
>
> > This one is easier than most because it's pretty much a do we or don't we
> > question, with the spelling semantics, all pretty much already decided.
>
> Is it to be added to lists, or lists and tuples? How about range
> objects?
>
> range(1, 9, 2).get(100)
>
> Strings and bytes?
>
> "abc".get(100)
>

I think at a minimum it should be in lists and tuples.

I can't see any reason not to add it to range, strings, bytes, deque,
array, etc. beyond implementation effort. I also think it's fine to leave
them out at first.

> Do we require 3rd party sequences to add this as well, in order to
> continue being called sequences?
>

I think this is pretty much the same question as the discussion about ABCs.
It seems like the answer is "no", or at least "not yet".

> > Though I haven't quite seen it said explicitly -- is this proposed to be
> > added to the Sequence ABC?
>
> If so, do we require that it return None as the default, or are types
> permitted to return whatever missing value they see fit?
>

This should be identical to Mapping.get. None isn't required to be the
default missing value - callers can specify any value they want - but None
is the default default, i.e. `def get(self, item, default=None)`. I don't
know of any Mapping or mapping-like class that has a different default
default. If such classes exist, then I guess None is just the default
default default :)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YWXT5AN2ADFEVEYR34Q2JMFDW6GNO4RG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: What about having a .get(index, default) method for arrays like we have for dicts?