Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Sjoerd Job Postmus]

On Thu, Oct 27, 2016 at 03:27:07AM +1100, Steven D'Aprano wrote:
> I think that there is zero hope of consistency for * the star operator. 
> That horse has bolted. It is already used for:
> 
> - ...
> - "zero or more of the previous element" in regular expressions
> - "zero or more of any character" in globbing
> - ...

After having read this multiple times, I still can't really understand
why these two matter to the discussion at hand. It's also used to mark
emphasised text in Markdown, lists in Markdown. You can also use it for
footnotes in plain text. It also has a special meaning in robots.txt
files.

Yes, I agree with you that the meaning of the asterisk symbol is quite
overloaded on the syntax level already. But I think that mentioning
regexes and globbing is a bit of a red herring.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Rob Cliffe]

On 14/10/2016 07:00, Greg Ewing wrote:

Neil Girdhar wrote:
At the end of this discussion it might be good to get a tally of how 
many people think the proposal is reasonable and logical.


I think it's reasonable and logical.


I concur.  Two points I personally find in favour, YMMV:
(1) [*subseq for subseq in seq] avoids the "conceptual hiatus" I 
described earlier in [elt for subseq in seq for elt in subseq]
  (I.e. I think the case for the proposal would be weaker if the 
loops in a list comprehension were written in reverse order.)
(2) This is admittedly a somewhat tangential argument, but: I didn't 
really know what "yield from" meant.  But when I read in an earlier post 
that someone had proposed "yield *" for it, I had a Eureka moment.  
Which suggests if  "*" is used to mean some sort of unpacking in more 
contexts, the more familiar and intuitive it may become.  I guess the 
word I'm groping for is 'consistency'.

Rob Cliffe
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Steven D'Aprano]

On Sat, Oct 15, 2016 at 05:38:15PM +, Neil Girdhar wrote:

> In ast.c, you can find:
> 
> if (is_dict) {
> ast_error(c, n, "dict unpacking cannot be used in "
> "dict comprehension");
> return NULL;
> }
> res = ast_for_dictcomp(c, ch);

[...]

Thanks for the pointer.


> > In any case, it isn't really the difficulty of implementation that
> > is being questioned. Many things are easy to implement, but we still
> > don't do them.
> 
> If it doesn't matter, why bring it up?

I never said it doesn't matter.

I brought it up because I'm curious. Because if it turns out that it 
actually is *difficult* to implement, that would be a point in my favour 
that *t doesn't naturally apply in list comps. And on the other hand, if 
it is *easy* to implement, that's a hint that perhaps I'm missing 
something and there is some natural interpretation of *t in a list comp 
that I've missed. Perhaps.

Just because I have a strong opinion doesn't mean I'm not willing to 
consider the possibility that I'm wrong.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Steven D'Aprano]

On Thu, Oct 13, 2016 at 01:30:45PM -0700, Neil Girdhar wrote:

> From a CPython implementation standpoint, we specifically blocked this code 
> path, and it is only a matter of unblocking it if we want to support this.

I find that difficult to believe. The suggested change seems like it 
should be much bigger than just removing a block. Can you point us to 
the relevant code?

In any case, it isn't really the difficulty of implementation that is 
being questioned. Many things are easy to implement, but we still 
don't do them. The real questions here are:

(1) Should we overload list comprehensions as sugar for a flatten() 
function?

(2) If so, should we spell that [*t for t in iterable]?


Actually the answer to (1) should be "we already do". We just spell it:

[x for t in iterable for x in t]



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Greg Ewing]

Neil Girdhar wrote:
At the end of this discussion it might be good to get a tally of how 
many people think the proposal is reasonable and logical.


I think it's reasonable and logical.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[David Mertz]

I've never used nor taught a * in a list display. I don't think they seem
so bad, but it's a step down a slippery slope towards forms that might as
well be Perl.

On Oct 13, 2016 10:33 PM, "Greg Ewing"  wrote:

> David Mertz wrote:
>
>> it would always be "Here's a Python wart to look out for if you see it in
>> other code... you should not ever use it yourself."
>>
>
> Do you currently tell them the same thing about the use
> of * in a list display?
>
> --
> Greg
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Greg Ewing]

David Mertz wrote:
it would always be "Here's a Python wart to look out 
for if you see it in other code... you should not ever use it yourself."


Do you currently tell them the same thing about the use
of * in a list display?

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[David Mertz]

Exactly with Paul!

As I mentioned, I teach software developers and scientists Python for a
living.  I get paid a lot of money to do that, and have a good sense of
what learners can easily understand and not (I've also written hundred of
articles and a few books about Python).  The people I write for and teach
are educated, smart, and generally have familiarity with multiple
programming languages.

In my opinion, this new construct—if added to the language—would be
difficult to teach, and most of my students would get it wrong most of the
time.

Yes, I understand the proposed semantics.  It is not *intuitive* to me, but
I could file the rule about the behavior if I had to.  But if I were forced
to teach it, it would always be "Here's a Python wart to look out for if
you see it in other code... you should not ever use it yourself."

On Thu, Oct 13, 2016 at 8:18 AM, Paul Moore  wrote:

> On 13 October 2016 at 15:32, Sven R. Kunze  wrote:
> > Steven, please. You seemed to struggle to understand the notion of the
> > [*] construct and many people (not just me) here tried their best to
> > explain their intuition to you.
>
> And yet, the fact that it's hard to explain your intuition to others
> (Steven is not the only one who's finding this hard to follow) surely
> implies that it's merely that - personal intuition - and not universal
> understanding.
>
> The *whole point* here is that not everyone understands the proposed
> notation the way the proposers do, and it's *hard to explain* to those
> people. Blaming the people who don't understand does not support the
> position that this notation should be added to the language. Rather,
> it reinforces the idea that the new proposal is hard to teach (and
> consequently, it may be a bad idea for Python).
>
> Paul
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Steven D'Aprano]

On Thu, Oct 13, 2016 at 03:28:27PM +, אלעזר wrote:

> It may also suggest that there are currently two ways to understand the
> *[...] construct, 

This thread is about allowing sequence unpacking as the internal 
expression of list comprehensions:

[ *(expr) for x in iterable ]

It isn't about unpacking lists:

*[...]

so I don't see what relevance your comment has.

There may be two or three or ten or 100 ways to (mis)understand list 
comprehensions in Python, but only one of them is the correct way. List 
comprehensions are (roughly) syntactic sugar for:

result = []
for x in iterable:
result.append(expression)

Any other understanding of them is incorrect.

Now if people wish to make an argument for changing the meaning of 
comprehensions so that the suggested internal unpacking makes sense, 
then by all means try making that argument! That's absolutely fine.

In the past, I've tried a similar thing: I argued for a variant list 
comprehension that halts early:

[expr for x in iterable while condition]

(as found in at least one other language), but had that knocked back 
because it doesn't fit the existing list comprehension semantics. I 
wasn't able to convince people that the value of this new comprehension 
was worth breaking the existing semantics of comprehensions.

Maybe you will be able to do better than me.

But understand that:

[*(expr) for x in iterable] 

also fails to fit the existing list comprehension semantics. To make it 
work requires changing the meaning of Python list comps. It isn't enough 
to just deny the existing meaning and insist that your own personal 
meaning is correct.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Paul Moore]

On 13 October 2016 at 15:32, Sven R. Kunze  wrote:
> Steven, please. You seemed to struggle to understand the notion of the
> [*] construct and many people (not just me) here tried their best to
> explain their intuition to you.

And yet, the fact that it's hard to explain your intuition to others
(Steven is not the only one who's finding this hard to follow) surely
implies that it's merely that - personal intuition - and not universal
understanding.

The *whole point* here is that not everyone understands the proposed
notation the way the proposers do, and it's *hard to explain* to those
people. Blaming the people who don't understand does not support the
position that this notation should be added to the language. Rather,
it reinforces the idea that the new proposal is hard to teach (and
consequently, it may be a bad idea for Python).

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Sven R. Kunze]

On 13.10.2016 16:10, Steven D'Aprano wrote:

On Thu, Oct 13, 2016 at 10:37:35AM +0200, Sven R. Kunze wrote:
Multiplication with only a single argument? Come on. 

You didn't say anything about a single argument. Your exact words are
shown above: "where have I seen * so far?". I'm pretty sure you've seen
* used for multiplication. I also could have mentioned regexes, globs,
and exponentiation.

I cannot respond to your intended meaning, only to what you actually
write. Don't blame the reader if you failed to communicate clearly and
were misunderstood.


Steven, please. You seemed to struggle to understand the notion of the 
[*] construct and many people (not just me) here tried their best to 
explain their intuition to you. But now it seems you don't even try to 
come behind the idea and instead try hard not to understand the help 
offered. If you don't want help or don't really want to understand the 
proposal, that's fine but please, do us a favor and don't divert the 
thread with nitpicking nonsensical details (like multiplication) and 
waste everybody's time.



The context of the proposal is about lists/dicts and the */** unpacking 
syntax. So, I actually expect you to put every post here into this very 
context. Discussions without context don't make much sense. So, I won't 
reply further.



Best,
Sven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[אלעזר]

On Thu, Oct 13, 2016 at 5:10 PM Steven D'Aprano  wrote:

> On Thu, Oct 13, 2016 at 10:37:35AM +0200, Sven R. Kunze wrote:
> > About the list constructor: we construct a list by writing [a,b,c] or by
> > writing [b for b in bs]. The end result is a list
>
> I construct lists using all sorts of ways:
>

I think there is a terminology problem here (again). "Constructor" in OOP
has a specific meaning, and "constructor" in functional terminology has a
slightly different meaning. I guess Sven uses the latter terminology
because pattern matching is the dual of the constructor - it is a
"destructor" - and it feels appropriate, although admittedly confusing. In
this terminology, map(), zip() etc. are definitely not constructors. there
is only one "constructor" (list()), and there are functions that may use it
as their implementation detail. In a way, [1, 2, 3] is just a syntactic
shorthand for list construction, so it is reasonable to a call it a
constructor.

This terminology is not a perfect fit into the object-oriented world of
Python, but it is very helpful in discussion of patterns how to apply them
uniformly, since they were pretty much invented in the functional world
(ML, I think, and mathematics). One only needs to be aware of the two
different meaning, and qualify if needed, so that we won't get lost in
terminology arguments again.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Steven D'Aprano]

On Thu, Oct 13, 2016 at 10:37:35AM +0200, Sven R. Kunze wrote:
> On 13.10.2016 01:29, Steven D'Aprano wrote:
> >On Wed, Oct 12, 2016 at 06:32:12PM +0200, Sven R. Kunze wrote:
> >>
> >>So, my reasoning would tell me: where have I seen * so far? *args and
> >>**kwargs!
> >And multiplication.
> 
> Multiplication with only a single argument? Come on.

You didn't say anything about a single argument. Your exact words are
shown above: "where have I seen * so far?". I'm pretty sure you've seen
* used for multiplication. I also could have mentioned regexes, globs,
and exponentiation.

I cannot respond to your intended meaning, only to what you actually 
write. Don't blame the reader if you failed to communicate clearly and 
were misunderstood.


[...]
> About the list constructor: we construct a list by writing [a,b,c] or by 
> writing [b for b in bs]. The end result is a list 

I construct lists using all sorts of ways:

list(argument)
map(func, sequence)
zip(a, b)
file.readlines()
dict.items()
os.listdir('.')
sorted(values)

and so on. Should I call them all "list constructors" just because they
return a list? No, I don't think so. Constructor has a specific meaning, 
and these are not all constructors -- and neither are list 
comprehensions.


> and that matters from 
> the end developer's point of view, no matter how fancy words you choose 
> for it.

These "fancy words" that you dismiss are necessary technical terms. 
Precision in language is something we should aim for, not dismiss as 
unimportant.

List comprehensions and list displays have different names, not to show 
off our knowledge of "fancy terms", but because they are different 
things which just happen to both return lists. Neither of them are what 
is commonly called a constructor. List displays are, in some senses, 
like a literal; list comprehensions are not, and are better understood 
as list builders, a process which builds a list. Emphasis should be on 
the *process* part: a comprehension is syntactic sugar for building a 
list using for-loop, not for a list display or list constructor.

The bottom line is that when you see a comprehension (list, set or dict) 
or a generator expression, you shouldn't think of list displays, but of 
a for-loop. That's one of the reasons why the analogy with argument 
unpacking fails: it doesn't match what comprehensions *actually* are.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Sven R. Kunze]

On 13.10.2016 01:29, Steven D'Aprano wrote:

On Wed, Oct 12, 2016 at 06:32:12PM +0200, Sven R. Kunze wrote:


So, my reasoning would tell me: where have I seen * so far? *args and
**kwargs!

And multiplication.


Multiplication with only a single argument? Come on.



And sequence unpacking.


We are on the right side of the = if any and not no the left side.




[...] is just the list constructor.

Also indexing: dict[key] or sequence[item or slice].


There's no name in front of [. So, I cannot be an index either.


Nothing else matches (in my head) and I also don't see any ambiguities. 
YMMV.


I remember a new co-worker, I taught how to use *args and **kwargs. It 
was unintuitive to him on the first time as well.


About the list constructor: we construct a list by writing [a,b,c] or by 
writing [b for b in bs]. The end result is a list and that matters from 
the end developer's point of view, no matter how fancy words you choose 
for it.


Cheers,
Sven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[אלעזר]

On Thu, Oct 13, 2016 at 2:35 AM Steven D'Aprano  wrote:

> On Wed, Oct 12, 2016 at 04:11:55PM +, אלעזר wrote:
>
> > Steve, you only need to allow multiple arguments to append(), then it
> makes
> > perfect sense.
>
> I think you're missing a step. What will multiple arguments given to
> append do? There are two obvious possibilities:
>
> - collect all the arguments into a tuple, and append the tuple;
>
> - duplicate the functionality of list.extend
>
>
> neither of which appeals to me.
>

The latter, of course. Similar to max(). Not unheard of.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Steven D'Aprano]

On Wed, Oct 12, 2016 at 04:11:55PM +, אלעזר wrote:

> Steve, you only need to allow multiple arguments to append(), then it makes
> perfect sense.

I think you're missing a step. What will multiple arguments given to 
append do? There are two obvious possibilities:

- collect all the arguments into a tuple, and append the tuple;

- duplicate the functionality of list.extend


neither of which appeals to me.


-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Steven D'Aprano]

On Wed, Oct 12, 2016 at 06:32:12PM +0200, Sven R. Kunze wrote:
> On 12.10.2016 17:41, Nick Coghlan wrote:
> >This particular proposal fails on the first question (as too many
> >people would expect it to mean the same thing as either "[*expr, for
> >expr in iterable]" or "[*(expr for expr in iterable)]")
> 
> So, my reasoning would tell me: where have I seen * so far? *args and 
> **kwargs! 

And multiplication. And sequence unpacking.

> [...] is just the list constructor. 

Also indexing: dict[key] or sequence[item or slice].

The list constructor would be either list(...) or possibly 
list.__new__.

[...] is either a list display:

[1, 2, 3, 4]

or a list comprehension. They are not the same thing, and they don't 
work the same way. The only similarity is that they use [ ] as 
delimiters, just like dict and sequence indexing. That doesn't mean that 
you can write:

mydict[x for x in seq if condition]

Not everything with [ ] is the same.


> So, putting those two pieces together is quite simple.

I don't see that it is simple at all. I don't see any connection between 
function *args and list comprehension loop variables.



> Furthermore, your two "interpretations" would yield the very same result 
> as [expr for expr in iterable] which doesn't match with my experience 
> with Python so far; especially when it comes to special characters. They 
> must mean something. So, a simple "no-op" would not match my expectations.

Just because something would otherwise be a no-op doesn't mean that it 
therefore has to have some magical meaning. Python has a few no-ops 
which are allowed, or required, by syntax but don't do anything.

   pass
   (x) # same as just x
   +1  # no difference between literals +1 and 1
   -0
   func((expr for x in iterable))  # redundant parens for generator expr

There may be more.



> >but it fails on the other two grounds as well.
> 
> Here I disagree with you. We use *args all the time, so we know what * 
> does. I don't understand why this should not work in between brackets [...].

By this logic, *t should work... everywhere?

while *args:
try:
raise *args
except *args:
del *args

That's not how Python works. Just because syntax is common, doesn't mean 
it has to work everywhere. We cannot write:


for x in import math:
...

even though importing is common.

*t doesn't work as the expression inside a list comprehension because 
that's not how list comps work. To make it work requires making this a 
special case and mapping 

[expr for t in iterable]

to a list append, while 

[*expr for t in iterable]

gets mapped to a list extend.

Its okay to want that as a special feature, but understand what you are 
asking for: you're not asking for some restriction to be lifted, which 
will then automatically give you the functionality you expect. You're 
asking for new functionality to be added.

Sequence unpacking inside list comprehensions as a way of flattening a 
sequence is completely new functionality which does not logically follow 
from the current semantics of comprehensions.



> >In most uses of *-unpacking it's adding entries to a comma-delimited
> >sequence, or consuming entries in a comma delimited sequence (the
> >commas are optional in some cases, but they're still part of the
> >relevant contexts). The expansions removed the special casing of
> >functions, and made these capabilities generally available to all
> >sequence definition operations.
> 
> I don't know what you mean by comma-delimited sequence. There are no 
> commas. It's just a list of entries. * adds entries to this list. (At 
> least from my point of view.)

Not all points of view are equally valid.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[אלעזר]

To be honest, I don't have a clear picture of what {**x for x in d.items()}
should be. But I do have such picture for

dict(**x for x in many_dictionaries)

Elazar

‪On Wed, Oct 12, 2016 at 11:37 PM ‫אלעזר‬‎  wrote:‬

> On Wed, Oct 12, 2016 at 11:26 PM David Mertz  wrote:
>
> On Wed, Oct 12, 2016 at 12:38 PM, אלעזר  wrote:
>
> What is the intuition behind [1, *x, 5]? The starred expression is
> replaced with a comma-separated sequence of its elements.
>
> I've never actually used the `[1, *x, 5]` form.  And therefore, of course,
> I've never taught it either (I teach Python for a living nowadays).  I
> think that syntax already perhaps goes too far, actually; but I can
> understand it relatively easily by analogy with:
>
>  a, *b, c = range(10)
>
>
> It's not exactly "analogy" as such - it is the dual notion. Here you are
> using the "destructor" (functional terminology) but we are talking about
> "constructors". But nevermind.
>
>
> But the way I think about or explain either of those is "gather the extra
> items from the sequence." That works in both those contexts.  In contrast:
>
> >>> *b = range(10)
> SyntaxError: starred assignment target must be in a list or tuple
>
> Since nothing was assigned to a non-unpacked variable, nothing is "extra
> items" in the same sense.  So failure feels right to me.  I understand that
> "convert an iterable to a list" is conceptually available for that line,
> but we already have `list(it)` around, so it would be redundant and
> slightly confusing.
>
>
> But that's not a uniform treatment. It might have good reasons from
> readability point of view, but it is an explicit exception for the rule.
> The desired behavior would be equivalent to
>
> b = tuple(range(10))
>
> and yes, there are Two Ways To Do It. I would think it should have been
> prohibited by PEP-8 and not by the compiler. Oh well.
>
> What seems to be wanted with `[*foo for foo in bar]` is basically just
> `flatten(bar)`.  The latter feels like a better spelling, and the recipes
> in itertools docs give an implementation already (a one-liner).
>
> We do have a possibility of writing this:
>
> >>>  [(*stuff,) for stuff in [range(-5,-1), range(5)]]
> [(-5, -4, -3, -2), (0, 1, 2, 3, 4)]
>
> That's not flattened, as it should not be.  But it is very confusing to
> have `[(*stuff) for stuff in ...]` behave differently than that.  It's much
> more natural—and much more explicit—to write:
>
> >>> [item for seq in [range(-5,-1), range(5)] for item in seq]
> [-5, -4, -3, -2, 0, 1, 2, 3, 4]
>
>
> The distinction between (x) and (x,) is already deep in the language. It
> has nothing to do with this thread
>
> >>> [1, *([2],), 3]
> [1, [2], 3]
> >>> [1, *([2]), 3]
> [1, 2, 3]
>
> So there. Just like in this proposal.
>
> Elazar.
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Sven R. Kunze]

On 12.10.2016 21:38, אלעזר wrote:


What is the intuition behind [1, *x, 5]? The starred expression is 
replaced with a comma-separated sequence of its elements.


The trailing comma Nick referred to is there, with the rule that [1,, 
5] is the same as [1, 5].




I have to admit that I have my problems with this "comma-separated 
sequence" idea. For me, lists are just collections of items. There are 
no commas involved. I also think that thinking about commas here 
complicates the matter.



What * does, it basically plugs in the items from the starred expression 
into its surroundings:


[*[1,2,3]] = [1,2,3]

Let's plug in two lists into its surrounding list:

[*[1,2,3], *[1,2,3]] = [1,2,3,1,2,3]

So, as the thing goes, it looks like as if * could just work anywhere 
inside those brackets:


[*[1,2,3] for _ in range(3)] = [*[1,2,3], *[1,2,3], *[1,2,3]] = 
[1,2,3,1,2,3,1,2,3]



I have difficulties to understand the problem of understanding the 
syntax. The * and ** variants just flow naturally whereas the "chain" 
equivalent is bit "meh".


Cheers,
Sven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Paul Moore]

On 12 October 2016 at 20:22, David Mertz  wrote:
> I've followed this discussion some, and every example given so far
> completely mystifies me and I have no intuition about what they should mean.

Same here.

On 12 October 2016 at 20:38, אלעזר  wrote:
> What is the intuition behind [1, *x, 5]? The starred expression is replaced
> with a comma-separated sequence of its elements.
>
> The trailing comma Nick referred to is there, with the rule that [1,, 5] is
> the same as [1, 5].
>
> All the examples follow this intuition, IIUC.

But intuition is precisely that - it's not based on rules, but on
people's instinctive understanding. When evaluating whether something
is intuitive, the *only* thing that matters is what people tell you
they do or don't understand by a given construct. And in this case,
people have been expressing differing interpretations, and confusion.
That says "not intuitive" loud and clear to me.

And yes, I find [1, *x, 5] intuitive. And I can't tell you why I find
it OK, but I find {**x for x in d.items()} non-intuitive. But just
because I can't explain it doesn't mean it's not true, or you can
"change my mind" about how I feel.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[David Mertz]

On Wed, Oct 12, 2016 at 12:38 PM, אלעזר  wrote:

> What is the intuition behind [1, *x, 5]? The starred expression is
> replaced with a comma-separated sequence of its elements.
>
I've never actually used the `[1, *x, 5]` form.  And therefore, of course,
I've never taught it either (I teach Python for a living nowadays).  I
think that syntax already perhaps goes too far, actually; but I can
understand it relatively easily by analogy with:

 a, *b, c = range(10)

But the way I think about or explain either of those is "gather the extra
items from the sequence." That works in both those contexts.  In contrast:

>>> *b = range(10)
SyntaxError: starred assignment target must be in a list or tuple

Since nothing was assigned to a non-unpacked variable, nothing is "extra
items" in the same sense.  So failure feels right to me.  I understand that
"convert an iterable to a list" is conceptually available for that line,
but we already have `list(it)` around, so it would be redundant and
slightly confusing.

What seems to be wanted with `[*foo for foo in bar]` is basically just
`flatten(bar)`.  The latter feels like a better spelling, and the recipes
in itertools docs give an implementation already (a one-liner).

We do have a possibility of writing this:

>>>  [(*stuff,) for stuff in [range(-5,-1), range(5)]]
[(-5, -4, -3, -2), (0, 1, 2, 3, 4)]

That's not flattened, as it should not be.  But it is very confusing to
have `[(*stuff) for stuff in ...]` behave differently than that.  It's much
more natural—and much more explicit—to write:

>>> [item for seq in [range(-5,-1), range(5)] for item in seq]
[-5, -4, -3, -2, 0, 1, 2, 3, 4]
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[אלעזר]

What is the intuition behind [1, *x, 5]? The starred expression is replaced
with a comma-separated sequence of its elements.

The trailing comma Nick referred to is there, with the rule that [1,, 5] is
the same as [1, 5].

All the examples follow this intuition, IIUC.

Elazar

בתאריך יום ד׳, 12 באוק' 2016, 22:22, מאת David Mertz ‏:

> I've followed this discussion some, and every example given so far
> completely mystifies me and I have no intuition about what they should mean.
>
> On Oct 12, 2016 8:43 AM, "Steven D'Aprano"  wrote:
>
> On Tue, Oct 11, 2016 at 02:42:54PM +0200, Martti Kühne wrote:
> > Hello list
> >
> > I love the "new" unpacking generalisations as of pep448. And I found
> > myself using them rather regularly, both with lists and dict.
> > Today I somehow expected that [*foo for foo in bar] was equivalent to
> > itertools.chain(*[foo for foo in bar]), which it turned out to be a
> > SyntaxError.
>
> To me, that's a very strange thing to expect. Why would you expect that
> unpacking items in a list comprehension would magically lead to extra
> items in the resulting list? I don't think that makes any sense.
>
> Obviously we could program list comprehensions to act that way if we
> wanted to, but that would not be consistent with the ordinary use of
> list comprehensions. It would introduce a special case of magical
> behaviour that people will have to memorise, because it doesn't follow
> logically from the standard list comprehension design.
>
> The fundamental design principle of list comps is that they are
> equivalent to a for-loop with a single append per loop:
>
> [expr for t in iterable]
>
> is equivalent to:
>
> result = []
> for t in iterable:
> result.append(expr)
>
>
> If I had seen a list comprehension with an unpacked loop variable:
>
> [*t for t in [(1, 'a'), (2, 'b'), (3, 'c')]]
>
>
> I never in a million years would expect that running a list
> comprehension over a three-item sequence would magically expand to six
> items:
>
> [1, 'a', 2, 'b', 3, 'c']
>
>
> I would expect that using the unpacking operator would give some sort
> of error, or *at best*, be a no-op and the result would be:
>
> [(1, 'a'), (2, 'b'), (3, 'c')]
>
>
> append() doesn't take multiple arguments, hence a error should be the
> most obvious result. But if not an error, imagine the tuple unpacked to
> two arguments 1 and 'a' (on the first iteration), then automatically
> packed back into a tuple (1, 'a') just as you started with.
>
> I think it is a clear, obvious and, most importantly, desirable property
> of list comprehensions with a single loop that they cannot be longer
> than the initial iterable that feeds them. They might be shorter, if you
> use the form
>
> [expr for t in iterable if condition]
>
> but they cannot be longer.
>
> So I'm afraid I cannot understand what reasoning lead you to
> expect that unpacking would apply this way. Wishful thinking
> perhaps?
>
>
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[אלעזר]

Steve, you only need to allow multiple arguments to append(), then it makes
perfect sense.

בתאריך יום ד׳, 12 באוק' 2016, 18:43, מאת Steven D'Aprano ‏<
st...@pearwood.info>:

> On Tue, Oct 11, 2016 at 02:42:54PM +0200, Martti Kühne wrote:
> > Hello list
> >
> > I love the "new" unpacking generalisations as of pep448. And I found
> > myself using them rather regularly, both with lists and dict.
> > Today I somehow expected that [*foo for foo in bar] was equivalent to
> > itertools.chain(*[foo for foo in bar]), which it turned out to be a
> > SyntaxError.
>
> To me, that's a very strange thing to expect. Why would you expect that
> unpacking items in a list comprehension would magically lead to extra
> items in the resulting list? I don't think that makes any sense.
>
> Obviously we could program list comprehensions to act that way if we
> wanted to, but that would not be consistent with the ordinary use of
> list comprehensions. It would introduce a special case of magical
> behaviour that people will have to memorise, because it doesn't follow
> logically from the standard list comprehension design.
>
> The fundamental design principle of list comps is that they are
> equivalent to a for-loop with a single append per loop:
>
> [expr for t in iterable]
>
> is equivalent to:
>
> result = []
> for t in iterable:
> result.append(expr)
>
>
> If I had seen a list comprehension with an unpacked loop variable:
>
> [*t for t in [(1, 'a'), (2, 'b'), (3, 'c')]]
>
>
> I never in a million years would expect that running a list
> comprehension over a three-item sequence would magically expand to six
> items:
>
> [1, 'a', 2, 'b', 3, 'c']
>
>
> I would expect that using the unpacking operator would give some sort
> of error, or *at best*, be a no-op and the result would be:
>
> [(1, 'a'), (2, 'b'), (3, 'c')]
>
>
> append() doesn't take multiple arguments, hence a error should be the
> most obvious result. But if not an error, imagine the tuple unpacked to
> two arguments 1 and 'a' (on the first iteration), then automatically
> packed back into a tuple (1, 'a') just as you started with.
>
> I think it is a clear, obvious and, most importantly, desirable property
> of list comprehensions with a single loop that they cannot be longer
> than the initial iterable that feeds them. They might be shorter, if you
> use the form
>
> [expr for t in iterable if condition]
>
> but they cannot be longer.
>
> So I'm afraid I cannot understand what reasoning lead you to
> expect that unpacking would apply this way. Wishful thinking
> perhaps?
>
>
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Nick Coghlan]

On 12 October 2016 at 23:58, Sven R. Kunze  wrote:
> Reading PEP448 it seems to me that it's already been considered:
> https://www.python.org/dev/peps/pep-0448/#variations
>
> The reason for not-inclusion were about concerns about acceptance because of
> "strong concerns about readability" but also received "mild support". I
> think your post strengthens the support given that you "expected it to just
> work". This shows at least to me that the concerns about
> readability/understandability are not justified much.

Readability isn't about "Do some people guess the same semantics for
what it would mean?", as when there are only a few plausible
interpretations, all the possibilities are going to get a respectable
number of folks picking them as reasonable behaviour.

Instead, readability is about:

- Do people consistently guess the *same* interpretation?
- Is that interpretation consistent with other existing uses of the syntax?
- Is it more readily comprehensible than existing alternatives, or is
it brevity for brevity's sake?

This particular proposal fails on the first question (as too many
people would expect it to mean the same thing as either "[*expr, for
expr in iterable]" or "[*(expr for expr in iterable)]"), but it fails
on the other two grounds as well.

In most uses of *-unpacking it's adding entries to a comma-delimited
sequence, or consuming entries in a comma delimited sequence (the
commas are optional in some cases, but they're still part of the
relevant contexts). The expansions removed the special casing of
functions, and made these capabilities generally available to all
sequence definition operations.

Comprehensions and generator expressions, by contrast, dispense with
the comma delimited format entirely, and instead use a format inspired
by mathematical set builder notation (just modified to use keywords
and Python expressions rather than symbols and mathematical
expressions): 
https://en.wikipedia.org/wiki/Set-builder_notation#Sets_defined_by_a_predicate

However, set builder notation doesn't inherently include the notion of
flattening lists-of-lists. Instead, that's a *consumption* operation
that happens externally after the initial list-of-lists has been
built, and that's exactly how it's currently spelled in Python:
"itertools.chain.from_iterable(subiter for subiter in iterable)".

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/