Re: [Python-ideas] Order of loops in list comprehension

[Greg Ewing]

Alexander Heger wrote:
For me the current behaviour does not seem unreasonable as it resembles 
the order in which you write out loops outside a comprehension


That's true, but the main reason for having comprehensions
syntax in the first place is so that it can be read
declaratively -- as a description of the list you want,
rather than a step-by-step sequence of instructions for
building it up.

If you have to stop and mentally transform it into nested
for-statements, that very purpose is undermined.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Python multi-dimensional array constructor

[Chris Barker]

On Wed, Oct 19, 2016 at 5:32 PM, Todd  wrote:

> If there is a problem with the current options (and I'm not convinced
>> there is) it's that it in'st a literal for multidimensional array, but
>> rather a literal for a bunch of nested lists -- the list themselves are
>> created, and so are all the "boxed" values in the array -- only to be
>> pulled out and unboxed to be put in the array.
>>
>>
> But as you said, that is not a multidimensional array.  We aren't
> comparing "a = [| 0, 1, 2 || 3, 4, 5 |]" and "a = [[0, 1, 2],[3, 4, 5]]",
> we are comparing "a = [| 0, 1, 2 || 3, 4, 5 |]" and "a = np.array([[0, 1,
> 2],[3, 4, 5]])".  That is a bigger difference.
>

Well then, you have mixed two proposals here:

1) a literal syntax for nd arrays -- that is not going to fly if there is
NO ndarray object builtin to python. I kinda think there should be, though
even then there need not be a literal for it (see Decimal). So I'd say --
get an nd array object into the standard library first, then we can talk
about the literal

2) what the syntax should be for such a literal. OK, in this case,
suggested that the way to hash that out is to start out with passing a
string to a function that constructs the array -- then you could try things
out without any additions to the language or the libraries -- it could be a
stand-alone module that extends numpy:

from ndarray_literal import nda

my array = nda('||| 3, 4, 5 || 6, 7, 8 |||')

(is that legal in your notation -- I honestly am not sure)

and yes, that still requires typing "nda('", which you are trying to avoid.
But honestly, I really have written a lot of numpy code, and writing:

np.array( . )

does not bother me at all. IF I did support a literal, it would be so that
the object could be constructed immediately rather than by creating other
python objects first (liss, usually), and then making an array from that.


If you do want to push the syntax idea further, I'd suggest going to the
numpy list and seeing what folks there think.

But as I can't help myself. It's clear from the other posts on the list
here that others find your proposed syntax as confusing as I do. but maybe
it could be made more clear. Taking a page from MATLAB:

1 2 3; 4 5 6

is a 2x3 2-d array. no in MATLAB, there only used to be matrixes, so this
was pretty nice, but a bit hard to extend to multiple dimensions. But the
principle is handy: one delimter for the first dimension,l a nother one for
the second, etc..

we probably dont want to go with trying colons, and ! and who knows what
else, so I liek your idea.

a 2-d array:

1 | 2 | 3 || 4 | 5 | 6

(Or better)

1 | 2 | 3 ||
4 | 5 | 6

a 3d array:

0 |  1 |  2 |  3 ||

4 |  5 |  6 |  7 ||

8 |  9 | 10 | 11 |||


12 | 13 | 14 | 15||

16 | 17 | 18 | 19||

20 | 21 | 22 | 23||

Notes:

1) guess how I wrote that? I did: np.arange(24).reshape((2,3,4)) and edited
the results -- making the point that maybe the current state of affairs is
not so bad...

2) These are delimiters, rather than brackets -- so don't go at the
beginning and optional at the end (like commas in python)

3) It doesn't use commas as all, as having a consistent system is clearer

4) Whitespace is insignificant as with the rest of Python -- though you
want to be able to use a line ending as insignificant whitespace, so this
may have to wrapped in a parentheses, or something to actually use it --
though a non-issue if it's a string

Hmm -- about point (3), maybe use only commas:

0,  1,  2,  3,,
4,  5,  6,  7,,
8,  9, 10, 11,,,

12, 13, 14, 15,,
16, 17, 18, 19,,
20, 21, 22, 23

That would be more consistent with the rest of python, and multiple commas
in a row are currently a syntax error.

Even if your original data is large, I often need smaller areas when
> processing, for example for broadcasting or as arguments to processing
> functions.
>

sure I do hard-coded arrays all teh time -- but not big ones, and I don't
think I've ever needed more than 2D and certainly not more than 3D. and not
large enough that performance matters.

It is:
>>
>
> r_[[0, 1, 2], [3, 4, 5]
>

no, that's a shorthand for "row stack" -- and really not much better than
the array() call, except a few less characters

I meant the np.matrix() function that Alexander pointed out -- which is
only really there to make folks coming from MATLAB happier...(and it makes
a Matrix object, which you likely don't want). The point was that it's easy
to make such a beast for your new syntax to try it out

b = np.array([[ 0, 1, 2 ],
>   [ 3, 4, 5 ]])
>
> The whole point of this is to avoid the "np.array" call.
>

again, trying to separate out the idea of a literal, from the syntax of the
literal.

but thinking now, Python already uses (), [], {}, and < and > -- so I don't
think there are any more brackets. but why not just use commas with square
brackets:

2Darr = [1, 2, 3,,
 4, 5, 6]

maybe too subtle?

Yes, I understand that.  But some projects are already doing that on their
> 

Re: [Python-ideas] Fwd: Fwd: Fwd: unpacking generalisations for list comprehension

[Chris Angelico]

On Fri, Oct 21, 2016 at 6:09 AM, Random832  wrote:
> On Tue, Oct 18, 2016, at 02:10, Nick Coghlan wrote:
>> Hi, I contributed the current list comprehension implementation (when
>> refactoring it for Python 3 to avoid leaking the iteration variable,
>> as requested in PEP 3100 [1]), and "comprehensions are syntactic sugar
>> for a series of nested for and if statements" is precisely my
>> understanding of how they work, and what they mean.
>
> But it's simply not true. It has never been true and it will never be
> true. Something is not "syntactic sugar" if it doesn't compile to the
> exact same sequence of operations as the thing it is supposedly
> syntactic sugar for. It's a useful teaching tool (though you've
> eventually got to teach the differences), but claiming that it's
> "syntactic sugar" - and "non-negotiably" so, at that - implies that it
> is a literal transformation.

But it is. There are two caveats to the transformation: firstly, it's
done in a nested function (as of Py3), and secondly, the core
operations are done with direct opcodes rather than looking up the
".append" method; but other than that, yes, it's exactly the same.
Here's the disassembly (in 2.7, to avoid the indirection of the nested
function):

>>> def f1(x):
... return [ord(ch) for ch in x]
...
>>> dis.dis(f1)
  2   0 BUILD_LIST   0
  3 LOAD_FAST0 (x)
  6 GET_ITER
>>7 FOR_ITER18 (to 28)
 10 STORE_FAST   1 (ch)
 13 LOAD_GLOBAL  0 (ord)
 16 LOAD_FAST1 (ch)
 19 CALL_FUNCTION1
 22 LIST_APPEND  2
 25 JUMP_ABSOLUTE7
>>   28 RETURN_VALUE
>>> def f2(x):
... ret = []
... for ch in x:
... ret.append(ord(ch))
... return ret
...
>>> dis.dis(f2)
  2   0 BUILD_LIST   0
  3 STORE_FAST   1 (ret)

  3   6 SETUP_LOOP  33 (to 42)
  9 LOAD_FAST0 (x)
 12 GET_ITER
>>   13 FOR_ITER25 (to 41)
 16 STORE_FAST   2 (ch)

  4  19 LOAD_FAST1 (ret)
 22 LOAD_ATTR0 (append)
 25 LOAD_GLOBAL  1 (ord)
 28 LOAD_FAST2 (ch)
 31 CALL_FUNCTION1
 34 CALL_FUNCTION1
 37 POP_TOP
 38 JUMP_ABSOLUTE   13
>>   41 POP_BLOCK

  5 >>   42 LOAD_FAST1 (ret)
 45 RETURN_VALUE
>>>


Okay, so what exactly is going on here? Looks to me like there's some
optimization happening in the list comp, but you can see that the same
code is being emitted. It's not *perfectly* identical, but that's
mainly because CPython doesn't take advantage of the fact that 'ret'
was initialized to a list - it still does the full "look up 'append',
then call it" work. I'm not sure why SETUP_LOOP exists in the full
version and not the comprehension, but I believe it's to do with the
break and continue keywords, which can't happen inside a
comprehension. So, again, it's optimizations that are possible in the
comprehension, but otherwise, the code is identical.

Maybe "syntactic sugar" is pushing it a bit, but there's no
fundamental difference between the two. Imagine if an optimizing
compiler could (a) notice that there's no use of break/continue, and
(b) do some static type analysis to see that 'ret' is always a list
(and not a subclass thereof), and optimize the multi-line version. At
that point, the two forms would look almost, or maybe completely,
identical. So I'd support the "syntactic sugar" label here.

Why is this discussion still on python-ideas? Shouldn't it be on
python-demanding-explanations-for-status-quo by now?

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Order of loops in list comprehension

[Alexander Heger]

For me the current behaviour does not seem unreasonable as it resembles the
order in which you write out loops outside a comprehension except that the
expression for generated values is provided first.

On 21 October 2016 at 05:03, Sven R. Kunze  wrote:

> On 19.10.2016 00:08, Rob Cliffe wrote:
>
>>
>> But it's far too late to change it now, sadly.
>>>
>> Indeed. :-(  But if I were ruler of the world and could have my own
>> wish-list for Python 4, this (as per the first example) would be on it.
>>
>
> I don't see no reason why we can't make it.
>
> Personally, I also dislike this behavior.
>
>
> Cheers,
> Sven
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: Fwd: unpacking generalisations for list comprehension

[Random832]

On Tue, Oct 18, 2016, at 02:10, Nick Coghlan wrote:
> Hi, I contributed the current list comprehension implementation (when
> refactoring it for Python 3 to avoid leaking the iteration variable,
> as requested in PEP 3100 [1]), and "comprehensions are syntactic sugar
> for a series of nested for and if statements" is precisely my
> understanding of how they work, and what they mean.

But it's simply not true. It has never been true and it will never be
true. Something is not "syntactic sugar" if it doesn't compile to the
exact same sequence of operations as the thing it is supposedly
syntactic sugar for. It's a useful teaching tool (though you've
eventually got to teach the differences), but claiming that it's
"syntactic sugar" - and "non-negotiably" so, at that - implies that it
is a literal transformation.

I said "limited" in reference to the specific claim - which was not
yours - that since "yield a, b" yields a tuple, "yield *x" [and
therefore (*x for...)] ought to also yield a tuple, and I stand by it.
It's the same kind of simplistic understanding that should lead one to
believe that not only the loop variable but also the "result" temporary
ought to exist after the comprehension is executed.

I was being entirely serious in saying that this is like objecting to
normal unpacking on the grounds that an ordinary list display should be
considered syntactic sugar for an unrolled sequence of append calls. In
both cases, the equivalence is not exact,  and there should be room to
at least discuss things that would merely require an additional rule to
be added (or changed, technically making it "result += [...]" would
cover both cases) to the transformation - a transformation which already
results in three different statements depending on whether it is a list
comprehension, a set comprehension, or a generator expression (and a
fourth if you count dict comprehensions, though that's a different
syntax too) - rather than simply declaring them "not negotiable".

Declaring this "not negotiable" was an incredibly hostile dismissal of
everyone else's position. Especially when what's being proposed wouldn't
invalidate the concept, it would just change the exact details of what
the transformation is. Which is more than can be said for not leaking
the variable.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: Fwd: unpacking generalisations for list comprehension

[Sven R. Kunze]

On 18.10.2016 08:23, Greg Ewing wrote:

If it were a namedtuple, for example, you could write

  [*t for t in fulltext_tuples if t.language == 'english']

or

  [x for t in fulltext_tuples if t.language == 'english' for x in t]

The latter is a bit unsatisfying, because we are having to
make up an arbitrary name 'x' to stand for an element of t.
Even though the two elements of t have quite different roles,
we can't use names that reflect those roles.


It's an intriguing idea to use namedtuples but in this case one should 
not over-engineer.


What I dislike most are the names of "fulltext_tuple", "x", "t". If I 
were to use it, I think my coworkers would tar and feather me. ;) This 
is one of the cases where it makes absolutely no sense to invent 
artificial names for the sake of naming. I can name a lot of (internal) 
examples where we tried really hard at inventing named concepts which 
make absolutely no sense half a year later even to those who invented 
them. Repeatedly, in the same part of the code.


Each newly named concept introduces another indirection. Thus, we always 
need to find a middle ground between naming and using language features, 
so I (personally) would be grateful for this particular feature. :)



Because of that, to my eyes the version with * makes it easier
to see what is going on.


That's a very nice phrase: "makes it easier to see what is going on". I 
need to remember that.


Cheers,
Sven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Order of loops in list comprehension

[Sven R. Kunze]

On 19.10.2016 00:08, Rob Cliffe wrote:



But it's far too late to change it now, sadly.
Indeed. :-(  But if I were ruler of the world and could have my own 
wish-list for Python 4, this (as per the first example) would be on it.


I don't see no reason why we can't make it.

Personally, I also dislike this behavior.


Cheers,
Sven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: Fwd: unpacking generalisations for list comprehension

[Sven R. Kunze]

On 18.10.2016 10:01, Daniel Moisset wrote:
So, for me, this feature is something that could be covered with a 
(new) function with no new syntax required. All you have to learn is 
that instead of [*...] you use flatten(...)


The main motivation is not "hey we need yet another way for xyz" but 
"could we remove that inconsistency?". You wrote "[*...]"; which already 
works. It just that it does not work for all kinds of "..." . I hope 
that makes the motivation clearer.



I for one don't need yet another function or function cascade to make 
things work. "list(itertools.chain.from_iterable(...))" just works fine 
for all kinds of "...".



Cheers,
Sven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Python multi-dimensional array constructor

[David Mertz]

I find the proposed syntax worse than the existing square brackets. The way
the NumPy does a repr of an array is a good model of clarity, and it's
correct current Python (except for larger arrays where visual ellipses are
used).

On Oct 20, 2016 12:01 AM, "Greg Ewing"  wrote:

> Matt Gilson wrote:
>
>> I think that it was mentioned that it might be possible for a user to
>> _register_ a callable that would then be used when this syntax was envoked
>> -- But having a global setting like that leads to contention.
>>
>
> I think for that to fly it would have to be a per-module
> thing. Then each module using the syntax would be able
> to choose the meaning of it.
>
> A simple way to do this would be for the compiler to
> translate it into something like
>
>__array__([[[ ... ]]])
>
> and then you would just define __array__ appropriately,
> e.g.
>
>from numpy import array as __array__
>
> Personally I'm not very enthusiastic about the whole
> thing, though. I don't find the new syntax to be much of
> an improvement, if any. Certainly nowhere near enough
> to be worth adding syntax.
>
> --
> Greg
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Python multi-dimensional array constructor

[Greg Ewing]

Todd wrote:
||| is the delimiter for the third dimension, || is the delimiter for 
the second dimension.


This seems a bit inconsistent. It appears the rule is
"n vertical bars is the delimiter for the nth dimension".
By that rule, the delimiter for the first dimension
should be a single vertical bar, but instead it's a
comma.

Also, it's not very clear why when you have a 2D array
with two rows you write

   [| 1,2,3 || 4,5,6 |]

i.e. with *one* vertical bar at each end, but when
there is only one row you write

   [|| 1,2,3 ||]

i.e. with *two* vertical bars at each end.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/