Re: [Numpy-discussion] [RFC] should we argue for a matrix power operator, @@?
Personally I did not like @@ in the first place. Sturla Nathaniel Smith n...@pobox.com wrote: Hi all, Here's the second thread for discussion about Guido's concerns about PEP 465. The issue here is that PEP 465 as currently written proposes two new operators, @ for matrix multiplication and @@ for matrix power (analogous to * and **): http://legacy.python.org/dev/peps/pep-0465/ The main thing we care about of course is @; I pushed for including @@ because I thought it was nicer to have than not, and I thought the analogy between * and ** might make the overall package more appealing to Guido's aesthetic sense. It turns out I was wrong :-). Guido is -0 on @@, but willing to be swayed if we think it's worth the trouble to make a solid case. Note that question now is *not*, how will @@ affect the reception of @. @ itself is AFAICT a done deal, regardless of what happens with @@. For this discussion let's assume @ can be taken for granted, and that we can freely choose to either add @@ or not add @@ to the language. The question is: which do we think makes Python a better language (for us and in general)? Some thoughts to start us off: Here are the interesting use cases for @@ that I can think of: - 'vector @@ 2' gives the squared Euclidean length (because it's the same as vector @ vector). Kind of handy. - 'matrix @@ n' of course gives the matrix power, which is of marginal use but does come in handy sometimes, e.g., when looking at graph connectivity. - 'matrix @@ -1' provides a very transparent notation for translating textbook formulas (with all their inverses) into code. It's a bit unhelpful in practice, because (a) usually you should use solve(), and (b) 'matrix @@ -1' is actually more characters than 'inv(matrix)'. But sometimes transparent notation may be important. (And in some cases, like using numba or theano or whatever, 'matrix @@ -1 @ foo' could be compiled into a call to solve() anyway.) (Did I miss any?) In practice it seems to me that the last use case is the one that's might matter a lot practice, but then again, it might not -- I'm not sure. For example, does anyone who teaches programming with numpy have a feeling about whether the existence of '@@ -1' would make a big difference to you and your students? (Alan? I know you were worried about losing the .I attribute on matrices if switching to ndarrays for teaching -- given that ndarray will probably not get a .I attribute, how much would the existence of @@ -1 affect you?) On a more technical level, Guido is worried about how @@'s precedence should work (and this is somewhat related to the other thread about @'s precedence and associativity, because he feels that if we end up giving @ and * different precedence, then that makes it much less clear what to do with @@, and reduces the strength of the */**/@/@@ analogy). In particular, if we want to argue for @@ then we'll need to figure out what expressions like a @@ b @@ c and a ** b @@ c and a @@ b ** c should do. A related question is what @@ should do if given an array as its right argument. In the current PEP, only integers are accepted, which rules out a bunch of the more complicated cases like a @@ b @@ c (at least assuming @@ is right-associative, like **, and I can't see why you'd want anything else). OTOH, in the brave new gufunc world, it technically would make sense to define @@ as being a gufunc with signature (m,m),()-(m,m), and the way gufuncs work this *would* allow the power to be an array -- for example, we'd have: mat = randn(m, m) pow = range(n) result = gufunc_matrix_power(mat, pow) assert result.shape == (n, m, m) for i in xrange(n): assert np.all(result[i, :, :] == mat ** i) In this case, a @@ b @@ c would at least be a meaningful expression to write. OTOH it would be incredibly bizarre and useless, so probably no-one would ever write it. As far as these technical issues go, my guess is that the correct rule is that @@ should just have the same precedence and the same (right) associativity as **, and in practice no-one will ever write stuff like a @@ b @@ c. But if we want to argue for @@ we need to come to some consensus or another here. It's also possible the answer is ugh, these issues are too complicated, we should defer this until later when we have more experience with @ and gufuncs and stuff. After all, I doubt anyone else will swoop in and steal @@ to mean something else! OTOH, if e.g. there's a strong feeling that '@@ -1' will make a big difference in pedagogical contexts, then putting that off for years might be a mistake. -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [RFC] should we argue for a matrix power operator, @@?
On Sat, Mar 15, 2014 at 4:32 AM, Nathaniel Smith n...@pobox.com wrote: For this discussion let's assume @ can be taken for granted, and that we can freely choose to either add @@ or not add @@ to the language. The question is: which do we think makes Python a better language (for us and in general)? The thread so far, it sounds like the consensus answer is meh, whatever. So I'm thinking we should just drop @@ from the PEP, and if it turns out that this is a problem we can always revisit it in the ~3.6/3.7 timeframe. -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [RFC] should we argue for a matrix power operator, @@?
On Mon, Mar 17, 2014 at 11:53 AM, Nathaniel Smith n...@pobox.com wrote: On Sat, Mar 15, 2014 at 4:32 AM, Nathaniel Smith n...@pobox.com wrote: For this discussion let's assume @ can be taken for granted, and that we can freely choose to either add @@ or not add @@ to the language. The question is: which do we think makes Python a better language (for us and in general)? The thread so far, it sounds like the consensus answer is meh, whatever. So I'm thinking we should just drop @@ from the PEP, and if it turns out that this is a problem we can always revisit it in the ~3.6/3.7 timeframe. +1. Thanks! -- Robert Kern ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
On Sat, Mar 15, 2014 at 7:01 PM, Alexander Belopolsky ndar...@mac.com wrote: On Sat, Mar 15, 2014 at 2:25 PM, Alexander Belopolsky ndar...@mac.com wrote: On Fri, Mar 14, 2014 at 11:41 PM, Nathaniel Smith n...@pobox.com wrote: Here's the main blocker for adding a matrix multiply operator '@' to Python: we need to decide what we think its precedence and associativity should be. I am not ready to form my own opinion, but I hope the following will help shaping the discussion. One more question that I think should be answered by the PEP and may influence the associativity decision is what happens if in an A @ B @ C expression, each operand has its own type that defines __matmul__ and __rmatmul__? For example, A can be an ndarray, B a sympy expression and C a pyoperator. The general rule in Python is that in a binary operation A # B, then first we try A.__special__, and if that doesn't exist or it returns NotImplemented, then we try B.__rspecial__. (The exception is that if B.__class__ is a proper subclass of A.__class__, then we do it in the reverse order.) -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
On Mon, Mar 17, 2014 at 11:48 AM, Nathaniel Smith n...@pobox.com wrote: One more question that I think should be answered by the PEP and may influence the associativity decision is what happens if in an A @ B @ C expression, each operand has its own type that defines __matmul__ and __rmatmul__? For example, A can be an ndarray, B a sympy expression and C a pyoperator. The general rule in Python is that in a binary operation A # B, then first we try A.__special__, and if that doesn't exist or it returns NotImplemented, then we try B.__rspecial__. (The exception is that if B.__class__ is a proper subclass of A.__class__, then we do it in the reverse order.) This is the simple case. My question was: what happens if in an A @ B @ C expression, each operand has its own type that defines __matmul__ and __rmatmul__? Are we going to recommend that other projects adopt numpy's __array_priority__? In mixed-type expressions, do you expect A @ B @ C to have type of A, B, or C? Does __matmul__ first then __rmatmul__ rule makes sense if @ becomes right-associative or should the order be reversed? ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
On Mon, Mar 17, 2014 at 3:48 PM, Nathaniel Smith n...@pobox.com wrote: On Sat, Mar 15, 2014 at 7:01 PM, Alexander Belopolsky ndar...@mac.com wrote: One more question that I think should be answered by the PEP and may influence the associativity decision is what happens if in an A @ B @ C expression, each operand has its own type that defines __matmul__ and __rmatmul__? For example, A can be an ndarray, B a sympy expression and C a pyoperator. The general rule in Python is that in a binary operation A # B, then first we try A.__special__, and if that doesn't exist or it returns NotImplemented, then we try B.__rspecial__. (The exception is that if B.__class__ is a proper subclass of A.__class__, then we do it in the reverse order.) Assuming that all combinations are possible and give no error: A @ B @ C == A.__matmul__(B.__matmul__(C)) # right A @ B @ C == A.__matmul__(B).__matmul__(C) # left Did you want to specify which permutations of X.__matmul__(Y) return NotImplemented? -- Robert Kern ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
On Mon, Mar 17, 2014 at 4:09 PM, Alexander Belopolsky ndar...@mac.com wrote: On Mon, Mar 17, 2014 at 11:48 AM, Nathaniel Smith n...@pobox.com wrote: One more question that I think should be answered by the PEP and may influence the associativity decision is what happens if in an A @ B @ C expression, each operand has its own type that defines __matmul__ and __rmatmul__? For example, A can be an ndarray, B a sympy expression and C a pyoperator. The general rule in Python is that in a binary operation A # B, then first we try A.__special__, and if that doesn't exist or it returns NotImplemented, then we try B.__rspecial__. (The exception is that if B.__class__ is a proper subclass of A.__class__, then we do it in the reverse order.) This is the simple case. My question was: what happens if in an A @ B @ C expression, each operand has its own type that defines __matmul__ and __rmatmul__? The point of associativity is that the complex case A @ B @ C gets turned into either A @ (B @ C) or else (A @ B) @ C, and then you're back in the simple case. Are we going to recommend that other projects adopt numpy's __array_priority__? In mixed-type expressions, do you expect A @ B @ C to have type of A, B, or C? Does __matmul__ first then __rmatmul__ rule makes sense if @ becomes right-associative or should the order be reversed? ** is right-associative and uses the left-then-right rule, so it seems fine to me. In general the left-then-right rule has no particular logic behind it, it's just chosen so as to have *some* rule. In practice all well-behaved classes have to make sure that they implement __special__ methods in such a way that all the different variations work, no matter which class ends up actually handling the operation. -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
On Mon, Mar 17, 2014 at 12:13 PM, Nathaniel Smith n...@pobox.com wrote: In practice all well-behaved classes have to make sure that they implement __special__ methods in such a way that all the different variations work, no matter which class ends up actually handling the operation. Well-behaved classes are hard to come by in practice. The @ operator may fix the situation with np.matrix, so take a look at MaskedArray with its 40-line __array_wrap__ and no end of bugs. Requiring superclass __method__ to handle creation of subclass results correctly is turning Liskov principle on its head. With enough clever tricks and tight control over the full class hierarchy you can make it work in some cases, but it is not a good design. I am afraid that making @ special among other binary operators that implement mathematically associative operations will create a lot of confusion. (The pow operator is special because the corresponding mathematical operation is non-associative.) Imagine teaching someone that a % b % c = (a % b) % c, but a @ b @ c = a @ (b @ c). What are the chances that they will correctly figure out what a // b // c means after this? ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [RFC] should we argue for a matrix power operator, @@?
On Mon, Mar 17, 2014 at 7:53 AM, Nathaniel Smith n...@pobox.com wrote: The thread so far, it sounds like the consensus answer is meh, whatever. So I'm thinking we should just drop @@ from the PEP, and if it turns out that this is a problem we can always revisit it in the ~3.6/3.7 timeframe. +1 from here. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
On Mon, Mar 17, 2014 at 12:50 PM, Alexander Belopolsky ndar...@mac.comwrote:
On Mon, Mar 17, 2014 at 12:13 PM, Nathaniel Smith n...@pobox.com wrote:
In practice all
well-behaved classes have to make sure that they implement __special__
methods in such a way that all the different variations work, no
matter which class ends up actually handling the operation.
Well-behaved classes are hard to come by in practice. The @ operator
may fix the situation with np.matrix, so take a look at MaskedArray with
its 40-line __array_wrap__ and no end of bugs.
Requiring superclass __method__ to handle creation of subclass results
correctly is turning Liskov principle on its head. With enough clever
tricks and tight control over the full class hierarchy you can make it work
in some cases, but it is not a good design.
I am afraid that making @ special among other binary operators that
implement mathematically associative operations will create a lot of
confusion. (The pow operator is special because the corresponding
mathematical operation is non-associative.)
Imagine teaching someone that a % b % c = (a % b) % c, but a @ b @ c = a @
(b @ c). What are the chances that they will correctly figure out what a
// b // c means after this?
One case where we need to keep track of left or right is type promotion
a.shape
(100,)
1. * a.dot(a)
-98.0
(1.*a).dot(a)
328350.0
a.dtype
dtype('int8')
1. * a @ a
???
similar to
1. * 2 / 3
0.
1. * (2 / 3) # I'm not in the `future`
0.0
Josef
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
1. * a.dot(a)
-98.0
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [RFC] should we argue for a matrix power operator, @@?
On Mon, Mar 17, 2014 at 10:01 AM, Aron Ahmadia a...@ahmadia.net wrote: On Mon, Mar 17, 2014 at 7:53 AM, Nathaniel Smith n...@pobox.com wrote: The thread so far, it sounds like the consensus answer is meh, whatever. So I'm thinking we should just drop @@ from the PEP, and if it turns out that this is a problem we can always revisit it in the ~3.6/3.7 timeframe. +1 from here. +1 too. Absent *clear* enthusiasm and support for new syntax/operators, I think being conservative and slow is the right approach. Just having @ will give us data and experience with this space, and it may become clear after one more cycle that we really need/want @@, or not, as the case may be. But it's easier to add it later if we really need it than to remove it if it proves to be a bad idea, so +1 for moving slowly on this. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
On Mon, Mar 17, 2014 at 1:18 PM, josef.p...@gmail.com wrote:
On Mon, Mar 17, 2014 at 12:50 PM, Alexander Belopolsky ndar...@mac.comwrote:
On Mon, Mar 17, 2014 at 12:13 PM, Nathaniel Smith n...@pobox.com wrote:
In practice all
well-behaved classes have to make sure that they implement __special__
methods in such a way that all the different variations work, no
matter which class ends up actually handling the operation.
Well-behaved classes are hard to come by in practice. The @ operator
may fix the situation with np.matrix, so take a look at MaskedArray with
its 40-line __array_wrap__ and no end of bugs.
Requiring superclass __method__ to handle creation of subclass results
correctly is turning Liskov principle on its head. With enough clever
tricks and tight control over the full class hierarchy you can make it work
in some cases, but it is not a good design.
I am afraid that making @ special among other binary operators that
implement mathematically associative operations will create a lot of
confusion. (The pow operator is special because the corresponding
mathematical operation is non-associative.)
Imagine teaching someone that a % b % c = (a % b) % c, but a @ b @ c = a
@ (b @ c). What are the chances that they will correctly figure out what a
// b // c means after this?
One case where we need to keep track of left or right is type promotion
a.shape
(100,)
1. * a.dot(a)
-98.0
(1.*a).dot(a)
328350.0
a.dtype
dtype('int8')
1. * a @ a
???
similar to
1. * 2 / 3
0.
1. * (2 / 3) # I'm not in the `future`
0.0
I thought of sending a message with I'm +-1 on either, but I'm not
I'm again in favor of left, because it's the simplest to understand
A.dot(B).dot(C) with some * mixed in
I understand now the computational argument in favor of right
x @ inv(x.T @ x) @ x.T @ y ( with shapes T,k k,k k,T T,1 )
or
x @ pinv(x) @ y(with shapes T,k k,T T,1 )
with with Tk (last 1 could be a m1 with Tm)
However, we don't write code like that most of the time.
Alan's students won't care much if some intermediate arrays blow up.
In library code like in statsmodels it's almost always a conscious choice
of where to set the parenthesis and, more often, which part of a long array
expression is taken out as a temporary or permanent variable.
I think almost the only uses of chain_dot(A, B, C) (which is right) is
for quadratic forms
xtxi = pinv(np.dot(exog.T, exog)) # k,k
xtdx = np.dot(exog.T * d[np.newaxis, :], exog) # k,k
vcov = chain_dot(xtxi, xtdx, xtxi) # kk, kk, kk
(from Quantreg)
I think optimizing this way is relatively easy
On the other hand, I worry a lot more about messy cases with different
dtypes or different classes involved as Alexander has pointed out. Cases
that might trip up medium to medium-advanced numpy users.
(Let's see, I have to read @ back to front, and * front to back, and why
did I put a sparse matrix in the middle and a masked array at the end. Oh
no, that's not a masked array it's a panda.)
compared to
(Somewhere there is a mistake, let's go through all terms from the
beginning to the end)
Josef
Josef
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
1. * a.dot(a)
-98.0
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
On Mon, Mar 17, 2014 at 2:55 PM, josef.p...@gmail.com wrote:
I'm again in favor of left, because it's the simplest to understand
A.dot(B).dot(C)
+1
Note that for many years to come the best option for repeated matrix
product will be A.dot(B).dot(C) ...
People who convert their dot(dot(dot('s to more readable method call syntax
now should not be forced to change the order or add parentheses when they
switch to @.
(Full disclosure: I am one of those people having recently converted a
large Numeric-based project to NumPy.)
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
In article CAPJVwBkLww7-ysZB76LMRZ+mmbyN_5T=ym_vu1pjgakrlbq...@mail.gmail.com, Nathaniel Smith n...@pobox.com wrote: OPTION 1 FOR @: Precedence: same as * Associativity: left My shorthand name for it: same-left (yes, very creative) This means that if you don't use parentheses, you get: a @ b @ c - (a @ b) @ c a * b @ c - (a * b) @ c a @ b * c - (a @ b) * c OPTION 2 FOR @: Precedence: more-weakly-binding than * Associativity: right My shorthand name for it: weak-right This means that if you don't use parentheses, you get: a @ b @ c - a @ (b @ c) a * b @ c - (a * b) @ c a @ b * c - a @ (b * c) OPTION 3 FOR @: Precedence: more-tightly-binding than * Associativity: right My shorthand name for it: tight-right This means that if you don't use parentheses, you get: a @ b @ c - a @ (b @ c) a * b @ c - a * (b @ c) a @ b * c - (a @ b) * c We need to pick which of which options we think is best, based on whatever reasons we can think of, ideally more than hmm, weak-right gives me warm fuzzy feelings ;-). (In principle the other 2 possible options are tight-left and weak-left, but there doesn't seem to be any argument in favor of either, so we'll leave them out of the discussion.) After seeing all the traffic on this thread, I am in favor of same-left because it is easiest to remember: - It introduces no new rules. - It is unambiguous. If we pick option 2 or 3 we have no strong reason to favor one over the other, leaving users to guess. To my mind, being able to easily reason about code you are reading is more important that hoping to increase efficiency for one common case when not using parenthesis. It also has the advantage that it needs the least justification. -- Russell ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
Hello, and what about something like that ? a @ b @ c - (a @ b) @ c a * b @ c - (a * b) @ c a @ b * c - a @ (b * c) Easy to remember. The *-product has priority to @-product, and then we just to @-product from left to right. An advantage of this is that parsers do job from left to right so I realy think that is a better choice than the weak-right. Christophe BAL 2014-03-17 21:37 GMT+01:00 Russell E. Owen ro...@uw.edu: In article CAPJVwBkLww7-ysZB76LMRZ+mmbyN_5T=ym_vu1pjgakrlbq...@mail.gmail.com, Nathaniel Smith n...@pobox.com wrote: OPTION 1 FOR @: Precedence: same as * Associativity: left My shorthand name for it: same-left (yes, very creative) This means that if you don't use parentheses, you get: a @ b @ c - (a @ b) @ c a * b @ c - (a * b) @ c a @ b * c - (a @ b) * c OPTION 2 FOR @: Precedence: more-weakly-binding than * Associativity: right My shorthand name for it: weak-right This means that if you don't use parentheses, you get: a @ b @ c - a @ (b @ c) a * b @ c - (a * b) @ c a @ b * c - a @ (b * c) OPTION 3 FOR @: Precedence: more-tightly-binding than * Associativity: right My shorthand name for it: tight-right This means that if you don't use parentheses, you get: a @ b @ c - a @ (b @ c) a * b @ c - a * (b @ c) a @ b * c - (a @ b) * c We need to pick which of which options we think is best, based on whatever reasons we can think of, ideally more than hmm, weak-right gives me warm fuzzy feelings ;-). (In principle the other 2 possible options are tight-left and weak-left, but there doesn't seem to be any argument in favor of either, so we'll leave them out of the discussion.) After seeing all the traffic on this thread, I am in favor of same-left because it is easiest to remember: - It introduces no new rules. - It is unambiguous. If we pick option 2 or 3 we have no strong reason to favor one over the other, leaving users to guess. To my mind, being able to easily reason about code you are reading is more important that hoping to increase efficiency for one common case when not using parenthesis. It also has the advantage that it needs the least justification. -- Russell ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
Sorry for all the misspellings... 2014-03-17 22:32 GMT+01:00 Christophe Bal projet...@gmail.com: Hello, and what about something like that ? a @ b @ c - (a @ b) @ c a * b @ c - (a * b) @ c a @ b * c - a @ (b * c) Easy to remember. The *-product has priority to @-product, and then we just to @-product from left to right. An advantage of this is that parsers do job from left to right so I realy think that is a better choice than the weak-right. Christophe BAL 2014-03-17 21:37 GMT+01:00 Russell E. Owen ro...@uw.edu: In article CAPJVwBkLww7-ysZB76LMRZ+mmbyN_5T=ym_vu1pjgakrlbq...@mail.gmail.com, Nathaniel Smith n...@pobox.com wrote: OPTION 1 FOR @: Precedence: same as * Associativity: left My shorthand name for it: same-left (yes, very creative) This means that if you don't use parentheses, you get: a @ b @ c - (a @ b) @ c a * b @ c - (a * b) @ c a @ b * c - (a @ b) * c OPTION 2 FOR @: Precedence: more-weakly-binding than * Associativity: right My shorthand name for it: weak-right This means that if you don't use parentheses, you get: a @ b @ c - a @ (b @ c) a * b @ c - (a * b) @ c a @ b * c - a @ (b * c) OPTION 3 FOR @: Precedence: more-tightly-binding than * Associativity: right My shorthand name for it: tight-right This means that if you don't use parentheses, you get: a @ b @ c - a @ (b @ c) a * b @ c - a * (b @ c) a @ b * c - (a @ b) * c We need to pick which of which options we think is best, based on whatever reasons we can think of, ideally more than hmm, weak-right gives me warm fuzzy feelings ;-). (In principle the other 2 possible options are tight-left and weak-left, but there doesn't seem to be any argument in favor of either, so we'll leave them out of the discussion.) After seeing all the traffic on this thread, I am in favor of same-left because it is easiest to remember: - It introduces no new rules. - It is unambiguous. If we pick option 2 or 3 we have no strong reason to favor one over the other, leaving users to guess. To my mind, being able to easily reason about code you are reading is more important that hoping to increase efficiency for one common case when not using parenthesis. It also has the advantage that it needs the least justification. -- Russell ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
Here is the translation. ;-) Hello, and what about something like that ? *a @ b @ c - (a @ b) @ c* *a * b @ c - (a * b) @ c* *a @ b * c - a @ (b * c)* Easy to remember: the *-product has priority regarding to the @-product, and we just do @-product from left to right. An advantage of this is that most parsers do analyze from left to right. So I really think that it is a better choice than the weak-right one. Christophe BAL 2014-03-17 22:34 GMT+01:00 Christophe Bal projet...@gmail.com: Sorry for all the misspellings... 2014-03-17 22:32 GMT+01:00 Christophe Bal projet...@gmail.com: Hello, and what about something like that ? a @ b @ c - (a @ b) @ c a * b @ c - (a * b) @ c a @ b * c - a @ (b * c) Easy to remember. The *-product has priority to @-product, and then we just to @-product from left to right. An advantage of this is that parsers do job from left to right so I realy think that is a better choice than the weak-right. Christophe BAL 2014-03-17 21:37 GMT+01:00 Russell E. Owen ro...@uw.edu: In article CAPJVwBkLww7-ysZB76LMRZ+mmbyN_5T=ym_vu1pjgakrlbq...@mail.gmail.com, Nathaniel Smith n...@pobox.com wrote: OPTION 1 FOR @: Precedence: same as * Associativity: left My shorthand name for it: same-left (yes, very creative) This means that if you don't use parentheses, you get: a @ b @ c - (a @ b) @ c a * b @ c - (a * b) @ c a @ b * c - (a @ b) * c OPTION 2 FOR @: Precedence: more-weakly-binding than * Associativity: right My shorthand name for it: weak-right This means that if you don't use parentheses, you get: a @ b @ c - a @ (b @ c) a * b @ c - (a * b) @ c a @ b * c - a @ (b * c) OPTION 3 FOR @: Precedence: more-tightly-binding than * Associativity: right My shorthand name for it: tight-right This means that if you don't use parentheses, you get: a @ b @ c - a @ (b @ c) a * b @ c - a * (b @ c) a @ b * c - (a @ b) * c We need to pick which of which options we think is best, based on whatever reasons we can think of, ideally more than hmm, weak-right gives me warm fuzzy feelings ;-). (In principle the other 2 possible options are tight-left and weak-left, but there doesn't seem to be any argument in favor of either, so we'll leave them out of the discussion.) After seeing all the traffic on this thread, I am in favor of same-left because it is easiest to remember: - It introduces no new rules. - It is unambiguous. If we pick option 2 or 3 we have no strong reason to favor one over the other, leaving users to guess. To my mind, being able to easily reason about code you are reading is more important that hoping to increase efficiency for one common case when not using parenthesis. It also has the advantage that it needs the least justification. -- Russell ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
On Mon, Mar 17, 2014 at 9:38 PM, Christophe Bal projet...@gmail.com wrote: Here is the translation. ;-) Hello, and what about something like that ? a @ b @ c - (a @ b) @ c a * b @ c - (a * b) @ c a @ b * c - a @ (b * c) Easy to remember: the *-product has priority regarding to the @-product, and we just do @-product from left to right. In the terminology we've been using in this thread, this is weak-left. An advantage of this is that most parsers do analyze from left to right. So I really think that it is a better choice than the weak-right one. We've mostly ignored this option because of assuming that if we want left-associativity, we should go with same-left instead of weak-left. Same-left is: a @ b @ c - (a @ b) @ c a * b @ c - (a * b) @ c a @ b * c - (a @ b) * c i.e., even more left-to-right than weak-left :-) Do you think weak-left is better than same-left? -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] GSoC project: draft of proposal
On 12.03.2014 17:52, Leo Mao wrote: Hi, The attachment is my draft of proposal. The project is vector math library integration. I think I need some feedback to make it solider. Any comment will be appreciated. Thanks in advance. hi, I finally had some time too properly look at your proposal, here are my comments. First of all I hope you are aware this is a very challenging project as you will have to deal with issues of several different areas: build systems, portability, low level performance, numerical issues, testing and the in some places quite daunting numpy codebase. I do fear that it might be too much for a first year student. Your proposal is lacking some information on your experiences. Are you already familiar with vectorization via SIMD? While the goal of this project is partly to avoid writing more vector code in NumPy it is still very useful if you are familiar with how it works. If you have no experience maybe add some time to learning the basics to the schedule. The numerical accuracy of the vector library needs to be evaluated, I suspect that this might be the biggest roadblock in adding support by default. The performance of the library over different value ranges also needs to investigated. What kind of hardware do you have at your disposal? SIMD vectorization performance is very hardware dependent, you probably want at least a intel sandy bridge or AMD bulldozer type cpu to get the most out of the library, those CPUs have the newish AVX SIMD instructions. While I think your schedule is already packed, another point you could add if you have extra time is extending the existing SSE vectorized code in numpy to AVX if the vector library does not provide an equivalent (e.g. probably the boolean stuff) The runtime feature detection vector libraries provide can be very useful for this. Regards, Julian Taylor ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
I think that weak-left is a little strange, just think a little of the operators used by mathematicians that always follow a hierarchy. A parser is mostly done using grammars : see http://docs.python.org/3.1/reference/grammar.html. Defining *-product to have stronger priority than the @-product, and this last having stronger priority than +, will make the changes in the grammar easier. I'm now convinced of the usefulness of @ and @@ too but I also think that you must think of other uses than only for numpy. In other words, numpy is a the good argument for this new operators, but this can also open new perspectives for other uses. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
On Mon, Mar 17, 2014 at 6:33 PM, Christophe Bal projet...@gmail.com wrote:
Defining *-product to have stronger priority than the @-product, and this
last having stronger priority than +, will make the changes in the grammar
easier.
The easiest is to give @ the same precedence as *. This will only require
changing
term: factor (('*'|'/'|'%'|'//') factor)*
to
term: factor (('*'|'/'|'%'|'//'|'@') factor)*
Anything else will require an extra rule, but in any case implementation is
trivial.
I don't think we need to worry about implementation details at this point.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
I'm now convinced of the usefulness of @ and @@ too but I also think that you must think of other uses than only for numpy. In other words, numpy is a the good argument for this new operators, but this can also open new perspectives for other uses. Speaking of `@@`, would the relative precedence of @ vs * be the same as @@ vs **? ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
First of all I'm must be very tired because I've written *I think that weak-left is a little strange...* instead of *I think that same-left is a little strange...*. It is the night in french... ;-) So I'm definitely for the weak-left ! Here is my answer to Alexander Belopolsky. You are right from a grammar point of view but for a human this looks too weird because * and @ are of different kinds contrary to * and / for example. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
If you see the operators as following a hierarchy, the answer is simply yes. 2014-03-18 0:16 GMT+01:00 Bago mrb...@gmail.com: I'm now convinced of the usefulness of @ and @@ too but I also think that you must think of other uses than only for numpy. In other words, numpy is a the good argument for this new operators, but this can also open new perspectives for other uses. Speaking of `@@`, would the relative precedence of @ vs * be the same as @@ vs **? ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
On Mon, Mar 17, 2014 at 11:16 PM, Bago mrb...@gmail.com wrote: Speaking of `@@`, would the relative precedence of @ vs * be the same as @@ vs **? This is one of the concerns that made Guido leery of @@ (but only one of them). Since we seem to be dropping @@: http://mail.scipy.org/pipermail/numpy-discussion/2014-March/069502.html we don't have to come up with an answer :-). -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
On Mon, Mar 17, 2014 at 10:33 PM, Christophe Bal projet...@gmail.com wrote: I think that weak-left is a little strange, just think a little of the operators used by mathematicians that always follow a hierarchy. Not sure what you mean -- I don't think most mathematicians think that scalar and matrix multiplication are above or below each other in precedence, for example. (Well, it's a strange question because scalar multiplication commutes, but even so, people often forget that these are even different operations.) A parser is mostly done using grammars : see http://docs.python.org/3.1/reference/grammar.html. Defining *-product to have stronger priority than the @-product, and this last having stronger priority than +, will make the changes in the grammar easier. I'm now convinced of the usefulness of @ and @@ too but I also think that you must think of other uses than only for numpy. In other words, numpy is a the good argument for this new operators, but this can also open new perspectives for other uses. No, that's not how this game is played :-). The way it works is, we figure out the best possible way to handle the use case that we've demonstrated a need for (matrix multiplication), and then once we've done that someone might or might not find some other uses too. If they do then cool, if not then too bad. This follows the principle that it's better to be great at some things than to be mediocre at everything. -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
On Mon, Mar 17, 2014 at 6:33 PM, Christophe Bal projet...@gmail.com wrote: I think that weak-left is a little strange, just think a little of the operators used by mathematicians that always follow a hierarchy. A parser is mostly done using grammars : see http://docs.python.org/3.1/reference/grammar.html. Defining *-product to have stronger priority than the @-product, and this last having stronger priority than +, will make the changes in the grammar easier. I'm now convinced of the usefulness of @ and @@ too but I also think that you must think of other uses than only for numpy. In other words, numpy is a the good argument for this new operators, but this can also open new perspectives for other uses. My main problem with weak-left (* higher) and tight-left (@ higher) compared to same-left is that I don't see any obvious choice between the weak and tight. I don't think I would have problems with readability. Wikipedia doesn't say anything about precedence of Hadamard versus matrix product. matlab, IDL and Gauss (I checked the manual) all use same-left, as Nathaniel pointed out. For scalar * together with dot product which is more common in formulas, we would just read it sequentially, i.e. same-left. I don't remember when I have seen dot-in-a-circle in a paper, but I don't think there was any precedence either. --- I guess the same applies for other (mis)uses of @ from math import sqrt class MyOp(object): def __init__(self, func): self.func = func def __at__(self, x): return [self.func(xi) for xi in x] myop = MyOp(lambda x: sqrt(x)) print myop.__at__(range(3)) # myop @ range(5) print myop.__at__(range(3) * 2) # myop @ (range(5) * 2) print myop.__at__(range(3)) * 3 # myop @ range(5) * 3 ''' [0.0, 1.0, 1.4142135623730951] [0.0, 1.0, 1.4142135623730951, 0.0, 1.0, 1.4142135623730951] [0.0, 1.0, 1.4142135623730951, 0.0, 1.0, 1.4142135623730951, 0.0, 1.0, 1.4142135623730951] ''' - Josef ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
This follows the principle that it's better to be great at some things than to be mediocre at everything. You're right. I think that weak-left is a little strange, just think a little of the operators used by mathematicians that always follow a hierarchy. Not sure what you mean -- I don't think most mathematicians think that scalar and matrix multiplication are above or below each other in precedence, for example. You're right but on the other hand, I've never seen mixed use of matrix and scalar products without parenthesis... Indeed in math, we can use Au , Bv for the scalar product of two matrix-vector products. But here, I think that the situation is different because we are talking about operators from arrays to array : mainly @ , * and + (elementwise for the two last). Whereas in the preceding example, the scalar product is from arrays to scalar. As a math user, I think at this point that the arrays-to-array operators must follows a hierarchy. Who is the guy who have asked such a complicated question about precedence ? :-) 2014-03-18 0:30 GMT+01:00 Nathaniel Smith n...@pobox.com: On Mon, Mar 17, 2014 at 10:33 PM, Christophe Bal projet...@gmail.com wrote: I think that weak-left is a little strange, just think a little of the operators used by mathematicians that always follow a hierarchy. Not sure what you mean -- I don't think most mathematicians think that scalar and matrix multiplication are above or below each other in precedence, for example. (Well, it's a strange question because scalar multiplication commutes, but even so, people often forget that these are even different operations.) A parser is mostly done using grammars : see http://docs.python.org/3.1/reference/grammar.html. Defining *-product to have stronger priority than the @-product, and this last having stronger priority than +, will make the changes in the grammar easier. I'm now convinced of the usefulness of @ and @@ too but I also think that you must think of other uses than only for numpy. In other words, numpy is a the good argument for this new operators, but this can also open new perspectives for other uses. No, that's not how this game is played :-). The way it works is, we figure out the best possible way to handle the use case that we've demonstrated a need for (matrix multiplication), and then once we've done that someone might or might not find some other uses too. If they do then cool, if not then too bad. This follows the principle that it's better to be great at some things than to be mediocre at everything. -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
On Tue, Mar 18, 2014 at 12:16 AM, Christophe Bal projet...@gmail.com wrote: I think that weak-left is a little strange, just think a little of the operators used by mathematicians that always follow a hierarchy. Not sure what you mean -- I don't think most mathematicians think that scalar and matrix multiplication are above or below each other in precedence, for example. You're right but on the other hand, I've never seen mixed use of matrix and scalar products without parenthesis... Indeed in math, we can use Au , Bv for the scalar product of two matrix-vector products. Not scalar product, scalar multiplication -- you're saying (I think) that 3 * Matrix1 * Matrix2 is just like 3 * Matrix1 + Matrix2 in the sense that mathematicians think of the 3 * Matrix1 part is very different from, and higher precedence than, the Matrix1 + Matrix2 part. And similarly that Matrix1 * Matrix2 * 3 is just like Matrix1 + Matrix2 * 3 But in fact I think if you asked most mathematicians which of the *'s in Matrix1 * Matrix2 * 3 is higher precedence, they would think this question very odd! -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
On Sat, Mar 15, 2014 at 6:28 PM, Nathaniel Smith n...@pobox.com wrote: Mathematica: instead of having an associativity, a @ b @ c gets converted into mdot([a, b, c]) So, I've been thinking about this (thanks to @rfateman for pointing it out), and wondering if Mathematica's approach is worth following up more. (It would need to make it past python-dev, of course, but worst case is just that they say no and we're back where we are now, so we might as well think it through.) Here's how it would work: Currently Python has 3 different kinds of ops: left-associative (most of them), right-associative (**), and chaining. Chaining is used for comparison ops. Example: a b c gets parsed to something like do_comparison(args=[a, b, c], ops=[lt, lt]) Notice this is very different from either of (a b) c a (b c) Which means that comparisons aren't left- OR right-associative, they're this other thing, chaining. So we could propose adding a 4th kind of op, calling grouping, which would be only @. And the idea is that a @ b @ c would be equivalent to operator.matmul((a, b, c)) which eventually (see below) becomes a call to a.__matmul__((a, b, c)) We'd use exactly the same parsing rules as the chaining ops, so you can still control evaluation order with parentheses if you want: a @ (b @ c) - matmul((a, matmul((b, c (a @ b) @ c - matmul((matmul((a, c)), c)) ...but if you don't specify, then each contiguous group of @ operators gets collected up and handed to __matmul__ together, and the __matmul__ implementation gets to decide which evaluation strategy to use. It's trivially fast for the computer to figure out the best evaluation order for matrix multiplication, so in practice I think this would mean that you could just stop worrying about parentheses for multiple contiguous calls to matmul. Fancier versions of __matmul__ defined on more specialized non-ndarray classes might even take into account their specialized knowledge of how expensive different evaluation orders are for their specific type -- I'm not sure if this actually happens in practice, but it might. (Like maybe the best way to evaluate a @ b @ c depends on the sparsity pattern in the various matrices, or maybe it depends on which matrices are currently on the GPU and which are in memory? Anyone have any real examples of this?) (Of course, this same evaluation-order problem arises for *all* expressions using numpy; being able to optimize whole expressions at a time is exactly what numexpr/numba/theano/etc. are useful for. So one could argue that baking it in to @ is pointless, if anyone gets tired of writing parentheses they should just use one of these libraries. Or maybe evaluation order problems arise so rarely for @ that no-one cares. But OTOH it would be so nice for once to just have a single best solution -- always use @ and be happy, it just works -- instead of all the caveats we normally do -- @ is good in some cases, but in other cases mdot is better, or if you know you can just use @ with the right parentheses) Of course, we still have to say something about what value a @ b @ c actually computes. In the PEP semantics, it isn't always associative -- specifically not if we do Mat @ vec @ Mat. So in this approach, we still need to decide what matmul((Mat, vec, Mat)) should return. But, this is actually a feature! Because obviously what *should* be returned in this case is *not* (Mat @ vec) @ Mat, *or* Mat @ (vec @ Mat). Both of those answers are terrible; it's just, if you have an ordinary left-/right-associative operator, those are your only options. What *should* be returned is an error. And in this scheme we get to see the whole @ expression at once, so we actually can raise an error for such things. So, this possibly has nicer performance characteristics, and is also possibly has nicer semantics. Now, how would this look in terms of the language definition? As far as the parser and AST go, this would use exactly the same rules as the chaining ops, so that's easy. Having parsed, we must evaluate. Likely the most contentious part of this approach is that we now have an n-arg operator, so the standard __X__/__rX__ dichotomy won't work, we need to do something like multiple dispatch. I haven't followed the debate on this issue in detail, but what I'd propose for this limited context is not to do anything like real multiple dispatch, but just directly generalize the familiar __rX__ rule to n arguments. The __rX__ rule is how Python's existing binary operators work: usually to evaluate a # b, you try a.__foo__, and then b.__foo__ EXCEPT if b is a proper subclass of a, you try b first. Generalized to 2 arguments, this looks like: def operator.matmul(args): candidates = list(args) while candidates: candidate = pop_next_candidate(candidates) if hasattr(candidate, __matmul__): result = candidate.__matmul__(args) if result is not NotImplemented:
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
On Mon, Mar 17, 2014 at 8:37 PM, Russell E. Owen ro...@uw.edu wrote: After seeing all the traffic on this thread, I am in favor of same-left because it is easiest to remember: - It introduces no new rules. - It is unambiguous. If we pick option 2 or 3 we have no strong reason to favor one over the other, leaving users to guess. To my mind, being able to easily reason about code you are reading is more important that hoping to increase efficiency for one common case when not using parenthesis. Personally I'm leaning in a similar direction (at least as far as left- versus right-associativity goes; I'm not sure yet what I think about the magic grouping thing I just posted :-)). The more I think about it, the weaker I find the avoiding-parentheses argument. If you're going to take the trouble to think about which ordering is best, you should write that down with parentheses no matter what the associativity is, so that when I have to read your code I'll see the parentheses and know that you thought about it! And certainly the slow part of this is not typing the parentheses, it's figuring out what order is best. (The potential advantage of grouping isn't that you don't have to write as many parentheses, it's that you don't have to *think* about parentheses.) The fact that Matlab et al get along fine with same-left also strikes me as strong evidence that right-associativity's benefits are at least not overwhelmingly compelling... -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
On Mon, Mar 17, 2014 at 8:54 PM, Nathaniel Smith n...@pobox.com wrote:
Currently Python has 3 different kinds of ops: left-associative (most
of them), right-associative (**), and chaining. Chaining is used for
comparison ops. Example:
a b c
gets parsed to something like
do_comparison(args=[a, b, c], ops=[lt, lt])
The actual parse tree is more like Compare(a, [lt, lt], [b, c]) with the
first aruments playing a distinct role:
ast.dump(ast.parse('abc'), annotate_fields=False)
Module([Expr(Compare(Name('a', Load()), [Lt(), Lt()], [Name('b', Load()),
Name('c', Load())]))])
Your idea is very interesting and IMO, worth considering independently from
the @ operator. I always wanted a vector between operator to be
available in numpy as low x high.
The only problem I see here is with mixed types, but we can follow the pow
precedent [1]: Note that ternary pow() will not try calling __rpow__()
(the coercion rules would become too complicated).
[1]
http://docs.python.org/3/reference/datamodel.html#emulating-numeric-types
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
On Mar 17, 2014 5:54 PM, Nathaniel Smith n...@pobox.com wrote: On Sat, Mar 15, 2014 at 6:28 PM, Nathaniel Smith n...@pobox.com wrote: Mathematica: instead of having an associativity, a @ b @ c gets converted into mdot([a, b, c]) So, I've been thinking about this (thanks to @rfateman for pointing it out), and wondering if Mathematica's approach is worth following up more. (It would need to make it past python-dev, of course, but worst case is just that they say no and we're back where we are now, so we might as well think it through.) Here's how it would work: Currently Python has 3 different kinds of ops: left-associative (most of them), right-associative (**), and chaining. Chaining is used for comparison ops. Example: a b c gets parsed to something like do_comparison(args=[a, b, c], ops=[lt, lt]) Notice this is very different from either of (a b) c a (b c) Which means that comparisons aren't left- OR right-associative, they're this other thing, chaining. So we could propose adding a 4th kind of op, calling grouping, which would be only @. And the idea is that a @ b @ c would be equivalent to operator.matmul((a, b, c)) which eventually (see below) becomes a call to a.__matmul__((a, b, c)) We'd use exactly the same parsing rules as the chaining ops, so you can still control evaluation order with parentheses if you want: a @ (b @ c) - matmul((a, matmul((b, c (a @ b) @ c - matmul((matmul((a, c)), c)) ...but if you don't specify, then each contiguous group of @ operators gets collected up and handed to __matmul__ together, and the __matmul__ implementation gets to decide which evaluation strategy to use. It's trivially fast for the computer to figure out the best evaluation order for matrix multiplication, so in practice I think this would mean that you could just stop worrying about parentheses for multiple contiguous calls to matmul. Fancier versions of __matmul__ defined on more specialized non-ndarray classes might even take into account their specialized knowledge of how expensive different evaluation orders are for their specific type -- I'm not sure if this actually happens in practice, but it might. (Like maybe the best way to evaluate a @ b @ c depends on the sparsity pattern in the various matrices, or maybe it depends on which matrices are currently on the GPU and which are in memory? Anyone have any real examples of this?) (Of course, this same evaluation-order problem arises for *all* expressions using numpy; being able to optimize whole expressions at a time is exactly what numexpr/numba/theano/etc. are useful for. So one could argue that baking it in to @ is pointless, if anyone gets tired of writing parentheses they should just use one of these libraries. Or maybe evaluation order problems arise so rarely for @ that no-one cares. But OTOH it would be so nice for once to just have a single best solution -- always use @ and be happy, it just works -- instead of all the caveats we normally do -- @ is good in some cases, but in other cases mdot is better, or if you know you can just use @ with the right parentheses) Of course, we still have to say something about what value a @ b @ c actually computes. In the PEP semantics, it isn't always associative -- specifically not if we do Mat @ vec @ Mat. So in this approach, we still need to decide what matmul((Mat, vec, Mat)) should return. But, this is actually a feature! Because obviously what *should* be returned in this case is *not* (Mat @ vec) @ Mat, *or* Mat @ (vec @ Mat). Both of those answers are terrible; it's just, if you have an ordinary left-/right-associative operator, those are your only options. What *should* be returned is an error. And in this scheme we get to see the whole @ expression at once, so we actually can raise an error for such things. So, this possibly has nicer performance characteristics, and is also possibly has nicer semantics. Now, how would this look in terms of the language definition? As far as the parser and AST go, this would use exactly the same rules as the chaining ops, so that's easy. Having parsed, we must evaluate. Likely the most contentious part of this approach is that we now have an n-arg operator, so the standard __X__/__rX__ dichotomy won't work, we need to do something like multiple dispatch. I haven't followed the debate on this issue in detail, but what I'd propose for this limited context is not to do anything like real multiple dispatch, but just directly generalize the familiar __rX__ rule to n arguments. The __rX__ rule is how Python's existing binary operators work: usually to evaluate a # b, you try a.__foo__, and then b.__foo__ EXCEPT if b is a proper subclass of a, you try b first. Generalized to 2 arguments, this looks like: def operator.matmul(args): candidates = list(args) while candidates: candidate =
Re: [Numpy-discussion] NumPy-Discussion Digest, Vol 90, Issue 56
Julian, I can see the need to recognize both column and row vectors, but why not with np.matrix? I can see no need for a new operator and hope to be able to comment more fully on PEP 465 in a few days. Colin W. On 17-Mar-2014 7:19 PM, numpy-discussion-requ...@scipy.org wrote: Send NumPy-Discussion mailing list submissions to numpy-discussion@scipy.org To subscribe or unsubscribe via the World Wide Web, visit http://mail.scipy.org/mailman/listinfo/numpy-discussion or, via email, send a message with subject or body 'help' to numpy-discussion-requ...@scipy.org You can reach the person managing the list at numpy-discussion-ow...@scipy.org When replying, please edit your Subject line so it is more specific than Re: Contents of NumPy-Discussion digest... Today's Topics: 1. Re: [help needed] associativity and precedence of '@' (Nathaniel Smith) 2. Re: GSoC project: draft of proposal (Julian Taylor) 3. Re: [help needed] associativity and precedence of '@' (Christophe Bal) 4. Re: [help needed] associativity and precedence of '@' (Alexander Belopolsky) 5. Re: [help needed] associativity and precedence of '@' (Bago) 6. Re: [help needed] associativity and precedence of '@' (Christophe Bal) 7. Re: [help needed] associativity and precedence of '@' (Christophe Bal) 8. Re: [help needed] associativity and precedence of '@' (Nathaniel Smith) -- Message: 1 Date: Mon, 17 Mar 2014 22:02:33 + From: Nathaniel Smith n...@pobox.com Subject: Re: [Numpy-discussion] [help needed] associativity and precedence of '@' To: Discussion of Numerical Python numpy-discussion@scipy.org Message-ID: CAPJVwB=zBazN+fiYWJeiWOL=4a9bf2xgxjgott8gftt-kdu...@mail.gmail.com Content-Type: text/plain; charset=UTF-8 On Mon, Mar 17, 2014 at 9:38 PM, Christophe Bal projet...@gmail.com wrote: Here is the translation. ;-) Hello, and what about something like that ? a @ b @ c - (a @ b) @ c a * b @ c - (a * b) @ c a @ b * c - a @ (b * c) Easy to remember: the *-product has priority regarding to the @-product, and we just do @-product from left to right. In the terminology we've been using in this thread, this is weak-left. An advantage of this is that most parsers do analyze from left to right. So I really think that it is a better choice than the weak-right one. We've mostly ignored this option because of assuming that if we want left-associativity, we should go with same-left instead of weak-left. Same-left is: a @ b @ c - (a @ b) @ c a * b @ c - (a * b) @ c a @ b * c - (a @ b) * c i.e., even more left-to-right than weak-left :-) Do you think weak-left is better than same-left? ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [help needed] associativity and precedence of '@'
On Mon, Mar 17, 2014 at 8:54 PM, Nathaniel Smith n...@pobox.com wrote: But, this is actually a feature! Because obviously what *should* be returned in this case is *not* (Mat @ vec) @ Mat, *or* Mat @ (vec @ Mat). Both of those answers are terrible; it's just, if you have an ordinary left-/right-associative operator, those are your only options. What *should* be returned is an error. And in this scheme we get to see the whole @ expression at once, so we actually can raise an error for such things. Sorry if this is a little off topic. But there's still something about the vector examples that bugs me, matrix@vector and vector@@2, keep popping up (this also applies to the matrix@matrix examples to a lesser extent). I'm a little unconformable looking at the shape to to decide what's a matrix and what's a vector. (Matlab has some problems like this) If it only has one or two dimensions it's easy, but I always find that if I've written code that works for 1 matrix or vector, 5 minutes later I want it to work for fields of matrices or vectors. If we're just going by shape there's no way to distinguish between a 2d field of matrices and a 3d field of vectors. I guess this is a repeat of part of what Eelco Hoogendoorn saying a few posts back I was just wondering if anyone sees a place, to get @ a little closer to Einsum, for some sort of array class that understands the difference between a 4D array of scalars, a 3D array of vectors, and a 2D array of matrices... The difference between the axes that broad-cast and the axes that can sum when you hit them with an @ ... or something like that. Just a thought. Einsum is fantastic by the way, totally worth learning and using. Mark Daoust ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
