What would be really useful here are *de-motivational* examples. What are some common operations that can't be done with this model? These would make fantastic test cases to see if sub-classing in SymPy is feasible or not.
Aaron brought up the associativity tests. These work just fine. What are some more complex tasks? An example that comes to mind is calling simplify on a matrix expression. We have lots of other *simp functions. Adding a matsimp function seems normal. Is simplify the kind of function for which it would be acceptable to add a "if expr.is_Matrix: return matsimp(expr)" statement? If this sort of thing isn't acceptable and if we want to allow the user to call the standard simplify function on matrix expressions then we have a problem. ---------- Forwarded message ---------- From: Brian Granger <[email protected]> Date: Wed, Jul 13, 2011 at 12:53 PM Subject: Re: [sympy] Re: Matrix Symbol To: [email protected] Matthew, On Wed, Jul 13, 2011 at 6:39 AM, Matthew Rocklin <[email protected]> wrote: > I have op_priority set to 11 so that basic arithmetic operations do use > MatMuls and MatAdds. MatMul and MatAdd check shape, then call regular > Mul/Add __new__ methods to take advantage of the already built flattening > functions. After that's done they go through and change classes back to > MatMul/MatAdd in case a normal Mul/Add was called within the __new__. So far > this has worked very well. All of the associativity cases 1-6 throw the > correct error. The more complex expressions that I've tried have also worked > fine. > Calling Mul or Add explicitly does break things. There's no entry point for > me to insert clever logic as I don't want to edit core functions. My > solution here is simply to say "Use MatMul or MatAdd instead, or, if you > must use Mul/Add, call matrixify on the expression afterwards". Matrixify > will be guaranteed to return a Matrix expression if MatrixSymbols exist > within a normal Expression. I plan to put it in a few key functions like > simplify and expand. > > Is this matrixify solution too much of a hack? Hopefully I can hide it away > within a few key functions so that the user never needs to know about it > unless they explicitly use Add or Mul. If they're using Add or Mul then I > expect them to be clever enough to handle matrixify. > I'm working on this branch . As I understand it Matrixify goes through an expression and makes sure that your Mul/Add/Pow subclasses are used. If that is the case, I advise against it because the second a user starts to do things with the expression they will have to call Matrixify again. Again, I would just use the default Mul/Add/Pow and provide standalone functions like check_shape that perform a specific type of action on an expression. Cheers, Brian > On Wed, Jul 13, 2011 at 12:25 AM, Aaron Meurer <[email protected]> wrote: >> >> One thing that I have noticed with regard to overriding __mul__ and >> __rmul__ (for example) is that you can never completely control what >> happens to your class because of association. For example, suppose >> that A and B are MatrixExprs and x is some Expr object (say, a >> Symbol). Suppose that A*B should give a ShapeError, and that x is a >> scalar, which does not matter where it is in the expression. Then you >> have to program it so that all of the following give ShapeError: >> >> 1. x*(A*B) >> 2. (x*A)*B >> 3. (A*x)*B >> 4. A*(x*B) >> 5. (A*B)*x >> 6. A*(B*x) >> >> Let us look at these. Obviously, 1 and 5 will work, since you have >> complete control over A.__mul__(B). Similarly, in 4 and 6, you will >> end up with A.__mul__(Mul(x, B)), but this is not a problem, since you >> can make A.__mul__ check for the left-most noncommutative in a Mul. >> >> But what about 2 and 3? Here, we have __mul__ being called by a Expr >> type, namely, Mul(x, A). You want to get B.__rmul__(Mul(x, A)), since >> you have implemented shape checking logic in MatrixExpr.__(r)mul__. >> But b.__rmul__(a) is called with a*b only if a.__mul__(b) returns >> NotImplemented. So, basically, we need Mul.__mul__(MatrixExpr) to >> return NotImplemented. Well, let us look at the code for Mul.__mul__ >> (actually Expr.__mul__): >> >> @_sympifyit('other', NotImplemented) >> @call_highest_priority('__rmul__') >> def __mul__(self, other): >> return Mul(self, other) >> >> Well, this is not very helpful, because the logic is buried in some >> decorators. So let us look at the code for those decorators. I will >> not bore you by pasting the entire code here (you can see it in >> sympy/core/decorators.py), but basically, _sympifyit makes tries to >> sympify other, and makes __mul__ return NotImplemented if it fails. I >> am not delving into this, because sympify would not fail for >> MatrixExpr (otherwise, we would actually have a problem doing any form >> of x*A*B). >> >> So call_highest_priority seems to be a better bet. This decorator I >> will also not paste here, but it basically lets you define >> ._op_priority on your object, and if it is greater than Expr's (which >> is quite arbitrarily set to 10.0), then it will call >> other.__rmul__(self). >> >> But wait, there's more. _op_priority can make the above cases work, >> but * is not the only way that SymPy multiplies things. You will also >> have to deal with variations on Mul(x, A, B). Mul completely ignores >> _op_priority. Unfortunately, even if this may seem like a more >> esoteric way to multiply things that you can just recommend users >> avoid, it is used internally a lot, because Mul(*args) is more >> efficient than reduce(operator.mul, args). >> >> Thus, you see that it is quite impossible to make things work 100% of >> the time without modifying the core. And actually, because of the Mul >> thing that would not work at all and that is called by so many core >> functions and methods, you will not even get something like things >> working 90% of the time, but instead things will break when used a >> certain way, and you will have hard to track bugs. >> >> Aaron Meurer >> >> On Tue, Jul 12, 2011 at 3:08 PM, Matthew Rocklin <[email protected]> >> wrote: >> > Subclassing Expr has some issues as well. This is what Brian was >> > referring >> > to. Within all of our code we use Add and Mul and don't check if instead >> > we >> > should use some subclass of Add or subclass of Mul. If I feed a matrix >> > expression into these objects then the special matrix structure is lost. >> > This happens if, for example, you call simplify on a matrix expression. >> > I think I can get around this though with a few well placed "matrixify" >> > functions. Matrixify is a function which goes through the expression >> > tree >> > and makes appropriate fixes. I've had good success so far with a very >> > very >> > basic Matrixify function. >> > Brian, did you have particular horror stories trying to subclass Expr? >> > I'm >> > enthusiastic about my approach but you seemed to have a bad experience. >> > Can >> > you suggest difficult test cases that I should think about? >> > >> > >> > On Fri, Jul 8, 2011 at 3:22 PM, Aaron Meurer <[email protected]> wrote: >> >> >> >> On Fri, Jul 8, 2011 at 4:00 AM, SherjilOzair <[email protected]> >> >> wrote: >> >> > There is something I'm doing as part of my project which maybe be >> >> > useful. I'm implementing a Matrix_ wrapper class which will wrap over >> >> > low-level matrices. Its being written to replace the current Matrix >> >> > class. Algorithmic code and user-level code is being separated into >> >> > different classes. Currently, we have three internal matrices, the >> >> > DenseMatrix( a modified form of the current Matrix class), DOKMatrix >> >> > and LILMatrix. These three are essentially Data internal matrices. >> >> > The MatrixSymbol can be another object that Matrix_ (later Matrix) >> >> > would use as internal. >> >> > >> >> > Of what I understand from this discussion about the Matrix Symbol, we >> >> > need to have an object that will be treated like a Matrix everywhere, >> >> > but without the internal data. Space is being made in the Matrix >> >> > module for such an object, but for it work nicely it needs to >> >> > interact >> >> > nicely with Expr objects. If its a problem subclassing Expr and >> >> > friends, I'm +1 about making a separate codebase for matrix >> >> > expressions. It is, after all, a different algebra. >> >> > >> >> > I'm not very familiar with Expr, Add, etc. but I think a separate >> >> > algebra MatrixExpr, MatrixAdd, etc. could be developed by >> >> > copy-pasting >> >> > some code from Expr, Add, and modifying according to need. I say >> >> > 'copy- >> >> > paste' and not subclassing as the code would be need to be modified >> >> > much before it can work on matrices. >> >> >> >> As has been discussed before, this is a bad idea. Just subclass Expr. >> >> >> >> Aaron Meurer >> >> >> >> > >> >> > Presently, I'm quite busy with this new encapsulation class Matrix_ >> >> > and can only think of working fulltime on MatrixSymbol after I've >> >> > merged in my work on DOKMatrix, LILMatrix and the ongoing work on >> >> > Matrix_. >> >> > >> >> > Matthew, you could set up a wiki page on 'matrix expressions' with >> >> > some example code in it. 'symbolic matrices' can mean a few number of >> >> > things, and is somewhat vague. It would be good if everyone could see >> >> > what sort of concrete things are expected from matrix expressions. >> >> > Approaches to implement Matrix Algebra, with pros and cons listed, >> >> > could be added there for discussion, suggestion and improvement. >> >> > >> >> > Regards, >> >> > Sherjil Ozair >> >> > >> >> > -- >> >> > You received this message because you are subscribed to the Google >> >> > Groups "sympy" group. >> >> > To post to this group, send email to [email protected]. >> >> > To unsubscribe from this group, send email to >> >> > [email protected]. >> >> > For more options, visit this group at >> >> > http://groups.google.com/group/sympy?hl=en. >> >> > >> >> > >> >> >> >> -- >> >> You received this message because you are subscribed to the Google >> >> Groups >> >> "sympy" group. >> >> To post to this group, send email to [email protected]. >> >> To unsubscribe from this group, send email to >> >> [email protected]. >> >> For more options, visit this group at >> >> http://groups.google.com/group/sympy?hl=en. >> >> >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups >> > "sympy" group. >> > To post to this group, send email to [email protected]. >> > To unsubscribe from this group, send email to >> > [email protected]. >> > For more options, visit this group at >> > http://groups.google.com/group/sympy?hl=en. >> > >> >> -- >> You received this message because you are subscribed to the Google Groups >> "sympy" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/sympy?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/sympy?hl=en. > -- Brian E. Granger Cal Poly State University, San Luis Obispo [email protected] and [email protected] -- You received this message because you are subscribed to the Google Groups "sympy" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/sympy?hl=en. -- You received this message because you are subscribed to the Google Groups "sympy" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/sympy?hl=en.
