On Thu, Apr 14, 2022 at 3:12 PM Oscar Benjamin <[email protected]> wrote: > > On Thu, 14 Apr 2022 at 20:30, Aaron Meurer <[email protected]> wrote: > > > > On Thu, Apr 14, 2022 at 12:26 PM Oscar Benjamin > > <[email protected]> wrote: > > > > > > As an example it would be trivial in SymPy to make substitutions > > > involving large expressions and many replacements much faster with a > > > small redesign. The problem though is backwards compatibility: each > > > expression class can implement its own _eval_subs method so for > > > backwards compatibility the subs implementation must recurse in the > > > same slow way once throughout the whole tree for each replacement that > > > is to be made. (There are ways to work around this but the design > > > makes it harder than it should be.) > > > > Just to be clear, isn't this already fixed with xreplace, which does > > only replacement with no smart substitution and no dispatching to > > _eval methods? That isn't to say xreplace itself couldn't be more > > performant, which I'm sure it could. > > Even xreplace runs the evaluation code which is baked into __new__.
Right, so the "redesign" isn't so much about the substitution but rather in removing automatic evaluation. You could write a substitution algorithm right now that completely ignores the constructors, and would be very fast. B then it would be easy to do (x + y).subs(y, x) and get x + x instead of 2*x, which in the current SymPy codebase can cause problems because everything expects evaluated expressions. > Also it recurses down through all of the args in the tree which can in > many cases be done much more efficiently if the expression is > represented as a DAG rather than a tree. Not sure that's so hard to do with the current design. I think it would be possible to add some optimizations to effectively de-duplicate identical subexpressions. It's already basically possible to do this, you just have to assume that == is not a slow operation (which it mostly isn't, although it could be faster). Aaron Meurer > > > > This is a small example of a more general problem. Using classes means > > > you have to use the interfaces of the class which means that efficient > > > algorithms needing something not provided by that public interface are > > > impossible. Even worse both SymPy and SymEngine allow the > > > *constructors* of those classes to do nontrivial work without > > > providing any good way to override that. This means that you can't > > > even represent a symbolic expression without allowing arbitrary code > > > execution: it's impossible to optimise higher-level algorithms if you > > > have so little control over execution from the outside. > > > > What's your suggested alternative? It seems like you on the one hand > > are saying that it's not extensible enough (I guess because any > > extension has to be in C++) and at the same time that it's too > > extensible, because custom classes can do anything. > > > > I think there is a balance in general between extensibility and > > performance. The only way to make something performant is to limit the > > domain of what can be expressed so that more efficient data structures > > can be used. > > I think it's possible to get more extensibility and more performance > by taking a different approach. It's not that SymPy can't be extended: > it's just not easy to extend. The Python/SymPy knowledge required to > be able to create new symbolic classes is too great for "ordinary > users" to make use of it. > > I think that what is needed is a core that is well-defined in its > scope but designed fundamentally around extensibility. The core > operations that manipulate expressions should be implemented in a > low-level language but it should be possible to define and control > their behaviour precisely from a higher level. > > -- > Oscar > > -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sympy/CAHVvXxS%2BCo%2BE5hxOxM4RfjZgH0XkryQtnTtH7n5Az21dNygr%2BQ%40mail.gmail.com. -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAKgW%3D6%2BEccpPcyXp4Cz%3DG%3DSUZYV97yMdtJmCZrtOoGy%2BMFMbYw%40mail.gmail.com.
