On Mon, Aug 3, 2020 at 5:01 PM Oscar Benjamin <oscar.j.benja...@gmail.com> wrote: > > On Mon, 3 Aug 2020 at 21:15, Aaron Meurer <asmeu...@gmail.com> wrote: > > > > - We have already done some degree of high-level planning in the past. > > It just hasn't been organized. For example, you can see there are some > > documents on the wiki that go over design ideas for various things. > > Some of them are implemented and some not. > > I took a look through the wiki and there are many different pages but > I don't see any that looks like a well specified design that has been > reviewed and agreed on. Since you mentioned the new assumptions > (below) I see that the new assumptions are one of the more discussed > topics having three separate pages: > https://github.com/sympy/sympy/wiki/_pages > https://github.com/sympy/sympy/wiki/Assumptions > https://github.com/sympy/sympy/wiki/Assumptions-Handlers > https://github.com/sympy/sympy/wiki/Assumptions-history > > Mostly though the pages on the wiki look like collections of > first-draft notes each written by a single author, reviewed by no one > and then never updated again. Those can be useful in many situations > but they're really not the kind of thing I'm thinking about.
Well usually if there was a discussion it happened on the list or on an issue, and wasn't necessarily put into the document. So I think having this sort of thing formalized so that the discussion always has to be summarized in the document is good. Here are some other examples that don't relate to assumptions: https://github.com/sympy/sympy/wiki/UD-series (as well as a bunch of other pages that start with "UD") https://github.com/sympy/sympy/wiki/Args-Invariant (this one actually does have some discussion on it) https://github.com/sympy/sympy/wiki/Canonicalization > > > And of course, every GSoC > > project starts with a GSoC proposal that outlines what will be done in > > the project. > > There are also lots of GSOC proposals on the wiki but those are also > different. A GSOC proposal is used to select a student who is then > supervised by the mentor(s) who will then drive the high-level design. > When evaluating proposals I treat them as a demonstration that the > student understands roughly what they are volunteering to do (under > instruction) rather than as a specification of what will actually be > done. I wasn't suggesting that the GSoC proposals are what this should look like. I'm only pointing out that the idea of writing down proposals for how to implement stuff in SymPy before doing it isn't completely new. GSoC proposals are actually a good place to look to get an idea of how far an original proposal can deviate from what was actually implemented, and why. > > > - I would look at how other similar projects do this, especially > > projects in the scientific ecosystem. NumPy, matplotlib, and > > scikit-learn > > ... > > We should also talk with members of those > > communities to see how they feel about the processes and what works > > and what doesn't. > > I'll give that a go. > > > I think Python PEPs aren't a great model to mimic > > directly. The Python community of developers is much larger, the > > community of users is *much* larger, and they also care much more > > about not breaking compatibility and being conservative about adding > > new features than we do. > > I think we *should* be more conservative about adding features. For > the most part, time would be better spent improving the existing > features and documentation. We should also be more conservative about > breaking compatibility. The community of users for sympy is growing > all of the time as is the number of downstream libraries. PyPI now > records on the order of a million downloads per month for sympy: > https://pypistats.org/packages/sympy > I'm sure the vast majority of that is CI but that shows how many > projects now have sympy as a dependency. Of course it's easier to be > more conservative about breaking compatibility if you are also more > conservative about adding features in the first place. Sure, any library should care about these things to some degree. But someone pointed out a PEP earlier in this thread for adding a single function to the Python standard library, a function which would probably be used by less than 1% of Python developers. I don't think we need to do that. If someone wants to add, say, a new well established special function to the functions submodule, we shouldn't need a design document to justify it. Actually, I would start out only using SymPEPs for very large changes, and see how it works. Then based on how it pans out we can see if it makes sense for more targeted changes. > > > - My biggest concern is this: it is often impossible to really know > > what will work unless we try to implement it. > > We would have to be prepared to discuss or allow changes as needed on > an ongoing basis until implementation is complete. I still think it's > good to start with a proper specification and consider that in full > with its rationale before implementing anything beyond proof of > concept. Problems will always emerge during implementation but better > design can help to avoid fundamental problems. > > > I think the assumptions > > are a great example of this. The idea of what the "new assumptions" > > should look like has changed several times over the years, because > > every time it has become clear that the current idea isn't tenable. > > This has led to a rather unfortunate state of things where we have two > > concurrent systems. But I don't really see how we could have come up > > with a "new assumptions" idea that would actually work without first > > trying out what we did and seeing why it didn't work. > > I wasn't around so I won't comment on discussion or design that took > place in the past. What I will say is that at this stage we really do > need something like a SymPEP to plan out what to do with assumptions > *going forwards*. There needs to be a discussion and decisions need to > be made and the result should be distilled and written down for all to > see. The implications for developers and users need to be spelled out > somewhere and there needs to be an implementation plan so that > contributors can get on with that work. > > Last year there was a GSOC student working on making the new > assumptions faster which is a straight-forward improvement that we can > all understand without needing to think about complicated design > questions. This year we have 6 GSOC students and none of them is > working on assumptions even though it is high priority. I think that's > quite simply because no one knows what a project on the new > assumptions would do at this stage. (Design not agreed, no > implementation plan) Some of the assumptions stuff predates me too (it's that old), but I believe the original idea was that the new assumptions would replace the old assumptions completely. I think it's now clear that this is a bad idea, because at least syntactically, the old assumptions are very convenient for the cases that they work for. And even ignoring that, swapping out a whole piece of the core like that is an impossible task, because it breaks too much (the devs from back then already went through this with the so-called "new core", so you think they would have known better). But there were also design missteps, like making the new assumptions handlers work too much like the old assumptions handlers, and not being careful about definitions. Making them separate from each other was also a mistake. If we had realized that the old assumptions syntax should stay, we could have made the new assumptions call the old assumptions as we do now, and avoided a lot of inconsistencies between them. There's also been disagreement, even now, about how good it is to have assumptions be separate from expressions. That's the basic design principle of the new assumptions. I don't know if trying to write a design document from the front would have avoided any of these issues, because it isn't clear to me that anyone knew they would be issues from the start. The main thing I think is that no one accurately predicted how difficult it would be to make any kind of change to the current (old) assumptions system. Even small changes that appear trivial can result in having to modify the entire codebase, and the performance of it is critical as well. Maybe we can do something now. We've definitely layed out some plans of how to proceed in previous GSoC projects, which were not all completely finished. I don't know if having a proposal would help to get someone to do it or not. If it does, I can think of a few high priority ideas that would be good to write SymPEPs for (see https://www.sympy.org/en/roadmap.html). > > > - I have a few gripes with the Python PEP process which I think we > > should try to avoid. > > All agreed. > > > - The SEP abbreviation is probably already used by some other > > projects. How about SymPEP? > > Yes, I like the pronunciation of that. > > > Oscar > > -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sympy+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sympy/CAHVvXxR7v%2BTmt91wsxX5AehXmEfLAL7Hu%2BUuqf92bdfJv2zvfA%40mail.gmail.com. -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sympy+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAKgW%3D6Jgkdruy7%3DJ5LS0bNLk2_cQW0pFOvwCaKenx%2BrBbA-2%3Dg%40mail.gmail.com.
