Re: [sage-devel] Patching policy
On Thursday, June 7, 2018 at 2:36:35 PM UTC+2, Timo Kaufmann wrote: > > In some cases where upstream has vanished and sage effectiely maintains > the project through patches anyways, it may be a better idea to just fork > the project. > It might be an idea to semi-automatize this, i.e., build a tool that takes a Sage version and creates forks of some packages A,B on github under sage/sage-A, sage/sage-B. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
Re: [sage-devel] Patching policy
2018-06-07 15:06 GMT+02:00 Jeroen Demeyer : > And I'm not saying there should be absolutely no patches, just that they >> should be the very last resort. >> > > I mostly agree with this, it's what I'm already doing. It just depends > where you put the borderline of "very last resort" and there we probably > differ. > For example the `backports.shutil_get_terminal_size` patch: It pretty much only fixes a formatting issue in the error messages right? I don't see that as last resort. > But I don't see how this would help distributions. As long as a package > has at least 1 essential patch (even a 1-liner in the case of GLPK), > distros have to deal with it. > That assumes that there always is an essential patch already. And even if there is, two patches don't have the same cost as one. They have to both be checked, understood, documented and maybe discussed with a maintainer (who does not see sage as his primary priority). -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
Re: [sage-devel] Patching policy
On 2018-06-07 14:36, Timo Kaufmann wrote: For what its worth, here is debians fix (self-labeled as "extra-hacky") for that issue: https://salsa.debian.org/science-team/sagemath/blob/master/debian/patches/dt-version-glpk-4.60-extra-hacky-fixes.patch That's not a fix at all. I could easily come up with a doctest that would break using Debian's "fix". So it's certainly not acceptable for Sage. And I'm not saying there should be absolutely no patches, just that they should be the very last resort. I mostly agree with this, it's what I'm already doing. It just depends where you put the borderline of "very last resort" and there we probably differ. But I don't see how this would help distributions. As long as a package has at least 1 essential patch (even a 1-liner in the case of GLPK), distros have to deal with it. Jeroen. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
Re: [sage-devel] Patching policy
2018-06-07 14:16 GMT+02:00 Jeroen Demeyer : > On 2018-06-07 13:24, Timo Kaufmann wrote: > >> I don't really agree but even if that was the case, the PARI stackwarn >> patch could have been handled through filtering within sage instead >> (which I proposed in that ticket). >> > > You proposed filtering *in the testsuite*. That comes back to my point > about fixing the testsuite: filtering in the testsuite only makes the tests > pass, but it doesn't really fix anything for end users. > That was because in that case I didn't actually see the warning as an error / something to avoid. I simply introduced filtering in the testsuite to avoid having to add warnings to all the tests and making them even more brittle. If you disagree (which you apparently do) we could've instead added filtering directly to the sage<->pari interface. > The problem with your proposal is that in many cases, it is hard or > impossible to work around the bug. For example, I have no idea how to "work > around" build/pkgs/glpk/patches/error_recovery.patch > For what its worth, here is debians fix (self-labeled as "extra-hacky") for that issue: https://salsa.debian.org/science-team/sagemath/blob/master/debian/patches/dt-version-glpk-4.60-extra-hacky-fixes.patch Which kind of supports my points that (1) distributions will have to work around it anyways and (2) the solutions of the distributions are probably not as good as the solutions the sage community could come up with. In that specific case I opted to adopt the glpk patch for nixos because I don't really understand the debian fix (I barely understand the dails of the problem). So if you don't want to patch the upstream package, then the only remaining > option is making Sage worse (either by not implementing the functionality > which requires the patch, or by accepting that a bug cannot be fixed). > And I'm not saying there should be absolutely no patches, just that they should be the very last resort. So if working around the problem is really not possible and there exists no other solution it'll have to be a patch. Just make sure upstream is aware that sage and probably most distributions shipping sage will have to patch their sotware, maybe that will make them reconsider. In some cases where upstream has vanished and sage effectiely maintains the project through patches anyways, it may be a better idea to just fork the project. > I'm biased of course. >> > > We are all biased in our own ways... > > I should also add that my opinion is my opinion only. Other SageMath > developers may have different opinions. Of course. Unless somebody else chimes in to add a different opinion, I think its relatively safe to assume that the other devs think similarly. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
[sage-devel] NTL 11.1.0
Just posted a new version of NTL, version 11.1.0. I've completely re-written the low-level small-prime FFT to use a truncated FFT algorithm. More details here: http://www.shoup.net/ntl/doc/tour-changes.html -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
Re: [sage-devel] Patching policy
On 2018-06-07 13:24, Timo Kaufmann wrote: I don't really agree but even if that was the case, the PARI stackwarn patch could have been handled through filtering within sage instead (which I proposed in that ticket). You proposed filtering *in the testsuite*. That comes back to my point about fixing the testsuite: filtering in the testsuite only makes the tests pass, but it doesn't really fix anything for end users. The problem with your proposal is that in many cases, it is hard or impossible to work around the bug. For example, I have no idea how to "work around" build/pkgs/glpk/patches/error_recovery.patch So if you don't want to patch the upstream package, then the only remaining option is making Sage worse (either by not implementing the functionality which requires the patch, or by accepting that a bug cannot be fixed). I'm biased of course. We are all biased in our own ways... I should also add that my opinion is my opinion only. Other SageMath developers may have different opinions. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
Re: [sage-devel] Patching policy
2018-06-07 11:12 GMT+02:00 Jeroen Demeyer : > I think that your post is focusing too much on tests, as if the only > purpose of Sage is to pass its testsuite. It's actually the other way > around: the purpose of the testsuite is to ensure that Sage functions > correctly. Of course. But the testsuite should always pass as long as sage is "working". Its the best proxy available. Sometimes the test suite is a bit too brittle. By patching the testsuite to accept buggy output anyway, you're not really > fixing anything, just hiding the problem. > If its a functional bug (`2 + 2 = 5`) I agree. But if its just a formatting bug, that should not fail the test suite in my opinion. By the way, the difference that you make between category 2 and 3 of > patches is not so relevant: I would argue that the PARI stackwarn.patch is > more essential (as in: more likely to affect users) than the > re_match_index-issue_27177.patch Python patch. > I don't really agree but even if that was the case, the PARI stackwarn patch could have been handled through filtering within sage instead (which I proposed in that ticket). Thats really my main point: use other solutions if possible. > I would propose to make it a policy to only include sage patches when >> *absolutely necessary*. If ugly workarounds or even monkey-patching is >> necessary >> for that, that sucks. But distributions will usually come up with even >> uglier >> workarounds (since they don't know the code) anyways, just resulting in >> duplication of effort. >> > > There is a constant ongoing tension between downstream (Sage), upstream > (PARI, Python, ...) and distributions (Debian, NixOS, ...). Nobody wants to > be the one needing to fix the problem. So upstream tries to convince > downstream that the problem is on their side, downstream tries to push the > problem to distributions and distributions probably bother both upstream > and downstream. > I'd argue distributions is the worst place to fix problems because that results in duplication of work and the people fixing the issues might not know what they're doing. > You seem to be blaming Sage for patching its dependencies, but in many > cases it would be even more valid to blame upstream for not accepting those > patches (or for not making a new release containing those patches). That > way, the problem is really fixed for everybody: not just Sage but all users > of a package. Or you could blame your distro for not applying that patch > too. > I really don't want to blame anyone (especially not you, I'm grateful that you already reviewed many of my patches). Sorry if it sounds that way. I'm just proposing to choose the most pragmatic solution. Its always best to fix the problem as far upstream as possible: Of course it would be ideal if the upstream packages would accept those patches. But if that is not the case for various reasons, I'm arguing that working around it in sage is the next best place. > Of all Sage developers, I could very well be the one adding most patches > to its packages. Whenever I feel the need for patching a package, I ask > myself what the best solution to the problem is: it could be an upstream > patch or it could be a change in Sage. When it's an upstream patch, I make > sure that it fixes a genuine upstream bug and I submit the patch. In most > cases, the patch is then accepted upstream. I very much agree with that process. As you said, fixing problems upstream fixes them for everybody. > However, when it's not accepted (for whatever reason), I don't want to do > the effort of figuring out a solution without patching the package. I feel > like that is just a waste of time since I already have a working solution > for the problem (patching the package) and working around a problem is > often harder than fixing it. > I'm arguing that (if distributions are considered first class citizens) it is very much not a waste of time. I think the ticket should be blocked until upstream has accepted the patch or it is clear they won't. If the ticket author doesn't work around the problem in the second case, distributions will have to. So the time is spent either way. Even if distributions just adopt the patch to the upstream package, figuring out which patch is needed, adopting it, testing it etc. takes significant time. I understand that your time is very valuable and you can do with it whatever you want and I'm not saying you should solve all the problems. I'm just saying we should adopt a policy and adhere to it as a community. What I'm trying to say is that patching a package in Sage is always a very > deliberate conscious choice and not just "whatever, let's patch this > package". So while I certainly understand the concerns of the > distributions, I'm personally not really willing to change anything. > I'm sad to hear that. I think it would be best for sage and gain it a lot of users and potential contributors if the community would invest some effort into being less
Re: [sage-devel] random_element and randtest_element
On Wed, 6 Jun 2018, Travis Scrimshaw wrote: Heisenfailures are very difficult to debug: One run, doctests fail. You make a small tweak (one that changes the random seed), which you think fixes the issue, and then tests pass. However, if it is a random test, then you have no idea if you have fixed the problem or not. True, we should somehow save the random seed used and give it with a failure notice. I have done something to help random testing: sage: from sage.tests.finite_poset import test_finite_lattice sage: L = posets.RandomLattice(10, 0.98) sage: test_finite_lattice(L) is None # Long time True We could check, for example, that matrix AB is invertible iff both A and B are, and so on. * * * Another thing, should we try to classify bugs we found? I mean that could we get insight of places to look for possible other bugs? -- Jori Mäntysalo
Re: [sage-devel] Patching policy
I think that your post is focusing too much on tests, as if the only purpose of Sage is to pass its testsuite. It's actually the other way around: the purpose of the testsuite is to ensure that Sage functions correctly. By patching the testsuite to accept buggy output anyway, you're not really fixing anything, just hiding the problem. By the way, the difference that you make between category 2 and 3 of patches is not so relevant: I would argue that the PARI stackwarn.patch is more essential (as in: more likely to affect users) than the re_match_index-issue_27177.patch Python patch. I would propose to make it a policy to only include sage patches when *absolutely necessary*. If ugly workarounds or even monkey-patching is necessary for that, that sucks. But distributions will usually come up with even uglier workarounds (since they don't know the code) anyways, just resulting in duplication of effort. There is a constant ongoing tension between downstream (Sage), upstream (PARI, Python, ...) and distributions (Debian, NixOS, ...). Nobody wants to be the one needing to fix the problem. So upstream tries to convince downstream that the problem is on their side, downstream tries to push the problem to distributions and distributions probably bother both upstream and downstream. You seem to be blaming Sage for patching its dependencies, but in many cases it would be even more valid to blame upstream for not accepting those patches (or for not making a new release containing those patches). That way, the problem is really fixed for everybody: not just Sage but all users of a package. Or you could blame your distro for not applying that patch too. Of all Sage developers, I could very well be the one adding most patches to its packages. Whenever I feel the need for patching a package, I ask myself what the best solution to the problem is: it could be an upstream patch or it could be a change in Sage. When it's an upstream patch, I make sure that it fixes a genuine upstream bug and I submit the patch. In most cases, the patch is then accepted upstream. However, when it's not accepted (for whatever reason), I don't want to do the effort of figuring out a solution without patching the package. I feel like that is just a waste of time since I already have a working solution for the problem (patching the package) and working around a problem is often harder than fixing it. What I'm trying to say is that patching a package in Sage is always a very deliberate conscious choice and not just "whatever, let's patch this package". So while I certainly understand the concerns of the distributions, I'm personally not really willing to change anything. Jeroen. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
[sage-devel] Re: Suggestion for the SageMath website
> > > On side note: Is there any problem with alphabetical order if the whole > list is there? > > Yes in my opinion. The animation takes 180 sec to display the whole list. I think that's way longer than the average time people stay on the home page. The result is a inequal exposure because the firsts in the list are the most viewed. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
