Thank you for working towards giving us extra tools to help us prevent regressions. It is definitely something we can put to good use. However, the total time it takes to test a file is somewhat of a bad metric as adding lots of (sometimes relatively computationally intensive) tests to a file will make it look like it had a regression, which is the case you mentioned for Lie algebras (e.g., see https://trac.sagemath.org/ticket/22883). Perhaps it would be better to normalize the differences based on the change in the number of doctests, so when doctests are added, it is less likely to show a large increase.
For me, the x-axis marks on the graphs are obscured by the top bar, so it makes it harder to understand what the commits are referring to. Best, Travis On Saturday, August 18, 2018 at 12:47:29 AM UTC+10, Julian Rüth wrote: > > Hello. > > At #25262 [1] I am trying to use airspeed velocity (asv) [2] to track > performance regressions in Sage. In a first metaclass hack, I had > asv benchmark all "long time" doctests in Sage for all the tags between > 8.2 and 8.3 (i.e., all the beta and rc versions of 8.3.) > > I ran this on my laptop computer over the past 48 or so hours. Since I > used it for quite some other stuff there is a lot of noise in the data > but you might still like to have a look at the result [3]. Strangely, > the 8.2 tag did not run so you can't really see 8.2 as a baseline > unfortunately. > > Btw., the tool detected serious regressions in > finite_dimensional_lie_algebras_with_basis and classical_lie_algebra. If > care about these you might want to double check what happened there. > > If you think it's a good idea to do this kind of regression tracking or > would like to see this being done differently, please join the > discussion at #25262 :) > > > julian > > [1] https://trac.sagemath.org/ticket/25262 > [2] https://github.com/airspeed-velocity/asv > [3] https://saraedum.github.io/sage/ > -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
