Hi Julian, On Saturday, August 18, 2018 at 5:59:55 AM UTC+2, Travis Scrimshaw wrote: >> >> Thank you for working towards giving us extra tools to help us prevent >> regressions. It is definitely something we can put to good use. However, >> the total time it takes to test a file is somewhat of a bad metric >> > Where does it show the total time for a file for you? Currently, this is > always for a block of doctest, i.e., typically all the tests defined by a > single docstring. When a docstring changes the old and the new docstring > are not going to show up as the same benchmark as the included hash value > has then changed. >
>From your post above, you said, 'all "long time" doctests,' which means to me all tests in the file. So if what you're saying is true, I have no idea how to interpret this data then. When these docstrings change, how does it keep track? What if something like just a variable name changes? How do we really know it is comparing apples to apples? > > >> as adding lots of (sometimes relatively computationally intensive) tests >> to a file will make it look like it had a regression, which is the case you >> mentioned for Lie algebras (e.g., see >> https://trac.sagemath.org/ticket/22883). Perhaps it would be better to >> normalize the differences based on the change in the number of doctests, so >> when doctests are added, it is less likely to show a large increase. >> > > >> For me, the x-axis marks on the graphs are obscured by the top bar, so it >> makes it harder to understand what the commits are referring to. >> > Could you provide me with a screenshot to explain the confusion? The > x-axis should be marked with the tags of the 8.3 cycle, i.e., 8.3.beta0 ... > 8.3.rc0 ... 8.3. (There is some noise due to some extra docker tags that > exist in my local repository.) > > See the attached screenshot. It works fine on my desktop with its higher screen resolution, but not on my laptop. Best, Travis -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
