Hi, I setup asv in Gitlab CI for https://gitlab.tiker.net/inducer/sumpy. It runs the PR head and master (cached) and errors if there is a test with 10% regression. Script is at https://gitlab.tiker.net/inducer/ci-support/blob/master/build-and-benchmark-py-project.sh. It runs on a dedicated Gitlab runner.
For sympy, you'll have to have a dedicated machine for benchmarking and some CI service that integrates with github installed. (Maybe Drone CI which is free for 15000 builds per year. https://drone.io/enterprise/starter/#features). You'll have to limit to 1 concurrent job to avoid multiple PRs affecting each other. Isuru On Mon, May 20, 2019 at 1:20 PM Aaron Meurer <asmeu...@gmail.com> wrote: > Hello everyone. > > I would like to start looking into setting up some kind of > benchmarking CI for SymPy. > > Ideally the CI would work just like Travis or codecov. It would run > the benchmark suite on every pull request, and report if there are any > major performance regressions. > > If anyone has any thoughts on how to go about doing this, let me know. > If anyone knows of any projects that already do this, let me know that > as well. > > Finally, if anyone is interested in helping out with this, please let me > know. > > Aaron Meurer > > -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sympy+unsubscr...@googlegroups.com. > To post to this group, send email to sympy@googlegroups.com. > Visit this group at https://groups.google.com/group/sympy. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sympy/CAKgW%3D6Lks6y8CKbiBRauwD1DQ5%3DcnMzQNbXuhhD9MefbN%3DQQrQ%40mail.gmail.com > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sympy+unsubscr...@googlegroups.com. To post to this group, send email to sympy@googlegroups.com. Visit this group at https://groups.google.com/group/sympy. To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CA%2B01voNuyrPRemoJ-3Q0HgucebO4%3DB1g0JScFqs0WwMj3-5odg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
