On Mon, May 20, 2019 at 3:34 PM Aaron Meurer <asmeu...@gmail.com> wrote:
> Thanks. My biggest question has been how we can do it in the cloud. > Most CI services run multiple concurrent jobs on the same machine, > making the performance inconsistent. Does drone.io let you have a > dedicated machine? > Drone.io has a shared cloud offering which we don't want. As you mentioned we can buy a cheap dedicated machine and install drone on it for free. Drone might support dedicated machines on the cloud, but I'm not sure. Travis-CI supports dedicated machines, but we looked at this for conda-forge and they were quite expensive. > If we can't do it in the cloud, another alternative would be to buy a > cheap dedicated machine that we can provision ourselves. We have funds > in our NumFOCUS account to do either of these, so the question is > which will be easier for us to set up and maintain. What do the sumpy > benchmarks run on? > It's a dedicated machine somewhere in the university running a gitlab runner service. Not a cloud machine. Isuru > Aaron Meurer > > On Mon, May 20, 2019 at 1:59 PM Isuru Fernando <isu...@gmail.com> wrote: > > > > Hi, > > > > I setup asv in Gitlab CI for https://gitlab.tiker.net/inducer/sumpy. It > runs the PR head and master (cached) and errors if there is a test with 10% > regression. > > Script is at > https://gitlab.tiker.net/inducer/ci-support/blob/master/build-and-benchmark-py-project.sh. > It runs on a dedicated Gitlab runner. > > > > For sympy, you'll have to have a dedicated machine for benchmarking and > some CI service that integrates with github installed. (Maybe Drone CI > which is free for 15000 builds per year. > https://drone.io/enterprise/starter/#features). You'll have to limit to 1 > concurrent job to avoid multiple PRs affecting each other. > > > > Isuru > > > > On Mon, May 20, 2019 at 1:20 PM Aaron Meurer <asmeu...@gmail.com> wrote: > >> > >> Hello everyone. > >> > >> I would like to start looking into setting up some kind of > >> benchmarking CI for SymPy. > >> > >> Ideally the CI would work just like Travis or codecov. It would run > >> the benchmark suite on every pull request, and report if there are any > >> major performance regressions. > >> > >> If anyone has any thoughts on how to go about doing this, let me know. > >> If anyone knows of any projects that already do this, let me know that > >> as well. > >> > >> Finally, if anyone is interested in helping out with this, please let > me know. > >> > >> Aaron Meurer > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups "sympy" group. > >> To unsubscribe from this group and stop receiving emails from it, send > an email to sympy+unsubscr...@googlegroups.com. > >> To post to this group, send email to sympy@googlegroups.com. > >> Visit this group at https://groups.google.com/group/sympy. > >> To view this discussion on the web visit > https://groups.google.com/d/msgid/sympy/CAKgW%3D6Lks6y8CKbiBRauwD1DQ5%3DcnMzQNbXuhhD9MefbN%3DQQrQ%40mail.gmail.com > . > >> For more options, visit https://groups.google.com/d/optout. > > > > -- > > You received this message because you are subscribed to the Google > Groups "sympy" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to sympy+unsubscr...@googlegroups.com. > > To post to this group, send email to sympy@googlegroups.com. > > Visit this group at https://groups.google.com/group/sympy. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/sympy/CA%2B01voNuyrPRemoJ-3Q0HgucebO4%3DB1g0JScFqs0WwMj3-5odg%40mail.gmail.com > . > > For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sympy+unsubscr...@googlegroups.com. > To post to this group, send email to sympy@googlegroups.com. > Visit this group at https://groups.google.com/group/sympy. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sympy/CAKgW%3D6L%3Ddx%3DXpxEH%2BBG%2B9nW%3DaLWceAsyRSv2xD6Zm-iQCHmDLw%40mail.gmail.com > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sympy+unsubscr...@googlegroups.com. To post to this group, send email to sympy@googlegroups.com. Visit this group at https://groups.google.com/group/sympy. To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CA%2B01voNzw3ZzZnan-%3DPMeYtGHD7zJ6O9UDrCdPsBMgMCYCi34A%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
