On Mon, May 20, 2019 at 2:45 PM Isuru Fernando <[email protected]> wrote: > > > On Mon, May 20, 2019 at 3:34 PM Aaron Meurer <[email protected]> wrote: >> >> Thanks. My biggest question has been how we can do it in the cloud. >> Most CI services run multiple concurrent jobs on the same machine, >> making the performance inconsistent. Does drone.io let you have a >> dedicated machine? > > > Drone.io has a shared cloud offering which we don't want. As you mentioned we > can buy a cheap dedicated machine and install drone on it for free. > > Drone might support dedicated machines on the cloud, but I'm not sure. > Travis-CI supports dedicated machines, but we looked at this for conda-forge > and they were quite expensive.
I see, so Drone is a CI software, similar to gitlab CI or Azure. We will need to decide which of those is the most appropriate to use. I've heard good things about GitLab CI. Can it not be used with GitHub repos? Regarding the hardware, the question is if there are cloud providers that provide dedicated machines, and how much they cost. If anyone has any suggestions for this let me know. Alternately, what's a reasonably cheap box we can put together to host ourselves, and what are the tradeoffs there? Aaron Meurer > >> >> If we can't do it in the cloud, another alternative would be to buy a >> cheap dedicated machine that we can provision ourselves. We have funds >> in our NumFOCUS account to do either of these, so the question is >> which will be easier for us to set up and maintain. What do the sumpy >> benchmarks run on? > > > It's a dedicated machine somewhere in the university running a gitlab runner > service. Not a cloud machine. > > Isuru > >> >> Aaron Meurer >> >> On Mon, May 20, 2019 at 1:59 PM Isuru Fernando <[email protected]> wrote: >> > >> > Hi, >> > >> > I setup asv in Gitlab CI for https://gitlab.tiker.net/inducer/sumpy. It >> > runs the PR head and master (cached) and errors if there is a test with >> > 10% regression. >> > Script is at >> > https://gitlab.tiker.net/inducer/ci-support/blob/master/build-and-benchmark-py-project.sh. >> > It runs on a dedicated Gitlab runner. >> > >> > For sympy, you'll have to have a dedicated machine for benchmarking and >> > some CI service that integrates with github installed. (Maybe Drone CI >> > which is free for 15000 builds per year. >> > https://drone.io/enterprise/starter/#features). You'll have to limit to 1 >> > concurrent job to avoid multiple PRs affecting each other. >> > >> > Isuru >> > >> > On Mon, May 20, 2019 at 1:20 PM Aaron Meurer <[email protected]> wrote: >> >> >> >> Hello everyone. >> >> >> >> I would like to start looking into setting up some kind of >> >> benchmarking CI for SymPy. >> >> >> >> Ideally the CI would work just like Travis or codecov. It would run >> >> the benchmark suite on every pull request, and report if there are any >> >> major performance regressions. >> >> >> >> If anyone has any thoughts on how to go about doing this, let me know. >> >> If anyone knows of any projects that already do this, let me know that >> >> as well. >> >> >> >> Finally, if anyone is interested in helping out with this, please let me >> >> know. >> >> >> >> Aaron Meurer >> >> >> >> -- >> >> You received this message because you are subscribed to the Google Groups >> >> "sympy" group. >> >> To unsubscribe from this group and stop receiving emails from it, send an >> >> email to [email protected]. >> >> To post to this group, send email to [email protected]. >> >> Visit this group at https://groups.google.com/group/sympy. >> >> To view this discussion on the web visit >> >> https://groups.google.com/d/msgid/sympy/CAKgW%3D6Lks6y8CKbiBRauwD1DQ5%3DcnMzQNbXuhhD9MefbN%3DQQrQ%40mail.gmail.com. >> >> For more options, visit https://groups.google.com/d/optout. >> > >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "sympy" group. >> > To unsubscribe from this group and stop receiving emails from it, send an >> > email to [email protected]. >> > To post to this group, send email to [email protected]. >> > Visit this group at https://groups.google.com/group/sympy. >> > To view this discussion on the web visit >> > https://groups.google.com/d/msgid/sympy/CA%2B01voNuyrPRemoJ-3Q0HgucebO4%3DB1g0JScFqs0WwMj3-5odg%40mail.gmail.com. >> > For more options, visit https://groups.google.com/d/optout. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "sympy" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at https://groups.google.com/group/sympy. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/sympy/CAKgW%3D6L%3Ddx%3DXpxEH%2BBG%2B9nW%3DaLWceAsyRSv2xD6Zm-iQCHmDLw%40mail.gmail.com. >> For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/sympy. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sympy/CA%2B01voNzw3ZzZnan-%3DPMeYtGHD7zJ6O9UDrCdPsBMgMCYCi34A%40mail.gmail.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/sympy. To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAKgW%3D6%2BQ6eHPqERxn6btbncGFYuf5pg_pHKnRvF6%2B5KQTEbVrw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
