Re: [sage-devel] Continuous benchmarking
On Sat, Jun 1, 2019 at 12:28 PM Volker Braun wrote: > > Any kind of shared cpu container solution like openstack is also going to be > unsuitable for benchmarking, I think. There is going to be lots of background > stuff that the test process has no control / visibility of. Yes-and-no. It depends on the types of benchmarks. If you want very fine-grained performance benchmarks on things like I/O or CPU time spent on a specific task it's a no-go. But the nice thing about ASV is it can also help you spot major trends in performance changes. Especially if you run it on multiple platforms. Then it can be very easy to spot common trends, e.g. when a specific change causes noticeable performance regressions in some algorithm or other. So for I think for something like ASV it can still be useful. But of course having dedicated machines for it is by far more ideal. > On Friday, May 31, 2019 at 5:31:22 PM UTC+2, E. Madison Bray wrote: >> >> On Thu, May 30, 2019 at 8:07 AM Samuel Lelièvre >> wrote: >> > >> > Dear sage-devel, >> > >> > This blog post on continuous benchmarking looks interesting! >> > >> > https://medium.com/@wolfv/building-an-open-source-continuous-benchmark-system-717839093962 >> > >> > There have been related discussions on this list in the past, >> > for example around airspeed velocity: >> > >> > https://groups.google.com/forum/#!searchin/sage-devel/airspeed$20velocity%7Csort:date >> >> I agree of course per previous discussions about it. It would be nice >> to have the infrastructure to do continuous *anything* outside the >> Sage buildbots. >> >> I've all but given up on Paris-Sud's OpenStack infrastructure. >> Everything I try to do with it regarding CI (be it the Cygwin >> buildbot, or gitlab runners) breaks constantly and I don't think it's >> anything I'm doing wrong. E.g. recently (within the last few weeks) >> they seem to have fiddled with some firewall rules such that now it >> terminates long-running TCP/IP connections. At least, that's my >> guess. I haven't been able to investigate thoroughly yet but it's >> basically causing all my CI processes to lose connections with their >> master servers mid-build. >> >> We need reliable infrastructure for Sage builds. > > -- > You received this message because you are subscribed to the Google Groups > "sage-devel" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sage-devel+unsubscr...@googlegroups.com. > To post to this group, send email to sage-devel@googlegroups.com. > Visit this group at https://groups.google.com/group/sage-devel. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sage-devel/7f24486d-8295-44e7-ab7f-fd99f88c4ff7%40googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAOTD34a%2BWAey_PxxO%3Dbx2_1rbeXauM3ZT2i4eZnwAy1yS4KojQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: [sage-devel] Continuous benchmarking
Any kind of shared cpu container solution like openstack is also going to be unsuitable for benchmarking, I think. There is going to be lots of background stuff that the test process has no control / visibility of. On Friday, May 31, 2019 at 5:31:22 PM UTC+2, E. Madison Bray wrote: > > On Thu, May 30, 2019 at 8:07 AM Samuel Lelièvre > > wrote: > > > > Dear sage-devel, > > > > This blog post on continuous benchmarking looks interesting! > > > > > https://medium.com/@wolfv/building-an-open-source-continuous-benchmark-system-717839093962 > > > > > There have been related discussions on this list in the past, > > for example around airspeed velocity: > > > > > https://groups.google.com/forum/#!searchin/sage-devel/airspeed$20velocity%7Csort:date > > > I agree of course per previous discussions about it. It would be nice > to have the infrastructure to do continuous *anything* outside the > Sage buildbots. > > I've all but given up on Paris-Sud's OpenStack infrastructure. > Everything I try to do with it regarding CI (be it the Cygwin > buildbot, or gitlab runners) breaks constantly and I don't think it's > anything I'm doing wrong. E.g. recently (within the last few weeks) > they seem to have fiddled with some firewall rules such that now it > terminates long-running TCP/IP connections. At least, that's my > guess. I haven't been able to investigate thoroughly yet but it's > basically causing all my CI processes to lose connections with their > master servers mid-build. > > We need reliable infrastructure for Sage builds. > -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/7f24486d-8295-44e7-ab7f-fd99f88c4ff7%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [sage-devel] Continuous benchmarking
On Thu, May 30, 2019 at 8:07 AM Samuel Lelièvre wrote: > > Dear sage-devel, > > This blog post on continuous benchmarking looks interesting! > > https://medium.com/@wolfv/building-an-open-source-continuous-benchmark-system-717839093962 > > There have been related discussions on this list in the past, > for example around airspeed velocity: > > https://groups.google.com/forum/#!searchin/sage-devel/airspeed$20velocity%7Csort:date I agree of course per previous discussions about it. It would be nice to have the infrastructure to do continuous *anything* outside the Sage buildbots. I've all but given up on Paris-Sud's OpenStack infrastructure. Everything I try to do with it regarding CI (be it the Cygwin buildbot, or gitlab runners) breaks constantly and I don't think it's anything I'm doing wrong. E.g. recently (within the last few weeks) they seem to have fiddled with some firewall rules such that now it terminates long-running TCP/IP connections. At least, that's my guess. I haven't been able to investigate thoroughly yet but it's basically causing all my CI processes to lose connections with their master servers mid-build. We need reliable infrastructure for Sage builds. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAOTD34ZS3fPN8MYAhmaDshaCYtyT-Fdu7p6tz7--Spf%2B0H77eQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
[sage-devel] Continuous benchmarking
Dear sage-devel, This blog post on continuous benchmarking looks interesting! https://medium.com/@wolfv/building-an-open-source-continuous-benchmark-system-717839093962 There have been related discussions on this list in the past, for example around airspeed velocity: https://groups.google.com/forum/#!searchin/sage-devel/airspeed$20velocity%7Csort:date Kind regards, Samuel -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAEcArF0nArs2Jos4U7BbpcyDS%2BqGx%3D%3DaHjN2s9fnY9GfCaYPQA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
