Sebastian Bazley from ASF INFRA would be good resource to contact to drive this.
In the past when I brought this with infra I think I understood from him that companies can donate funds and explicitly state that are for the OpenWhisk project then the INFRA team will allocate VMs to OpenWhisk to run load/performance tests, same way that Hadoop project has decimated VMs in Jenkins today. — Carlos On Fri, Apr 6, 2018 at 9:38 AM Michael Marth <mma...@adobe.com.invalid> wrote: > Hi mentors (and others), > > Had an offline discussion this week in which the question came up how an > ASF project should best go about running performance/throughput/scalability > tests – i.e. tests that cannot be run locally and require a repeatable > environment. > > Some options: > * interested companies run the tests on their own infra and publish > results. Pretty lame, especially because typically only that company’s > engineers can access the env and investigate further. > * interested companies donate cash to sponsor compute resources, > committers can run and investigate the tests. Ideal from tech perspective, > but I have no idea how that cash would make its way from the ASF to a > particular project. > * maybe a middle-ground: interested party that happens to have a public > cloud offering gives credentials to committers > > I am mainly interested to learn if there are other ASF projects (e.g. in > the Big Data/Hadoop ecosystem) that do something similar. Or if there is an > ASF-recommended way to do this. Or else, where I could ask this question? > > Thanks! > Michael > > From: Michael Marth <mma...@adobe.com> > Date: Wednesday 3 May 2017 20:57 > To: "dev@openwhisk.apache.org" <dev@openwhisk.apache.org> > Subject: Re: Performance tests for OpenWhisk > > Markus, > > Quick update: sent the below to users@infra. So far no reaction. The > archive is here [1] but Bertrand tells me only ASF member have access - > for whatever reason. > > Michael > > [1] > https://lists.apache.org/thread.html/70999f9233dac9b416ef9dedc97c0ef196a938c05d6a407b94ba3479@%3Cusers.infra.apache.org%3E > > > On Fri, Apr 28, 2017 at 2:23 PM, Michael Marth <mma...@adobe.com<mailto: > mma...@adobe.com>> wrote: > Dear Infra team, > > I am enquiring on behalf of the OpenWhisk project (currently in Incubator) > [1]. > > We would like to periodically run performance tests on a distributed > environment (OpenWhisk typically runs on more than 1 machine). So we are > basically looking for an ability to spin up/tear down a number of (virtual) > machines and exclusively use them for a certain amount of time (so that the > VMs are not shared and the performance test results are comparable over > time). > The order of magnitude would be ~5-10 VMs for 1 hour 3 times a week. > > I would like to find out if there is an ASF-supported mechanism to do that. > For example, can Infra provide such infrastructure? Or is there a cloud > provider (like Azure) that might sponsor such efforts with VMs? Or maybe > there is an established way for commercial companies that are interested in > an ASF project to sponsor (fund) such tests? > > If none of the above exists, then it would also be helpful for us to get to > know how other projects run such sort of tests. > > Thanks a lot! > Michael > > > [1] > > https://lists.apache.org/thread.html/b66ab5b438f2db5cdc8c5f5eabece201b4ad090058fa3a9a3bd09d12@%3Cdev.openwhisk.apache.org%3E > > > > > From: Markus Thömmes <markusthoem...@me.com<mailto:markusthoem...@me.com>> > Reply-To: "dev@openwhisk.apache.org<mailto:dev@openwhisk.apache.org>" < > dev@openwhisk.apache.org<mailto:dev@openwhisk.apache.org>> > Date: Wednesday 26 April 2017 12:59 > To: "dev@openwhisk.apache.org<mailto:dev@openwhisk.apache.org>" < > dev@openwhisk.apache.org<mailto:dev@openwhisk.apache.org>> > Subject: Re: Performance tests for OpenWhisk > > Hi Michael, > > yeah that sounds pretty much spot on. I'd like to have at least 2 VMs with > 4+ cores and 8GB memory. One VM would host the management stack while one > would be dedicated to an Invoker only. That way we could assert > single-invoker performance the easiest. > > Thanks for helping! > > Cheers, > Markus > > Am 26. April 2017 um 11:36 schrieb Michael Marth <mma...@adobe.com<mailto: > mma...@adobe.com>>: > Markus, > > Does what I describe reflect what you are looking for? > If yes, I am happy to ask on infra. > > Let me know > Michael > > > > On 26/04/17 07:52, "Bertrand Delacretaz" <bdelacre...@apache.org<mailto: > bdelacre...@apache.org>> wrote: > > > Hi Michael, > > On Tue, Apr 25, 2017 at 6:52 PM, Michael Marth <mma...@adobe.com<mailto: > mma...@adobe.com>> wrote: > ...Maybe our mentors can chime in. Has this been discussed in the ASF > board or so?... > > Best would be to ask the ASF infrastructure team via > us...@infra.apache.org<mailto:us...@infra.apache.org> - briefly describe > what you need to see what's > possible. > > -Bertrand >
