Re: Performance tests of OpenWhisk

2018-03-19 Thread Rodric Rabbah
Agreed that we can’t yet everything but perhaps these may be sensible. 1. Latency regression because of a cold start related bug (might need at least two invokers and controllers) 2. For a single invoker (non black box), some expected l/sustained throughout range (rps) 3. If we disable the

Re: Performance tests of OpenWhisk

2018-03-19 Thread Christian Bickel
As I said, we only have a very small travis machine. But I think it's at least worth a try, if we get consistent performance numbers in travis, to set a threshold and fail if it's below this threshold. Greetings Christian March 19, 2018 12:03 PM, "Rodric Rabbah" wrote: >

Re: Performance tests of OpenWhisk

2018-03-19 Thread Rodric Rabbah
Great - are you considering also a Travis CI matrix? -r