Thanks Markus. I created a PR for the gatling tests https://github.com/markusthoemmes/openwhisk-performance/pull/1
Currently the tests I have do both the admin (delete/create actions) and user facing (action invocation) workflows - I have mixed feelings about that, but it was partly an experiment to see what a more complex test would look like. It is convenient to have the test run fully encapsulated, but it adds some odor to the code, like coordinating the invocation loop to wait for the setup to be complete (not sure if there is a better way, I didn’t find any info that gatling could be coaxed into running multiple user populations in sequence). Comments welcome. Thanks Tyson On May 9, 2017, at 10:38 PM, Markus Thömmes <[email protected]<mailto:[email protected]>> wrote: Hi Tyson, awesome, we had the same train of thought :). I looked into Gatling as well, mainly because it's in Scala so it'd be easy for us to read and build new scenarios. Gatling also has a Jenkins plugin (we use Jenkins) which makes it very easy to integrate test results. I also liked how you define a scenario in Gatling and it seemed to fit OpenWhisk quite well, especially when looking at more sophisticated use-cases with different load-patterns. My initial experiments are here: https://github.com/markusthoemmes/gatling-example<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmarkusthoemmes%2Fgatling-example&data=02%7C01%7C%7C90f3400224ca48fdfb7108d49766e4bb%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636299915569507148&sdata=yr%2BpEDO592E3N8M%2FzqF6UU9bIGk7aiFI8eM%2Ft6%2B5LYI%3D&reserved=0>. I also have a branch somewhere which builds a docker container around the test. That's mainly why I used env variables for configuration. Yes I think we should have those in the openwhisk-performance repo. Eventually Gatling/Locust (have to look at that) tests might even subsume the existing very basic ones. Cheers Markus Am 10. Mai 2017 um 06:46 schrieb Tyson Norris <[email protected]<mailto:[email protected]>>: Hi - I spent some time experimenting with gatling and locust.io<http://locust.io><http://locust.io> load tests, and wanted to find out if people here have some preference? From my limited usage, some differences are: - gatling has some good facilities for managing complex tests and decent reports with little effort, compared to locust - locust has some distributed testing capabilities (generate load from multiple hosts in a coordinated fashion), but is lighter on report details and test scenario complexity. Branches in my fork are here, I can create pull requests if people are interested in pulling this into the openwhisk-performance repo? https://github.com/tysonnorris/openwhisk-performance/tree/gatling-tests<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftysonnorris%2Fopenwhisk-performance%2Ftree%2Fgatling-tests&data=02%7C01%7C%7C90f3400224ca48fdfb7108d49766e4bb%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636299915569507148&sdata=QTQ9U0oZEiFBJ1b%2FRKYC8HA6dmB1PlrOI7Xmt4k45tI%3D&reserved=0> https://github.com/tysonnorris/openwhisk-performance/tree/locust-tests<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftysonnorris%2Fopenwhisk-performance%2Ftree%2Flocust-tests&data=02%7C01%7C%7C90f3400224ca48fdfb7108d49766e4bb%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636299915569507148&sdata=kTdvDe55f7iXm8SFSDEJp%2FwmAk7toOiubO%2FjSLzzEnU%3D&reserved=0> (In each branch I created a separate readme with pertinent details of that test harness, open to merge with main README.md before creating a PR) Its also an option to include both, if people are interested. Thanks Tyson
