This is an automated email from the ASF dual-hosted git repository. markusthoemmes pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-performance.git
commit cd838d38834e0ccaf160a46816c59710eedacaf8 Author: Markus Thoemmes <[email protected]> AuthorDate: Tue Apr 25 08:10:09 2017 +0200 Documentation on using the testsuites --- README.md | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 2d208f9..7b4c214 100644 --- a/README.md +++ b/README.md @@ -2,14 +2,18 @@ A few simple but efficient performance test suites for Apache OpenWhisk. Determines the maximum throughput and end-user latency of the system. ## Test setup -- A standard OpenWhisk system is deployed. Note that the edge nginx router and API Gateway are left out currently. +- A standard OpenWhisk system is deployed. Note that the edge nginx router and API Gateway are left out currently. As a consequence, the tests talk directly to the controller. - Limits are set to 999999 each, for the test's load that means: No throttling at all. - The deployment uses a docker setup as proposed by the OpenWhisk development team: `overlay` driver and HTTP API enabled via a UNIX port. +The load is driven by the beautiful [`loadtest`](https://www.npmjs.com/package/loadtest) module. + ### Travis' machine setup The [machine provided by Travis](https://docs.travis-ci.com/user/ci-environment/#Virtualization-environments) has ~2 CPU cores (likely shared through virtualization) and 7.5GB memory. -## Latency test +## Suites + +### Latency test The latency test determines the end-to-end latency a user experiences when doing a blocking invocation. The action used is a noop so the numbers returned are plain overhead of the OpenWhisk system. - 1 HTTP request at a time (concurrency: 1) @@ -18,9 +22,22 @@ The latency test determines the end-to-end latency a user experiences when doing **Note:** The throughput number has a 100% correlation with the latency in this case. This test does not serve to determine the maximum throughput of the system. -## Throughput test +### Throughput test The throughput test determines the maximum throughput a user can get out of the system while using a single action. The action used again is a noop, so the numbers are plain OpenWhisk overhead. Note that the throughput does not directly correlate to end-to-end latency here, as the system does more processing in the background as it shows to the user in a blocking invocation. - 4 HTTP requests at a time (concurrency: 4) (basically CPU cores * 2 to exploit some buffering) - 10.000 samples with a single user -- noop action \ No newline at end of file +- noop action + +## Running the suites yourself +To run the suites against your own system (or any other OpenWhisk deployment really), simply use the `test.sh` script. + +It takes a couple of parameters, defined as follows: + +``` +> test.sh HOST CREDENTIALS CONCURRENCY_LEVEL +``` + +- **HOST**: The host to reach OpenWhisk. Should include the protocol (`http`/`https`) and the port if applicable. +- **CREDENTIALS**: Credentials for OpenWhisk in USER:PASS form. +- **CONCURRENCY_LEVEL**: Concurrency level for the loadtest. This number should be a multiple of the number of CPU cores all invoker machines have in total. \ No newline at end of file -- To stop receiving notification emails like this one, please contact "[email protected]" <[email protected]>.
