This is an automated email from the ASF dual-hosted git repository.
markusthoemmes pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk.git
The following commit(s) were added to refs/heads/master by this push:
new f3f1cb9 Move performance repository to core repository. (#3475)
f3f1cb9 is described below
commit f3f1cb9171e0b95a27ac8f43dd8eafd5bce2f938
Author: Christian Bickel <git...@cbickel.de>
AuthorDate: Wed Apr 4 15:55:05 2018 +0200
Move performance repository to core repository. (#3475)
Co-authored-by: Markus Thömmes <markusthoem...@me.com>
Co-authored-by: Jeremias Werner <jeremias.wer...@gmail.com>
Co-authored-by: Alexandre Nicastro <kryn...@gmail.com>
---
performance/.travis.yml | 14 ++++++++++++++
performance/README.md | 33 +++++++++++++++++++++++++++++++++
performance/preparation/create.sh | 17 +++++++++++++++++
performance/preparation/deploy.sh | 25 +++++++++++++++++++++++++
performance/wrk_tests/latency.sh | 12 ++++++++++++
performance/wrk_tests/post.lua | 1 +
performance/wrk_tests/throughput.sh | 31 +++++++++++++++++++++++++++++++
7 files changed, 133 insertions(+)
diff --git a/performance/.travis.yml b/performance/.travis.yml
new file mode 100644
index 0000000..b425c60
--- /dev/null
+++ b/performance/.travis.yml
@@ -0,0 +1,14 @@
+sudo: required
+language: java
+services:
+ - docker
+
+env:
+ - TERM=dumb
+
+install:
+ - ./preparation/deploy.sh
+
+script:
+ - ./wrk_tests/latency.sh "http://172.17.0.1:10001"; "$(cat
openwhisk/ansible/files/auth.guest)" 2m
+ - ./wrk_tests/throughput.sh "http://172.17.0.1:10001"; "$(cat
openwhisk/ansible/files/auth.guest)" 4 2 2m
diff --git a/performance/README.md b/performance/README.md
new file mode 100644
index 0000000..c1d668f
--- /dev/null
+++ b/performance/README.md
@@ -0,0 +1,33 @@
+# :electric_plug: Apache OpenWhisk - Performance Tests
+A few simple but efficient test suites for determining the maximum throughput
and end-user latency of the Apache OpenWhisk system.
+
+## Workflow
+- A standard OpenWhisk system is deployed. (_Note that the edge NGINX router
and API Gateway are currently left out. As a consequence, the tests talk
directly to the controller._)
+- All limits are set to 999999, which in our current use case means "No
throttling at all".
+- The deployment is using the docker setup proposed by the OpenWhisk
development team: `overlay` driver and HTTP API enabled via a UNIX port.
+
+The load is driven by the blazingly fast [`wrk`](https://github.com/wg/wrk).
+
+#### Travis Machine Setup
+The [machine provided by
Travis](https://docs.travis-ci.com/user/ci-environment/#Virtualization-environments)
has ~2 CPU cores (likely shared through virtualization) and 7.5GB memory.
+
+## Suites
+
+### Latency Test
+Determines the end-to-end latency a user experience when doing a blocking
invocation. The action used is a no-op so the numbers returned are the plain
overhead of the OpenWhisk system.
+
+- 1 HTTP request at a time (concurrency: 1)
+- You can specify how long this test will run. Default are 30s.
+- no-op action
+
+**Note:** The throughput number has a 100% correlation with the latency in
this case. This test does not serve to determine the maximum throughput of the
system.
+
+### Throughput Test
+Determines the maximum throughput a user can get out of the system while using
a single action. The action used is a no-op, so the numbers are plain OpenWhisk
overhead. Note that the throughput does not directly correlate to end-to-end
latency here, as the system does more processing in the background as it shows
to the user in a blocking invocation.
+
+- 4 HTTP requests at a time (concurrency: 4) (using CPU cores * 2 to exploit
some buffering)
+- 10.000 samples with a single user
+- no-op action
+
+## Running tests against your own system is simple too!
+All you have to do is use the corresponding script located in /*_tests folder,
remembering that the parameters are defined inline.
diff --git a/performance/preparation/create.sh
b/performance/preparation/create.sh
new file mode 100755
index 0000000..1a79649
--- /dev/null
+++ b/performance/preparation/create.sh
@@ -0,0 +1,17 @@
+#!/bin/sh
+set -e
+
+# Host to use. Needs to include the protocol.
+host=$1
+# Credentials to use for the test. USER:PASS format.
+credentials=$2
+# Name of the action to create and test.
+action=$3
+
+# create a noop action
+echo "Creating action $action"
+curl -k -u "$credentials" "$host/api/v1/namespaces/_/actions/$action" -XPUT -d
'{"namespace":"_","name":"test","exec":{"kind":"nodejs:default","code":"function
main(){return {};}"}}' -H "Content-Type: application/json"
+
+# run the noop action
+echo "Running $action once to assert an intact system"
+curl -k -u "$credentials"
"$host/api/v1/namespaces/_/actions/$action?blocking=true" -XPOST
diff --git a/performance/preparation/deploy.sh
b/performance/preparation/deploy.sh
new file mode 100755
index 0000000..5628176
--- /dev/null
+++ b/performance/preparation/deploy.sh
@@ -0,0 +1,25 @@
+#!/bin/sh
+
+set -e
+currentDir="$(cd "$(dirname "$0")"; pwd)"
+
+# common docker setup
+sudo gpasswd -a travis docker
+sudo -E bash -c 'echo '\''DOCKER_OPTS="-H tcp://0.0.0.0:4243 -H
unix:///var/run/docker.sock --storage-driver=overlay
--userns-remap=default"'\'' > /etc/default/docker'
+sudo service docker restart
+
+# install ansible
+pip install --user ansible==2.4.2.0
+
+cd $currentDir/../../ansible
+ANSIBLE_CMD="ansible-playbook -i environments/local -e
docker_image_prefix=openwhisk -e docker_registry=docker.io/ -e
limit_invocations_per_minute=999999 -e limit_invocations_concurrent=999999 -e
limit_invocations_concurrent_system=999999"
+
+$ANSIBLE_CMD setup.yml
+$ANSIBLE_CMD prereq.yml
+$ANSIBLE_CMD couchdb.yml
+$ANSIBLE_CMD initdb.yml
+$ANSIBLE_CMD wipe.yml
+
+$ANSIBLE_CMD kafka.yml
+$ANSIBLE_CMD controller.yml
+$ANSIBLE_CMD invoker.yml
diff --git a/performance/wrk_tests/latency.sh b/performance/wrk_tests/latency.sh
new file mode 100755
index 0000000..bee706b
--- /dev/null
+++ b/performance/wrk_tests/latency.sh
@@ -0,0 +1,12 @@
+#!/bin/sh
+set -e
+currentDir="$(cd "$(dirname "$0")"; pwd)"
+
+# Host to use. Needs to include the protocol.
+host=$1
+# Credentials to use for the test. USER:PASS format.
+credentials=$2
+# How long to run the test
+duration=${3:-30s}
+
+$currentDir/throughput.sh $host $credentials 1 1 $duration
diff --git a/performance/wrk_tests/post.lua b/performance/wrk_tests/post.lua
new file mode 100644
index 0000000..e3e664c
--- /dev/null
+++ b/performance/wrk_tests/post.lua
@@ -0,0 +1 @@
+wrk.method = "POST"
\ No newline at end of file
diff --git a/performance/wrk_tests/throughput.sh
b/performance/wrk_tests/throughput.sh
new file mode 100755
index 0000000..6ed2453
--- /dev/null
+++ b/performance/wrk_tests/throughput.sh
@@ -0,0 +1,31 @@
+#!/bin/sh
+set -e
+currentDir="$(cd "$(dirname "$0")"; pwd)"
+
+# Host to use. Needs to include the protocol.
+host=$1
+# Credentials to use for the test. USER:PASS format.
+credentials=$2
+# concurrency level of the throughput test: How many requests should
+# open in parallel.
+concurrency=$3
+# How many threads to utilize, directly correlates to the number
+# of CPU cores
+threads=${4:-4}
+# How long to run the test
+duration=${5:-30s}
+
+action="noopThroughput"
+"$currentDir/../preparation/create.sh" "$host" "$credentials" "$action"
+
+# run throughput tests
+encodedAuth=$(echo "$credentials" | tr -d '\n' | base64 | tr -d '\n')
+docker run --pid=host --userns=host --rm -v "$currentDir":/data williamyeh/wrk
\
+ --threads "$threads" \
+ --connections "$concurrency" \
+ --duration "$duration" \
+ --header "Authorization: basic $encodedAuth" \
+ "$host/api/v1/namespaces/_/actions/$action?blocking=true" \
+ --latency \
+ --timeout 10s \
+ --script post.lua
--
To stop receiving notification emails like this one, please contact
markusthoem...@apache.org.
