This is an automated email from the ASF dual-hosted git repository.
pdesai pushed a commit to branch master
in repository
https://gitbox.apache.org/repos/asf/incubator-openwhisk-devtools.git
The following commit(s) were added to refs/heads/master by this push:
new ddbbdc2 Fix Knative dedicated function request pre-processing logic
(#253)
ddbbdc2 is described below
commit ddbbdc25af80ba38b91c8f17f26e0ec7aa009b33
Author: Matt Rutkowski <[email protected]>
AuthorDate: Thu Apr 25 13:29:14 2019 -0500
Fix Knative dedicated function request pre-processing logic (#253)
* cleanup README to be more clear and handoff properly to language runtime
READMEs
* cleanup README to be more clear and handoff properly to language runtime
READMEs
* cleanup README to be more clear and handoff properly to language runtime
READMEs
* cleanup README to be more clear and handoff properly to language runtime
READMEs
* javascript README fix to path
* Fix Knative dedicated pre-procssing logic path
* Fix Knative dedicated pre-procssing logic path
* Fix Knative dedicated pre-procssing logic path
---
knative-build/README.md | 14 ++-
knative-build/runtimes/javascript/README.md | 20 ++--
.../runtimes/javascript/platform/knative.js | 108 +++++++++++----------
.../tests/helloworld/payload-knative-init.http | 13 +++
.../tests/helloworld/payload-knative-run.http | 17 ++++
.../payload-knative-init-run.http | 2 +-
6 files changed, 111 insertions(+), 63 deletions(-)
diff --git a/knative-build/README.md b/knative-build/README.md
index a0b94dc..22739fd 100644
--- a/knative-build/README.md
+++ b/knative-build/README.md
@@ -152,7 +152,6 @@ status:
phase: Active
```
-
**Note**: If you do not see the **istio-injection** label, verify you issued
the 'kubectl' command to set this label to the default namespace. See
[Troubleshooting](#troubleshooting) section for more information.
# Building and Serving OpenWhisk Runtime Build Templates
@@ -163,7 +162,7 @@ All OpenWhisk Runtime Build Templates require a valid
Kubernetes **Service Accou
```bash
$ git clone https://github.com/apache/incubator-openwhisk-devtools.git
-$ cd knative-build
+$ cd incubator-openwhisk-devtools/knative-build
```
## Register Secrets for Docker Hub
@@ -181,7 +180,7 @@ $ export DOCKERHUB_PASSWORD_BASE64_ENCODED=`echo -n
"${DOCKERHUB_PASSWORD_PLAIN_
# make sure that DOCKERHUB_PASSWORD_BASE64_ENCODED is set to something similar
to t80szzToPshrpDr3sdMn==
```
-On your local system, copy the file
[docker-secret.yaml.tmpl](docker-secret.yaml.tmpl) to `docker-secrets.yaml` and
replace the **username** and **password** values with the base64 encoded
versions of your Docker Hub username and password using following `sed` command:
+Use the following `sed` command which will generate a `docker-secrets.yaml`
file from the file [docker-secret.yaml.tmpl](docker-secret.yaml.tmpl) and
replace the **username** and **password** values with the base64 encoded
versions of your Docker Hub username and password from the environment
variables you exported above:
```
sed -e
's/${DOCKERHUB_USERNAME_BASE64_ENCODED}/'"$DOCKERHUB_USERNAME_BASE64_ENCODED"'/'
-e
's/${DOCKERHUB_PASSWORD_BASE64_ENCODED}/'"$DOCKERHUB_PASSWORD_BASE64_ENCODED"'/'
docker-secret.yaml.tmpl > docker-secret.yaml
@@ -219,6 +218,13 @@ NAME SECRETS AGE
openwhisk-runtime-builder 2 3m46s
```
+## Building Knative runtimes and executing functions
+
+At this point, you have completed all the pre-reqs to build and run OpenWhisk
runtimes on Knative. In order to actually build and run them, you will need to
continue following the instructions on the READMEs for the respective language
runtimes listed below:
+- [NodeJS Runtime](./runtimes/javascript/#OpenWhisk-NodeJS-Runtime-for-Knative)
+
+---
+
## Troubleshooting
### Knative and Istio Install
@@ -268,7 +274,7 @@ statefulset "prometheus-system" created
#### PROBLEM: Knative build fails initializing at `Init:1/4`
-Check the GitHub revision is set to right branch:
+Check the GitHub revision is set to right branch within your ```build.yaml```
file:
```
source:
diff --git a/knative-build/runtimes/javascript/README.md
b/knative-build/runtimes/javascript/README.md
index 413f571..2592646 100644
--- a/knative-build/runtimes/javascript/README.md
+++ b/knative-build/runtimes/javascript/README.md
@@ -169,7 +169,7 @@ metadata:
We will use the simple "helloworld" test case to demonstrate how to use
Knative to Build your function into container image and then deploy it as a
Service.
The testcase resides within this repo. at:
--
[/runtimes/javascript/tests/helloworld/](/runtimes/javascript/tests/helloworld/)
+-
[tests/helloworld/](tests/helloworld/#Hello-World-Test-for-OpenWhisk-NodeJS-Runtime-using-Knative)
For a complete listing of testcases, please view the [README](tests/README.md)
in the tests subdirectory.
@@ -186,8 +186,11 @@ To do this,
If you wish to run repeated tests you MAY set an environment variable and use
```sed``` to replace the ```${DOCKER_USERNAME}``` within any of the test's
Kubernetes Build YAML files as follows:
```
-export DOCKER_USERNAME="myusername"
-sed 's/${DOCKER_USERNAME}/'"$DOCKER_USERNAME"'/' build.yaml.tmpl > build.yaml
+# If you have not already exported your Dockerhub username, use the following
command:
+export DOCKERHUB_USERNAME_PLAIN_TEXT="myusername"
+
+# Generate the build file from the template using your Dockerhub username set
in the environment
+sed 's/${DOCKER_USERNAME}/'"$DOCKERHUB_USERNAME_PLAIN_TEXT"'/' build.yaml.tmpl
> build.yaml
```
<details>
@@ -225,7 +228,12 @@ spec:
kubectl apply -f build.yaml
```
-This creates a pod with a NodeJS runtime and all the action metadata (action
code, main function name, etc) integrated into the container image. If for any
reason there is a failure creating the pod, we can troubleshoot the deployment
with:
+This creates a pod with a NodeJS runtime and all the action metadata (action
code, main function name, etc) integrated into the container image.
+
+
+#### Troubleshootinh the build
+
+If for any reason there is a failure creating the pod, we can troubleshoot the
deployment with:
#### `kubectl get pods`
@@ -280,8 +288,8 @@ To do this,
As described for 'build.yaml.tmpl', you MAY set an environment variable and
use ```sed``` to replace the ```${DOCKER_USERNAME}``` within any of the test's
Kubernetes Build YAML files as follows:
```
-export DOCKER_USERNAME="myusername"
-sed 's/${DOCKER_USERNAME}/'"$DOCKER_USERNAME"'/' service.yaml.tmpl >
service.yaml
+export DOCKERHUB_USERNAME_PLAIN_TEXT="myusername"
+sed 's/${DOCKER_USERNAME}/'"$DOCKERHUB_USERNAME_PLAIN_TEXT"'/'
service.yaml.tmpl > service.yaml
```
<details>
diff --git a/knative-build/runtimes/javascript/platform/knative.js
b/knative-build/runtimes/javascript/platform/knative.js
index a537aa5..194f4be 100644
--- a/knative-build/runtimes/javascript/platform/knative.js
+++ b/knative-build/runtimes/javascript/platform/knative.js
@@ -78,76 +78,78 @@ function removeInitData(body) {
}
}
+
/**
- * Pre-process the incoming
+ * Create request init data from the process environment
*/
-function preProcessInitData(env, initdata, valuedata, activationdata) {
+function createInitDataFromEnvironment(env) {
DEBUG.functionStart();
try {
- // Set defaults to use INIT data not provided on the request
- // Look first to the process (i.e., Container's) environment variables.
- var main = (typeof env.__OW_ACTION_MAIN === 'undefined') ? "main" :
env.__OW_ACTION_MAIN;
+ var initdata = {};
+ initdata.main = (typeof env.__OW_ACTION_MAIN === 'undefined') ? "main"
: env.__OW_ACTION_MAIN;
// TODO: Throw error if CODE is NOT defined!
- var code = (typeof env.__OW_ACTION_CODE === 'undefined') ? "" :
env.__OW_ACTION_CODE;
- var binary = (typeof env.__OW_ACTION_BINARY === 'undefined') ? false :
env.__OW_ACTION_BINARY.toLowerCase() === "true";
+ initdata.code = (typeof env.__OW_ACTION_CODE === 'undefined') ? "" :
env.__OW_ACTION_CODE;
+ initdata.binary = (typeof env.__OW_ACTION_BINARY === 'undefined') ?
false : env.__OW_ACTION_BINARY.toLowerCase() === "true";
// TODO: default to empty?
- var actionName = (typeof env.__OW_ACTION_NAME === 'undefined') ? "" :
env.__OW_ACTION_NAME;
- var raw = (typeof env.__OW_ACTION_RAW === 'undefined') ? false :
env.__OW_ACTION_RAW.toLowerCase() === "true";
+ initdata.actionName = (typeof env.__OW_ACTION_NAME === 'undefined') ?
"" : env.__OW_ACTION_NAME;
+ initdata.raw = (typeof env.__OW_ACTION_RAW === 'undefined') ? false :
env.__OW_ACTION_RAW.toLowerCase() === "true";
- DEBUG.dumpObject(actionName, "Action name");
- DEBUG.dumpObject(main, "Action main");
- DEBUG.dumpObject(code, "Action code");
- DEBUG.dumpObject(binary, "Action binary");
- DEBUG.dumpObject(raw, "Action Raw");
+ DEBUG.dumpObject(initdata, "initdata");
+ return initdata;
+ } catch(e){
+ console.error(e);
+ DEBUG.functionEndError(e.message);
+ throw("Unable to process Initialization data: " + e.message);
+ }
+ DEBUG.functionEnd();
+}
+
+
+/**
+ * Pre-process the init data from the request
+ */
+function preProcessInitData(initdata, valuedata, activationdata) {
+ DEBUG.functionStart();
+ try {
// Look for init data within the request (i.e., "stem cell" runtime,
where code is injected by request)
if (typeof(initdata) !== "undefined") {
- if (initdata.name && typeof initdata.name === 'string') {
- actionName = initdata.name;
- }
+
if (initdata.main && typeof initdata.main === 'string') {
- main = initdata.main;
+ valuedata.main = initdata.main;
}
if (initdata.code && typeof initdata.code === 'string') {
- code = initdata.code;
+ valuedata.code = initdata.code;
}
if (initdata.binary) {
if (typeof initdata.binary === 'boolean') {
- binary = initdata.binary;
+ valuedata.binary = initdata.binary;
} else {
throw ("Invalid Init. data; expected boolean for key
'binary'.");
}
}
- if (initdata.raw ) {
+ if (initdata.raw) {
if (typeof initdata.raw === 'boolean') {
- raw = initdata.raw;
+ valuedata.raw = initdata.raw;
} else {
throw ("Invalid Init. data; expected boolean for key
'raw'.");
}
}
- }
- // Move the init data to the request body under the "value" key.
- // This will allow us to reuse the "openwhisk" /init route handler
function
- valuedata.main = main;
- valuedata.code = code;
- valuedata.binary = binary;
- valuedata.raw = raw;
-
- // Action name is a special case, as we have a key collision on "name"
between init. data and request
- // param. data (as they both appear within "body.value") so we must
save it to its final location
- // as the default Action name as part of the activation data
- // NOTE: if action name is not present in the action data, we will set
it regardless even if an empty string
- if (typeof(activationdata) !== "undefined" ) {
- if (typeof(activationdata.action_name) === "undefined" ||
- (typeof(activationdata.action_name) === "string" &&
activationdata.action_name.length == 0)){
- activationdata.action_name = actionName;
+ // Action name is a special case, as we have a key collision on
"name" between init. data and request
+ // param. data (as they both appear within "body.value") so we
must save it to its final location
+ // as the default Action name as part of the activation data
+ if (initdata.name && typeof initdata.name === 'string') {
+ if (typeof (activationdata) !== "undefined") {
+ if (typeof (activationdata.action_name) === "undefined" ||
+ (typeof (activationdata.action_name) === "string" &&
+ activationdata.action_name.length === 0)) {
+ activationdata.action_name = initdata.name;
+ }
+ }
}
+ DEBUG.dumpObject(valuedata, "valuedata");
}
- DEBUG.dumpObject(valuedata.main, "valuedata.main");
- DEBUG.dumpObject(valuedata.code , "valuedata.code");
- DEBUG.dumpObject(valuedata.binary, "valuedata.binary");
- DEBUG.dumpObject(valuedata.raw, "valuedata.raw");
} catch(e){
console.error(e);
@@ -267,7 +269,7 @@ function preProcessRequest(req){
preProcessInitData(env, initData, valueData, activationData);
}
- if( hasActivationData(req)) {
+ if(hasActivationData(req)) {
// process HTTP request header and body to make it available to
function as parameter data
preProcessHTTPContext(req, valueData);
@@ -351,7 +353,7 @@ function postProcessResponse(req, result, res) {
delete body['binary'];
}
- //When the content-type is defined, check if the response is binary data or
+ // When the content-type is defined, check if the response is binary data
or
// plain text and decode the plain text using a base64 decoder whenever
needed.
// Should the body fail to decoded correctly, return an error to the
caller.
if (contentTypeInHeader && headers[CONTENT_TYPE].lastIndexOf("image", 0)
=== 0) {
@@ -414,10 +416,15 @@ function PlatformKnativeImpl(platformFactory) {
if (hasInitData(req) && !isStemCell(process.env))
throw ("Cannot initialize a runtime with a dedicated
function.");
- if(hasInitData(req) && hasActivationData(req)){
+ // If this is a dedicated, uninitialized runtime, then copy INIT
data from env. into the request
+ if( !isStemCell(process.env) && !service.initialized()){
+ let body = req.body || {};
+ body.init = createInitDataFromEnvironment(process.env);
+ }
- // Process request and process env. variables to provide them
in the manner
- // an OpenWhisk Action expects them, as well as enable
additional Http features.
+ // Different pre-processing logic based upon request data needed
due Promise behavior
+ if(hasInitData(req) && hasActivationData(req)){
+ // Request has both Init and Run (activation) data
preProcessRequest(req);
// Invoke the OW "init" entrypoint
service.initCode(req).then(function () {
@@ -437,9 +444,7 @@ function PlatformKnativeImpl(platformFactory) {
}
});
} else if(hasInitData(req)){
-
- // Process request and process env. variables to provide them
in the manner
- // an OpenWhisk Action expects them, as well as enable
additional Http features.
+ // Request has ONLY Init data
preProcessRequest(req);
// Invoke the OW "init" entrypoint
service.initCode(req).then(function (result) {
@@ -454,8 +459,7 @@ function PlatformKnativeImpl(platformFactory) {
}
});
} else if(hasActivationData(req)){
- // Process request and process env. variables to provide them
in the manner
- // an OpenWhisk Action expects them, as well as enable
additional Http features.
+ // Request has ONLY Run (activation) data
preProcessRequest(req);
// Invoke the OW "run" entrypoint
service.runCode(req).then(function (result) {
diff --git
a/knative-build/runtimes/javascript/tests/helloworld/payload-knative-init.http
b/knative-build/runtimes/javascript/tests/helloworld/payload-knative-init.http
new file mode 100644
index 0000000..a2b27d9
--- /dev/null
+++
b/knative-build/runtimes/javascript/tests/helloworld/payload-knative-init.http
@@ -0,0 +1,13 @@
+POST http://localhost:8080/ HTTP/1.1
+content-type: application/json
+
+{
+ "init": {
+ "name": "nodejs-helloworld",
+ "main": "main",
+ "binary": false,
+ "code": "function main() {return {payload: 'Hello World!'};}"
+ }
+}
+
+###
diff --git
a/knative-build/runtimes/javascript/tests/helloworld/payload-knative-run.http
b/knative-build/runtimes/javascript/tests/helloworld/payload-knative-run.http
new file mode 100644
index 0000000..e83b884
--- /dev/null
+++
b/knative-build/runtimes/javascript/tests/helloworld/payload-knative-run.http
@@ -0,0 +1,17 @@
+POST http://localhost:8080/ HTTP/1.1
+content-type: application/json
+
+{
+ "activation": {
+ "namespace": "default",
+ "action_name": "nodejs-helloworld",
+ "api_host": "",
+ "api_key": "",
+ "activation_id": "",
+ "deadline": "4102498800000"
+ },
+ "value": {
+ }
+}
+
+###
diff --git
a/knative-build/runtimes/javascript/tests/helloworldwithparams/payload-knative-init-run.http
b/knative-build/runtimes/javascript/tests/helloworldwithparams/payload-knative-init-run.http
index 7b4a421..61d9ef6 100644
---
a/knative-build/runtimes/javascript/tests/helloworldwithparams/payload-knative-init-run.http
+++
b/knative-build/runtimes/javascript/tests/helloworldwithparams/payload-knative-init-run.http
@@ -6,7 +6,7 @@ content-type: application/json
"name" : "nodejs-helloworld-with-params",
"main" : "main",
"binary": false,
- "code" : "function main(params) {return {payload: 'Hello ' + params.name +
' from ' + params.place + '!'};}"
+ "code" : "function main(params) {return {payload: 'Hello ' + params.name +
' from ' + params.place + '!'};}"
},
"activation": {
"namespace": "default",
