This is an automated email from the ASF dual-hosted git repository.
rabbah pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/openwhisk-package-kafka.git
The following commit(s) were added to refs/heads/master by this push:
new b4ff0f7 Architecture docs (#273)
b4ff0f7 is described below
commit b4ff0f740cc34497d6e8ebd967fa20f23f47260e
Author: Carlos Santana <[email protected]>
AuthorDate: Sat May 23 18:59:26 2020 -0400
Architecture docs (#273)
---
.gitignore | 10 ++-
README.md | 5 +-
docs/arch/README.md | 85 +++++++++++++++++++++++++
docs/arch/images/Arch-Provider-MHV1-Create.png | Bin 0 -> 156566 bytes
docs/arch/images/Arch-Provider-MHV1-Delete.png | Bin 0 -> 118502 bytes
docs/arch/images/Arch-Provider-MHV1-Read.png | Bin 0 -> 96355 bytes
docs/arch/images/Arch-Provider-MHV1-Update.png | Bin 0 -> 151310 bytes
docs/arch/images/Arch-Provider.xml | 1 +
devGuide.md => docs/dev/README.md | 0
9 files changed, 99 insertions(+), 2 deletions(-)
diff --git a/.gitignore b/.gitignore
index 15822ad..5fc58a2 100644
--- a/.gitignore
+++ b/.gitignore
@@ -8,5 +8,13 @@ action/node_modules/
action/package-lock.json
package-lock.json
+# Eclipse
+bin/
+**/.project
+.settings/
+.classpath
+.cache-main
+.cache-tests
+
.idea/
-out/
\ No newline at end of file
+out/
diff --git a/README.md b/README.md
index 620a16f..fe42133 100644
--- a/README.md
+++ b/README.md
@@ -379,8 +379,11 @@ The action caller (you, or your code) must first Base64
encode the data, for exa
### Integrating OpenWhisk with IBM Message Hub, Node Red, IBM Watson IoT, IBM
Object Storage, and IBM Data Science Experience
Example that integrates OpenWhisk with IBM Message Hub, Node Red, IBM Watson
IoT, IBM Object Storage, IBM Data Science Experience (Spark) service can be
[found
here](https://medium.com/openwhisk/transit-flexible-pipeline-for-iot-data-with-bluemix-and-openwhisk-4824cf20f1e0).
+## Architecture
+Archtecture documentation and diagrams, please refer to the [Architecture
Docs](docs/arch/README.md)
+
## Development and Testing
-If you wish to deploy the feed service yourself, please refer to the
[Development Guide](devGuide.md).
+If you wish to deploy the feed service yourself, please refer to the
[Development Guide](docs/dev/README.md).
## References
- [OpenWhisk](https://www.ibm.com/cloud-computing/bluemix/openwhisk)
diff --git a/docs/arch/README.md b/docs/arch/README.md
new file mode 100644
index 0000000..0eb6b0b
--- /dev/null
+++ b/docs/arch/README.md
@@ -0,0 +1,85 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+## MessageHub Trigger Provider Architecture
+
+### Create Trigger Feed
+
+
+**Scenario:** User wants to create a trigger `trigger1` for MessageHub service
instance `instance1`, using Credentials `Credential-1` and use rule `rule1` to
invoke action `action1` with messages from topic `topic1`.
+
+1. Developer creates a MessageHub service `instance1`
+2. Developer creates topic `topic1` in MessageHub service `instance1`
+3. Developer creates Credential key `Credential-1` for MessageHub `instance1`
+4. Developer creates trigger `trigger1` on OpenWhisk, the trigger stores the
annotation `feed` with the feedAction name from system package or binded
package.(`/whisk.system/messagingWeb/messageHubFeed`).
+5. Developer invokes action feedAction to create trigger feed passing input
parameters (lifeCycle:`CREATE`, `trigger1`, Credentials1, Options:`topic1`)
+6. The feedAction invokes feedWebAction forwarding input parameter.
+7. The feedWebAction inserts trigger feed doc into DB for worker group 0
(feedWebAction protects DB credentials)
+8. DB insertion notifies workers group 0 via Cloudant/CouchDB changes API,
workers listen on DB view with a filter for their group `worker0` and gets the
DB doc.
+9. Kafka Consumer is created on each worker in a consumer group and starts
polling for messages on `topic1` from `instance1` using `Credentials-1`.
+10. Developer creates `rule1` indicating that when `trigger1` fires invoke
`action1`.
+11. Event source produces messages on `topic1`.
+12. Both consumers will batch the messages from `topic1` and fire `trigger1`.
+ - The fire is done with an http request containing the batch of messages
in the body.
+ - Consumer will not poll for more messages and will not commit batch of
messages until the http request gets a response from OpenWhisk trigger endpoint.
+ - Consumers in the same consumer group get assigned a set of partitions,
each consumer on each worker host will get a unique set of messages avoiding
duplicate messages being included in trigger fires.
+9. OpenWhisk will process the trigger fire for `trigger1` and finds the
`rule1` and invokes `action1` with messages from topic `topic1`.
+
+### Update Trigger Feed
+
+
+**Scenario:** User wants to update trigger `trigger1` to change from topic
`topic1` to topic `topic2`.
+
+1. Developer creates topic `topic2` in MessageHub service `instance1`.
+2. Developer gets the annotation `feed` from trigger `trigger1`.
+3. Developer invokes feedAction to update trigger feed passing input
parameters (lifeCycle:`UPDATE`, `trigger1`, Options:`topic2`).
+4. The feedAction invokes feedWebAction forwarding input parameter.
+5. The feedWebAction inserts trigger feed doc into DB for worker group 0
(feedWebAction protects DB credentials).
+6. DB insertion notifies workers group 0 via Cloudant/CouchDB changes API,
workers listen on DB view with a filter for their group `worker0` and gets the
DB doc.
+7. Kafka Consumer is re-created on each worker in a consumer group and starts
polling for messages on `topic2` from `instance1` using `Credentials-1`.
+8. Event source produces messages on `topic2`.
+9. Both consumers will now handle `topic2` instead of `topic1`.
+10. OpenWhisk will process the trigger fire for `trigger1`, finds the rule
`rule1` and invokes `action1` with messages from topic `topic2`.
+
+### Read Trigger Feed
+
+
+**Scenario:** User wants to read the configuration and staus for trigger
`trigger1`.
+
+1. Developer gets the annotation `feed` from trigger `trigger1`.
+2. Developer invokes feedAction to read the trigger feed passing input
parameters (lifeCycle:`READ`, `trigger1`).
+3. The feedAction invokes feedWebAction forwarding input parameter.
+4. The feedWebAction gets the trigger feed doc from DB (feedWebAction protects
DB credentials).
+5. The DB returns the trigger feed doc for `trigger1`.
+6. The feedWebAction returns a response to feedAction.
+7. The feedAction returns response (config, status) to Developer.
+
+### Delete Trigger Feed
+
+
+**Scenario:** User wants to delete trigger `trigger1`.
+
+1. Developer deletes rule `rule1`
+2. Developer gets the annotation `feed` from trigger `trigger1`.
+3. Developer invokes feedAction to delete the trigger feed passing input
parameters (lifeCycle:`DELETE`, `trigger1`).
+4. The feedAction invokes feedWebAction forwarding input parameter.
+5. The feedWebAction updates the trigger feed doc into DB with a field
`delete:true`(feedWebAction protects DB credentials).
+6. DB update notifies workers group 0 via Cloudant/CouchDB changes API,
workers listen on DB view with a filter for their group `worker0` and gets the
DB doc. The Kafka consumers for `trigger1/topic2` get destroyed.
+7. The feedWebAction deletes the trigger feed doc from DB.
+8. The Developer deletes trigger `trigger1`
diff --git a/docs/arch/images/Arch-Provider-MHV1-Create.png
b/docs/arch/images/Arch-Provider-MHV1-Create.png
new file mode 100644
index 0000000..8d5aec5
Binary files /dev/null and b/docs/arch/images/Arch-Provider-MHV1-Create.png
differ
diff --git a/docs/arch/images/Arch-Provider-MHV1-Delete.png
b/docs/arch/images/Arch-Provider-MHV1-Delete.png
new file mode 100644
index 0000000..abfa261
Binary files /dev/null and b/docs/arch/images/Arch-Provider-MHV1-Delete.png
differ
diff --git a/docs/arch/images/Arch-Provider-MHV1-Read.png
b/docs/arch/images/Arch-Provider-MHV1-Read.png
new file mode 100644
index 0000000..80db17c
Binary files /dev/null and b/docs/arch/images/Arch-Provider-MHV1-Read.png differ
diff --git a/docs/arch/images/Arch-Provider-MHV1-Update.png
b/docs/arch/images/Arch-Provider-MHV1-Update.png
new file mode 100644
index 0000000..3a38cef
Binary files /dev/null and b/docs/arch/images/Arch-Provider-MHV1-Update.png
differ
diff --git a/docs/arch/images/Arch-Provider.xml
b/docs/arch/images/Arch-Provider.xml
new file mode 100644
index 0000000..0b98596
--- /dev/null
+++ b/docs/arch/images/Arch-Provider.xml
@@ -0,0 +1 @@
+<mxfile userAgent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"
version="8.8.7" editor="www.draw.io" type="device"><diagram
id="8b90fb75-adc8-f8ed-5786-064473bb2ab5"
name="Page-1">7V1Zk9s2Ev41qnIehgWA9+McmWRrHce1zq6TJxclQhJjStSS1Bz59QFIgiLR0A1wZCuaskyBIAh2N74+gAZH9v3i5ac8Ws1/yWKajgiKX0b2w4gQ7NiI/cdLXuuS0MF1wSxP4qbSpuBT8hdtCpvrZuskpkWvYpllaZms+oWTbLmkk7JXFuV59tyvNs3S/l1X0YyCgk+TKIWln5O4nNelgYs25T/TZDYXd8aoOTOOJl9
[...]
\ No newline at end of file
diff --git a/devGuide.md b/docs/dev/README.md
similarity index 100%
rename from devGuide.md
rename to docs/dev/README.md
