sjwiesman commented on a change in pull request #263: URL: https://github.com/apache/flink-statefun/pull/263#discussion_r697486471
########## File path: docs/content/docs/modules/overview.md ########## @@ -0,0 +1,66 @@ +--- +title: 'Overview' +weight: 1 +type: docs +aliases: + - /modules/ +permalink: /modules/index.html +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Application Modules + +An application module consists of multiple [components]({{< ref "docs/concepts/application-building-blocks" >}}) +that take part in a StateFun application. It includes the endpoints where functions can be reached along with ingress +and egress definitions. + +Modules are defined using a YAML file. For example, below is a module that defines an HTTP function endpoint as well as +a Kafka ingress and egress: + +```yaml +kind: io.statefun.endpoints.v2/http +spec: + functions: com.example/* + urlPathTemplate: https://bar.foo.com/{function.name} +--- +kind: io.statefun.kafka.v1/ingress +spec: + id: com.example/my-ingress + address: kafka-broker:9092 + consumerGroupId: my-consumer-group + topics: + - topic: message-topic + valueType: io.statefun.types/string + targets: + - com.example/greeter +--- +kind: io.statefun.kafka.v1/egress +spec: + id: com.example/my-egress + address: kafka-broker:9092 + deliverySemantic: + type: exactly-once + transactionTimeout: 15min +--- +``` + +A module YAML file can contain multiple YAML documents, separated by `---`, each representing a component to be included in the +application. Each component is defined by a `kind` typename string, and a `spec` object containing the properties of the +component. Review comment: ```suggestion A module YAML file can contain multiple YAML documents, separated by `---`, each representing a component to be included in the application. Each component is defined by a kind typename string and a spec object containing the component's properties. ``` ########## File path: docs/content/docs/modules/io/overview.md ########## @@ -0,0 +1,36 @@ +--- +title: 'Overview' +weight: 1 +type: docs +aliases: + - /modules/io/ +permalink: /modules/io/index.html +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# I/O Module + +Stateful Functions' I/O modules allow functions to receive and send messages to external systems. +Based on the concept of Ingress (input) and Egress (output) points, and built on top of the Apache Flink® connector ecosystem, I/O modules enable functions to interact with the outside world through the style of message passing. + +Commonly used I/O modules are bundled into the runtime by default and can be configured direclty via the applications [module configuration]({{< ref "docs/modules/overview" >}}). Review comment: ```suggestion Commonly used I/O modules are bundled into the runtime by default and can be configured directly via the applications [module configuration]({{< ref "docs/modules/overview" >}}). ``` ########## File path: docs/content/docs/modules/http-endpoint.md ########## @@ -0,0 +1,206 @@ +--- +title: 'HTTP Function Endpoint' +weight: 2 +type: docs +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# HTTP Function Endpoint + +A HTTP Function Endpoint component defines the endpoint URL that the Stateful Functions runtime should connect to for +invoking a given function, or for a more dynamic deployment, functions within a specified namespace. + +Below is an example of an HTTP endpoint definition in an application's module configuration: + +{{< tabs "8951ef0a-cdd4-40d1-bda8-dec1299aaf41" >}} +{{< tab "v2 (latest)" >}} +```yaml +kind: io.statefun.endpoints.v2/http +spec: + functions: com.example/* + urlPathTemplate: https://bar.foo.com:8080/{function.name} + transport: + timeouts: + call: 1 min + read: 10 sec + write: 10 sec +``` +{{< /tab >}} +{{< tab "v1" >}} +```yaml +kind: io.statefun.endpoints.v1/http +spec: + functions: com.example/* + urlPathTemplate: https://bar.foo.com:8080/{function.name} + timeouts: + call: 1 min + read: 10 sec + write: 10 sec +``` +{{< /tab >}} +{{< /tabs >}} + +In this example, an endpoint for a function within the logical namespace `com.example` is declared. +The runtime will invoke all functions under this namespace with the endpoint URL template. + +### URL Template + +The URL template name may contain template parameters that are filled in based on the function's specific type. +In the example below, a message sent to message type `com.example/greeter` will be sent to `http://bar.foo.com/greeter`. + +```yaml +spec: + functions: com.example/* + urlPathTemplate: https://bar.foo.com/{function.name} +``` + +Templating parameterization works well with load balancers and service gateways. +Suppose `http://bar.foo.com` was an [NGINX](https://www.nginx.com/) server, you can use the different paths to physical systems. Users may now deploy some functions on Kubernetes, others AWS Lambda, while others still on physical servers. + +{{< img src="/fig/dispatch.png" alt="function dispatch" width="75%" >}} + +### Transport + +Switching between different transport clients is supported since `v2` of HTTP function endpoint definitions. + +The transport client to use is specified using `spec.transport.type`. If not specified, by default, Stateful Functions uses [OkHttp](https://square.github.io/okhttp/). + +All fields under `spec.transport` is used as the properties to configure the transport client. For example, the example below configures various timeout settings for the default `OkHttp` transport: + +```yaml +spec: + transport: + timeouts: + call: 1 min + read: 30 sec + write: 20 sec +``` + +#### Asynchronous HTTP transport (Beta) + +Alternatively, Stateful Functions also ships a transport option based on asynchronous non-blocking IO, implemented with [Netty](https://netty.io/). Review comment: We need one more sentence here motivating the feature. ########## File path: docs/content/docs/modules/http-endpoint.md ########## @@ -0,0 +1,206 @@ +--- +title: 'HTTP Function Endpoint' +weight: 2 +type: docs +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# HTTP Function Endpoint + +A HTTP Function Endpoint component defines the endpoint URL that the Stateful Functions runtime should connect to for +invoking a given function, or for a more dynamic deployment, functions within a specified namespace. + +Below is an example of an HTTP endpoint definition in an application's module configuration: + +{{< tabs "8951ef0a-cdd4-40d1-bda8-dec1299aaf41" >}} +{{< tab "v2 (latest)" >}} +```yaml +kind: io.statefun.endpoints.v2/http +spec: + functions: com.example/* + urlPathTemplate: https://bar.foo.com:8080/{function.name} + transport: + timeouts: + call: 1 min + read: 10 sec + write: 10 sec +``` +{{< /tab >}} +{{< tab "v1" >}} +```yaml +kind: io.statefun.endpoints.v1/http +spec: + functions: com.example/* + urlPathTemplate: https://bar.foo.com:8080/{function.name} + timeouts: + call: 1 min + read: 10 sec + write: 10 sec +``` +{{< /tab >}} +{{< /tabs >}} + +In this example, an endpoint for a function within the logical namespace `com.example` is declared. +The runtime will invoke all functions under this namespace with the endpoint URL template. + +### URL Template + +The URL template name may contain template parameters that are filled in based on the function's specific type. +In the example below, a message sent to message type `com.example/greeter` will be sent to `http://bar.foo.com/greeter`. + +```yaml +spec: + functions: com.example/* + urlPathTemplate: https://bar.foo.com/{function.name} +``` + +Templating parameterization works well with load balancers and service gateways. +Suppose `http://bar.foo.com` was an [NGINX](https://www.nginx.com/) server, you can use the different paths to physical systems. Users may now deploy some functions on Kubernetes, others AWS Lambda, while others still on physical servers. Review comment: ```suggestion Suppose `http://bar.foo.com` was an [NGINX](https://www.nginx.com/) server; you can use the different paths to physical systems. Users may now deploy some functions on Kubernetes, others AWS Lambda, while others are still on physical servers. ``` ########## File path: docs/content/docs/modules/http-endpoint.md ########## @@ -0,0 +1,206 @@ +--- +title: 'HTTP Function Endpoint' +weight: 2 +type: docs +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# HTTP Function Endpoint + +A HTTP Function Endpoint component defines the endpoint URL that the Stateful Functions runtime should connect to for Review comment: English is a dumb language, even though HTTP doesn't start with a vowel it should still be "An HTTP". ```suggestion An HTTP Function Endpoint component defines the endpoint URL that the Stateful Functions runtime should connect to for ``` ########## File path: docs/content/docs/modules/http-endpoint.md ########## @@ -0,0 +1,206 @@ +--- +title: 'HTTP Function Endpoint' +weight: 2 +type: docs +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# HTTP Function Endpoint + +A HTTP Function Endpoint component defines the endpoint URL that the Stateful Functions runtime should connect to for +invoking a given function, or for a more dynamic deployment, functions within a specified namespace. + +Below is an example of an HTTP endpoint definition in an application's module configuration: + +{{< tabs "8951ef0a-cdd4-40d1-bda8-dec1299aaf41" >}} +{{< tab "v2 (latest)" >}} +```yaml +kind: io.statefun.endpoints.v2/http +spec: + functions: com.example/* + urlPathTemplate: https://bar.foo.com:8080/{function.name} + transport: + timeouts: + call: 1 min + read: 10 sec + write: 10 sec +``` +{{< /tab >}} +{{< tab "v1" >}} +```yaml +kind: io.statefun.endpoints.v1/http +spec: + functions: com.example/* + urlPathTemplate: https://bar.foo.com:8080/{function.name} + timeouts: + call: 1 min + read: 10 sec + write: 10 sec +``` +{{< /tab >}} +{{< /tabs >}} + +In this example, an endpoint for a function within the logical namespace `com.example` is declared. +The runtime will invoke all functions under this namespace with the endpoint URL template. + +### URL Template + +The URL template name may contain template parameters that are filled in based on the function's specific type. Review comment: ```suggestion The URL template name may contain template parameters filled in dynamically based on the function's specific type. ``` ########## File path: docs/content/docs/modules/overview.md ########## @@ -0,0 +1,66 @@ +--- +title: 'Overview' +weight: 1 +type: docs +aliases: + - /modules/ +permalink: /modules/index.html +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Application Modules + +An application module consists of multiple [components]({{< ref "docs/concepts/application-building-blocks" >}}) +that take part in a StateFun application. It includes the endpoints where functions can be reached along with ingress +and egress definitions. Review comment: ```suggestion An application module consists of multiple [components]({{< ref "docs/concepts/application-building-blocks" >}}) that take part in a StateFun application. It includes the endpoints where the runtime can reach functions, along with ingress and egress definitions. ``` ########## File path: docs/content/docs/modules/io/overview.md ########## @@ -0,0 +1,36 @@ +--- +title: 'Overview' +weight: 1 +type: docs +aliases: + - /modules/io/ +permalink: /modules/io/index.html +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# I/O Module + +Stateful Functions' I/O modules allow functions to receive and send messages to external systems. +Based on the concept of Ingress (input) and Egress (output) points, and built on top of the Apache Flink® connector ecosystem, I/O modules enable functions to interact with the outside world through the style of message passing. + +Commonly used I/O modules are bundled into the runtime by default and can be configured direclty via the applications [module configuration]({{< ref "docs/modules/overview" >}}). +Additionally, custom connectors for other systems can be [plugged in]({{< ref "docs/modules/io/flink-connectors" >}}) to the runtime. + +Keep in mind that to use one of these connectors in an application, additional third party components are usually required, e.g. servers for the data stores or message queues. Review comment: ```suggestion Remember, to use one of these connectors in an application, third-party components are usually required, e.g., servers for the data stores or message queues. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
