tzulitai opened a new pull request #247:
URL: https://github.com/apache/flink-statefun/pull/247
This PR achieves the following goals:
- Makes system code that parses `module.yaml` definition files responsible
only up to the point of extracting multiple objects which each represent
application components (e.g. function / ingress / egress) that should be bound
to the application universe. The system code should not handle parsing of each
individual object. These component objects are represented by the
`ModuleComponent` class, and their YAML document format is the following:
```
kind: com.foo.bar.v5/some.component
spec:
...
```
- As shown above, each `ModuleComponent` should define a binder kind. For
example, an http endpoint component defines the `io.statefun.endpoints/http`
kind. This kind typename maps directly to a `ComponentBinder` implementation.
The `ComponentBinder` understand how to parse the object, resolves them into
application components, and binds them to the universe. `ComponentBinder`s are
extensions added to the system using `ExtensionModule`s.
- Each `ComponentBinder` implementation should be located close to where it
is actually meaningful, instead of being all clotted in the
`o.a.f.statefun.flink.core.jsonmodule` package like before. For example, http
endpoint binders live within the same package as the `HttpFunctionProvider`s,
and Kafka / Kinesis IO binders live together within the Kafka / Kinesis
packages in `flink-statefun-io-bundle`. The
`o.a.f.statefun.flink.core.jsonmodule` package now really just contains
system-level code, and treating all module components as black boxes to be
handled by the individual `ComponentBinder`s located elsewhere.
- Moreover, this makes extending `module.yaml` specifications trivial, as it
just requires implementing a new `ComponentBinder` and binding it as an
extension to the system. The same goes for user custom components they wish to
add to `module.yaml`, for example custom ingresses / egresses.
## Shipped `ComponentBinder`s
As a result, with this PR, StateFun currently ships out-of-box the following
`ComponentBinder`s that are available to be used:
### Function Endpoints
- `io.statefun.endpoints.v1/http` or `io.statefun.endpoints/http`:
corresponding binder is `HttpFunctionEndpointComponentBinderV1`.
- `io.statefun.endpoints.v2/http`: corresponding binder is
`HttpFunctionEndpointComponentBinderV2`.
### Ingresses / Egresses
- `io.statefun.kafka.v1/ingress` or `io.statefun.kafka/ingress`:
corresponding binder is `AutoRoutableKafkaIngressComponentBinderV1`.
- `io.statefun.kafka.v1/egress` or `io.statefun.kafka/egress`: corresponding
binder is `GenericKafkaEgressComponentBinderV1`.
- `io.statefun.kinesis.v1/ingress` or `io.statefun.kinesis/ingress`:
corresponding binder is `AutoRoutableKinesisIngressComponentBinderV1`.
- `io.statefun.kinesis.v1/egress` or `io.statefun.kinesis/egress`:
corresponding binder is `GenericKinesisEgressComponentBinderV1`.
## Follow-up: Adopting k8s-style resource formats for the `module.yaml`
definition
With this change, we are essentially making `module.yaml` definition files
simply a collection of YAML documents, each representing a component. This
actually brings us very close to the format of k8s resource definitions.
With this in mind, the second commit in this PR leverages this fact to
further simplify `module.yaml`s to the following format:
```
kind: com.foo.bar.v5/some.component
spec:
...
---
kind: com.foo.bar/some.component.2
spec:
...
---
kind: com.foo.bar.v4/some.component.3
spec:
...
---
```
This is much more lean than the single root structure we had before.
Moreover, it is almost trivial to just copy-and-paste the exact same contents
and port it as a k8s custom resource definition. This would become very handy
when we decide to implement our own custom k8s StateFun resource / deployment
definition.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
