tillrohrmann commented on a change in pull request #309:
URL: https://github.com/apache/flink-statefun/pull/309#discussion_r836216926
##########
File path: docs/content/docs/modules/overview.md
##########
@@ -61,3 +61,36 @@ spec:
A module YAML file can contain multiple YAML documents, separated by `---`,
each representing a component to be included in the application.
Each component is defined by a kind typename string and a spec object
containing the component's properties.
+
+# Configuration string interpolation
+You can use `${placeholders}` inside `spec` elements. These will be replaced
by entries from a configuration map, consisting of:
+1. System properties
+2. Environment variables
+3. flink-conf.yaml entries with prefix 'statefun.module.global-config.'
+4. Command line args
Review comment:
I think it is more common to give env variables precedence over
`flink-conf.yaml` values.
##########
File path: docs/content/docs/modules/overview.md
##########
@@ -61,3 +61,36 @@ spec:
A module YAML file can contain multiple YAML documents, separated by `---`,
each representing a component to be included in the application.
Each component is defined by a kind typename string and a spec object
containing the component's properties.
+
+# Configuration string interpolation
+You can use `${placeholders}` inside `spec` elements. These will be replaced
by entries from a configuration map, consisting of:
+1. System properties
+2. Environment variables
+3. flink-conf.yaml entries with prefix 'statefun.module.global-config.'
+4. Command line args
+
+where (4) override (3) which override (2) which override (1).
+
+Example:
+```yaml
+kind: io.statefun.endpoints.v2/http
+spec:
+ functions: com.example/*
+ urlPathTemplate: ${FUNC_PROTOCOL}://${FUNC_DNS}/{function.name}
+---
+kind: io.statefun.kafka.v1/ingress
+spec:
+ id: com.example/my-ingress
+ address: ${KAFKA_ADDRESS}:${KAFKA_PORT}
+ consumerGroupId: my-consumer-group
+ topics:
+ - topic: ${KAFKA_INGRESS_TOPIC}
+ (...)
+ properties:
+ - ssl.truststore.location: ${SSL_TRUSTSTORE_LOCATION}
+ - ssl.truststore.password: ${SSL_TRUSTSTORE_PASSWORD}
+ (...)
Review comment:
I really like the flexibility the templating mechanism gives to the
user. The thing I am asking myself is whether it gives too much power so that
the user can shoot himself into his foot.
The danger I see is that we add level of indirections that make it harder to
reason about what the effective yaml will look like for a user. The underlying
problem the issue wants to solve is to pass in information that is only
available to the process that runs SF but not the user (e.g. secrets). I am
wondering whether there is an alternative to achieve the same but with a bit
less power (e.g. allowing substitution only for selected fields (also
confusing)). I don't have a perfect answer here. I mainly wanted to hear your
and @igalshilman's opinion here.
Maybe one idea could be to log the effective/resolved yaml somewhere so that
the user sees what is actually run, if this does not happen already.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
