[
https://issues.apache.org/jira/browse/KAFKA-18054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932736#comment-17932736
]
Lorcan commented on KAFKA-18054:
--------------------------------
Hi [~ableegoldman], I’ve done some investigation into this issue and wanted to
check my understanding if possible.
>From my reading of the description I thought a potential implementation would
>involve comparing the topologyConfigs and the stream/application configs and
>check if there’s a mismatch for the topology specific configuration.
The tests that are written in KafkaStreamsTest create a Topology by using
StreamsBuild.build(), which returns a Topology that has been created with the
properties that have been passed to the build method.
This then uses the internalStreamsBuilder.buildAndOptimizeTopology() method
with the properties passed to it however it does not seem to save these
properties in the TopologyConfigs.
The notes for the TopologyConfig refer to passing in a topologyConfig to
constructors for some config needed when the topology is being built, but it
seems like this isn’t necessary for StreamsBuilder.build with properties passed
to it.
As the topologyConfig isn’t updated with the value of the
topology_optimization_config, I’m not sure how this would be check during/
after the creation of the KafkaStream object for the purpose of checking for
mis-configurations.
I’m still quite new to the codebase and am not 100% clear on the above, so
wanted to check if this makes sense. Cheers
> Automatically detect missed configs needed by a topology
> --------------------------------------------------------
>
> Key: KAFKA-18054
> URL: https://issues.apache.org/jira/browse/KAFKA-18054
> Project: Kafka
> Issue Type: Sub-task
> Components: streams
> Reporter: A. Sophie Blee-Goldman
> Assignee: Lorcan
> Priority: Major
>
> See umbrella ticket for problem summary.
> This ticket would help tackle the first issue and would be a signifciant
> improvement without requiring a KIP.
> Basically, we should help users avoid accidentally misconfiguring their
> application by setting one of the topology-specific configs in their
> StreamsConfig but only passing it into the KafkaStreams constructor and not
> into the topology where it actually is applied.
> We should be able to detect this case pretty easily: when the topology and
> application configs are passed into the KafkaStreams constructor, we just
> compare the topology's configs with the provided application configs and look
> for each of the topology-specific configs (see umbrella ticket for the list).
> If any of these configs were set by the user in the application configs but
> not set in the topology's configs, we can throw an exception.
> We should also log a warning if the config is set in both cases but have a
> different value.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
