A. Sophie Blee-Goldman created KAFKA-18053:
----------------------------------------------
Summary: Clean up TopologyConfig and API for supplying configs
needed by the topology
Key: KAFKA-18053
URL: https://issues.apache.org/jira/browse/KAFKA-18053
Project: Kafka
Issue Type: Improvement
Components: streams
Reporter: A. Sophie Blee-Goldman
As mentioned briefly in KIP-1112, the current API around configs which need to
be applied prior to building the topology can be cumbersome and easy to
implement incorrectly. This leads to awkward code and confused users at best,
and applications that silently drop some configs at worst.
There are three main problems we want to solve:
# The most serious issue is that the KafkaStreams constructor requires the
application configs whereas the topology can be built without them, so most
people will only pass their configs into KafkaStreams. However certain configs
have to be passed into the topology as well, which is easy for users to miss
and leads to a silent misconfiguration that can easily go unnoticed
# Over time we've added several new APIs for configs that need to be applied
at different/earlier stages of the topology building process. So now users are
faced with multiple ways of passing these configs into the topology, and
configs that may work with any of them or may need to be used with a specific
API
# At some point we also made the TopologyConfig class public, which was
originally intended only for internal use. Furthermore it's only purpose was to
separate configs between different topologies, a feature that has now been
deprecated and will soon be removed, at which point TopologyConfig serves no
purpose and should be removed as well. Unfortunately, the newest APIs for
passing configs into a topology are a Topology and StreamsBuilder constructor
that have a TopologyConfig parameter, rather than the StreamsConfig parameter
used in the previous API (ie StreamsBuilder#build). This is unnecessary since
StreamsConfig would work just as well, and in fact a TopologyConfig has to be
constructed from a StreamsConfig so users are just adding a wrapper layer
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
