Jason Gustafson created KAFKA-12280:
---------------------------------------
Summary: Consider internal topic configuration when created
through `CreateTopic`
Key: KAFKA-12280
URL: https://issues.apache.org/jira/browse/KAFKA-12280
Project: Kafka
Issue Type: Improvement
Reporter: Jason Gustafson
Currently internal topics such as __consumer_offsets and __transaction_state
are created dynamically when we receive a `FindCoordinator` request from a
client. It is also possible to create these topics manually using the
`CreateTopic` API. We leverage this capability in some integration test cases,
but I am not sure if it is generally useful.
The interesting thing when these topics are created manually is which topic
configuration will be used. Currently, if the request itself does not indicate
overrides, then the broker will apply the standard topic defaults. It seems
like it would be more logical to use the explicit configuration overrides such
as `offsets.topic.num.partitions`.
I think this raises a few questions:
1. Do we want to support manual creation of the internal topics going forward?
I think we have found this convenient in performance testing when we did not
want to conflate internal topic creation overhead with consumer performance.
2. If we do want to support it, the next question is if we should allow the
configurations like `offsets.topic.num.partitions` to be overridden? This does
create some complexity in `GroupMetadataManager` since we currently don't
support re-partitioning of this topic.
3. Finally, assuming we still support this, we probably at a minimum should
change the behavior to use the overrides from the configuration.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
