Aman Harish Gandhi created KAFKA-15266:
------------------------------------------
Summary: Log configs ignore static configs set non primary synonyms
Key: KAFKA-15266
URL: https://issues.apache.org/jira/browse/KAFKA-15266
Project: Kafka
Issue Type: Bug
Components: core
Affects Versions: 2.6.0
Reporter: Aman Harish Gandhi
Assignee: Aman Harish Gandhi
In our server.properties we had the following config
{code:java}
log.retention.hours=48
{code}
We noticed that after running alter configs to update broker level config(for a
config unrelated to retention) we were only deleting data after 7 days instead
of the configured 2.
The alterconfig we had ran was similar to this
{code:java}
sh kafka-config.sh --bootstrap-server localhost:9092 --alter --add-config
"log.segment.bytes=500000"
{code}
Digging deeper the issue could be pin pointed to the reconfigure block of
DynamicLogConfig inside DynamicBrokerConfig. Here we only look at the "primary"
KafkaConfig synonym of the LogConfig and if it is not set then we remove the
value set in default log config as well. This eventually leads to the
retention.ms not being set in the default log config and that leads to the
default value of 7 days being used. The value set in "log.retention.hours" is
completely ignored in this case.
Pasting the relevant code block here
{code:java}
newConfig.valuesFromThisConfig.forEach { (k, v) =>
if (DynamicLogConfig.ReconfigurableConfigs.contains(k)) {
DynamicLogConfig.KafkaConfigToLogConfigName.get(k).foreach { configName =>
if (v == null)
newBrokerDefaults.remove(configName)
else
newBrokerDefaults.put(configName, v.asInstanceOf[AnyRef])
}
}
} {code}
In the above block `DynamicLogConfig.ReconfigurableConfigs` contains only
log.retention.ms. It does not contain the other synonyms like
`log.retention.minutes` or `log.retention.hours`.
This issue seems prevalent in all cases where there are more than 1 KafkaConfig
synonyms for the LogConfig.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
