[
https://issues.apache.org/jira/browse/KAFKA-13473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeel Jotaniya reassigned KAFKA-13473:
-------------------------------------
Assignee: Jeel Jotaniya
> Log cleaner Dynamic configs aren't applied after a restart
> ----------------------------------------------------------
>
> Key: KAFKA-13473
> URL: https://issues.apache.org/jira/browse/KAFKA-13473
> Project: Kafka
> Issue Type: Bug
> Components: config, core
> Affects Versions: 2.8.1
> Reporter: Tim Patterson
> Assignee: Jeel Jotaniya
> Priority: Minor
>
> Upon restarting kafka, dynamically configured log cleaner configs aren't
> picked up and applied.
>
> Here are some logs from a local kafka when I up the threads to 2 using the
> kafka-config tool - Noting the last 2 lines where it starts up 2 log cleaner
> threads.
>
> {code:java}
> [2021-11-23 21:09:50,044] INFO [Admin Manager on Broker 1001]: Updating
> brokers with new configuration : log.cleaner.threads -> 2
> (kafka.server.ZkAdminManager)
> [2021-11-23 21:09:50,092] INFO Processing override for entityPath:
> brokers/<default> with config: HashMap(log.cleaner.threads -> 2)
> (kafka.server.DynamicConfigManager) log.cleaner.threads = 2
> [2021-11-23 21:09:50,113] INFO Shutting down the log cleaner.
> (kafka.log.LogCleaner)
> [2021-11-23 21:09:50,114] INFO [kafka-log-cleaner-thread-0]: Shutting down
> (kafka.log.LogCleaner)
> [2021-11-23 21:09:50,116] INFO [kafka-log-cleaner-thread-0]: Stopped
> (kafka.log.LogCleaner)
> [2021-11-23 21:09:50,116] INFO [kafka-log-cleaner-thread-0]: Shutdown
> completed (kafka.log.LogCleaner)
> [2021-11-23 21:09:50,119] INFO Starting the log cleaner (kafka.log.LogCleaner)
> [2021-11-23 21:09:50,178] INFO [kafka-log-cleaner-thread-0]: Starting
> (kafka.log.LogCleaner)
> [2021-11-23 21:09:50,181] INFO [kafka-log-cleaner-thread-1]: Starting
> (kafka.log.LogCleaner){code}
> And now after a restart, at no point does it ever start 2 threads, even
> though it clearly knows about the configs
>
> {code:java}
> [2021-11-23 21:10:46,659] INFO Starting the log cleaner (kafka.log.LogCleaner)
> [2021-11-23 21:10:46,723] INFO [kafka-log-cleaner-thread-0]: Starting
> (kafka.log.LogCleaner)
> [2021-11-23 21:10:48,124] INFO Processing override for entityPath:
> brokers/<default> with config: HashMap(log.cleaner.threads -> 2)
> (kafka.server.DynamicConfigManager) log.cleaner.backoff.ms = 15000
> log.cleaner.dedupe.buffer.size = 150000000 log.cleaner.delete.retention.ms =
> 86400000 log.cleaner.enable = true log.cleaner.io.buffer.load.factor = 0.9
> log.cleaner.io.buffer.size = 524288 log.cleaner.io.max.bytes.per.second =
> 1.7976931348623157E308 log.cleaner.max.compaction.lag.ms =
> 9223372036854775807 log.cleaner.min.cleanable.ratio = 0.5
> log.cleaner.min.compaction.lag.ms = 0 log.cleaner.threads = 2{code}
>
> When investigating from the kafka config tool all looks well.
>
> {code:java}
> kafka-configs --bootstrap-server $BROKER_URL --entity-type brokers
> --entity-default --describe --all | grep log.cleaner.threads
> log.cleaner.threads=2 sensitive=false
> synonyms={DYNAMIC_DEFAULT_BROKER_CONFIG:log.cleaner.threads=2}{code}
>
> But if you try change the config you soon find out that all is not well(note
> here it mentions the current value is 1 in the validation message)
>
> {code:java}
> kafka-configs --bootstrap-server $BROKER_URL --entity-type brokers
> --entity-default --alter --add-config log.cleaner.threads=3 Error while
> executing config command with args '--bootstrap-server profile_kafka:9093
> --entity-type brokers --entity-default --alter --add-config
> log.cleaner.threads=3' java.util.concurrent.ExecutionException:
> org.apache.kafka.common.errors.InvalidRequestException: Invalid config value
> for resource ConfigResource(type=BROKER, name=''): Invalid value
> org.apache.kafka.common.config.ConfigException: Log cleaner threads cannot be
> increased to more than double the current value 1 for configuration Invalid
> dynamic configuration{code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
