Kuan Po Tseng created KAFKA-19264:
-------------------------------------
Summary: Remove fallback for thread pool sizes in
RemoteLogManagerConfig
Key: KAFKA-19264
URL: https://issues.apache.org/jira/browse/KAFKA-19264
Project: Kafka
Issue Type: Task
Components: Tiered-Storage
Reporter: Kuan Po Tseng
Assignee: Kuan Po Tseng
The fallback mechanism was first introduced in
[KIP-950|https://cwiki.apache.org/confluence/x/joqzDw]. According to the
proposal, if no thread values are set for
{{remote.log.manager.copier.thread.pool.size}} and
{{{}remote.log.manager.expiration.thread.pool.size{}}}, these two configs would
default to using the value of {{{}remote.log.manager.thread.pool.size{}}}.
As quoted from the KIP:
{quote}If no thread values are set for the two new configurations presented
later on in the document we will default to using the same number of threads in
each pool as detailed by remote.log.manager.thread.pool.size.
{quote}
This fallback behavior was implemented in
[https://github.com/apache/kafka/commit/84ab3b9a5c4930f5ae047df088e38c456c7cde54].
However, this approach was abandoned in
[KIP-1030|https://cwiki.apache.org/confluence/x/FAqpEQ], where the default
values for {{copier}} and {{expiration}} thread pool sizes were changed from
{{-1}} to {{{}10{}}}. The related commit can be found in
[https://github.com/apache/kafka/commit/3b1bd3812e48d488e4b6b53a9085d6552e8adf02].
Additionally, both {{remote.log.manager.copier.thread.pool.size}} and
{{remote.log.manager.expiration.thread.pool.size}} now include a configuration
validator that enforces a minimum value of {{{}1{}}}. This means the fallback
mechanism should be removed from
[{{RemoteLogManagerConfig.java}}|https://issues.apache.org/jira/issues/RemoteLogManagerConfig.java]
to align with the new defaults and validation.
In short, RemoteLogManagerConfig should apply the following changes:
{code:java}
public int remoteLogManagerCopierThreadPoolSize() {
- int size =
config.getInt(REMOTE_LOG_MANAGER_COPIER_THREAD_POOL_SIZE_PROP);
- return size == -1 ? remoteLogManagerThreadPoolSize() : size;
+ return config.getInt(REMOTE_LOG_MANAGER_COPIER_THREAD_POOL_SIZE_PROP);
}
public int remoteLogManagerExpirationThreadPoolSize() {
- int size =
config.getInt(REMOTE_LOG_MANAGER_EXPIRATION_THREAD_POOL_SIZE_PROP);
- return size == -1 ? remoteLogManagerThreadPoolSize() : size;
+ return
config.getInt(REMOTE_LOG_MANAGER_EXPIRATION_THREAD_POOL_SIZE_PROP);
}{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
