[jira] [Created] (KAFKA-20901) [KRaft] cordoned.log.dirs is accepted but not enforced when log.dirs uses a relative path

[Apple Min (Jira)]

Apple Min created KAFKA-20901:
---------------------------------

             Summary: [KRaft] cordoned.log.dirs is accepted but not enforced 
when log.dirs uses a relative path
                 Key: KAFKA-20901
                 URL: https://issues.apache.org/jira/browse/KAFKA-20901
             Project: Kafka
          Issue Type: Bug
          Components: core
    Affects Versions: 4.3.1
            Reporter: Apple Min


h2. Description

When {{log.dirs}} is configured with a relative path, setting 
{{cordoned.log.dirs}} has inconsistent behavior:
 # Setting {{cordoned.log.dirs}} to the absolute path returned by 
{{DescribeLogDirs}} is rejected because it does not exactly match the relative 
value in {{{}log.dirs{}}}.
 # Setting {{cordoned.log.dirs}} to the relative path succeeds and the dynamic 
configuration is persisted.
 # However, the directory is not actually treated as cordoned, and replicas for 
newly created topics can still be assigned to it.

This makes disk cordoning ineffective when {{log.dirs}} contains relative paths.
h2. Environment

 

{{Apache Kafka: 4.3.1
Mode: KRaft
OS: Linux
Broker ID: 1

Kafka working directory:
/root/k431/kafka_2.13-4.3.1_b1

server.properties:
log.dirs=kraft-combined-logs

Resolved runtime log directory:
/root/k431/kafka_2.13-4.3.1_b1/kraft-combined-logs}}
h2. Steps to reproduce
h3. 1. Start a KRaft cluster with a relative {{log.dirs}}

Broker 1 configuration:

 

{{node.id=1
process.roles=broker,controller
log.dirs=kraft-combined-logs}}

The broker resolves this directory at runtime to:

 

{{/root/k431/kafka_2.13-4.3.1_b1/kraft-combined-logs}}

{{DescribeLogDirs}} also returns the absolute path.
h3. 2. Try to cordon the absolute path

Execute an incremental broker configuration update equivalent to:

 

{{bin/kafka-configs.sh \
  --bootstrap-server localhost:9092 \
  --alter \
  --entity-type brokers \
  --entity-name 1 \
  --add-config \
  'cordoned.log.dirs=/root/k431/kafka_2.13-4.3.1_b1/kraft-combined-logs'}}

The update fails with:

 

{{requirement failed: All entries in cordoned.log.dirs must be present
in log.dirs or log.dir.

Missing entries:
/root/k431/kafka_2.13-4.3.1_b1/kraft-combined-logs}}
h3. 3. Cordon the relative path instead

 

{{bin/kafka-configs.sh \
  --bootstrap-server localhost:9092 \
  --alter \
  --entity-type brokers \
  --entity-name 1 \
  --add-config 'cordoned.log.dirs=kraft-combined-logs'}}

This update succeeds.

Describing the broker configuration confirms that the value is persisted:

 

{{cordoned.log.dirs=kraft-combined-logs}}
h3. 4. Create a new topic

Create a topic without an explicit replica assignment:

 

{{bin/kafka-topics.sh \
  --bootstrap-server localhost:9092 \
  --create \
  --topic relative-cordon-test \
  --partitions 12 \
  --replication-factor 1}}

Describe the topic:

 

{{bin/kafka-topics.sh \
  --bootstrap-server localhost:9092 \
  --describe \
  --topic relative-cordon-test}}
h2. Actual behavior

Broker 1 remains eligible for new replica placement.

New replicas are assigned to broker 1 and are created under:

 

{{/root/k431/kafka_2.13-4.3.1_b1/kraft-combined-logs}}

The {{cordoned.log.dirs}} dynamic configuration is present, but the directory 
is not effectively cordoned.
h2. Expected behavior

When the configured value is:

 

{{log.dirs=kraft-combined-logs}}

and the broker accepts:

 

{{cordoned.log.dirs=kraft-combined-logs}}

Kafka should resolve both values consistently and report the corresponding 
directory ID as cordoned.

The Controller should then exclude that directory from new replica placement.

The equivalent absolute path should also either:
 # be accepted after path normalization; or
 # produce a clear validation error stating that relative paths are unsupported 
for disk cordoning.

A configuration update must not succeed while silently failing to cordon the 
directory.
h2. Code-path analysis

The issue appears to be caused by inconsistent path representations.
h3. Validation uses the original configuration strings

In:

 

{{core/src/main/scala/kafka/server/DynamicBrokerConfig.scala
DynamicLogConfig.validateReconfiguration}}

the validation checks whether each entry from {{cordoned.log.dirs}} is directly 
contained in {{{}newConfig.logDirs(){}}}.

This allows:

 

{{kraft-combined-logs == kraft-combined-logs}}

but rejects the equivalent absolute path.
h3. Runtime directory IDs are indexed by absolute path

In:

 

{{core/src/main/scala/kafka/log/LogManager.scala
LogManager.loadDirectoryIds}}

the directory ID map is populated using:

 

{{logDir.getAbsolutePath}}

{{LogManager.directoryId}} is also documented as accepting an absolute path.
h3. The relative path is passed directly to {{directoryId}}

In:

 

{{core/src/main/scala/kafka/server/DynamicBrokerConfig.scala
DynamicLogConfig.reconfigure}}

the configured strings are passed directly to:

 

{{logManager.directoryId(dir)}}

With the example configuration, this effectively performs:

 

{{directoryId("kraft-combined-logs")}}

while the map key is:

 

{{/root/k431/kafka_2.13-4.3.1_b1/kraft-combined-logs}}

The lookup returns no directory ID, so the Broker does not report the directory 
as cordoned to the Controller.

There is a similar representation mismatch in the per-directory cordon metric, 
which compares the configured set against {{{}dir.getAbsolutePath{}}}.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)