[
https://issues.apache.org/jira/browse/NIFI-16077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wojciech Trefon updated NIFI-16077:
-----------------------------------
Description:
The Kafka components (ConsumeKafka, PublishKafka, and the Kafka Connection
Service) expose a "Security Protocol" property whose default value is
currently PLAINTEXT (KafkaClientComponent.SECURITY_PROTOCOL).
TLS is the norm for production Kafka deployments today, so a plaintext
default is both an insecure and a fragile starting point. When a client is
left on a plaintext protocol but the broker actually requires TLS, the Kafka
client reads the broker's TLS alert record as a message-length field
(0x15030300 ≈ 336 MB) and attempts to allocate that amount per broker
connection, exhausting the heap and driving the process into an
OutOfMemoryError crash-loop within roughly 30-60 seconds. The root cause is
in the Kafka client library (see KAFKA-4090, open and unresolved), so it
cannot be prevented on the NiFi side — but shipping a secure default makes it
far less likely for users to fall into this trap by omission.
Proposed change: set the default value of the Security Protocol property to
SSL in the shared KafkaClientComponent definition. Because the property
descriptor is shared, this single change covers all Kafka processors and the
Kafka connection service. The property is required and is always persisted
explicitly in existing flows, so this only affects newly added components.
References:
- KAFKA-4090: https://issues.apache.org/jira/browse/KAFKA-4090
was:
The Kafka components (ConsumeKafka, PublishKafka, and the Kafka Connection
Service) expose a "Security Protocol" property whose default value is
currently PLAINTEXT (KafkaClientComponent.SECURITY_PROTOCOL).
TLS is the norm for production Kafka deployments today, so a plaintext
default is both an insecure and a fragile starting point. When a client is
left on a plaintext protocol but the broker actually requires TLS, the Kafka
client reads the broker's TLS alert record as a message-length field
(0x15030300 ≈ 336 MB) and attempts to allocate that amount per broker
connection, exhausting the heap and driving the process into an
OutOfMemoryError crash-loop within roughly 30-60 seconds. The root cause is
in the Kafka client library (see KAFKA-4090, open and unresolved), so it
cannot be prevented on the NiFi side — but shipping a secure default makes it
far less likely for users to fall into this trap by omission.
Proposed change: set the default value of the Security Protocol property to
SASL_SSL in the shared KafkaClientComponent definition. Because the property
descriptor is shared, this single change covers all Kafka processors and the
Kafka connection service. The property is required and is always persisted
explicitly in existing flows, so this only affects newly added components.
References:
- KAFKA-4090: https://issues.apache.org/jira/browse/KAFKA-4090
> Change default Security Protocol for Kafka components from PLAINTEXT to SSL
> ---------------------------------------------------------------------------
>
> Key: NIFI-16077
> URL: https://issues.apache.org/jira/browse/NIFI-16077
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Wojciech Trefon
> Priority: Minor
> Labels: kafka
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The Kafka components (ConsumeKafka, PublishKafka, and the Kafka Connection
> Service) expose a "Security Protocol" property whose default value is
> currently PLAINTEXT (KafkaClientComponent.SECURITY_PROTOCOL).
> TLS is the norm for production Kafka deployments today, so a plaintext
> default is both an insecure and a fragile starting point. When a client is
> left on a plaintext protocol but the broker actually requires TLS, the Kafka
> client reads the broker's TLS alert record as a message-length field
> (0x15030300 ≈ 336 MB) and attempts to allocate that amount per broker
> connection, exhausting the heap and driving the process into an
> OutOfMemoryError crash-loop within roughly 30-60 seconds. The root cause is
> in the Kafka client library (see KAFKA-4090, open and unresolved), so it
> cannot be prevented on the NiFi side — but shipping a secure default makes it
> far less likely for users to fall into this trap by omission.
> Proposed change: set the default value of the Security Protocol property to
> SSL in the shared KafkaClientComponent definition. Because the property
> descriptor is shared, this single change covers all Kafka processors and the
> Kafka connection service. The property is required and is always persisted
> explicitly in existing flows, so this only affects newly added components.
> References:
> - KAFKA-4090: https://issues.apache.org/jira/browse/KAFKA-4090
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
