Maximilian Michels created FLINK-38381:
------------------------------------------
Summary: Autoscaler does not enumerate partitions across kafka
clusters
Key: FLINK-38381
URL: https://issues.apache.org/jira/browse/FLINK-38381
Project: Flink
Issue Type: Bug
Components: Autoscaler, Kubernetes Operator
Affects Versions: kubernetes-operator-1.12.1, kubernetes-operator-1.12.0
Reporter: Maximilian Michels
Assignee: Maximilian Michels
Fix For: kubernetes-operator-1.13.0
When using the DynamicKafkaSink, topics can be spread across multiple clusters.
This used to work fine, but a regression has been added which considers
partitions across different clusters to be identical. This limits the scale out
of the source operator.
Here is an example:
{noformat}
"1.Source__Kafka_Source_(testTopic).kafkaCluster.my-cluster-1.KafkaSourceReader.topic.testTopic.partition.0.currentOffset",
"1.Source__Kafka_Source_(testTopic).kafkaCluster.my-cluster-2.KafkaSourceReader.topic.testTopic.partition.0.currentOffset"{noformat}
Those would result be treated as one partition, but there are two partitions
from separate kafka clusters.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
