[
https://issues.apache.org/jira/browse/FLINK-38381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-38381:
-----------------------------------
Labels: pull-request-available (was: )
> Autoscaler does not enumerate partitions across kafka clusters
> --------------------------------------------------------------
>
> Key: FLINK-38381
> URL: https://issues.apache.org/jira/browse/FLINK-38381
> Project: Flink
> Issue Type: Bug
> Components: Autoscaler, Kubernetes Operator
> Affects Versions: kubernetes-operator-1.12.0, kubernetes-operator-1.12.1
> Reporter: Maximilian Michels
> Assignee: Maximilian Michels
> Priority: Major
> Labels: pull-request-available
> Fix For: kubernetes-operator-1.13.0
>
>
> When using the DynamicKafkaSink, topics can be spread across multiple
> clusters. This used to work fine, but a regression has been added which
> considers partitions across different clusters to be identical. This limits
> the scale out of the source operator.
> Here is an example:
> {noformat}
> "1.Source__Kafka_Source_(testTopic).kafkaCluster.my-cluster-1.KafkaSourceReader.topic.testTopic.partition.0.currentOffset",
> "1.Source__Kafka_Source_(testTopic).kafkaCluster.my-cluster-2.KafkaSourceReader.topic.testTopic.partition.0.currentOffset"{noformat}
> Those would result be treated as one partition, but there are two partitions
> from separate kafka clusters.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
