[
https://issues.apache.org/jira/browse/FLINK-8181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16276365#comment-16276365
]
ASF GitHub Bot commented on FLINK-8181:
---------------------------------------
Github user tzulitai commented on the issue:
https://github.com/apache/flink/pull/5108
This is not an actual user-reported problem.
I considered this a regression due to the fact that the original
`FlinkFixedPartitioner` implementation did not switch partitions at all between
recoveries, even in the case of scaling out Kafka.
Let me explain a bit the history of the fixed partitioner with a bit of
code (sorry for not have provided this information earlier). The previous
implementation:
https://github.com/apache/flink/blob/release-1.2/flink-connectors/flink-connector-kafka-base/src/main/java/org/apache/flink/streaming/connectors/kafka/partitioner/FixedPartitioner.java.
As you can see, the original implementation of the fixed partitioner
already has the following problems:
1. State of the partitioner (i.e., the `targetPartition`) is lost on
failure, and re-determined on restore.
2. The API for the partitioner itself is problematic, in the case of
multiple topics. Please see
[FLINK-6288](https://issues.apache.org/jira/browse/FLINK-6288) for a
description on that.
It, however, does have the guarantee that provides stickiness between
recoveries.
Now, the issue this PR is fixing occurred when fixing problem 2. in 9ed9b68.
The `FlinkFixedPartitioner` no longer provided stickiness between
recoveries; the partitioner will switch partitions in the case of scaling out
Kafka.
> Kafka producer's FlinkFixedPartitioner returns different partitions when a
> target topic is rescaled
> ---------------------------------------------------------------------------------------------------
>
> Key: FLINK-8181
> URL: https://issues.apache.org/jira/browse/FLINK-8181
> Project: Flink
> Issue Type: Bug
> Components: Kafka Connector
> Affects Versions: 1.4.0, 1.5.0
> Reporter: Tzu-Li (Gordon) Tai
> Assignee: Tzu-Li (Gordon) Tai
> Priority: Blocker
> Fix For: 1.4.0, 1.5.0
>
>
> On fixing FLINK-6288 and migrating the original {{FlinkFixedPartitioner}} to
> the new partitioning API (commit 9ed9b68397b51bfd2b0f6e532212a82f771641bd),
> the {{FlinkFixedPartitioner}} no longer returns identical target partitions
> once a target topic is rescaled.
> This results in a behavioral regression when the {{FlinkFixedPartitioner}} is
> used.
> The {{FlinkFixedPartitionerTest}} should also be strengthened to cover the
> target topic rescaling case.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
