[
https://issues.apache.org/jira/browse/FLINK-21507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Iaroslav Zeigerman updated FLINK-21507:
---------------------------------------
Description:
When I use multiple custom partitioning operations in a row like this:
{code:java}
stream
.partitionCustom(<custom_partitioner1>, _.key1)
.mapWithState(...)
.partitionCustom(<custom_partitioner2>, _.key2)
.map(...)
....{code}
I see that only last partitioning operation (custom_partitioner2) is reflected
in the DAG while the 1st one is ignored entirely.
I've also confirmed that the 1st partitioning wasn't applied at runtime from
application logs.
**UPD**
Seems like the problem is caused by
was:
When I use multiple custom partitioning operations in a row like this:
{code:java}
stream
.partitionCustom(<custom_partitioner1>, _.key1)
.mapWithState(...)
.partitionCustom(<custom_partitioner2>, _.key2)
.map(...)
....{code}
I see that only last partitioning operation (custom_partitioner2) is reflected
in the DAG while the 1st one is ignored entirely.
I've also confirmed that the 1st partitioning wasn't applied at runtime from
application logs.
> Reinterpreting stream as keyed breaks the upstream partitioning
> ---------------------------------------------------------------
>
> Key: FLINK-21507
> URL: https://issues.apache.org/jira/browse/FLINK-21507
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream, API / Scala
> Affects Versions: 1.11.0
> Reporter: Iaroslav Zeigerman
> Priority: Major
>
> When I use multiple custom partitioning operations in a row like this:
> {code:java}
> stream
> .partitionCustom(<custom_partitioner1>, _.key1)
> .mapWithState(...)
> .partitionCustom(<custom_partitioner2>, _.key2)
> .map(...)
> ....{code}
> I see that only last partitioning operation (custom_partitioner2) is
> reflected in the DAG while the 1st one is ignored entirely.
> I've also confirmed that the 1st partitioning wasn't applied at runtime from
> application logs.
> **UPD**
> Seems like the problem is caused by
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
