[
https://issues.apache.org/jira/browse/FLINK-4578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-4578:
----------------------------------
Labels: auto-deprioritized-major stale-minor (was:
auto-deprioritized-major)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Minor but is unassigned and neither itself nor its Sub-Tasks have been updated
for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is
still Minor, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> AggregateOperator incorrectly sets ForwardedField with nested composite types
> -----------------------------------------------------------------------------
>
> Key: FLINK-4578
> URL: https://issues.apache.org/jira/browse/FLINK-4578
> Project: Flink
> Issue Type: Bug
> Components: API / DataSet
> Reporter: Gábor Gévay
> Priority: Minor
> Labels: auto-deprioritized-major, stale-minor
>
> When an aggregation is called on a grouped DataSet,
> {{AggregateOperator.translateToDataFlow}} tries to determine whether the
> field that is being aggregated is the same field that the grouping is based
> on. If this is not the case, then it adds the ForwardedField property for the
> key field.
> However, the mechanism that makes this decision breaks when there are nested
> composite types involved, because it gets the key positions with
> {{getKeys().computeLogicalKeyPositions()}}, which returns the _flat_
> positions, whereas the position of the field to aggregate is counted only on
> the outer type.
> Example code: https://github.com/ggevay/flink/tree/agg-bad-forwarded-fields
> Here, I have changed the WordCount example to have the type
> {{Tuple3<Tuple2<Byte,Byte>, String, Integer>}}, and do {{.groupBy(1).sum(2)}}
> (which groups by the String field and sums the Integer field). If you set a
> breakpoint into {{AggregateOperator.translateToDataFlow}}, you can see that
> {{logicalKeyPositions}} contains 2, and {{fields}} also contains 2, which
> causes {{keyFieldUsedInAgg}} to be erroneously set to true. The problem is
> caused by the Tuple2 being counted as 2 fields in {{logicalKeyPositions}},
> but only 1 field in {{fields}}.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
