[
https://issues.apache.org/jira/browse/FLINK-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376777#comment-14376777
]
ASF GitHub Bot commented on FLINK-1656:
---------------------------------------
GitHub user fhueske opened a pull request:
https://github.com/apache/flink/pull/525
[FLINK-1656] Filter ForwardedField properties for group-at-a-time operators
in Optimizer
Restricts forward field information for group-wise operators.
- For `GroupReduce`, `GroupCombine`, and `CoGroup` operators forward field
information is restricted to grouping key fields.
- For `MapPartition` operators all forwarded field information is discarded.
Extended documentation accordingly.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/fhueske/flink groupKeySemProps
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/525.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #525
----
commit a49967db7ecea782860d7a0fa430cb2281653b03
Author: Fabian Hueske <[email protected]>
Date: 2015-03-23T10:55:34Z
[FLINK-1656] Filter ForwardedField properties for group-at-a-time operators
in Optimizer.
----
> Filtered Semantic Properties for Operators with Iterators
> ---------------------------------------------------------
>
> Key: FLINK-1656
> URL: https://issues.apache.org/jira/browse/FLINK-1656
> Project: Flink
> Issue Type: Bug
> Components: Documentation
> Affects Versions: 0.9
> Reporter: Fabian Hueske
> Assignee: Fabian Hueske
> Priority: Critical
>
> The documentation of ForwardedFields is incomplete for operators with
> iterator inputs (GroupReduce, CoGroup).
> This should be fixed ASAP, because it can lead to incorrect program execution.
> The conditions for forwarded fields on operators with iterator input are:
> 1) forwarded fields must be emitted in the order in which they are received
> through the iterator
> 2) all forwarded fields of a record must stick together, i.e., if your
> function builds record from field 0 of the 1st, 3rd, 5th, ... and field 1 of
> the 2nd, 4th, ... record coming through the iterator, these are not valid
> forwarded fields.
> 3) it is OK to completely filter out records coming through the iterator.
> The reason for these conditions is that the optimizer uses forwarded fields
> to reason about physical data properties such as order and grouping. Mixing
> up the order of records or emitting records which are composed from different
> input records, might destroy a (secondary) order or grouping.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
