[
https://issues.apache.org/jira/browse/FLINK-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14381564#comment-14381564
]
ASF GitHub Bot commented on FLINK-1656:
---------------------------------------
Github user fhueske commented on the pull request:
https://github.com/apache/flink/pull/525#issuecomment-86400852
I updated the PR as discussed:
- GlobalProperties are filtered with the user-specified semantic properties.
- LocalProperties for are filtered with forward field info for key fields
only (sorting/grouping can only be preserved for key fields and is completely
destroyed for key-less operators such as AllReduce or MapPartition).
Documentation is adapted accordingly.
> Filtered Semantic Properties for Operators with Iterators
> ---------------------------------------------------------
>
> Key: FLINK-1656
> URL: https://issues.apache.org/jira/browse/FLINK-1656
> Project: Flink
> Issue Type: Bug
> Components: Documentation
> Affects Versions: 0.9
> Reporter: Fabian Hueske
> Assignee: Fabian Hueske
> Priority: Critical
>
> The documentation of ForwardedFields is incomplete for operators with
> iterator inputs (GroupReduce, CoGroup).
> This should be fixed ASAP, because it can lead to incorrect program execution.
> The conditions for forwarded fields on operators with iterator input are:
> 1) forwarded fields must be emitted in the order in which they are received
> through the iterator
> 2) all forwarded fields of a record must stick together, i.e., if your
> function builds record from field 0 of the 1st, 3rd, 5th, ... and field 1 of
> the 2nd, 4th, ... record coming through the iterator, these are not valid
> forwarded fields.
> 3) it is OK to completely filter out records coming through the iterator.
> The reason for these conditions is that the optimizer uses forwarded fields
> to reason about physical data properties such as order and grouping. Mixing
> up the order of records or emitting records which are composed from different
> input records, might destroy a (secondary) order or grouping.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
