[
https://issues.apache.org/jira/browse/FLINK-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15060984#comment-15060984
]
ASF GitHub Bot commented on FLINK-3166:
---------------------------------------
GitHub user greghogan opened a pull request:
https://github.com/apache/flink/pull/1464
[FLINK-3166] [runtime] The first program in ObjectReuseITCase has the wrong
expected result, and it succeeds
- TestEnvironment now honors configuration of object reuse
- Fixed reduce transformations to allow the user to modify and return
either input
This is neither clean nor complete but I wanted to get some feedback before
continuing.
ObjectReuseITCase was forcing object reuse, this was fixed by updating
TestEnvironment.
ReduceCombineDriver and ReduceDriver both requests the next record and call
the user's reduce() and so can handle whichever of the two inputs objects the
user might write to.
In the second test case DataSourceTask (which requests the next record)
calls ChainedAllReduceDriver.collect (which calls the user's reduce()). There
is no feedback so DataSourceTask must cycle through `n`+1 reusable objects for
unknown `n`.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/greghogan/flink 3166_fix_objectreuseitcase
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1464.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1464
----
commit 78dc073440ed8ecfec51fbe33ce665562786bf20
Author: Greg Hogan <[email protected]>
Date: 2015-12-16T19:42:06Z
[FLINK-3166] [runtime] The first program in ObjectReuseITCase has the wrong
expected result, and it succeeds
- TestEnvironment now honors configuration of object reuse
- Fixed reduce transformations to allow the user to modify and return
either input
----
> The first program in ObjectReuseITCase has the wrong expected result, and it
> succeeds
> -------------------------------------------------------------------------------------
>
> Key: FLINK-3166
> URL: https://issues.apache.org/jira/browse/FLINK-3166
> Project: Flink
> Issue Type: Bug
> Components: Distributed Runtime, Documentation, Tests
> Reporter: Gabor Gevay
> Assignee: Greg Hogan
> Priority: Critical
>
> The first program in ObjectReuseITCase has the following input:
> a,1
> a,2
> a,3
> a,4
> a,50
> There is a groupBy on field 0, and then a reduce, so the result should be
> 1+2+3+4+50 = 60. But the hardcoded expected result is 100, and running the
> Flink program also produces this.
> The problem is caused my mismatched assumptions between
> ReduceCombineDriver.sortAndCombine and the ReduceFunction in the test about
> object reuse rules of ReduceFunctions:
> ReduceCombineDriver.sortAndCombine has the following comment:
> "The user function is expected to return the first input as the result."
> While the ReduceFunction in the test is modifying and returning the second
> input. (And the second program in the test also has the same problem.)
> I can't find the assumption that is stated in the comment in any
> documentation. For example, the javadoc of ReduceFunction should make the
> user aware of this. Or, alternatively, the code of the driver should be
> modified to not make this assumption. I'm not sure which solution is better.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
