[
https://issues.apache.org/jira/browse/FLINK-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15063854#comment-15063854
]
ASF GitHub Bot commented on FLINK-3166:
---------------------------------------
Github user StephanEwen commented on a diff in the pull request:
https://github.com/apache/flink/pull/1464#discussion_r48017305
--- Diff:
flink-tests/src/test/java/org/apache/flink/test/javaApiOperators/ObjectReuseITCase.java
---
@@ -124,26 +125,30 @@ public static String runProgram(int progId, String
resultPath) throws Exception
env.execute();
// return expected result
- return "a,100\n";
+ return "a,60\n";
}
case 2: {
+ // Global reduce
final ExecutionEnvironment env =
ExecutionEnvironment.getExecutionEnvironment();
- env.getConfig().enableObjectReuse();
DataSet<Tuple2<String, Integer>> input =
env.readCsvFile(inReducePath).types(String.class,
Integer.class).setParallelism(1);
- DataSet<Tuple2<String, Integer>> result = input
- .reduce(new
ReduceFunction<Tuple2<String, Integer>>() {
+ DataSet<Tuple2<String, Integer>> result =
input.reduce(new ReduceFunction<Tuple2<String, Integer>>() {
@Override
public Tuple2<String,
Integer> reduce(
Tuple2<String, Integer> value1,
Tuple2<String, Integer> value2) throws Exception {
- value2.f1 +=
value1.f1;
- return value2;
+ if (value1.f1 %
2 == 0) {
--- End diff --
How about changing this to `% 3`, so that it does not "accidentally" work
by perfectly following the rotation of the "reuse1" and "reuse2" variables in
the driver?
> The first program in ObjectReuseITCase has the wrong expected result, and it
> succeeds
> -------------------------------------------------------------------------------------
>
> Key: FLINK-3166
> URL: https://issues.apache.org/jira/browse/FLINK-3166
> Project: Flink
> Issue Type: Bug
> Components: Distributed Runtime, Documentation, Tests
> Reporter: Gabor Gevay
> Assignee: Greg Hogan
> Priority: Critical
>
> The first program in ObjectReuseITCase has the following input:
> a,1
> a,2
> a,3
> a,4
> a,50
> There is a groupBy on field 0, and then a reduce, so the result should be
> 1+2+3+4+50 = 60. But the hardcoded expected result is 100, and running the
> Flink program also produces this.
> The problem is caused my mismatched assumptions between
> ReduceCombineDriver.sortAndCombine and the ReduceFunction in the test about
> object reuse rules of ReduceFunctions:
> ReduceCombineDriver.sortAndCombine has the following comment:
> "The user function is expected to return the first input as the result."
> While the ReduceFunction in the test is modifying and returning the second
> input. (And the second program in the test also has the same problem.)
> I can't find the assumption that is stated in the comment in any
> documentation. For example, the javadoc of ReduceFunction should make the
> user aware of this. Or, alternatively, the code of the driver should be
> modified to not make this assumption. I'm not sure which solution is better.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
