[
https://issues.apache.org/jira/browse/FLINK-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15140937#comment-15140937
]
ASF GitHub Bot commented on FLINK-2237:
---------------------------------------
Github user ggevay commented on a diff in the pull request:
https://github.com/apache/flink/pull/1517#discussion_r52468223
--- Diff:
flink-tests/src/test/java/org/apache/flink/test/manual/ReducePerformance.java
---
@@ -0,0 +1,191 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.test.manual;
+
+import org.apache.flink.api.common.functions.ReduceFunction;
+import org.apache.flink.api.common.operators.base.ReduceOperatorBase;
+import org.apache.flink.api.java.DataSet;
+import org.apache.flink.api.java.ExecutionEnvironment;
+import org.apache.flink.api.java.tuple.Tuple2;
+import org.apache.flink.api.java.typeutils.TupleTypeInfo;
+import org.apache.flink.util.SplittableIterator;
+
+import java.io.Serializable;
+import java.util.Iterator;
+import java.util.Random;
+
+public class ReducePerformance {
+
+ public static void main(String[] args) throws Exception {
+
+ ExecutionEnvironment env =
ExecutionEnvironment.getExecutionEnvironment();
+ //env.getConfig().enableObjectReuse();
+ //env.setParallelism(1);
+
+ @SuppressWarnings("unchecked")
+ DataSet<Tuple2<Integer, Integer>> output =
+ env.fromParallelCollection(new
SplittableRandomIterator(40 * 1000 * 1000, new TupleIntIntIterator(4 * 1000 *
1000)),
+ TupleTypeInfo.<Tuple2<Integer,
Integer>>getBasicTupleTypeInfo(Integer.class, Integer.class))
+ .groupBy("0")
+ .reduce(new SumReducer(),
ReduceOperatorBase.ReduceHint.HASH);
+
+// DataSet<Tuple2<Integer, Integer>> output =
--- End diff --
OK, sorry, I'll split the two versions to different methods.
The purpose here was to comment this in and comment out the similar part
above when someone wants to run the test with `TupleStringIntIterator` instead
of `TupleIntIntIterator`, but I realize now that this is not a good way to do
this.
> Add hash-based Aggregation
> --------------------------
>
> Key: FLINK-2237
> URL: https://issues.apache.org/jira/browse/FLINK-2237
> Project: Flink
> Issue Type: New Feature
> Reporter: Rafiullah Momand
> Assignee: Gabor Gevay
> Priority: Minor
>
> Aggregation functions at the moment are implemented in a sort-based way.
> How can we implement hash based Aggregation for Flink?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
