fhueske commented on a change in pull request #8034: [FLINK-11733] Provide
HadoopMapFunction for org.apache.hadoop.mapreduce.Mapper
URL: https://github.com/apache/flink/pull/8034#discussion_r283882266
##########
File path:
flink-connectors/flink-hadoop-compatibility/src/main/java/org/apache/flink/hadoopcompatibility/mapreduce/HadoopMapFunction.java
##########
@@ -42,35 +41,34 @@
* This wrapper maps a Hadoop Mapper (mapreduce API) to a Flink
FlatMapFunction.
*/
@SuppressWarnings("rawtypes")
-@Public
+@PublicEvolving
public class HadoopMapFunction<KEYIN, VALUEIN, KEYOUT, VALUEOUT>
extends RichFlatMapFunction<Tuple2<KEYIN, VALUEIN>, Tuple2<KEYOUT,
VALUEOUT>>
implements ResultTypeQueryable<Tuple2<KEYOUT, VALUEOUT>>, Serializable {
private static final long serialVersionUID = 1L;
- private transient HadoopProxyMapper<KEYIN, VALUEIN, KEYOUT, VALUEOUT>
hadoopProxyMapper;
+ private HadoopProxyMapper<KEYIN, VALUEIN, KEYOUT, VALUEOUT>
hadoopProxyMapper;
+ private HadoopProxyMapper.HadoopDummyMapperContext mapperContext;
+
private transient Mapper<KEYIN, VALUEIN, KEYOUT, VALUEOUT> hadoopMapper;
- private transient Job jobConf;
- private transient HadoopProxyMapper.HadoopDummyMapperContext
mapperContext;
+ private transient org.apache.hadoop.conf.Configuration jobConf;
public HadoopMapFunction(Mapper<KEYIN, VALUEIN, KEYOUT, VALUEOUT>
hadoopMapper) throws IOException {
- this(hadoopMapper, Job.getInstance());
+ this(hadoopMapper, new org.apache.hadoop.conf.Configuration());
}
- public HadoopMapFunction(Mapper<KEYIN, VALUEIN, KEYOUT, VALUEOUT>
hadoopMapper, Job conf) {
+ public HadoopMapFunction(Mapper<KEYIN, VALUEIN, KEYOUT, VALUEOUT>
hadoopMapper, org.apache.hadoop.conf.Configuration conf) {
Review comment:
I think we can still accept a `Job` as parameter (if this is more common in
the Hadoop ecosystem), but we only need to store `Job.getConfiguration()`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
