[
https://issues.apache.org/jira/browse/SPARK-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen updated SPARK-5795:
-----------------------------
Priority: Minor (was: Major)
When you say "doesn't compile", you should show the compilation error. Although
I think I know what it is. There's a workaround but I agree we can look at
fixing it. If it breaks binary compatibility, it would have to wait until later.
> api.java.JavaPairDStream.saveAsNewAPIHadoopFiles may not friendly to java
> -------------------------------------------------------------------------
>
> Key: SPARK-5795
> URL: https://issues.apache.org/jira/browse/SPARK-5795
> Project: Spark
> Issue Type: Bug
> Components: Streaming
> Affects Versions: 1.2.1
> Reporter: Littlestar
> Priority: Minor
>
> import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat;
> the following code can't compile on java.
> JavaPairDStream<Integer, Integer> rs =....
> rs.saveAsNewAPIHadoopFiles("prefix", "txt", Integer.class, Integer.class,
> TextOutputFormat.class, jobConf);
> but similar code in JavaPairRDD works ok.
> JavaPairRDD<String, String> counts =.......
> counts.saveAsNewAPIHadoopFile("out", Text.class, Text.class,
> TextOutputFormat.class, jobConf);
> ====================
> mybe the
> def saveAsNewAPIHadoopFiles(
> prefix: String,
> suffix: String,
> keyClass: Class[_],
> valueClass: Class[_],
> outputFormatClass: Class[_ <: NewOutputFormat[_, _]],
> conf: Configuration = new Configuration) {
> dstream.saveAsNewAPIHadoopFiles(prefix, suffix, keyClass, valueClass,
> outputFormatClass, conf)
> }
> =====>
> def saveAsNewAPIHadoopFiles[F <: NewOutputFormat[_, _]](
> prefix: String,
> suffix: String,
> keyClass: Class[_],
> valueClass: Class[_],
> outputFormatClass: Class[F],
> conf: Configuration = new Configuration) {
> dstream.saveAsNewAPIHadoopFiles(prefix, suffix, keyClass, valueClass,
> outputFormatClass, conf)
> }
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
