[
https://issues.apache.org/jira/browse/SPARK-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-5795.
------------------------------
Resolution: Fixed
Fix Version/s: 1.3.0
Issue resolved by pull request 4608
[https://github.com/apache/spark/pull/4608]
> api.java.JavaPairDStream.saveAsNewAPIHadoopFiles may not friendly to java
> -------------------------------------------------------------------------
>
> Key: SPARK-5795
> URL: https://issues.apache.org/jira/browse/SPARK-5795
> Project: Spark
> Issue Type: Bug
> Components: Streaming
> Affects Versions: 1.2.1
> Reporter: Littlestar
> Priority: Critical
> Fix For: 1.3.0
>
> Attachments: TestStreamCompile.java
>
>
> import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat;
> the following code can't compile on java.
> JavaPairDStream<Integer, Integer> rs =....
> rs.saveAsNewAPIHadoopFiles("prefix", "txt", Integer.class, Integer.class,
> TextOutputFormat.class, jobConf);
> but similar code in JavaPairRDD works ok.
> JavaPairRDD<String, String> counts =.......
> counts.saveAsNewAPIHadoopFile("out", Text.class, Text.class,
> TextOutputFormat.class, jobConf);
> ====================
> mybe the
> def saveAsNewAPIHadoopFiles(
> prefix: String,
> suffix: String,
> keyClass: Class[_],
> valueClass: Class[_],
> outputFormatClass: Class[_ <: NewOutputFormat[_, _]],
> conf: Configuration = new Configuration) {
> dstream.saveAsNewAPIHadoopFiles(prefix, suffix, keyClass, valueClass,
> outputFormatClass, conf)
> }
> =====>
> def saveAsNewAPIHadoopFiles[F <: NewOutputFormat[_, _]](
> prefix: String,
> suffix: String,
> keyClass: Class[_],
> valueClass: Class[_],
> outputFormatClass: Class[F],
> conf: Configuration = new Configuration) {
> dstream.saveAsNewAPIHadoopFiles(prefix, suffix, keyClass, valueClass,
> outputFormatClass, conf)
> }
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
