First you create the file:
final File outputFile = new File(outputPath);
Then you write to it:
Files.append(counts + "\n", outputFile, Charset.defaultCharset());
Cheers
On Fri, Aug 14, 2015 at 4:38 PM, Mohit Anchlia
wrote:
> I thought prefix meant the output path? What's the purpo
I thought prefix meant the output path? What's the purpose of prefix and
where do I specify the path if not in prefix?
On Fri, Aug 14, 2015 at 4:36 PM, Ted Yu wrote:
> Please take a look at JavaPairDStream.scala:
> def saveAsHadoopFiles[F <: OutputFormat[_, _]](
> prefix: String,
>
Please take a look at JavaPairDStream.scala:
def saveAsHadoopFiles[F <: OutputFormat[_, _]](
prefix: String,
suffix: String,
keyClass: Class[_],
valueClass: Class[_],
outputFormatClass: Class[F]) {
Did you intend to use outputPath as prefix ?
Cheers
On Fri, Aug 14
Spark 1.3
Code:
wordCounts.foreachRDD(*new* *Function2, Time,
Void>()* {
@Override
*public* Void call(JavaPairRDD rdd, Time time) *throws*
IOException {
String counts = "Counts at time " + time + " " + rdd.collect();
System.*out*.println(counts);
System.*out*.println("Appending to " + output
Which Spark release are you using ?
Can you show us snippet of your code ?
Have you checked namenode log ?
Thanks
> On Aug 13, 2015, at 10:21 PM, Mohit Anchlia wrote:
>
> I was able to get this working by using an alternative method however I only
> see 0 bytes files in hadoop. I've verifi
I was able to get this working by using an alternative method however I
only see 0 bytes files in hadoop. I've verified that the output does exist
in the logs however it's missing from hdfs.
On Thu, Aug 13, 2015 at 10:49 AM, Mohit Anchlia
wrote:
> I have this call trying to save to hdfs 2.6
>
>
I have this call trying to save to hdfs 2.6
wordCounts.saveAsNewAPIHadoopFiles("prefix", "txt");
but I am getting the following:
java.lang.RuntimeException: class scala.runtime.Nothing$ not
org.apache.hadoop.mapreduce.OutputFormat
