First you create the file:
final File outputFile = new File(outputPath);
Then you write to it:
Files.append(counts + \n, outputFile, Charset.defaultCharset());
Cheers
On Fri, Aug 14, 2015 at 4:38 PM, Mohit Anchlia mohitanch...@gmail.com
wrote:
I thought prefix meant the output
Please take a look at JavaPairDStream.scala:
def saveAsHadoopFiles[F : OutputFormat[_, _]](
prefix: String,
suffix: String,
keyClass: Class[_],
valueClass: Class[_],
outputFormatClass: Class[F]) {
Did you intend to use outputPath as prefix ?
Cheers
On Fri, Aug
Spark 1.3
Code:
wordCounts.foreachRDD(*new* *Function2JavaPairRDDString, Integer, Time,
Void()* {
@Override
*public* Void call(JavaPairRDDString, Integer rdd, Time time) *throws*
IOException {
String counts = Counts at time + time + + rdd.collect();
System.*out*.println(counts);
I thought prefix meant the output path? What's the purpose of prefix and
where do I specify the path if not in prefix?
On Fri, Aug 14, 2015 at 4:36 PM, Ted Yu yuzhih...@gmail.com wrote:
Please take a look at JavaPairDStream.scala:
def saveAsHadoopFiles[F : OutputFormat[_, _]](
prefix:
Which Spark release are you using ?
Can you show us snippet of your code ?
Have you checked namenode log ?
Thanks
On Aug 13, 2015, at 10:21 PM, Mohit Anchlia mohitanch...@gmail.com wrote:
I was able to get this working by using an alternative method however I only
see 0 bytes files in
I have this call trying to save to hdfs 2.6
wordCounts.saveAsNewAPIHadoopFiles(prefix, txt);
but I am getting the following:
java.lang.RuntimeException: class scala.runtime.Nothing$ not
org.apache.hadoop.mapreduce.OutputFormat
I was able to get this working by using an alternative method however I
only see 0 bytes files in hadoop. I've verified that the output does exist
in the logs however it's missing from hdfs.
On Thu, Aug 13, 2015 at 10:49 AM, Mohit Anchlia mohitanch...@gmail.com
wrote:
I have this call trying to
