Hi,
I have a Streaming app which reads from inputs, does some text
transformation and try to output to a HDFS text file by using
saveAsTextFiles in DSteam object.
But the method saves nothing (not even an empty file), the jobs successfully
run, i.e. no error/warning. When I replace the save-to-file part with
"print()", it prints out contents on the screen. Also tried
"saveAsTextFiles" in SC RDD, works.
Not sure why, did anyone get "saveAsTextFiles" working with DStream?
Here is the line of code I use for output:
actions.saveAsTextFiles("hdfs://nn1:8020/user/ds/actions/test", "test")
I'm using Spark 0.9.0, hadoop2.0.0-cdh4.5.0.
thanks
Robin
-----
-- Robin Li
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/DStream-saveAsTextFiles-saves-nothing-tp1666.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
