Please take a look
at
streaming/src/main/scala/org/apache/spark/streaming/dstream/PairDStreamFunctions.scala
def saveAsHadoopFiles[F <: OutputFormat[K, V]](
prefix: String,
suffix: String
)(implicit fm: ClassTag[F]): Unit = ssc.withScope {
Cheers
On Sat, Jul 4, 2015 at 5:23 PM, ravi tella <ddpis...@gmail.com> wrote:
> Hello,
> How should I write a text file stream DStream to HDFS.
> I tried the the following
> val lines = ssc.textFileStream("hdfs:/user/hadoop/spark/streaming/input/")
> lines.saveAsTextFile("hdfs:/user/hadoop/output1")
>
> val lines = ssc.textFileStream("hdfs:/user/hadoop/spark/streaming/input/")
> lines.saveAsHadoopFiles("hdfs:/user/hadoop/output1","out")
>
> in both cases i received compile errors saying:
> is not a member of org.apache.spark.streaming.dstream.DStream[String]
>
> Thanks in advance for help.
>
