yes it is a streaming dataset. so what is the problem with following code? Dataset<String> ds = dataset.toJSON().map(()->{some function that returns a string}); StreamingQuery query = ds.writeStream().start(); query.awaitTermination();
On Sat, Sep 9, 2017 at 4:20 PM, Felix Cheung <felixcheun...@hotmail.com> wrote: > What is newDS? > If it is a Streaming Dataset/DataFrame (since you have writeStream there) > then there seems to be an issue preventing toJSON to work. > > ------------------------------ > *From:* kant kodali <kanth...@gmail.com> > *Sent:* Saturday, September 9, 2017 4:04:33 PM > *To:* user @spark > *Subject:* Queries with streaming sources must be executed with > writeStream.start() > > Hi All, > > I have the following code and I am not sure what's wrong with it? I > cannot call dataset.toJSON() (which returns a DataSet) ? I am using spark > 2.2.0 so I am wondering if there is any work around? > > Dataset<String> ds = newDS.toJSON().map(()->{some function that returns a > string}); > StreamingQuery query = ds.writeStream().start(); > query.awaitTermination(); > >