See https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#recovering-from-failures-with-checkpointing
Though I think that this currently doesn't work with the console sink. On Tue, Aug 15, 2017 at 9:40 AM, purna pradeep <purna2prad...@gmail.com> wrote: > Hi, > >> >> I'm trying to restart a streaming query to refresh cached data frame >> >> Where and how should I restart streaming query >> > > > val sparkSes = SparkSession > > .builder > > .config("spark.master", "local") > > .appName("StreamingCahcePoc") > > .getOrCreate() > > > > import sparkSes.implicits._ > > > > val dataDF = sparkSes.readStream > > .schema(streamSchema) > > .csv("testData") > > > > > > val query = counts.writeStream > > .outputMode("complete") > > .format("console") > > .start() > > > query.awaittermination() > > > >> >> >>