Yes, check out mapWithState:https://databricks.com/blog/2016/02/01/faster-stateful-stream-processing-in-apache-spark-streaming.html
_____________________________ From: Nikhil Goyal <nownik...@gmail.com> Sent: Monday, May 23, 2016 23:28 Subject: Timed aggregation in Spark To: <user@spark.apache.org> Hi all, I want to aggregate my data for 5-10 min and then flush the aggregated data to some database like vertica. updateStateByKey is not exactly helpful in this scenario as I can't flush all the records at once, neither can I clear the state. I wanted to know if anyone else has faced a similar issue and how did they handle it. ThanksNikhil