In a continuous processing pipeline with dataframes is there any way to
checkpoint the processing state (by the user) at periodic intervals. The
thought process behind this is to rewind to any particular checkpoint and then
fast forward processing thereon.
Date: Wed, 29 Jun 2016 23:17:47 -0700
F
Is there a way to checkpoint sink(s) to facilitate rewind processing from a
specific offset.
For example a continuous query aggregated by month.
On the 10 month would like to re-compute information between 4th to 8th
months.
--
View this message in context:
http://apache-spark-user-list.10015