Yes, all checkpoints are forward compatible.
However, you do need to restart the query if you want to update the code of
the query. This downtime can be in less than a second (if you just restart
the query without stopping the application/Spark driver) or 10s of seconds
(if you have to stop the application and resubmit your application to the
cluster).
On Thu, Mar 8, 2018 at 12:11 PM, Georg Heiler <georg.kf.hei...@gmail.com>
wrote:
> Hi
>
> What is the state of spark structured streaming jobs and upgrades?
>
> Can checkpoints of version 1 be read by version 2 of a job? Is downtime
> required to upgrade the job?
>
> Thanks
>
