Yes, all checkpoints are forward compatible. However, you do need to restart the query if you want to update the code of the query. This downtime can be in less than a second (if you just restart the query without stopping the application/Spark driver) or 10s of seconds (if you have to stop the application and resubmit your application to the cluster).
On Thu, Mar 8, 2018 at 12:11 PM, Georg Heiler <georg.kf.hei...@gmail.com> wrote: > Hi > > What is the state of spark structured streaming jobs and upgrades? > > Can checkpoints of version 1 be read by version 2 of a job? Is downtime > required to upgrade the job? > > Thanks >
