Checkpointing is only used for failure recovery not for app upgrades. You need to manually code the unload/load and save it to a persistent store
Le mar. 18 déc. 2018 à 17:29, Priya Matpadi <pmatp...@gmail.com> a écrit : > Using checkpointing for graceful updates is my understanding as well, > based on the writeup in > https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#recovering-from-failures-with-checkpointing, > and some prototyping. Have you faced any missed events? > > On Mon, Dec 17, 2018 at 6:56 PM Yuta Morisawa < > yu-moris...@kddi-research.jp> wrote: > >> Hi >> >> Now I'm trying to update my structured streaming application. >> But I have no idea how to update it gracefully. >> >> Should I stop it, replace a jar file then restart it? >> In my understanding, in that case, all the state will be recovered if I >> use checkpoints. >> Is this correct? >> >> Thank you, >> >> >> -- >> >> >> --------------------------------------------------------------------- >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >> >>
