I totally missed this new feature. Thanks for the pointer Le mar. 18 déc. 2018 à 21:18, Priya Matpadi <[email protected]> a écrit :
> Changes in streaming query that allow or disallow recovery from checkpoint > is clearly provided in > https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#recovery-semantics-after-changes-in-a-streaming-query > . > > On Tue, Dec 18, 2018 at 9:45 AM vincent gromakowski < > [email protected]> wrote: > >> Checkpointing is only used for failure recovery not for app upgrades. You >> need to manually code the unload/load and save it to a persistent store >> >> Le mar. 18 déc. 2018 à 17:29, Priya Matpadi <[email protected]> a >> écrit : >> >>> Using checkpointing for graceful updates is my understanding as well, >>> based on the writeup in >>> https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#recovering-from-failures-with-checkpointing, >>> and some prototyping. Have you faced any missed events? >>> >>> On Mon, Dec 17, 2018 at 6:56 PM Yuta Morisawa < >>> [email protected]> wrote: >>> >>>> Hi >>>> >>>> Now I'm trying to update my structured streaming application. >>>> But I have no idea how to update it gracefully. >>>> >>>> Should I stop it, replace a jar file then restart it? >>>> In my understanding, in that case, all the state will be recovered if I >>>> use checkpoints. >>>> Is this correct? >>>> >>>> Thank you, >>>> >>>> >>>> -- >>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe e-mail: [email protected] >>>> >>>>
