+user <[email protected]> On Mon, Sep 21, 2020 at 9:16 AM Luke Cwik <[email protected]> wrote:
> You need the "sources" to stop and advance the watermark to infinity and > have that propagate through the entire pipeline. There are propoosals for > pipeline drain[1] and also for snapshot and update[2] for Apache Beam. We > would love contributions in this space. > > Max shared some more details about how Flink users typically do this[3], > does that apply to Spark? > > 1: > https://docs.google.com/document/d/1NExwHlj-2q2WUGhSO4jTu8XGhDPmm3cllSN8IMmWci8 > 2: > https://docs.google.com/document/d/1UWhnYPgui0gUYOsuGcCjLuoOUlGA4QaY91n8p3wz9MY > 3: > https://lists.apache.org/thread.html/864eb7b4e7192706074059eef1e116146382552fa885dd6054ef4988%40%3Cuser.beam.apache.org%3E > > On Mon, Sep 21, 2020 at 7:43 AM Sunny, Mani Kolbe <[email protected]> wrote: > >> Forgot to mention, we are using spark runner. >> >> >> >> *From:* Sunny, Mani Kolbe <[email protected]> >> *Sent:* Monday, September 21, 2020 12:33 PM >> *To:* [email protected] >> *Subject:* How to gracefully stop a beam application >> >> >> >> *CAUTION:* This email originated from outside of D&B. Please do not >> click links or open attachments unless you recognize the sender and know >> the content is safe. >> >> >> >> Hello Beam community, >> >> >> >> When you are running a Beam application in full stream mode, it is >> continuously running. What is the recommended way to stop it gracefully for >> say maintenance/upgrades etc? When I say gracefully, I mean (1) without >> data loss and (2) application existing with exit 0 code. >> >> >> >> Regards, >> >> Mani >> >
