Hi Nicholas, Keyong, Thanks for taking a look. Keyong, sounds great. With both of you aligned, we'll keep the contract and plugins to own their I/O timeouts.
Nicholas, agreed! Here's my plan. 1. Lifecycle contract. You're right, getReason() runs before the trigger today, so a blocked getReason() can hold back a check that already returned true. I'll trigger first and log the reason after. The CIP-22 now clarifies the contract for all three methods. shouldDecommission(), getReason() and close() must return quickly and put their own timeouts on I/O. close() can overlap a check that ignored the interrupt and outlived the 5s wait, so it has to be thread-safe against that check and must not wait on it. 2. Timing. I hadn't separated entering InDecommission from rejecting writes, and they're a few steps apart. The hook sends ReportWorkerDecommission synchronously and only then sets shutdown. With the defaults (askTimeout 60s, maxRetries 15), a hung master can stretch that to minutes. The CIP now clarifies each step, says the 80s/105s numbers assume a fast check and a responsive master, and adds an unavailable-master test. REST and master-driven decommission share this ordering, so I've filed CELEBORN-2538 to fix it for all triggers rather than inside CIP-22. Does that work? 3. Exit race. Agreed, the first accepted exit should decide both the mode and the timeout. I filed CELEBORN-2537 and linked it from the CIP. Will add regression test (graceful accepted first, a later decommission rejected, timeout unchanged) along with the fix. 4. Validation. I'll add unit tests for a checker that ignores interrupts and for blocked getReason() and close(). And you're right that WorkerSuite can't prove the shutdown path, since it stubs out System.exit. The test plan now has cluster cases with real worker processes: SIGTERM during polling, the hook and file cleanup, and an unavailable master. CIP doc is now updated, and will address the changes in the PR shortly. Regards Venkata krishnan On Sun, Nov 1, 2026 at 11:59 PM Keyong Zhou via dev <[email protected]> wrote: > Hi Venkata, > > Thanks for the proposal! The design looks good to me. For the open question > in the document, I'm also inclined to keep the contract. > > Regards, > Keyong Zhou. > > Venkatakrishnan Sowrirajan <[email protected]> 于2026年11月2日周一 14:56写道: > > > Hi Celeborn Devs, > > > > I would like to propose adding support for a pluggable worker > > self-decommission mechanism based on an external signal which is > especially > > useful in the context of cloud environments with spot instances, K8s > > environments with planned maintenance etc. > > > > Here is the proposal doc/design of CIP-22: > > > > > https://docs.google.com/document/d/1cfNhyz4ABzxFSNpY8ISND1nefKHlLC9X8BNHv7Mki-E/edit?usp=sharing > > > > Please share your review comments or questions in the CIP-22 doc. > > > > Regards > > Venkata krishnan > > >
