DanielLeens commented on issue #12886: URL: https://github.com/apache/seatunnel/issues/12886#issuecomment-6315601585
Thank you for completing the PoC and for stating its failure-model boundary clearly. Classification: D / Zeta Kubernetes Application Mode HA design. The sequential Pod-loss results are useful evidence that the proposed direction can reuse the existing Zeta election, backup, checkpoint, and member-loss paths. In particular, validating active/standby master loss, worker loss, the stable endpoint, status/cancel, cleanup, and the 480-record LocalFile run is a good basis for the design discussion. It is not yet approval to start a production implementation PR. STIP-41's merged v1 contract is a single master `Job`, with application-owned Service/Secret/worker resources tied to that Job; it explicitly excludes master failover and worker replacement. An owner/observer Job plus master/worker StatefulSets changes the platform ownership, terminal-status, cleanup, and native-application-ID contracts, so those parts need a reviewed STIP follow-up before code is claimed. Please add that follow-up under #11857 and make the following decisions and evidence explicit: 1. The exact controller graph and ownership model: which resource is authoritative for the Kubernetes application ID, which resources own the master/worker StatefulSets and Service, and how normal completion, cancellation, and partial startup clean them up without deleting user-owned checkpoints. 2. The active-master handoff contract: how the active endpoint is selected, how status/cancel find the live master after replacement, and how an exiting/replaced master is prevented from running the normal terminal cleanup path. 3. The supported failure model: the guarantee is sequential Pod loss with accessible durable checkpoints. Node/storage outages, loss of quorum, and network-partition/fencing safety remain out of scope until they have a separately specified and tested model; do not describe the PoC as covering them. 4. The configuration contract and validation point for HA replicas and `backup-count`, including clear pre-submission errors for invalid combinations. 5. A reproducible K3s/Kubernetes matrix with manifests, commands, expected status/cancel behavior, checkpoint-restore evidence, record-integrity checks, and cleanup assertions for active-master, standby-master, worker, and owner/controller loss. 6. The LocalFile restore prerequisite as its own linked issue/PR, and PostgreSQL fencing as a separate follow-up, before either is treated as part of this HA implementation. Once that STIP update and matrix are reviewed, we can split the implementation into bounded follow-up PRs. No new implementation claim, assignment, or label change is made in this comment. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
