roryqi commented on PR #13386: URL: https://github.com/apache/gravitino/pull/13386#issuecomment-5772764923
Could we explicitly list the fault-tolerance models before finalizing the claim design? 1. **Best effort:** a successful commit may permanently lose its maintenance trigger after a failure. 2. **At-least-once latest-state evaluation:** after recovery, the table's latest state is eventually evaluated; multiple commits may be coalesced, and evaluation may repeat. 3. **At-least-once per-commit evaluation:** every successful commit is eventually represented and evaluated; evaluation may repeat. 4. **Exactly-once job effect:** maintenance jobs are neither missed nor submitted twice. Stating these models would clarify what “no missed trigger” means, expose the availability and complexity costs of each guarantee, and give us concrete failure tests. Based on our discussion, best effort does not meet the requirement. We should decide whether coalescing commits is acceptable for each maintenance strategy. Please also describe the recovery behavior at each failure boundary: - Iceberg commit succeeds before the event is durable. - The event is durable before the callback or claim runs. - A node acquires a claim and fails during evaluation. - A Job is accepted before its `job_id` is recorded. - Evaluation is deferred because a Job is running or the minimum interval has not elapsed. For each boundary, which durable state drives recovery? Can evaluation be delayed, repeated, or permanently lost? The per-policy claim addresses concurrent execution, but it does not by itself establish a delivery guarantee. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
