[
https://issues.apache.org/jira/browse/CASSSIDECAR-485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrés Beck-Ruiz updated CASSSIDECAR-485:
-----------------------------------------
Labels: CEP-53 (was: )
> Fail operational job nodes whose owning Sidecar is permanently down
> ----------------------------------------------------------------------
>
> Key: CASSSIDECAR-485
> URL: https://issues.apache.org/jira/browse/CASSSIDECAR-485
> Project: Sidecar for Apache Cassandra
> Issue Type: Improvement
> Reporter: Andrés Beck-Ruiz
> Priority: Major
> Labels: CEP-53
>
> CASSSIDECAR-378 adds a per-node execution timeout so a {{RUNNING}} node
> always reaches a terminal state and cannot wedge an operation. That timeout
> is enforced by the node's co-located Sidecar (only it holds the local job
> handle and drives the node), so it only covers the case where that Sidecar is
> alive. If the owning Sidecar goes down and does not return, nothing enforces
> the bound: a node that never started stays CREATED, and a node that was
> mid-execution stays RUNNING, indefinitely. Because finalization requires
> every node in the execution order to be terminal, the operation never
> finalizes and the datacenter-scoped active operation lock for that operation
> type is never released, wedging all future operations of that type.
> This is not solvable with a timeout, as a peer Sidecar cannot safely assume
> another node's work has stopped just because time has passed (the owner may
> be slow, GC-paused, or network-partitioned rather than dead). Closing the gap
> requires a node-liveness mechanism so that some other participant can
> determine an owning Sidecar is genuinely gone and mark its node FAILED,
> allowing the operation to finalize and release the lock.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
