Dale Richardson created YUNIKORN-3417:
-----------------------------------------
Summary: RM event replies are unbuffered sends made under the
ClusterContext write lock
Key: YUNIKORN-3417
URL: https://issues.apache.org/jira/browse/YUNIKORN-3417
Project: Apache YuniKorn
Issue Type: Bug
Components: core - scheduler
Reporter: Dale Richardson
The three infrastructure event handlers in {{context.go}}
({{processRMRegistrationEvent}}, {{processRMConfigUpdateEvent}},
{{removePartitionsByRMID}}) reply to the RM proxy with {{event.Channel <-
&rmevent.Result{...}}} while holding {{cc.Lock()}}. The channel is unbuffered
and nothing guarantees the reply is collected. If the goroutine that posted the
event is gone, or the reply is sent on a channel nobody reads, the send blocks
for ever with the cluster context write lock held, and the scheduling cycle,
the REST handlers and the health check stall with it.
Today every producer parks unconditionally on the reply, so this needs the
pairing to fail first; nothing reproduces it and no test exercises a dropped
reply. It is the lock-side view of the plumbing in YUNIKORN-3365 (reply never
drained on shutdown): there a parked goroutine is harmless, here a failed
rendezvous is cluster-wide. Filed so the ten suppressed sends have a home
rather than because it has bitten anyone.
Fix: compute the result under the lock, release it, then send; or give the
reply channels a buffer of one so the send cannot block.
Marker: one commented site plus nine sibling sends in the same three handlers
carry {{+lockblockingignore}} for this JIRA; the fix removes them.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
