Dale Richardson created YUNIKORN-3425:
-----------------------------------------
Summary: Admission controller serving goroutine reads the server
field Shutdown nils
Key: YUNIKORN-3425
URL: https://issues.apache.org/jira/browse/YUNIKORN-3425
Project: Apache YuniKorn
Issue Type: Bug
Components: shim - kubernetes
Reporter: Dale Richardson
{{WebHook.Startup}} assigns {{wh.server}} under the mutex and then starts a
goroutine that calls {{wh.server.ListenAndServeTLS}} after the lock is
released. {{WebHook.Shutdown}} sets the field to {{nil}} under the same mutex.
The reload loop in {{main}} runs {{Shutdown()}} then {{Startup()}} on
{{SIGUSR1}} and on the certificate-expiry timer, so every rotation exercises
the sequence.
Reproduced on the fork fix branch: the pre-fix test produces both the data race
report and the nil-receiver SIGSEGV ({{ListenAndServeTLS(0x0, ...)}}), and the
controller's own sequential reload cycle trips the detector without any timing
luck. A crash here takes admission down until the pod restarts. In the same
critical section {{Shutdown}} holds the mutex across
{{http.Server.Shutdown(context.Background())}}, draining in-flight admissions
with no timeout while the reload's {{Startup}} waits for the lock.
Fix: capture the server in a local under the lock at both ends so neither
goroutine touches the field outside the critical section; keep the drain
synchronous so the port is free before rebinding. A drain timeout is a separate
change. A fix exists on the tigerquoll fork (PR #10 there) and will be filed
once this has a number.
Marker: the serving goroutine in {{cmd/admissioncontroller/main.go}} carries
this JIRA; the fix removes it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
