janhoy opened a new issue, #871:
URL: https://github.com/apache/solr-operator/issues/871
The operator sets a `preStop` lifecycle hook of `solr stop -p <podPort>` on
the Solr container. In the official Solr container image this command can never
succeed, because `bin/solr stop` resolves the process through
`$SOLR_PID_DIR/solr-<port>.pid` and that file does not exist — Solr runs in the
foreground as PID 1 and foreground mode writes no PID file. Every pod deletion
therefore emits a `FailedPreStopHook` warning event.
## Reproduce
Create any SolrCloud, then delete one of its pods (or trigger any rolling
update):
```
kubectl get events --field-selector type=Warning
LAST SEEN TYPE REASON OBJECT MESSAGE
3s Warning FailedPreStopHook pod/example-solrcloud-0
PreStopHook failed
```
Inside a running pod:
```
$ echo $SOLR_PID_DIR
/var/solr
$ ls /var/solr/solr-8983.pid
ls: cannot access '/var/solr/solr-8983.pid': No such file or directory
```
Observed on operator v0.9.1 and v0.10.0-prerelease, with Solr 9.7.0, 9.11.0
and 10.1.0, on Kubernetes 1.36.1. It is not a regression — it appears to have
always been the case.
## Cause
`controllers/util/solr_util.go:442` sets the hook unconditionally:
```go
Command: []string{"solr", "stop", "-p", strconv.Itoa(solrPodPort)},
```
`bin/solr` looks the PID up in `$SOLR_PID_DIR/solr-$THE_PORT.pid` (around
lines 489-490) and exits non-zero when that file is missing. The container
entrypoint starts Solr in the foreground, so nothing ever writes it.
## Impact
Mostly noise, but not nothing. Every pod deletion produces a
`FailedPreStopHook` warning event, which is misleading during incident review
and pollutes alerting that watches for warning events. The hook also suggests
that graceful shutdown is handled by `solr stop` when it is not: shutdown
actually relies on the SIGTERM the kubelet sends to PID 1 afterwards. (I have
not measured whether that SIGTERM path shuts Solr down cleanly in all cases —
it is standard Jetty behaviour, but worth confirming before deciding on the
fix.)
## Suggested fix
Either drop the `preStop` hook and rely on SIGTERM to PID 1, or replace it
with something that works in foreground mode. If the hook is meant to guarantee
an orderly shutdown rather than just stop the process, dropping it silently
would change behaviour, so it is worth deciding which of the two it is intended
to be.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
