tjololo opened a new issue, #2768:
URL: https://github.com/apache/activemq/issues/2768
**Affected:** `apache/activemq` images 6.2.1+ / 5.19.2+ (since AMQ-9588, PR
#1665), including 6.3.3, 6.4.0 and current `main`.
**File:** `assembly/src/docker/entrypoint.sh`
#### Problem
The entrypoint runs the broker in the background, traps TERM/INT and waits:
```bash
trap _term TERM INT
"$@" &
child_pid=$!
wait "${child_pid}"
exit $?
```
When bash receives a signal it has a trap for while it is in `wait`, `wait`
returns immediately with a status above 128, and then the trap runs (see the
bash manual, "Signals"). `_term` forwards SIGTERM to the broker, and the script
carries on to `exit $?` straight away. Because the script is PID 1, the
container stops when it exits, and the runtime SIGKILLs the broker. This
happens while the broker's shutdown hook is still stopping transports and
closing KahaDB.
The result is that every `docker stop` or Kubernetes pod termination ends in
an unclean shutdown, whatever grace period is configured. On the next start,
KahaDB has to recover the journal, which gets slower as the store grows.
#### Steps to reproduce
```sh
docker run -d --name amq apache/activemq:6.3.3
sleep 25 # let the broker start
time docker stop -t 60 amq
docker logs amq 2>&1 | tail -n 15
docker rm amq
```
**Expected:** after "Received signal, stopping ActiveMQ...", the logs show
the broker's shutdown sequence, ending with "Apache ActiveMQ … is shutdown",
and the container exits once that is done.
**Actual:** the logs stop at or right after "Received signal, stopping
ActiveMQ...", with no completed shutdown sequence, and the container exits
immediately with status 143.
```sh
INFO | Starting KahaDBStore
INFO | Opening MessageDatabase
INFO | Starting Temp Data Store
INFO | PListStore:[/opt/apache-activemq/data/localhost/tmp_storage] started
INFO | Starting Job Scheduler Store
INFO | Persistence Adapter successfully started
INFO | Apache ActiveMQ 6.3.3 (localhost,
ID:370510bedabb-33937-1793196812484-0:1) is starting
INFO | Listening for connections at:
tcp://370510bedabb:61616?maximumConnections=1000&wireFormat.maxFrameSize=10485760
INFO | Connector openwire started
INFO | Apache ActiveMQ 6.3.3 (localhost,
ID:370510bedabb-33937-1793196812484-0:1) started
INFO | For help or more information please see: http://activemq.apache.org
WARN | Store limit is 102400 mb (current store usage is 0 mb). The data
directory: /opt/apache-activemq/data/kahadb only has 90197 mb of usable space.
- resetting to maximum available disk space: 90197 mb
INFO | Jetty Server listening on: http://0.0.0.0:8161/
Received signal, stopping ActiveMQ...
```
For comparison, running the same image without the entrypoint produces the
full clean shutdown sequence, because `activemq console` execs the JVM:
```sh
docker run -d --name amq-direct --entrypoint activemq apache/activemq:6.3.3
console
sleep 25
docker stop -t 60 amq-direct
docker logs amq-direct 2>&1 | tail -n 15
docker rm amq-direct
```
#### Suggested fix
Wait again after the trap has fired, so PID 1 stays alive until the broker
exits:
```bash
"$@" &
child_pid=$!
wait "${child_pid}"
trap - TERM INT
wait "${child_pid}"
exit $?
```
Another option is to go back to `exec "$@"` at the end of the script.
`bin/activemq console` already execs the JVM, so the JVM would become PID 1 and
handle SIGTERM through its own shutdown hook. The non-root changes from
AMQ-9588 (`USER activemq`, `chown`) don't depend on the background/trap pattern.
#### Workaround
Bypass the entrypoint, e.g. in Kubernetes:
```yaml
command: ["activemq", "console"]
```
This also skips the entrypoint's `ACTIVEMQ_OPTS` defaults, including
`-Djetty.host=0.0.0.0`, and its `ACTIVEMQ_*_USER` / `ACTIVEMQ_*_PASSWORD`
credential handling.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
For further information, visit: https://activemq.apache.org/contact
