imbajin commented on issue #3043:
URL: https://github.com/apache/hugegraph/issues/3043#issuecomment-4583123376
Thanks for opening this. I think this is a valid follow-up to #3025, not
something already fixed by it.
#3025 intentionally kept runtime behavior unchanged and focused on Docker
build/cache hygiene. This issue is about a different layer: Docker runtime
lifecycle and process supervision.
## Core Context
The main problem is that the Java process lifecycle is currently decoupled
from the container lifecycle.
At the moment, the Docker entrypoints start the HugeGraph Java process in
the background, then keep the container alive with `tail -f /dev/null`.
That means when Java dies, Docker still sees the container as running, so
`restart: unless-stopped` does not trigger.
```text
Current behavior
┌──────────────────────────────┐
│ dumb-init / docker-entrypoint │
└──────────────┬───────────────┘
│
▼
start-*.sh starts Java &
│
▼
tail -f /dev/null keeps running
│
▼
Java crashes or exits
│
▼
container still appears Up
│
▼
Docker restart policy is not triggered
│
▼
users get connection refused
```
Expected Docker-native behavior should be closer to:
```text
Expected behavior
┌──────────────────────────────┐
│ dumb-init / docker-entrypoint │
└──────────────┬───────────────┘
│
▼
supervise actual Java process
│
▼
Java crashes or exits
│
▼
entrypoint exits
│
▼
container exits
│
▼
Docker restart policy restarts it
```
## Conclusion
I agree this should be treated as a real Docker runtime lifecycle bug.
The key points are:
- `tail -f /dev/null` hides Java process failure from Docker.
- Docker restart policies only trigger when the container exits.
- A Compose/Docker `healthcheck` helps observability, but ordinary Docker
does not restart a container just because it is unhealthy.
- The legacy cron monitor is not a good Docker supervision model unless cron
is actually started, and even then it is less aligned with Docker-native
lifecycle handling.
- PD and Store need a real foreground mode.
- Server needs extra care because the hstore path has post-start checks such
as `wait-partition.sh`; it likely needs the pattern: start in background, run
post-start checks, then wait on the actual Java PID.
## Suggested Fix Direction
A clean fix should probably be split like this:
1. Fix startup script foreground semantics
- Keep daemon mode as the default for existing callers.
- Fix `start-hugegraph.sh -d false` so it propagates the Java exit code
correctly.
- Add foreground mode to `start-hugegraph-pd.sh`.
- Add foreground mode to `start-hugegraph-store.sh`.
2. Fix Docker entrypoints
- PD / Store: run the Java process in foreground mode and remove `tail -f
/dev/null`.
- Server: preserve the current post-start hstore checks, then wait on the
real HugeGraphServer PID instead of tailing forever.
3. Document the lifecycle model
- `HEALTHCHECK` reports health.
- `restart: unless-stopped` restarts only after container exit.
- Docker deployments should rely on container lifecycle, not a hidden
background Java process plus `tail`.
## Validation
The fix should include a runtime restart validation, not only a Docker build
check.
Example validation flow:
```bash
docker build -f hugegraph-server/Dockerfile -t hugegraph/server:test .
HUGEGRAPH_VERSION=test docker compose -f docker/docker-compose.yml up -d
# Kill the Java process, not the container or entrypoint.
docker exec hg-server pkill -f HugeGraphServer
# The container should exit and be restarted by Docker policy.
docker inspect hg-server \
--format "{{.RestartCount}} {{.State.Status}} {{if
.State.Health}}{{.State.Health.Status}}{{end}}"
# Service should eventually recover.
curl -fsS http://localhost:8080/versions
```
Expected result after the fix:
- The Java process death causes the entrypoint to exit.
- The container exits instead of staying `Up` with a dead service.
- `RestartCount` increases.
- `/versions` becomes reachable again after restart.
The same lifecycle check should be covered for Server, PD, and Store. For
Server with hstore, we should also verify that the existing post-start
partition wait logic is not skipped.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
