imbajin commented on code in PR #3258: URL: https://github.com/apache/hugegraph/pull/3258#discussion_r4213597666
########## docker/README.md: ########## @@ -230,6 +230,22 @@ Standalone stores RocksDB data at `/hugegraph-server/rocksdb-data`. The HStore t `docker compose down` keeps named-volume data. `docker compose down -v` intentionally deletes it. +### Server logs and crash files + +This section describes images built from current master. Images from 1.7.0 and earlier do not set `STDOUT_MODE`: `docker logs` shows only the entrypoint's output, and the service logs stay in `logs/` inside the container. + +Both Server images (`hugegraph/hugegraph` and `hugegraph/server`) write the HugeGraph log to container stdout. WARN and above from Hadoop, ZooKeeper, SOFA, Netty and Commons also reaches stdout. INFO from Hadoop, Netty and Commons, the audit log and the slow-query log stay in files only; ZooKeeper and SOFA log nothing below WARN. Launcher errors that stop the Server before Java starts, such as an unsupported JDK or too little free memory, and fatal bootstrap errors go to stderr as well as `hugegraph-server.log`, so when a Server exits during startup, `docker logs` or `kubectl logs --previous` shows why. Every start also prints `Picked up JAVA_TOOL_OPTIONS: ...` with the crash-file defaults below. + +`/hugegraph-server/logs` holds `hugegraph-server.log`, JVM crash logs (`hs_err_pid<pid>_<host>_<launch time>.log`) and out-of-memory heap dumps (`heapdump_<host>_<launch time>/java_pid<pid>.hprof`), where `<launch time>` is the container's local time as `YYYYMMDD-HHMMSS`. Each start gets its own `heapdump_*` directory, so the Server and the JVMs it starts, such as computer jobs, write separate dumps. The host name (the pod name on Kubernetes, unless the pod uses `hostNetwork`) and a `-1`, `-2`, ... counter keep names from clashing. If the directory cannot be created, for example on a full volume, the Server still starts and dumps go to `logs/` itself as `java_pid<pid>.hprof`; in that case a restarted container that reuses a PID cannot write over an earlier dump with the same name. Review Comment: 🧹 Blocking: no. Summary: Kubernetes derives the Pod hostname from metadata.name (or spec.hostname); hostNetwork changes networking and does not make the node hostname the Pod hostname. Please remove this exception or describe the actual hostname override. Evidence: [Pod hostname docs](https://kubernetes.io/docs/concepts/workloads/pods/pod-hostname/) and [kubelet hostname builder](https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/kubelet_pods.go#L3256-L3297). ########## docker/README.md: ########## @@ -230,6 +230,22 @@ Standalone stores RocksDB data at `/hugegraph-server/rocksdb-data`. The HStore t `docker compose down` keeps named-volume data. `docker compose down -v` intentionally deletes it. +### Server logs and crash files + +This section describes images built from current master. Images from 1.7.0 and earlier do not set `STDOUT_MODE`: `docker logs` shows only the entrypoint's output, and the service logs stay in `logs/` inside the container. + +Both Server images (`hugegraph/hugegraph` and `hugegraph/server`) write the HugeGraph log to container stdout. WARN and above from Hadoop, ZooKeeper, SOFA, Netty and Commons also reaches stdout. INFO from Hadoop, Netty and Commons, the audit log and the slow-query log stay in files only; ZooKeeper and SOFA log nothing below WARN. Launcher errors that stop the Server before Java starts, such as an unsupported JDK or too little free memory, and fatal bootstrap errors go to stderr as well as `hugegraph-server.log`, so when a Server exits during startup, `docker logs` or `kubectl logs --previous` shows why. Every start also prints `Picked up JAVA_TOOL_OPTIONS: ...` with the crash-file defaults below. + +`/hugegraph-server/logs` holds `hugegraph-server.log`, JVM crash logs (`hs_err_pid<pid>_<host>_<launch time>.log`) and out-of-memory heap dumps (`heapdump_<host>_<launch time>/java_pid<pid>.hprof`), where `<launch time>` is the container's local time as `YYYYMMDD-HHMMSS`. Each start gets its own `heapdump_*` directory, so the Server and the JVMs it starts, such as computer jobs, write separate dumps. The host name (the pod name on Kubernetes, unless the pod uses `hostNetwork`) and a `-1`, `-2`, ... counter keep names from clashing. If the directory cannot be created, for example on a full volume, the Server still starts and dumps go to `logs/` itself as `java_pid<pid>.hprof`; in that case a restarted container that reuses a PID cannot write over an earlier dump with the same name. Review Comment: 🧹 Blocking: no. Summary: When the logs volume is full, falling back to HeapDumpPath=${LOGS} may still leave no space for the heap dump. Please describe this as a best-effort fallback rather than a guaranteed saved dump. Evidence: On mkdir failure, the launcher sets HEAP_DUMP_PATH to ${LOGS} at hugegraph-server.sh:322-326. ########## docker/README.md: ########## @@ -230,6 +230,22 @@ Standalone stores RocksDB data at `/hugegraph-server/rocksdb-data`. The HStore t `docker compose down` keeps named-volume data. `docker compose down -v` intentionally deletes it. +### Server logs and crash files + +This section describes images built from current master. Images from 1.7.0 and earlier do not set `STDOUT_MODE`: `docker logs` shows only the entrypoint's output, and the service logs stay in `logs/` inside the container. + +Both Server images (`hugegraph/hugegraph` and `hugegraph/server`) write the HugeGraph log to container stdout. WARN and above from Hadoop, ZooKeeper, SOFA, Netty and Commons also reaches stdout. INFO from Hadoop, Netty and Commons, the audit log and the slow-query log stay in files only; ZooKeeper and SOFA log nothing below WARN. Launcher errors that stop the Server before Java starts, such as an unsupported JDK or too little free memory, and fatal bootstrap errors go to stderr as well as `hugegraph-server.log`, so when a Server exits during startup, `docker logs` or `kubectl logs --previous` shows why. Every start also prints `Picked up JAVA_TOOL_OPTIONS: ...` with the crash-file defaults below. + +`/hugegraph-server/logs` holds `hugegraph-server.log`, JVM crash logs (`hs_err_pid<pid>_<host>_<launch time>.log`) and out-of-memory heap dumps (`heapdump_<host>_<launch time>/java_pid<pid>.hprof`), where `<launch time>` is the container's local time as `YYYYMMDD-HHMMSS`. Each start gets its own `heapdump_*` directory, so the Server and the JVMs it starts, such as computer jobs, write separate dumps. The host name (the pod name on Kubernetes, unless the pod uses `hostNetwork`) and a `-1`, `-2`, ... counter keep names from clashing. If the directory cannot be created, for example on a full volume, the Server still starts and dumps go to `logs/` itself as `java_pid<pid>.hprof`; in that case a restarted container that reuses a PID cannot write over an earlier dump with the same name. + +Both images declare `VOLUME /hugegraph-server`, so `docker restart` and a Compose recreate keep the files in that anonymous volume. `docker compose down` leaves the volume behind but the next `up` does not attach it, so the files become hard to reach and keep using disk until `docker compose down -v` deletes them; mount a named volume or a bind mount at `/hugegraph-server/logs` to keep them reachable. Kubernetes starts a new container on every restart and does not turn the image's `VOLUME` into a pod volume, so mount an `emptyDir` or a PersistentVolumeClaim at `/hugegraph-server/logs`. Give each pod its own volume, or a shared PVC with `subPathExpr: $(POD_NAME)`: crash files get distinct names, but `hugegraph-server.log`, the audit log and the slow-query log have fixed names, and pods sharing them would roll them over for each other. The Helm chart in [`helm/`](../helm/hugegraph) does not mount a logs volume yet, so with it these files are lost when a container restarts. Review Comment: ⚠️ Blocking: no. Summary: subPathExpr: $(POD_NAME) isolates replicas only when POD_NAME is provided to the container, but the current Helm Server template defines POD_IP only. Please show the Downward API fieldRef for metadata.name or state that users must add it. Evidence: [Kubernetes documents subPathExpr as using Downward API environment variables](https://kubernetes.io/docs/concepts/storage/volumes/#using-subpath-with-expanded-environment-variables). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
