imbajin commented on PR #3355:
URL: https://github.com/apache/hugegraph/pull/3355#issuecomment-6422165850
Hi @byteayan (after I check it again, the PR need changes):
Please simplify PR #3355 without weakening the original OOM fix.
Focus on these changes:
1. **Keep the automatic RocksDB memory budget**, including heap, direct
memory, container/cgroup limit, native reserve, and the shared Raft log
`WriteBufferManager`. These protections are necessary.
2. **Reduce the cgroup implementation weight**:
- Move cgroup v1/v2 detection into a small independent helper such as
`CgroupMemoryLimit`.
- Keep `RocksdbMemoryBudget` responsible only for parsing the setting and
calculating the budget.
- Treat hidden parent limits as an observability boundary: use the lowest
visible limit and warn when the hierarchy is incomplete. Do not add more
heuristics to guess invisible limits.
3. **Improve the configuration semantics**:
- Prefer `rocksdb.total_memory_size: auto` in the shipped configuration.
- Keep missing, blank, and legacy `0` as backward-compatible aliases for
`auto`.
- Keep positive integers as explicit byte values.
- Allow `HG_STORE_ROCKSDB_TOTAL_MEMORY_SIZE=auto`.
- Do not add unit parsing such as `4GB` in this PR.
4. **Simplify PD/Spring precedence handling**:
- When a JSON or `-Drocksdb.total_memory_size` override exists, remove
only this key from the PD property source before adding it.
- Let Spring preserve its native precedence instead of creating another
synthetic property source.
5. **Separate unrelated calculations**:
- Move `disruptorBufferSize()` and `writeBufferSize()` out of
`RocksdbMemoryBudget`, since they are Store tuning rules rather than cgroup or
process-memory detection.
6. **Make constants explicit and reusable**:
- Separate the 1 GiB Raft log cache reserve from the 512 MiB other-native
reserve.
- Avoid duplicating the Raft cache size between the budget calculation
and `RaftRocksdbOptions`.
7. **Improve startup diagnostics**:
Log the selected budget and its inputs in readable units, for example:
```text
RocksDB memory budget: 512 MiB
mode=auto
physical-limit=4 GiB
source=cgroup-v2
heap=1 GiB
direct=512 MiB
reserve=1.5 GiB
cgroup-view=complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
