dataroaring opened a new pull request, #61332: URL: https://github.com/apache/doris/pull/61332
## Summary Add two thread-safe primitive-key concurrent hash maps built on fastutil, designed as drop-in replacements for `ConcurrentHashMap<Long, V>` and `ConcurrentHashMap<Long, Long>` in memory-sensitive FE paths. - **`ConcurrentLong2ObjectHashMap<V>`** — replaces `ConcurrentHashMap<Long, V>` - **`ConcurrentLong2LongHashMap`** — replaces `ConcurrentHashMap<Long, Long>` ### Why `ConcurrentHashMap<Long, V>` costs ~64 bytes per entry due to Long boxing, Node wrapper, and segment overhead. These fastutil-based maps reduce that to ~16 bytes per entry — a **4x memory reduction**. In Doris FE, several critical data structures use `ConcurrentHashMap<Long, V>` at tablet/partition scale (millions of entries), making this a significant memory optimization opportunity. ### Design - **Segment-based locking** (default 16 segments) for concurrent throughput, similar to Java 7's ConcurrentHashMap design - Full `Map` interface compatibility for drop-in replacement - Atomic operations: `putIfAbsent`, `computeIfAbsent`, `replace`, `remove(key, value)` - Thread-safe iteration via snapshot-based `entrySet()`/`keySet()`/`values()` ### Memory comparison | Collection | Per-entry overhead | 1M entries | |------------|-------------------|------------| | `ConcurrentHashMap<Long, V>` | ~64 bytes | ~61 MB | | `ConcurrentLong2ObjectHashMap<V>` | ~16 bytes | ~15 MB | | `ConcurrentHashMap<Long, Long>` | ~80 bytes | ~76 MB | | `ConcurrentLong2LongHashMap` | ~16 bytes | ~15 MB | ## Test plan - [x] `ConcurrentLong2ObjectHashMapTest` — 432 lines covering put/get/remove, putIfAbsent, computeIfAbsent, replace, concurrent writes from multiple threads, iteration consistency, empty map edge cases - [x] `ConcurrentLong2LongHashMapTest` — 455 lines covering CRUD, default value semantics, concurrent operations, atomic operations, iteration, edge cases 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
