void-ptr974 commented on PR #4732: URL: https://github.com/apache/bookkeeper/pull/4732#issuecomment-4641878881
I added a JMH benchmark branch to make the cache-size tradeoff easier to evaluate: https://github.com/void-ptr974/bookkeeper/tree/pr4732-last-entry-cache-benchmark The benchmark models the mixed workload this PR is targeting: - 10,000 ledgers - 128 entries per ledger - 12 reader threads calling `getLastEntryInLedger` - 4 writer threads calling `addLocation` - different `dbStorage_lastEntryCacheMaxSize` values - JMH `readWrite` group, with `-prof gc` The numbers below were collected on the same machine with non-forked JMH runs, so they should be read as relative trend data. | cache max size | read latency | allocation | |---:|---:|---:| | 10000 | ~0.072 us/op | ~0 B/op | | 9500 | ~8.819 us/op | ~38.1 B/op | | 9000 | ~10.417 us/op | ~47.0 B/op | | 8000 | ~13.562 us/op | ~60.1 B/op | | 7000 | ~16.220 us/op | ~69.8 B/op | | 5000 | ~19.316 us/op | ~86.7 B/op | | 3000 | ~22.244 us/op | ~100.9 B/op | | 1000 | ~25.065 us/op | ~113.9 B/op | | 256 | ~25.761 us/op | ~118.5 B/op | | 32 | ~26.158 us/op | ~119.9 B/op | The cache is very effective when it can cover the active ledger working set. Once the configured cache size is materially smaller than the active ledger count, the benefit drops quickly and the workload approaches the RocksDB lookup path again. This suggests the setting is useful, but the default and documentation should make it clear that the value needs to be sized close to the expected active ledger working set to get the full benefit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
