He-Pin opened a new pull request, #3023: URL: https://github.com/apache/pekko/pull/3023
### Motivation The Artery compression system uses a `CountMinSketch` (128KB per connection, 16×1024×8 bytes) for heavy hitter detection in actor ref and class manifest compression tables. Pekko already has `FastFrequencySketch` in `pekko-actor` (used for cluster sharding entity passivation) which is: - **~32x more memory efficient** (~4KB vs ~128KB per inbound connection) - **Includes TinyLFU reset** — periodic halving of all counters provides natural aging, enabling the heavy hitters table to adapt to changing traffic patterns over time - **Already battle-tested** — used in production cluster sharding passivation strategies This addresses [akka/akka-core#31093](https://github.com/akka/akka-core/issues/31093). ### Modification - **`InboundCompressions.scala`**: Replace `CountMinSketch` with `FastFrequencySketch[T]` in `InboundCompression`. The `increment` method now calls `frequencySketch.increment(value)` per occurrence and uses `frequencySketch.frequency(value)` as the estimated weight for heavy hitter detection. - **`TopHeavyHitters.scala`**: - `update()` now always allows weight updates for existing heavy hitters (previously only allowed increases). This is necessary because `FrequencySketch`'s periodic reset halves all counters. - `updateExistingHeavyHitter()` supports both weight increase (push down) and decrease (bubble up) in the heap. - New `fixHeapUp()` method for bidirectional heap property restoration. - `isHeavy()` check now only gates insertion of **new** entries. - **`CountMinSketch.java`**: Removed — was `INTERNAL API` with no external users. - **`CountMinSketchBenchmark.scala`**: Updated to benchmark `FastFrequencySketch` instead. - **Legal files**: Removed CountMinSketch/stream-lib attribution (no longer bundled). ### Result Artery compression uses ~4KB instead of ~128KB per inbound connection. Heavy hitters now adapt to changing traffic patterns via TinyLFU aging rather than tracking monotonically increasing absolute counts. ### Tests - `remote / testOnly ...HeavyHittersSpec`: 13/13 passed (includes 2 new tests for weight decrease and heap upward restoration) - `remote / testOnly ...CompressionTableSpec`: 3/3 passed - `remote / testOnly ...OutboundCompressionSpec`: 2/2 passed ### References Refs akka/akka-core#31093 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
