dajac opened a new pull request, #23710: URL: https://github.com/apache/kafka/pull/23710
`CachedTopicResolver.id` and `CachedTopicResolver.name` pass a lambda capturing the key to `HashMap.computeIfAbsent`, so every lookup allocates one, cache hits included. The JIT does not eliminate it: `HashMap.computeIfAbsent` is above C2's inlining limit, so the lambda escapes. `TopicIds` calls `id` for every topic it iterates and `name` in `contains`, so an assignor walking the subscriptions of a heterogeneous group allocates one lambda per member and subscribed topic. This patch reads the cache with `get` and only looks up the image and calls `put` on a miss. As before, an unknown topic resolves to `null` and is not cached. Bytes allocated per assignment (`gc.alloc.rate.norm`) with heterogeneous subscriptions, before and after. `ConsumerAssignorBenchmark`, 5000 members, 20000 partitions, `FULL`: | Assignor | Subscription | Topics | Before | After | |----------|------------------------|--------|------------|----------| | UNIFORM | HETEROGENEOUS_DISJOINT | 500 | 47.3 MB | 35.5 MB | | UNIFORM | HETEROGENEOUS_NESTED | 500 | 126.2 MB | 90.4 MB | | UNIFORM | HETEROGENEOUS_DISJOINT | 5000 | 422.7 MB | 302.8 MB | | UNIFORM | HETEROGENEOUS_NESTED | 5000 | 1,211.9 MB | 851.9 MB | | RANGE | HETEROGENEOUS_DISJOINT | 500 | 38.0 MB | 14.5 MB | | RANGE | HETEROGENEOUS_NESTED | 500 | 110.1 MB | 38.7 MB | | RANGE | HETEROGENEOUS_DISJOINT | 5000 | 362.6 MB | 123.2 MB | | RANGE | HETEROGENEOUS_NESTED | 5000 | 1,083.0 MB | 363.3 MB | `TargetAssignmentBuilderBenchmark`, 10 partitions per member, `HETEROGENEOUS`: | Members | Topics | Before | After | |---------|--------|----------|----------| | 1000 | 100 | 2.4 MB | 1.7 MB | | 1000 | 1000 | 17.3 MB | 12.4 MB | | 10000 | 100 | 22.1 MB | 15.6 MB | | 10000 | 1000 | 162.5 MB | 112.7 MB | The `STABLE` event gives similar reductions. Homogeneous subscriptions resolve the topics of a single member, so the saving there is at most one lambda per topic. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
