lhotari commented on code in PR #24833:
URL: https://github.com/apache/pulsar/pull/24833#discussion_r2436985881
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java:
##########
@@ -2610,6 +2581,99 @@ protected void
handleGetTopicsOfNamespace(CommandGetTopicsOfNamespace commandGet
}
}
+ private void internalHandleGetTopicsOfNamespace(String namespace,
NamespaceName namespaceName, long requestId,
+
CommandGetTopicsOfNamespace.Mode mode,
+ Optional<String>
topicsPattern, Optional<String> topicsHash,
+ Semaphore lookupSemaphore)
{
+ BooleanSupplier isPermitRequestCancelled = () ->
!ctx().channel().isActive();
+ TopicListSizeResultCache.ResultHolder
+ listSizeHolder =
service.getTopicListSizeResultCache().getTopicListSize(namespaceName.toString(),
mode);
+ listSizeHolder.getSizeAsync().thenAccept(initialSize -> {
+ maxTopicListInFlightLimiter.withAcquiredPermits(initialSize,
+ AsyncDualMemoryLimiter.LimitType.HEAP_MEMORY,
isPermitRequestCancelled, initialPermits -> {
+ return
getBrokerService().pulsar().getNamespaceService()
+ .getListOfUserTopics(namespaceName, mode)
+ .thenAccept(topics -> {
+ long actualSize = topics.stream()
+ .mapToInt(ByteBufUtil::utf8Bytes)
// convert character count to bytes
+ .map(n -> n + 32) // add 32 bytes
overhead for each entry
Review Comment:
In JVMs, an object header typically consumes 16 bytes and a reference
consumes 4 bytes or 8 bytes. This is improve the accuracy of the estimation of
consumed heap. I guess there's also padding involved.
I think that there's at least 20 bytes of overhead per entry.
One detail here is that the memory limiter doesn't do an accurate
calculation about the actual consumed heap. The purpose is to have a way to
ensure that there is a limit for heap memory. In many cases the referenced
objects are shared and therefore they won't duplicate the usage of heap memory.
It would be possible to address that, but it would unnecessarily complicate the
solution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
