tarun11Mavani opened a new pull request, #18980: URL: https://github.com/apache/pinot/pull/18980
Add UPSERT_PRIMARY_KEY_MAP_SIZE_IN_BYTES ServerGauge to complement the existing key-count gauge. Tables with few large-string PKs and tables with many small UUID-like PKs can have very different memory footprints despite similar counts, so byte-size observability helps with capacity monitoring. UpsertUtils.estimatePrimaryKeyMapSizeInBytes samples up to 1000 keys to average PK content size (via PrimaryKey.asBytes() / ByteArray.length()), adds a per-entry overhead constant for the hashmap node + RecordLocation object, and extrapolates over the full key count, keeping the cost bounded regardless of map size. Computed at the same segment-lifecycle cadence as the existing count gauge (add/replace/remove/preload/close), not on the per-record ingest path. Wired into both ConcurrentMapPartitionUpsertMetadataManager and ConcurrentMapPartitionUpsertMetadataManagerForConsistentDeletes via a new abstract getPrimaryKeyMapSizeInBytes() hook on BasePartitionUpsertMetadataManager. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
