johnsolomonj commented on code in PR #18185:
URL: https://github.com/apache/pinot/pull/18185#discussion_r3388353077
##########
pinot-controller/src/main/java/org/apache/pinot/controller/util/ServerSegmentMetadataReader.java:
##########
@@ -144,6 +160,62 @@ public TableMetadataInfo
getAggregatedTableMetadataFromServer(String tableNameWi
}
return l;
}));
+ // Aggregate per-column compression stats from server responses
+ List<ColumnCompressionStatsInfo> serverColStats =
tableMetadataInfo.getColumnCompressionStats();
+ if (serverColStats != null) {
+ for (ColumnCompressionStatsInfo info : serverColStats) {
+ // Skip columns with no codec — these are old raw segments built
before compression stats
+ // tracking was enabled and carry no meaningful data.
+ if (info.getCodec() == null) {
+ continue;
+ }
+ String col = info.getColumn();
+ long[] accum = columnCompressionAccum.computeIfAbsent(col, k ->
new long[2]);
+ // Only accumulate uncompressed size when it is a real value (not
the -1 sentinel from dict columns)
+ if (info.getRawIngestSizeInBytes() >= 0) {
+ accum[0] += info.getRawIngestSizeInBytes();
+ }
+ accum[1] += info.getOnDiskSizeInBytes();
+ if (info.getCodec() != null) {
+ columnCodecMap.merge(col, info.getCodec(),
+ (existing, incoming) -> existing.equals(incoming) ? existing
: "MIXED");
Review Comment:
When `codec="MIXED"` in the response, the `codecBreakdown` map provides the
full per-codec detail — segment count, `rawIngestSizeInBytes`, and
`onDiskSizeInBytes` for each codec. So the list of codecs and their sizes are
already available via `codecBreakdown`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
