xiangfu0 opened a new pull request, #19474:
URL: https://github.com/apache/pinot/pull/19474
## What
`PhysicalColumnIndexContainer` stored its readers in an `IndexTypeMap`
holding a span-sized `IndexReader[]`: with forward index at numeric id 2 and
null vector at 8, a column with two readers allocated seven slots. It now keeps
a `long` presence mask plus a densely packed, exactly sized reader array, so
`getIndex` is a shift, mask and `Long.bitCount` instead of a range check, at
the same O(1) cost. Construction also drops the transient per-column
`ArrayList`s.
A `Preconditions.checkState` rejects a numeric index id of 64 or above at
construction rather than silently misbehaving; OSS uses 13 ids today.
## Tests
`PhysicalColumnIndexContainerTest`: forward only, forward plus null vector,
and a mix spanning the lowest and a high id, asserting `getIndex` returns the
created reader for present types and null for absent ones, that
`forwardIndexOnly` filtering still applies, and that `close` closes each reader
exactly once.
## Why
A server keeps one metadata object graph per (segment, column) for as long
as the segment is loaded, so on wide tables the per-column footprint decides
how many segments a server can hold. Measured end to end on a 1000-column
segment, this series takes the heap retained at load from **4.08 MB to 0.175 MB
per segment** (4,080 to 174 bytes per column), with a fully compacting
collector on both sides. No on-disk format change, the
`/tables/{table}/segments/{segment}/metadata` JSON stays byte-identical, and
every public and SPI signature keeps working.
## Stack
Part 4 of 9, based on `xiangfu0/data-3221-2-intern-parse`. Review only this
part's own commits; the rest of the diff belongs to the parts below it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
