Hi all,

I opened PR #16358 a few weeks back and would appreciate some feedback:

https://github.com/apache/lucene/pull/16358

*Current pain point:*


   - Adding any new stat to the skip index today requires a format version
   bump, recalculating the fixed byte offsets, and gating reads behind version
   checks because the layout is a fixed schema where every byte position is
   predetermined. So it makes it hard to add new stats, or write a custom
   implementation if someone wants to.
   - Additionally, DocValuesSkipper as of today currently carries both
   navigation and stats, so a caller that only wants field-level stats has to
   open the .dvs file to get them



*The PR* first replaces the fixed layout with length-prefixed, type-tagged
entries, so readers skip stats they don't recognize and adding one needs no
version bump or offset math.


Second, it adds DocValuesField and SkipStat<T>: field-level stats come from
.dvm with no I/O, and callers can check which optional stats a field has
before opening the skipper.


Thanks,
Sagar

Reply via email to