deepthi912 opened a new pull request, #18761:
URL: https://github.com/apache/pinot/pull/18761
## Summary
Adds a new table-level config `tableIndexConfig.skipSecondaryIndexes`
(default `false`). When enabled:
1. **New segments are built with only the forward index.**
`BaseSegmentCreator` skips every non-forward index creator during the per-row
build loop and short-circuits `updatePostSegmentCreationIndexes` (so star-tree
is also skipped). Dictionary is still written for dict-encoded columns because
the forward index needs it to resolve dict-ids.
2. **Existing segments load as-is and preprocess does not add/remove
secondary indexes.** `SegmentPreProcessor.process()` runs only the
`ForwardIndexHandler` (still allows forward-encoding upgrades) and
default-column materialization; it skips the loop over all other index
handlers, plus star-tree and multi-col text. Min/max generation is also skipped
— it is purely a query-time pruning hint. `needProcess()` mirrors the same
skips so preprocess isn't triggered just to add a secondary index. The on-disk
loader (`PhysicalColumnIndexContainer`) is intentionally untouched, so segments
with pre-existing secondary indexes keep serving them.
## Motivation
Useful for workloads that don't need secondary indexes during initial ingest
(or for an async "build indexes later" flow where the flag is later flipped off
and a reload runs the missing handlers).
## Config
```json
{
"tableIndexConfig": {
"skipSecondaryIndexes": true
}
}
```
## Test plan
- [ ] Unit test: build a segment with the flag on; verify only `<col>.fwd`
(+ `<col>.dict` for dict-encoded columns) is present.
- [ ] Unit test: load an existing segment with secondary indexes under the
flag; verify on-disk indexes still serve queries.
- [ ] Unit test: with the flag on, add a default column to schema and
trigger reload; verify the default column is materialized but no secondary
indexes are built.
- [ ] Unit test: flip flag off and reload; verify previously-skipped indexes
get built.
- [ ] Integration test: star-tree config + flag on → star-tree not built;
flip off → star-tree built on next reload.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
