Akanksha-kedia opened a new pull request, #18920:
URL: https://github.com/apache/pinot/pull/18920
## Summary
Closes #10506.
- **JsonIndexHandler**: After creating a JSON index, the active
`JsonIndexConfig` is serialised to JSON and stored in `metadata.properties`
under `column.<name>.jsonIndexConfig`. On the next segment reload
`needUpdateIndices()` reads the stored config, compares it with the current
table config using `equals()`, and triggers a rebuild when they differ.
Segments indexed before this change carry no stored config and are treated as
unchanged (backward compatibility — no spurious rebuild).
- **JsonIndexConfig.equals / hashCode**: Fixed to include `_indexPaths` and
`_maxBytesSize`, which were previously missing. Without this fix, config
changes to those two fields would be invisible to any equality-based comparison
(including the new detection logic).
- **Performance**: `metadata.properties` is loaded once per
`needUpdateIndices()` / `updateIndices()` call; the persist step in
`updateIndices()` batches all `setProperty` calls and writes the file once at
the end rather than once per rebuilt column.
## Test plan
- [ ] New `JsonIndexHandlerTest` (5 unit tests):
- No rebuild when stored config equals current config
- Rebuild triggered when `maxLevels` changes
- Rebuild triggered when `excludeArray` changes
- Rebuild triggered when `maxBytesSize` changes (would have silently
failed before the `equals` fix)
- No rebuild (backward compat) when no stored config is present
- [ ] Existing `SegmentPreProcessorTest` JSON index tests continue to pass
unmodified
cc @Jackie-Jiang @xiangfu0 @yashmayya
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
