Akanksha-kedia commented on code in PR #18920:
URL: https://github.com/apache/pinot/pull/18920#discussion_r3535376527
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/invertedindex/JsonIndexHandler.java:
##########
@@ -65,12 +70,21 @@ public boolean needUpdateIndices(SegmentDirectory.Reader
segmentReader) {
String segmentName = _segmentDirectory.getSegmentMetadata().getName();
Set<String> columnsToAddIdx = new HashSet<>(_jsonIndexConfigs.keySet());
Set<String> existingColumns =
segmentReader.toSegmentDirectory().getColumnsWithIndex(StandardIndexes.json());
- // Check if any existing index need to be removed.
+ // Load metadata properties once to avoid repeated disk reads when
checking multiple columns.
+ PropertiesConfiguration properties = loadMetadataProperties();
+ // Check if any existing index need to be removed or if the config changed.
for (String column : existingColumns) {
if (!columnsToAddIdx.remove(column)) {
LOGGER.info("Need to remove existing json index from segment: {},
column: {}", segmentName, column);
return true;
}
+ // Column exists in both existing indexes and config; check if config
changed.
+ JsonIndexConfig currentConfig = _jsonIndexConfigs.get(column);
+ JsonIndexConfig storedConfig = readStoredJsonIndexConfig(column,
properties);
+ if (storedConfig != null && !storedConfig.equals(currentConfig)) {
Review Comment:
Fixed. The condition is now `properties != null && (storedConfig == null ||
!storedConfig.equals(currentConfig))`.
- `properties == null` (in-memory segment) → skip rebuild, same as before.
- `properties != null && storedConfig == null` (legacy segment with no
persisted config) → trigger a one-time rebuild to backfill the stored config
and catch any config drift.
- `properties != null && !storedConfig.equals(currentConfig)` (config
changed) → rebuild as before.
The test `testNeedUpdateReturnsTrueWhenNoStoredConfig` was also updated to
assert `true` for legacy segments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
