vvivekiyer commented on code in PR #9454:
URL: https://github.com/apache/pinot/pull/9454#discussion_r984020908
##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessorTest.java:
##########
@@ -304,6 +306,28 @@ public void testEnableFSTIndexOnExistingColumnDictEncoded()
checkFSTIndexCreation(EXISTING_STRING_COL_DICT, 9, 4,
_newColumnsSchemaWithFST, false, false, 26);
}
+ @Test
+ public void testForwardIndexHandler()
+ throws Exception {
+ Map<String, ChunkCompressionType> compressionConfigs = new HashMap<>();
+ ChunkCompressionType newCompressionType = ChunkCompressionType.ZSTANDARD;
+ compressionConfigs.put(EXISTING_STRING_COL_RAW, newCompressionType);
+ _indexLoadingConfig.setCompressionConfigs(compressionConfigs);
+ _indexLoadingConfig.setNoDictionaryColumns(new HashSet<String>() {{
+ add(EXISTING_STRING_COL_RAW);
+ }});
+
+ // Test1: Rewriting forward index will be a no-op for v1 segments. Default
LZ4 compressionType will be retained.
+ constructV1Segment();
+ checkForwardIndexCreation(EXISTING_STRING_COL_RAW, 5, 3, _schema, false,
false, false, 0, ChunkCompressionType.LZ4);
+
+ // Convert the segment to V3.
+ new SegmentV1V2ToV3FormatConverter().convert(_indexDir);
+
+ // Test2: Now forward index will be rewritten with ZSTANDARD
compressionType.
Review Comment:
> I am guessing this will read the new forward index (because old one is no
longer accessible via the _columnEntries map).
That is right. Subsequent handlers (like range index, inv index, etc) will
only see the newly created forward because all lookups are done using the
in-memory `_columnEntries` in `SingleFileIndexDirectory.java`
Given that this test verifies text index creation on the same column where
compressionType has been changed, we should be good here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
