Jackie-Jiang commented on code in PR #10905:
URL: https://github.com/apache/pinot/pull/10905#discussion_r1232927284
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessor.java:
##########
@@ -243,10 +243,8 @@ private void processStarTrees(File indexDir)
if (starTreeMetadataList != null) {
// There are existing star-trees
if
(StarTreeBuilderUtils.shouldRemoveExistingStarTrees(starTreeBuilderConfigs,
starTreeMetadataList)) {
Review Comment:
(minor) Let's rename this method to something like
`shouldModifyExistingStarTrees()` and also modify the documentation to reflect
the change
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessor.java:
##########
@@ -243,10 +243,8 @@ private void processStarTrees(File indexDir)
if (starTreeMetadataList != null) {
// There are existing star-trees
if
(StarTreeBuilderUtils.shouldRemoveExistingStarTrees(starTreeBuilderConfigs,
starTreeMetadataList)) {
- // Remove the existing star-trees
- LOGGER.info("Removing star-trees from segment: {}",
_segmentMetadata.getName());
- StarTreeBuilderUtils.removeStarTrees(indexDir);
- _segmentMetadata = new SegmentMetadataImpl(indexDir);
+ LOGGER.info("Change detected in star-trees for segment: {}",
_segmentMetadata.getName());
+ shouldGenerateStarTree = true;
Review Comment:
We can simply remove the existing star trees when `shouldGenerateStarTree`
is false (user wants to remove all the star-trees)
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilder.java:
##########
@@ -115,13 +117,33 @@ public MultipleTreesBuilder(@Nullable
List<StarTreeIndexConfig> indexConfigs, bo
}
}
+ private SeparatedStarTreesMetadata getExistingStarTrees()
+ throws Exception {
+ try {
+ if (_metadataProperties.containsKey(MetadataKey.STAR_TREE_COUNT)) {
+ // Extract existing startrees
+ // clean star-tree related files and configs once all the star-trees
are separated and extracted
+ SeparatedStarTreesMetadata existingStarTrees = extractStarTrees();
+ StarTreeBuilderUtils.removeStarTrees(_segmentDirectory);
Review Comment:
`removeStarTrees()` should take `indexDir` instead of `segmentDirectory`
(even though it works if we pass in the `segmentDirectory`). `segmentDirectory`
could be the `v3` sub-directory under the `indexDir`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
