Github user qiuchenjian commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2713#discussion_r242430992
--- Diff:
datamap/bloom/src/main/java/org/apache/carbondata/datamap/bloom/BloomCoarseGrainDataMapFactory.java
---
@@ -218,56 +218,46 @@ public DataMapBuilder createBuilder(Segment segment,
String shardName,
this.bloomFilterSize, this.bloomFilterFpp, bloomCompress);
}
- /**
- * returns all shard directories of bloom index files for query
- * if bloom index files are merged we should get only one shard path
- */
- private Set<String> getAllShardPaths(String tablePath, String segmentId)
{
- String dataMapStorePath = CarbonTablePath.getDataMapStorePath(
- tablePath, segmentId, dataMapName);
- CarbonFile[] carbonFiles =
FileFactory.getCarbonFile(dataMapStorePath).listFiles();
- Set<String> shardPaths = new HashSet<>();
+
+ private boolean isAllShardsMerged(String dmSegmentPath) {
+ boolean mergeShardExist = false;
boolean mergeShardInprogress = false;
- CarbonFile mergeShardFile = null;
+ CarbonFile[] carbonFiles =
FileFactory.getCarbonFile(dmSegmentPath).listFiles();
for (CarbonFile carbonFile : carbonFiles) {
- if
(carbonFile.getName().equals(BloomIndexFileStore.MERGE_BLOOM_INDEX_SHARD_NAME))
{
- mergeShardFile = carbonFile;
- } else if
(carbonFile.getName().equals(BloomIndexFileStore.MERGE_INPROGRESS_FILE)) {
+ String fileName = carbonFile.getName();
+ if
(fileName.equals(BloomIndexFileStore.MERGE_BLOOM_INDEX_SHARD_NAME)) {
+ mergeShardExist = true;
+ } else if
(fileName.equals(BloomIndexFileStore.MERGE_INPROGRESS_FILE)) {
mergeShardInprogress = true;
--- End diff --
If MERGE_INPROGRESS_FILE exists, shard's index file will be deleted
sometime, so this scene need to be focus on, but this question shows up before
this PR
---
