vinothchandar commented on a change in pull request #513: [WIP] Improving Tag 
location using interval trees for index files
URL: https://github.com/apache/incubator-hudi/pull/513#discussion_r272710682
 
 

 ##########
 File path: 
hoodie-client/src/main/java/com/uber/hoodie/index/bloom/HoodieBloomIndex.java
 ##########
 @@ -300,57 +295,70 @@ public boolean isImplicitWithStorage() {
   }
 
   /**
-   * if we dont have key ranges, then also we need to compare against the 
file. no other choice if
-   * we do, then only compare the file if the record key falls in range.
+   * if we dont have key ranges, then also we need to compare against the 
file. no other choice if we do, then only
+   * compare the file if the record key falls in range.
    */
   boolean shouldCompareWithFile(BloomIndexFileInfo indexInfo, String 
recordKey) {
     return !indexInfo.hasKeyRanges() || indexInfo.isKeyInRange(recordKey);
   }
 
 
   /**
-   * For each incoming record, produce N output records, 1 each for each file 
against which the
-   * record's key needs to be checked. For datasets, where the keys have a 
definite insert order
-   * (e.g: timestamp as prefix), the number of files to be compared gets cut 
down a lot from range
-   * pruning.
+   * For each incoming record, produce N output records, 1 each for each file 
against which the record's key needs to be
+   * checked. For datasets, where the keys have a definite insert order (e.g: 
timestamp as prefix), the number of files
+   * to be compared gets cut down a lot from range pruning.
    *
-   * Sub-partition to ensure the records can be looked up against files & also 
prune
-   * file<=>record comparisons based on recordKey
-   * ranges in the index info.
+   * Sub-partition to ensure the records can be looked up against files & also 
prune file<=>record comparisons based on
+   * recordKey ranges in the index info.
    */
   @VisibleForTesting
   JavaPairRDD<String, Tuple2<String, HoodieKey>> 
explodeRecordRDDWithFileComparisons(
       final Map<String, List<BloomIndexFileInfo>> partitionToFileIndexInfo,
       JavaPairRDD<String, String> partitionRecordKeyPairRDD) {
+    Map<String, IndexLookUpTree> partitionToFileIndexLookUpTree = new 
HashMap<>();
+    Set<String> filesWithNoRanges = new HashSet<>();
 
 Review comment:
   ` IndexFileFilter` has been introduced 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[email protected]


With regards,
Apache Git Services

Reply via email to