[GitHub] [incubator-hudi] lamber-ken commented on issue #1469: [HUDI-686] Implement BloomIndexV2 that does not depend on memory caching

Sat, 11 Apr 2020 18:10:26 -0700 

lamber-ken commented on issue #1469: [HUDI-686] Implement BloomIndexV2 that 
does not depend on memory caching
URL: https://github.com/apache/incubator-hudi/pull/1469#issuecomment-612546567
 
 
   > understood :).. but I am saying the difference you see is outside the 
indexing piece.. Can you post the stages UI?
   
   Got, let me show you the spark stage page, amazing 👍. 
   From the ui, we can see that time-consuming, shuffle write, shuffle read are 
better.
   
   BTW, env: 4core, 6GB driver, local mode, don't forget `"hoodie.index.type" 
-> "BLOOM_V2"`
   
   #### `HoodieBloomIndex`: cost about 20min
   
   
![image](https://user-images.githubusercontent.com/20113411/79058033-0f7f5680-7c9b-11ea-9f7d-c6f3e94e62bb.png)
   
   #### `HoodieBloomIndexV2`: cost about 3min
   
   
![image](https://user-images.githubusercontent.com/20113411/79058182-1eff9f00-7c9d-11ea-9e4e-059fcef568bc.png)
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[email protected]


With regards,
Apache Git Services

Reply via email to