Ted Malaska created HBASE-10983: ----------------------------------- Summary: Enhance LoadIncrementalHFile with option to do splitting in a distributed fashon Key: HBASE-10983 URL: https://issues.apache.org/jira/browse/HBASE-10983 Project: HBase Issue Type: Improvement Reporter: Ted Malaska Priority: Minor
Currently LoadIncrementalHFile supports splitting HFiles if they don't match up with the current regions of the table being imported too. However this functionality of reading and rewriting the HFile is done through a single JVM, which limits the overall speed of the splitting process. This jira will allow the user to set a flag or a threshold (on the total size of the HFiles to be split) that may trigger the splitting logic to be executed through a Map Only job as opposed to the existing thread pool in a single JVM. I will have the following goals when writing this patch: 1. Extend LoadIncrementalHFile 2. Reuse as much code from LoadIncrementalHFile as possible -- This message was sent by Atlassian JIRA (v6.2#6252)