[jira] [Commented] (HBASE-4650) Update LoadIncrementalHFiles to use atomic bulk load RS mechanism
[ https://issues.apache.org/jira/browse/HBASE-4650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13139049#comment-13139049 ] Hadoop QA commented on HBASE-4650: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12501392/hbase-4650.v3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause mvn compile goal to fail. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/90//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/90//console This message is automatically generated. Update LoadIncrementalHFiles to use atomic bulk load RS mechanism - Key: HBASE-4650 URL: https://issues.apache.org/jira/browse/HBASE-4650 Project: HBase Issue Type: Sub-task Components: regionserver Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.92.0 Attachments: 0001-HBASE-4650-Update-LoadIncrementalHFiles-to-use-atomi.patch, 0001-HBASE-4650-Update-LoadIncrementalHFiles-to-use-atomi.prelim.patch, hbase-4650.v2.patch, hbase-4650.v3.patch MR jobs and command line bulk load execution runs use the LoadIncrementalHFile.doBulkLoad. This needs to be updated to group HFiles by row/region so that rows can be atomically loaded multiple column families. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4650) Update LoadIncrementalHFiles to use atomic bulk load RS mechanism
[ https://issues.apache.org/jira/browse/HBASE-4650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136094#comment-13136094 ] Hadoop QA commented on HBASE-4650: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12500899/hbase-4650.v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/71//console This message is automatically generated. Update LoadIncrementalHFiles to use atomic bulk load RS mechanism - Key: HBASE-4650 URL: https://issues.apache.org/jira/browse/HBASE-4650 Project: HBase Issue Type: Sub-task Components: regionserver Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.92.0 Attachments: 0001-HBASE-4650-Update-LoadIncrementalHFiles-to-use-atomi.patch, 0001-HBASE-4650-Update-LoadIncrementalHFiles-to-use-atomi.prelim.patch, hbase-4650.v2.patch MR jobs and command line bulk load execution runs use the LoadIncrementalHFile.doBulkLoad. This needs to be updated to group HFiles by row/region so that rows can be atomically loaded multiple column families. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4650) Update LoadIncrementalHFiles to use atomic bulk load RS mechanism
[ https://issues.apache.org/jira/browse/HBASE-4650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134253#comment-13134253 ] Jonathan Hsieh commented on HBASE-4650: --- I'm in the process of cleaning up the modifications to LoadIncrementalHFiles and adding more tests before submitting HBASE-4650 for a for review. This cut passes the two tests that use LoadIncrementalHFiles (TestLoadIncrementalHFiles and TestHFileOutputFormat). I'll post a preliminary version for those interested. In the code, there are significant logic changes due to grouping so I've chosen to take out the concurrency on the first cut because gathering and splitting HFiles into proper groups introduces a synchronization point that prevents some of the concurrency as before. This is because groups need to be fully gathered before bulk loads in a region is attempted. I'll include comments where concurrency is ok. Before I spend effort to parallelize this implementation more, I want to add another test to verify that this works while splits are going on. Update LoadIncrementalHFiles to use atomic bulk load RS mechanism - Key: HBASE-4650 URL: https://issues.apache.org/jira/browse/HBASE-4650 Project: HBase Issue Type: Sub-task Components: regionserver Reporter: Jonathan Hsieh Fix For: 0.92.0 MR jobs and command line bulk load execution runs use the LoadIncrementalHFile.doBulkLoad. This needs to be updated to group HFiles by row/region so that rows can be atomically loaded multiple column families. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4650) Update LoadIncrementalHFiles to use atomic bulk load RS mechanism
[ https://issues.apache.org/jira/browse/HBASE-4650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134553#comment-13134553 ] Jonathan Hsieh commented on HBASE-4650: --- initial parallel implementation here: https://reviews.apache.org/r/2557/ Update LoadIncrementalHFiles to use atomic bulk load RS mechanism - Key: HBASE-4650 URL: https://issues.apache.org/jira/browse/HBASE-4650 Project: HBase Issue Type: Sub-task Components: regionserver Reporter: Jonathan Hsieh Fix For: 0.92.0 Attachments: 0001-HBASE-4650-Update-LoadIncrementalHFiles-to-use-atomi.patch, 0001-HBASE-4650-Update-LoadIncrementalHFiles-to-use-atomi.prelim.patch MR jobs and command line bulk load execution runs use the LoadIncrementalHFile.doBulkLoad. This needs to be updated to group HFiles by row/region so that rows can be atomically loaded multiple column families. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
