[
https://issues.apache.org/jira/browse/HBASE-6358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427786#comment-13427786
]
Hadoop QA commented on HBASE-6358:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12538970/HBASE-6358-trunk-v2.diff
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
-1 javac. The applied patch generated 5 javac compiler warnings (more than
the trunk's current 4 warnings).
-1 findbugs. The patch appears to introduce 10 new Findbugs (version
1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestHRegionServerBulkLoad
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles
org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/2492//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2492//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2492//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2492//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2492//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2492//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/2492//console
This message is automatically generated.
> Bulkloading from remote filesystem is problematic
> -------------------------------------------------
>
> Key: HBASE-6358
> URL: https://issues.apache.org/jira/browse/HBASE-6358
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.94.0
> Reporter: Dave Revell
> Assignee: Dave Revell
> Attachments: HBASE-6358-trunk-v1.diff, HBASE-6358-trunk-v2.diff,
> HBASE-6358-trunk-v3.diff
>
>
> Bulk loading hfiles that don't live on the same filesystem as HBase can cause
> problems for subtle reasons.
> In Store.bulkLoadHFile(), the regionserver will copy the source hfile to its
> own filesystem if it's not already there. Since this can take a long time for
> large hfiles, it's likely that the client will timeout and retry. When the
> client retries repeatedly, there may be several bulkload operations in flight
> for the same hfile, causing lots of unnecessary IO and tying up handler
> threads. This can seriously impact performance. In my case, the cluster
> became unusable and the regionservers had to be kill -9'ed.
> Possible solutions:
> # Require that hfiles already be on the same filesystem as HBase in order
> for bulkloading to succeed. The copy could be handled by
> LoadIncrementalHFiles before the regionserver is called.
> # Others? I'm not familiar with Hadoop IPC so there may be tricks to extend
> the timeout or something else.
> I'm willing to write a patch but I'd appreciate recommendations on how to
> proceed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
