[
https://issues.apache.org/jira/browse/HBASE-18885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16182216#comment-16182216
]
Hadoop QA commented on HBASE-18885:
-----------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
10s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} branch-1 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
4s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
3s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} branch-1 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
40s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
20m 14s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}269m 57s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 3m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}312m 41s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.regionserver.TestCompactionInDeadRegionServer |
| | hadoop.hbase.master.balancer.TestStochasticLoadBalancer2 |
| | hadoop.hbase.client.TestAdmin2 |
| Timed out junit tests | org.apache.hadoop.hbase.mapreduce.TestWALPlayer |
| | org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat |
| | org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy
|
| | org.apache.hadoop.hbase.master.TestMasterFailover |
| | org.apache.hadoop.hbase.TestHBaseTestingUtility |
| | org.apache.hadoop.hbase.master.TestAssignmentManagerMetrics |
| | org.apache.hadoop.hbase.wal.TestWALFiltering |
| | org.apache.hadoop.hbase.regionserver.TestRegionReplicas |
| | org.apache.hadoop.hbase.regionserver.TestRegionServerAbort |
| | org.apache.hadoop.hbase.client.TestMetaWithReplicas |
| | org.apache.hadoop.hbase.regionserver.wal.TestLogRollAbort |
| | org.apache.hadoop.hbase.master.TestTableLockManager |
| | org.apache.hadoop.hbase.master.procedure.TestWALProcedureStoreOnHDFS |
| | org.apache.hadoop.hbase.util.TestMiniClusterLoadEncoded |
| | org.apache.hadoop.hbase.TestJMXConnectorServer |
| | org.apache.hadoop.hbase.master.TestDeadServer |
| | org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential |
| | org.apache.hadoop.hbase.regionserver.wal.TestLogRolling |
| |
org.apache.hadoop.hbase.regionserver.throttle.TestCompactionWithThroughputController
|
| | org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush |
| |
org.apache.hadoop.hbase.regionserver.throttle.TestFlushWithThroughputController
|
| | org.apache.hadoop.hbase.TestAcidGuarantees |
| | org.apache.hadoop.hbase.master.TestZKBasedOpenCloseRegion |
| | org.apache.hadoop.hbase.mapreduce.TestCopyTable |
| | org.apache.hadoop.hbase.client.TestReplicaWithCluster |
| |
org.apache.hadoop.hbase.client.replication.TestReplicationAdminWithTwoDifferentZKClusters
|
| | org.apache.hadoop.hbase.wal.TestWALOpenAfterDNRollingStart |
| | org.apache.hadoop.hbase.master.TestRestartCluster |
| | org.apache.hadoop.hbase.regionserver.TestHRegion |
| | org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel |
| | org.apache.hadoop.hbase.io.hfile.TestCacheOnWrite |
| | org.apache.hadoop.hbase.master.procedure.TestMasterProcedureEvents |
| | org.apache.hadoop.hbase.TestHColumnDescriptorDefaultVersions |
| | org.apache.hadoop.hbase.client.TestTableSnapshotScanner |
| | org.apache.hadoop.hbase.client.TestMultiParallel |
| | org.apache.hadoop.hbase.TestMetaTableAccessor |
| |
org.apache.hadoop.hbase.client.replication.TestReplicationAdminWithClusters |
| | org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover |
| | org.apache.hadoop.hbase.master.handler.TestEnableTableHandler |
| | org.apache.hadoop.hbase.master.handler.TestCreateTableHandler |
| | org.apache.hadoop.hbase.wal.TestWALFactory |
| | org.apache.hadoop.hbase.mapreduce.TestTableSnapshotInputFormat |
| | org.apache.hadoop.hbase.replication.TestMasterReplication |
| | org.apache.hadoop.hbase.io.TestFileLink |
| | org.apache.hadoop.hbase.mapred.TestTableSnapshotInputFormat |
| | org.apache.hadoop.hbase.util.TestFSUtils |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:6f1cc2c |
| JIRA Issue | HBASE-18885 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12889216/HBASE-18885.branch-1.001.patch
|
| Optional Tests | asflicense shadedjars javac javadoc unit findbugs
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux b3de9c824e36 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-1 / 3abc045 |
| Default Java | 1.7.0_151 |
| Multi-JDK versions | /usr/lib/jvm/java-8-oracle:1.8.0_144
/usr/lib/jvm/java-7-openjdk-amd64:1.7.0_151 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/8814/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8814/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8814/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> HFileOutputFormat2 hardcodes default FileOutputCommitter
> --------------------------------------------------------
>
> Key: HBASE-18885
> URL: https://issues.apache.org/jira/browse/HBASE-18885
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Reporter: Shaofeng SHI
> Assignee: Shaofeng SHI
> Attachments: HBASE-18885.branch-1.001.patch,
> HBASE-18885.master.001.patch
>
>
> Apache Kylin uses HBase's HFileOutputFormat2.java to configure the MR job.
> The original reporting is in KYLIN-2788[1]. After some investigation, we
> found this class always uses the default "FileOutputCommitter", see [2],
> regardless of the job's configuration; so it always writing to "_temporary"
> folder. Since AWS EMR configured to use DirectOutputCommitter for S3, then
> this problem occurs: Hadoop expects to see the file directly under output
> path, while the RecordWriter generates them in "_temporary" folder. This
> caused no data be loaded to HTable.
> Seems this problem exists in all versions so far.
> [1] https://issues.apache.org/jira/browse/KYLIN-2788
> [2]
> https://github.com/apache/hbase/blob/master/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat2.java#L193
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
