[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+
[ https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16530352#comment-16530352 ] Sean Busbey commented on HBASE-16110: - Hi [~timoha]! Please bring issues running HBase to the user@hbase mailing list. JIRA is for tracking specific work tasks and this one is unlikely to get responses since it was resolved a couple of years ago. > AsyncFS WAL doesn't work with Hadoop 2.8+ > - > > Key: HBASE-16110 > URL: https://issues.apache.org/jira/browse/HBASE-16110 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-16110-v1.patch, HBASE-16110.patch > > > The async wal implementation doesn't work with Hadoop 2.8+. Fails compilation > and will fail running. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+
[ https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16530323#comment-16530323 ] Andrey Elenskiy commented on HBASE-16110: - Hello, we are running HBase 2.0.1 with official Hadoop 2.8.4 jars and hadoop 2.8.4 client (http://central.maven.org/maven2/org/apache/hadoop/hadoop-client/2.8.4/). Got the following exception on regionserver which brings it down: ``` 18/07/02 18:51:06 WARN concurrent.DefaultPromise: An exception was thrown by org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete() java.lang.Error: Couldn't properly initialize access to HDFS internals. Please update your WAL Provider to not make use of the 'asyncfs' provider. See HBASE-16110 for more information. at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:268) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:661) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:720) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:715) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:638) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:676) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:552) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:394) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:304) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:232) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:262) ... 18 more ``` FYI, we don't have encryption enabled. > AsyncFS WAL doesn't work with Hadoop 2.8+ > - > > Key: HBASE-16110 > URL: https://issues.apache.org/jira/browse/HBASE-16110 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-16110-v1.patch, HBASE-16110.patch > > > The async wal implementation doesn't work with Hadoop 2.8+. Fails compilation > and will fail running. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+
[ https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383720#comment-15383720 ] Duo Zhang commented on HBASE-16110: --- Good. Thanks [~busbey]. > AsyncFS WAL doesn't work with Hadoop 2.8+ > - > > Key: HBASE-16110 > URL: https://issues.apache.org/jira/browse/HBASE-16110 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-16110-v1.patch, HBASE-16110.patch > > > The async wal implementation doesn't work with Hadoop 2.8+. Fails compilation > and will fail running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+
[
https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372217#comment-15372217
]
Hadoop QA commented on HBASE-16110:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
30s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 4s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 21s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 97m 42s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 142m 37s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.replication.TestMasterReplication |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12817327/HBASE-16110-v1.patch |
| JIRA Issue | HBASE-16110 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / ccf293d |
| Default Java | 1.7.0_80 |
| Multi-JDK vers
[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+
[ https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357109#comment-15357109 ] Sean Busbey commented on HBASE-16110: - sorry, missed the review board link in the JIRA outage yesterday. I'll work through things later today. > AsyncFS WAL doesn't work with Hadoop 2.8+ > - > > Key: HBASE-16110 > URL: https://issues.apache.org/jira/browse/HBASE-16110 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-16110.patch > > > The async wal implementation doesn't work with Hadoop 2.8+. Fails compilation > and will fail running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+
[ https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15356594#comment-15356594 ] Duo Zhang commented on HBASE-16110: --- Ping [~busbey] > AsyncFS WAL doesn't work with Hadoop 2.8+ > - > > Key: HBASE-16110 > URL: https://issues.apache.org/jira/browse/HBASE-16110 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-16110.patch > > > The async wal implementation doesn't work with Hadoop 2.8+. Fails compilation > and will fail running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+
[ https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353217#comment-15353217 ] Sean Busbey commented on HBASE-16110: - a review board link would help, yes. I'd also like to make sure we document here which upstream changes in Hadoop caused our various breakages. > AsyncFS WAL doesn't work with Hadoop 2.8+ > - > > Key: HBASE-16110 > URL: https://issues.apache.org/jira/browse/HBASE-16110 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-16110.patch > > > The async wal implementation doesn't work with Hadoop 2.8+. Fails compilation > and will fail running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+
[ https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352049#comment-15352049 ] Duo Zhang commented on HBASE-16110: --- Need a reviewboard link? > AsyncFS WAL doesn't work with Hadoop 2.8+ > - > > Key: HBASE-16110 > URL: https://issues.apache.org/jira/browse/HBASE-16110 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-16110.patch > > > The async wal implementation doesn't work with Hadoop 2.8+. Fails compilation > and will fail running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+
[ https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351519#comment-15351519 ] Sean Busbey commented on HBASE-16110: - I have more feedback on the patch but am on mobile. Will post tonight. > AsyncFS WAL doesn't work with Hadoop 2.8+ > - > > Key: HBASE-16110 > URL: https://issues.apache.org/jira/browse/HBASE-16110 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-16110.patch > > > The async wal implementation doesn't work with Hadoop 2.8+. Fails compilation > and will fail running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+
[ https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351513#comment-15351513 ] Sean Busbey commented on HBASE-16110: - If Hadoop publishes SNAPSHOT jars then we can build against branch-2.8 by adding 2.8.0-SNAPSHOT to our list of Hadoop versions. > AsyncFS WAL doesn't work with Hadoop 2.8+ > - > > Key: HBASE-16110 > URL: https://issues.apache.org/jira/browse/HBASE-16110 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-16110.patch > > > The async wal implementation doesn't work with Hadoop 2.8+. Fails compilation > and will fail running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+
[
https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15350754#comment-15350754
]
Hadoop QA commented on HBASE-16110:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
54s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 44s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
56s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 54s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 8s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 90m 54s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 131m 59s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12813611/HBASE-16110.patch |
| JIRA Issue | HBASE-16110 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / fc4b8aa |
| Default Java | 1.7.0_80 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/home/jenkins/jenkins-slave/tools/hudson.model
[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+
[ https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15350552#comment-15350552 ] Duo Zhang commented on HBASE-16110: --- [~busbey] Can we have a pre-commit which builds against hadoop-2.8? Thanks. > AsyncFS WAL doesn't work with Hadoop 2.8+ > - > > Key: HBASE-16110 > URL: https://issues.apache.org/jira/browse/HBASE-16110 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-16110.patch > > > The async wal implementation doesn't work with Hadoop 2.8+. Fails compilation > and will fail running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+
[ https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349466#comment-15349466 ] Duo Zhang commented on HBASE-16110: --- And for hadoop 3, [~stack] and I are trying to push AsyncFSOutput related code into H3. No big progress yet... But seems the endless quarrel on AsyncFS API has ended finally. Will continue tracking it and prepare a patch at the right time. Thanks. > AsyncFS WAL doesn't work with Hadoop 2.8+ > - > > Key: HBASE-16110 > URL: https://issues.apache.org/jira/browse/HBASE-16110 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Priority: Blocker > Fix For: 2.0.0 > > > The async wal implementation doesn't work with Hadoop 2.8+. Fails compilation > and will fail running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+
[ https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349097#comment-15349097 ] Duo Zhang commented on HBASE-16110: --- Got it. Let me fix this. > AsyncFS WAL doesn't work with Hadoop 2.8+ > - > > Key: HBASE-16110 > URL: https://issues.apache.org/jira/browse/HBASE-16110 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Priority: Blocker > Fix For: 2.0.0 > > > The async wal implementation doesn't work with Hadoop 2.8+. Fails compilation > and will fail running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+
[ https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349053#comment-15349053 ] Sean Busbey commented on HBASE-16110: - No, it's not out. It's been "coming soon" since like November of last year. These same changes are in Hadoop 3, however, and that's supposed to have alphas in a month or two. I've run into this while working on updating HADOOP-11804 for current hbase master. I think I'll have all of these things worked out ~Tuesday. > AsyncFS WAL doesn't work with Hadoop 2.8+ > - > > Key: HBASE-16110 > URL: https://issues.apache.org/jira/browse/HBASE-16110 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Priority: Blocker > Fix For: 2.0.0 > > > The async wal implementation doesn't work with Hadoop 2.8+. Fails compilation > and will fail running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+
[ https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348959#comment-15348959 ] Duo Zhang commented on HBASE-16110: --- Yeah it does not work. Need some changes. Is hadoop 2.8.0 out? Or is there a final due date? Thanks. > AsyncFS WAL doesn't work with Hadoop 2.8+ > - > > Key: HBASE-16110 > URL: https://issues.apache.org/jira/browse/HBASE-16110 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Priority: Blocker > Fix For: 2.0.0 > > > The async wal implementation doesn't work with Hadoop 2.8+. Fails compilation > and will fail running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+
[ https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348794#comment-15348794 ] Sean Busbey commented on HBASE-16110: - Second issue: o.a.h.hdfs.server.common.HdfsServerConstants.READ_TIMEOUT was moved by HDFS-8803 to o.a.h.hdfs.protocol.HdfsConstants > AsyncFS WAL doesn't work with Hadoop 2.8+ > - > > Key: HBASE-16110 > URL: https://issues.apache.org/jira/browse/HBASE-16110 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Priority: Blocker > Fix For: 2.0.0 > > > The async wal implementation doesn't work with Hadoop 2.8+. Fails compilation > and will fail running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+
[ https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348760#comment-15348760 ] Sean Busbey commented on HBASE-16110: - It would be useful to know if this could be restricted to working with fewer versions of Hadoop, btw. Barring a discussion on dev@ I'm going to work against 2.4+. > AsyncFS WAL doesn't work with Hadoop 2.8+ > - > > Key: HBASE-16110 > URL: https://issues.apache.org/jira/browse/HBASE-16110 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Priority: Blocker > Fix For: 2.0.0 > > > The async wal implementation doesn't work with Hadoop 2.8+. Fails compilation > and will fail running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+
[ https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348757#comment-15348757 ] Sean Busbey commented on HBASE-16110: - I'm going through getting this to work slowly, but if someone else wants to take over that's great by me so I'm not assigning to myself. > AsyncFS WAL doesn't work with Hadoop 2.8+ > - > > Key: HBASE-16110 > URL: https://issues.apache.org/jira/browse/HBASE-16110 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Priority: Blocker > Fix For: 2.0.0 > > > The async wal implementation doesn't work with Hadoop 2.8+. Fails compilation > and will fail running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16110) AsyncFS WAL doesn't work with Hadoop 2.8+
[ https://issues.apache.org/jira/browse/HBASE-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348755#comment-15348755 ] Sean Busbey commented on HBASE-16110: - First issue: The IA.Private DFSClient.Conf class was removed in HDFS-8100. This causes compilation against version 2.8.0-SNAPSHOT+ to fail. If we compile against an earlier release and then run in a cluster running 2.8+ versions, the static initializer in FanOutOneBlockAsyncDFSOutputHelper that sets CREATE_CHECKSUM will throw a ClassNotFoundException and bring down the RS. > AsyncFS WAL doesn't work with Hadoop 2.8+ > - > > Key: HBASE-16110 > URL: https://issues.apache.org/jira/browse/HBASE-16110 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.0.0 >Reporter: Sean Busbey >Priority: Blocker > Fix For: 2.0.0 > > > The async wal implementation doesn't work with Hadoop 2.8+. Fails compilation > and will fail running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
