[jira] [Assigned] (HBASE-19861) Avoid using RPCs when querying table infos for master status pages
[
https://issues.apache.org/jira/browse/HBASE-19861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang reassigned HBASE-19861:
-
Assignee: Xiaolin Ha
> Avoid using RPCs when querying table infos for master status pages
> --
>
> Key: HBASE-19861
> URL: https://issues.apache.org/jira/browse/HBASE-19861
> Project: HBase
> Issue Type: Improvement
> Components: monitoring
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
>
> When querying table information for master status pages, currently method is
> using admin interfaces. For example, when list user tables, codes are as
> follows.
> Connection connection = master.getConnection();
> Admin admin = connection.getAdmin();
> try {
> tables = admin.listTables();
> } finally {
> admin.close();
> }
> But actually, we can get all user tables from master's memory.
> Using admin interfaces means using RPCs, which has a low efficiency.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19803) False positive for the HBASE-Find-Flaky-Tests job
[
https://issues.apache.org/jira/browse/HBASE-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338855#comment-16338855
]
Duo Zhang commented on HBASE-19803:
---
Checked the stacktrace a bit
{noformat}
org.apache.hadoop.hbase.HConstants$ExitException: There is no escape!
at
org.apache.hadoop.hbase.HConstants$NoExitSecurityManager.checkExit(HConstants.java:63)
at java.lang.Runtime.halt(Runtime.java:273)
at
org.apache.maven.surefire.booter.ForkedBooter.kill(ForkedBooter.java:300)
at
org.apache.maven.surefire.booter.ForkedBooter.kill(ForkedBooter.java:294)
at
org.apache.maven.surefire.booter.ForkedBooter.access$300(ForkedBooter.java:68)
at
org.apache.maven.surefire.booter.ForkedBooter$4.update(ForkedBooter.java:247)
at
org.apache.maven.surefire.booter.CommandReader$CommandRunnable.insertToListeners(CommandReader.java:475)
at
org.apache.maven.surefire.booter.CommandReader$CommandRunnable.run(CommandReader.java:421)
at java.lang.Thread.run(Thread.java:748)
{noformat}
https://github.com/apache/maven-surefire/blob/surefire-2.20.1/surefire-api/src/main/java/org/apache/maven/surefire/booter/CommandReader.java#L421
https://github.com/apache/maven-surefire/blob/surefire-2.20.1/surefire-booter/src/main/java/org/apache/maven/surefire/booter/ForkedBooter.java#L247
I think it is obvious that the UT is killed by the surefire itself.
CommandReader is a class that keep reading data from stdin to get commands from
the plugin process. And in the ForkedBooter, we actually do the kill job by
calling System.exit, see the below code.
{code}
DumpErrorSingleton.getSingleton()
.dumpText( "Killing self fork JVM. Received
SHUTDOWN command from Maven shutdown hook." );
kill();
{code}
This is not a kill by signal since the command is gotten from stdin. I think
this is correct direction to go, find out why the plugin process(or called
maven master process?) wants to kill the forked test process.
Thanks.
> False positive for the HBASE-Find-Flaky-Tests job
> -
>
> Key: HBASE-19803
> URL: https://issues.apache.org/jira/browse/HBASE-19803
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Priority: Major
> Attachments: 2018-01-24T17-45-37_000-jvmRun1.dumpstream,
> HBASE-19803.master.001.patch
>
>
> It reports two hangs for TestAsyncTableGetMultiThreaded, but I checked the
> surefire output
> https://builds.apache.org/job/HBASE-Flaky-Tests/24830/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt
> This one was likely to be killed in the middle of the run within 20 seconds.
> https://builds.apache.org/job/HBASE-Flaky-Tests/24852/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt
> This one was also killed within about 1 minutes.
> The test is declared as LargeTests so the time limit should be 10 minutes. It
> seems that the jvm may crash during the mvn test run and then we will kill
> all the running tests and then we may mark some of them as hang which leads
> to the false positive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-19861) Avoid using RPCs when querying table infos for master status pages
Xiaolin Ha created HBASE-19861:
--
Summary: Avoid using RPCs when querying table infos for master
status pages
Key: HBASE-19861
URL: https://issues.apache.org/jira/browse/HBASE-19861
Project: HBase
Issue Type: Improvement
Components: monitoring
Reporter: Xiaolin Ha
When querying table information for master status pages, currently method is
using admin interfaces. For example, when list user tables, codes are as
follows.
Connection connection = master.getConnection();
Admin admin = connection.getAdmin();
try {
tables = admin.listTables();
} finally {
admin.close();
}
But actually, we can get all user tables from master's memory.
Using admin interfaces means using RPCs, which has a low efficiency.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-15666) shaded dependencies for hbase-testing-util
[
https://issues.apache.org/jira/browse/HBASE-15666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338828#comment-16338828
]
DM edited comment on HBASE-15666 at 1/25/18 7:30 AM:
-
[~mdrob] - I am using version 1.2.0. I was trying it as a part of big project.
But this can be easily reproduced using a toy project placed in
https://github.com/debraj-manna/reproduce .
You can download the maven project and just do {{mvn clean install}}. You will
see the exception in the console logs like below
{code:java}
2018-01-25 12:17:51,160 ERROR [main] master (Slf4jLog.java:warn(87)) -
unavailable
java.lang.ClassCastException:
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.servlet.Context$SContext
cannot be cast to org.mortbay.jetty.handler.ContextHandler$SContext
at
org.mortbay.jetty.servlet.DefaultServlet.init(DefaultServlet.java:154)
at javax.servlet.GenericServlet.init(GenericServlet.java:241)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:440)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.servlet.ServletHolder.doStart(ServletHolder.java:263)
at
org.apache.hadoop.hbase.shaded.org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:736)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
at
org.apache.hadoop.hbase.shaded.org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
at
org.apache.hadoop.hbase.shaded.org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.Server.doStart(Server.java:224)
at
org.apache.hadoop.hbase.shaded.org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at org.apache.hadoop.hbase.http.HttpServer.start(HttpServer.java:954)
at org.apache.hadoop.hbase.http.InfoServer.start(InfoServer.java:91)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.putUpWebUI(HRegionServer.java:1794)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:602)
at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:364)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:139)
at
org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:219)
at
org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:155)
at
org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:214)
at
org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1031)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:991)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:863)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:857)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:801)
at com.repro.hbase.TestHBase.testHbase(TestHBase.java:12)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
[jira] [Comment Edited] (HBASE-15666) shaded dependencies for hbase-testing-util
[
https://issues.apache.org/jira/browse/HBASE-15666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338828#comment-16338828
]
DM edited comment on HBASE-15666 at 1/25/18 7:15 AM:
-
[~mdrob] - I am using version 1.2.0. I was trying it as a part of big project.
But this can be easily reproduced using a toy project placed in
https://github.com/debraj-manna/reproduce .
You can unzip the maven project and run {{TestHbase}}. You will see the
exception in the console logs like below
{code:java}
2018-01-25 12:17:51,160 ERROR [main] master (Slf4jLog.java:warn(87)) -
unavailable
java.lang.ClassCastException:
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.servlet.Context$SContext
cannot be cast to org.mortbay.jetty.handler.ContextHandler$SContext
at
org.mortbay.jetty.servlet.DefaultServlet.init(DefaultServlet.java:154)
at javax.servlet.GenericServlet.init(GenericServlet.java:241)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:440)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.servlet.ServletHolder.doStart(ServletHolder.java:263)
at
org.apache.hadoop.hbase.shaded.org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:736)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
at
org.apache.hadoop.hbase.shaded.org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
at
org.apache.hadoop.hbase.shaded.org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.Server.doStart(Server.java:224)
at
org.apache.hadoop.hbase.shaded.org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at org.apache.hadoop.hbase.http.HttpServer.start(HttpServer.java:954)
at org.apache.hadoop.hbase.http.InfoServer.start(InfoServer.java:91)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.putUpWebUI(HRegionServer.java:1794)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:602)
at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:364)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:139)
at
org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:219)
at
org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:155)
at
org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:214)
at
org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1031)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:991)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:863)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:857)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:801)
at com.repro.hbase.TestHBase.testHbase(TestHBase.java:12)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
[jira] [Commented] (HBASE-15666) shaded dependencies for hbase-testing-util
[
https://issues.apache.org/jira/browse/HBASE-15666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338828#comment-16338828
]
DM commented on HBASE-15666:
[~mdrob] - I am using version 1.2.0. I was trying it as a part of big project.
But this can be easily reproduced using the attached toy project
{{reproduce.zip}}
You can unzip the maven project and run {{TestHbase}}. You will see the
exception in the console logs like below
{code:java}
2018-01-25 12:17:51,160 ERROR [main] master (Slf4jLog.java:warn(87)) -
unavailable
java.lang.ClassCastException:
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.servlet.Context$SContext
cannot be cast to org.mortbay.jetty.handler.ContextHandler$SContext
at
org.mortbay.jetty.servlet.DefaultServlet.init(DefaultServlet.java:154)
at javax.servlet.GenericServlet.init(GenericServlet.java:241)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:440)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.servlet.ServletHolder.doStart(ServletHolder.java:263)
at
org.apache.hadoop.hbase.shaded.org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:736)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
at
org.apache.hadoop.hbase.shaded.org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
at
org.apache.hadoop.hbase.shaded.org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
at
org.apache.hadoop.hbase.shaded.org.mortbay.jetty.Server.doStart(Server.java:224)
at
org.apache.hadoop.hbase.shaded.org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at org.apache.hadoop.hbase.http.HttpServer.start(HttpServer.java:954)
at org.apache.hadoop.hbase.http.InfoServer.start(InfoServer.java:91)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.putUpWebUI(HRegionServer.java:1794)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:602)
at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:364)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:139)
at
org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:219)
at
org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:155)
at
org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:214)
at
org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1031)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:991)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:863)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:857)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:801)
at com.repro.hbase.TestHBase.testHbase(TestHBase.java:12)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
[jira] [Commented] (HBASE-19723) hbase-thrift declares slf4j-api twice
[
https://issues.apache.org/jira/browse/HBASE-19723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338807#comment-16338807
]
Chia-Ping Tsai commented on HBASE-19723:
[~Jan Hentschel] Why not push this fix to branch-2?
> hbase-thrift declares slf4j-api twice
> -
>
> Key: HBASE-19723
> URL: https://issues.apache.org/jira/browse/HBASE-19723
> Project: HBase
> Issue Type: Improvement
> Components: Thrift
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Trivial
> Fix For: 3.0.0
>
> Attachments: HBASE-19723.master.001.patch
>
>
> Currently *hbase-thrift* declares the dependency {{slf4j-api}} twice
> resulting in the following warning
> {code}
> [WARNING] Some problems were encountered while building the effective model
> for org.apache.hbase:hbase-thrift:jar:3.0.0-SNAPSHOT
> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must
> be unique: org.slf4j:slf4j-api:jar -> duplicate declaration of version (?) @
> org.apache.hbase:hbase-thrift:[unknown-version],
> /Users/jan/Documents/Projects/github/hbase/hbase-thrift/pom.xml, line 250,
> column 17
> [WARNING]
> [WARNING] It is highly recommended to fix these problems because they
> threaten the stability of your build.
> [WARNING]
> [WARNING] For this reason, future Maven versions might no longer support
> building such malformed projects.
> {code}
> One should be removed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19803) False positive for the HBASE-Find-Flaky-Tests job
[ https://issues.apache.org/jira/browse/HBASE-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338779#comment-16338779 ] stack commented on HBASE-19803: --- What you thinking then [~appy] The dumpStream is archived IIRC. The mangled thread dump in it is useful? Is it for the surefire driver or is it forked JVM? I'd be +1 for trying it for a while to see if it turns up anything. Anything better than driving blind sir. > False positive for the HBASE-Find-Flaky-Tests job > - > > Key: HBASE-19803 > URL: https://issues.apache.org/jira/browse/HBASE-19803 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Priority: Major > Attachments: 2018-01-24T17-45-37_000-jvmRun1.dumpstream, > HBASE-19803.master.001.patch > > > It reports two hangs for TestAsyncTableGetMultiThreaded, but I checked the > surefire output > https://builds.apache.org/job/HBASE-Flaky-Tests/24830/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was likely to be killed in the middle of the run within 20 seconds. > https://builds.apache.org/job/HBASE-Flaky-Tests/24852/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was also killed within about 1 minutes. > The test is declared as LargeTests so the time limit should be 10 minutes. It > seems that the jvm may crash during the mvn test run and then we will kill > all the running tests and then we may mark some of them as hang which leads > to the false positive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19806) Lower max versions for table column family of hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-19806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338771#comment-16338771
]
stack commented on HBASE-19806:
---
Is this a problem? Evidence?
> Lower max versions for table column family of hbase:meta
>
>
> Key: HBASE-19806
> URL: https://issues.apache.org/jira/browse/HBASE-19806
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Trivial
> Fix For: 2.0.0-beta-2
>
> Attachments: 19806.v1.txt, lower-namespace-max-vers.txt
>
>
> On an hbase 2 cluster, I got the description of hbase:meta table:
> {code}
> {NAME => 'info', VERSIONS => '3', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
> CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
> 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'NONE', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'true', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '81
> 92'}
> ...
> {NAME => 'table', VERSIONS => '10', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
> CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
> 'FOREVER', MIN_VERSIONS => '0'
> , REPLICATION_SCOPE => '0', BLOOMFILTER => 'NONE', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'true', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '
> 8192'}
> {code}
> You can see that 'table' family has MAX VERSIONS much higher than the other
> families.
> The MAX VERSIONS value should be brought in sync with the other families.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19806) Lower max versions for table column family of hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-19806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-19806:
--
Priority: Trivial (was: Minor)
> Lower max versions for table column family of hbase:meta
>
>
> Key: HBASE-19806
> URL: https://issues.apache.org/jira/browse/HBASE-19806
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Trivial
> Fix For: 2.0.0-beta-2
>
> Attachments: 19806.v1.txt, lower-namespace-max-vers.txt
>
>
> On an hbase 2 cluster, I got the description of hbase:meta table:
> {code}
> {NAME => 'info', VERSIONS => '3', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
> CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
> 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'NONE', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'true', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '81
> 92'}
> ...
> {NAME => 'table', VERSIONS => '10', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
> CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
> 'FOREVER', MIN_VERSIONS => '0'
> , REPLICATION_SCOPE => '0', BLOOMFILTER => 'NONE', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'true', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '
> 8192'}
> {code}
> You can see that 'table' family has MAX VERSIONS much higher than the other
> families.
> The MAX VERSIONS value should be brought in sync with the other families.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19840) Flakey TestMetaWithReplicas
[ https://issues.apache.org/jira/browse/HBASE-19840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338763#comment-16338763 ] stack commented on HBASE-19840: --- I removed the patch I committed that added debug, 2.001. .001 should fix this flakey. It does make an interesting change though. Lets see if any repercussions; i.e. on shutdown, we stop the procedure executor. Fix two issues: # Meta Replicas can all be assigned to the same server. This will call the test to hang when we do our kill of the server hosting meta because there'll be no replicas to read from as test intends. Check is to look for this condition on startup and adjust if we come across it. Replicas cross-cut assignment. They need work. # Other issue was shutdown. The master started toward the end of the test may not have come up fully by the time shutdown is called. We could be stuck assigning the meta replicas. Have shutdown shutdown the procedure executor engine. There is other cleanup and notes in the below. M HMaster Remove the silly stops in startup now we have real means of shutting down Master during init. M hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterMetaBootstrap.java This replica stuff was doing stuff it shouldn't be doing like setting core Master state flags. It may have made sense once but now meta is assigned by a Pv2 Procedure so the flag setting in here is meddlesome. Clear out methods no longer needed. M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java Remove unused methods. Changes local variable names so they align w/ our naming elsewhere in code base. M hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaWithReplicas.java Check for all replicas on the one server. > Flakey TestMetaWithReplicas > --- > > Key: HBASE-19840 > URL: https://issues.apache.org/jira/browse/HBASE-19840 > Project: HBase > Issue Type: Sub-task > Components: flakey, test >Reporter: stack >Assignee: stack >Priority: Major > Attachments: HBASE-19840.master.001.patch > > > Failing about 15% of the time.. In testShutdownHandling.. > [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests-branch2.0/lastSuccessfulBuild/artifact/dashboard.html] > > Adding some debug. Its hard to follow what is going on in this test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19840) Flakey TestMetaWithReplicas
[ https://issues.apache.org/jira/browse/HBASE-19840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19840: -- Attachment: HBASE-19840.master.001.patch > Flakey TestMetaWithReplicas > --- > > Key: HBASE-19840 > URL: https://issues.apache.org/jira/browse/HBASE-19840 > Project: HBase > Issue Type: Sub-task > Components: flakey, test >Reporter: stack >Priority: Major > Attachments: HBASE-19840.master.001.patch > > > Failing about 15% of the time.. In testShutdownHandling.. > [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests-branch2.0/lastSuccessfulBuild/artifact/dashboard.html] > > Adding some debug. Its hard to follow what is going on in this test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-19840) Flakey TestMetaWithReplicas
[ https://issues.apache.org/jira/browse/HBASE-19840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reassigned HBASE-19840: - Assignee: stack > Flakey TestMetaWithReplicas > --- > > Key: HBASE-19840 > URL: https://issues.apache.org/jira/browse/HBASE-19840 > Project: HBase > Issue Type: Sub-task > Components: flakey, test >Reporter: stack >Assignee: stack >Priority: Major > Attachments: HBASE-19840.master.001.patch > > > Failing about 15% of the time.. In testShutdownHandling.. > [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests-branch2.0/lastSuccessfulBuild/artifact/dashboard.html] > > Adding some debug. Its hard to follow what is going on in this test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19840) Flakey TestMetaWithReplicas
[ https://issues.apache.org/jira/browse/HBASE-19840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19840: -- Attachment: (was: HBASE-19840.branch-2.001.patch) > Flakey TestMetaWithReplicas > --- > > Key: HBASE-19840 > URL: https://issues.apache.org/jira/browse/HBASE-19840 > Project: HBase > Issue Type: Sub-task > Components: flakey, test >Reporter: stack >Priority: Major > Attachments: HBASE-19840.master.001.patch > > > Failing about 15% of the time.. In testShutdownHandling.. > [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests-branch2.0/lastSuccessfulBuild/artifact/dashboard.html] > > Adding some debug. Its hard to follow what is going on in this test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19806) Lower max versions for table column family of hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-19806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19806:
---
Attachment: lower-namespace-max-vers.txt
> Lower max versions for table column family of hbase:meta
>
>
> Key: HBASE-19806
> URL: https://issues.apache.org/jira/browse/HBASE-19806
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 2.0.0-beta-2
>
> Attachments: 19806.v1.txt, lower-namespace-max-vers.txt
>
>
> On an hbase 2 cluster, I got the description of hbase:meta table:
> {code}
> {NAME => 'info', VERSIONS => '3', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
> CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
> 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'NONE', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'true', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '81
> 92'}
> ...
> {NAME => 'table', VERSIONS => '10', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
> CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
> 'FOREVER', MIN_VERSIONS => '0'
> , REPLICATION_SCOPE => '0', BLOOMFILTER => 'NONE', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'true', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '
> 8192'}
> {code}
> You can see that 'table' family has MAX VERSIONS much higher than the other
> families.
> The MAX VERSIONS value should be brought in sync with the other families.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19806) Lower max versions for table column family of hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-19806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19806:
---
Attachment: (was: lower-namespace-max-vers.txt)
> Lower max versions for table column family of hbase:meta
>
>
> Key: HBASE-19806
> URL: https://issues.apache.org/jira/browse/HBASE-19806
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 2.0.0-beta-2
>
> Attachments: 19806.v1.txt, lower-namespace-max-vers.txt
>
>
> On an hbase 2 cluster, I got the description of hbase:meta table:
> {code}
> {NAME => 'info', VERSIONS => '3', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
> CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
> 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'NONE', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'true', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '81
> 92'}
> ...
> {NAME => 'table', VERSIONS => '10', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
> CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
> 'FOREVER', MIN_VERSIONS => '0'
> , REPLICATION_SCOPE => '0', BLOOMFILTER => 'NONE', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'true', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '
> 8192'}
> {code}
> You can see that 'table' family has MAX VERSIONS much higher than the other
> families.
> The MAX VERSIONS value should be brought in sync with the other families.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-17079) HBase build fails on windows, hbase-archetype-builder is reason for failure
[
https://issues.apache.org/jira/browse/HBASE-17079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338755#comment-16338755
]
Hudson commented on HBASE-17079:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4463 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/4463/])
HBASE-17079 HBase build fails on windows, hbase-archetype-builder is (apurtell:
rev 98cae45d2ba0854592956e8d7ed5976d3eaa5da6)
* (edit) hbase-archetypes/hbase-archetype-builder/pom.xml
* (edit) pom.xml
> HBase build fails on windows, hbase-archetype-builder is reason for failure
> ---
>
> Key: HBASE-17079
> URL: https://issues.apache.org/jira/browse/HBASE-17079
> Project: HBase
> Issue Type: Bug
> Components: build
> Environment: windows
>Reporter: Mohammad Arshad
>Assignee: Mohammad Arshad
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.3.2, 1.4.1, 1.5.0
>
> Attachments: HBASE-17079.branch-1.3.002.patch,
> HBASE-17079.branch-1.3.Verification.jpg, HBASE-17079.master.001.patch,
> HBASE-17079.master.002.patch, screenshot-1.png
>
>
> HBase buid fails on windows, hbase-archetype-builder is reason for failure.
> Cygwin is installed so the shell scripts should execute successfully.
> Here is build failure log
> {code}
> [INFO] Apache HBase - Archetype builder ... FAILURE [ 1.014
> s]
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 06:13 min
> [INFO] Finished at: 2016-11-12T18:12:26+05:30
> [INFO] Final Memory: 235M/1012M
> [INFO]
>
> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:exec
> (make-scripts-executable) on project hbase-archetype-builder: Command
> execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :hbase-archetype-builder
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19756) Master NPE during completed failed proc eviction
[
https://issues.apache.org/jira/browse/HBASE-19756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338756#comment-16338756
]
Hudson commented on HBASE-19756:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4463 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/4463/])
HBASE-19756 Master NPE during completed failed proc eviction (apurtell: rev
ce50830a0af29e0ad2be24528629965923ef1cbf)
* (edit)
hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java
* (add)
hbase-server/src/test/java/org/apache/hadoop/hbase/procedure/TestFailedProcCleanup.java
> Master NPE during completed failed proc eviction
>
>
> Key: HBASE-19756
> URL: https://issues.apache.org/jira/browse/HBASE-19756
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.0, 1.3.1
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.3.2, 1.4.1, 1.5.0
>
> Attachments: HBASE-19756.branch-1.4.001.patch,
> HBASE-19756.branch-1.4.002.patch, HBASE-19756.branch-1.4.003.patch,
> HBASE-19756.master.001.patch
>
>
> When procedures like Create table fails due to say AccessDeniedException,
> then a rollback procedure is created. When the rollback is being cleaned up,
> it results in an NPE because those nonce procs aren't persisted
> Stack trace when this happens:
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.procedure2.store.ProcedureStoreTracker.delete(ProcedureStoreTracker.java:385)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.updateStoreTracker(WALProcedureStore.java:547)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.pushData(WALProcedureStore.java:504)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.delete(WALProcedureStore.java:453)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$CompletedProcedureCleaner.periodicExecute(ProcedureExecutor.java:184)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.timeoutLoop(ProcedureExecutor.java:995)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$500(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$3.run(ProcedureExecutor.java:507)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19860) Run branch-2 nightly job more frequently
[ https://issues.apache.org/jira/browse/HBASE-19860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338752#comment-16338752 ] stack commented on HBASE-19860: --- I like the frequent runs... especially now when we are trying to kill the flakies. Thanks [~appy] > Run branch-2 nightly job more frequently > > > Key: HBASE-19860 > URL: https://issues.apache.org/jira/browse/HBASE-19860 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy >Priority: Minor > Attachments: HBASE-19860.branch-2.001.patch > > > For time being, running the job more frequently will give us more data faster > to debug flaky tests . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19553) Old replica regions should be cleared from AM memory after primary region split or merge
[ https://issues.apache.org/jira/browse/HBASE-19553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pankaj Kumar updated HBASE-19553: - Attachment: HBASE-19553-branch-1-v4.patch > Old replica regions should be cleared from AM memory after primary region > split or merge > > > Key: HBASE-19553 > URL: https://issues.apache.org/jira/browse/HBASE-19553 > Project: HBase > Issue Type: Bug > Components: master >Reporter: huaxiang sun >Assignee: Pankaj Kumar >Priority: Minor > Fix For: 1.5.0 > > Attachments: HBASE-19553-branch-1-v2.patch, > HBASE-19553-branch-1-v3.patch, HBASE-19553-branch-1-v4.patch, > HBASE-19553-branch-1.patch > > > Similar to HBASE-18025, the replica parent's info is not removed from master. > Actually I think it can be removed after replica region is split or merged, I > will check the logic and apply one patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19806) Lower max versions for table column family of hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-19806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338744#comment-16338744
]
Anoop Sam John commented on HBASE-19806:
bq.namespace entries should not change much.
Ya with that my doubt was whether we should be worried abt this 10 max size at
all. Just leave it is ok?
When seeing the code, we have config for META block size also. Same apply to
NS also? Just asking. So my suggestion would be , if we create a new jira to
handle the NS specially (like META) for these settings, let us take all and
propose a full patch.
> Lower max versions for table column family of hbase:meta
>
>
> Key: HBASE-19806
> URL: https://issues.apache.org/jira/browse/HBASE-19806
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 2.0.0-beta-2
>
> Attachments: 19806.v1.txt, lower-namespace-max-vers.txt
>
>
> On an hbase 2 cluster, I got the description of hbase:meta table:
> {code}
> {NAME => 'info', VERSIONS => '3', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
> CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
> 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'NONE', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'true', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '81
> 92'}
> ...
> {NAME => 'table', VERSIONS => '10', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
> CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
> 'FOREVER', MIN_VERSIONS => '0'
> , REPLICATION_SCOPE => '0', BLOOMFILTER => 'NONE', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'true', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '
> 8192'}
> {code}
> You can see that 'table' family has MAX VERSIONS much higher than the other
> families.
> The MAX VERSIONS value should be brought in sync with the other families.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19806) Lower max versions for table column family of hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-19806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338730#comment-16338730
]
Ted Yu commented on HBASE-19806:
lower-namespace-max-vers.txt shows what I propose.
> Lower max versions for table column family of hbase:meta
>
>
> Key: HBASE-19806
> URL: https://issues.apache.org/jira/browse/HBASE-19806
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 2.0.0-beta-2
>
> Attachments: 19806.v1.txt, lower-namespace-max-vers.txt
>
>
> On an hbase 2 cluster, I got the description of hbase:meta table:
> {code}
> {NAME => 'info', VERSIONS => '3', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
> CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
> 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'NONE', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'true', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '81
> 92'}
> ...
> {NAME => 'table', VERSIONS => '10', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
> CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
> 'FOREVER', MIN_VERSIONS => '0'
> , REPLICATION_SCOPE => '0', BLOOMFILTER => 'NONE', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'true', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '
> 8192'}
> {code}
> You can see that 'table' family has MAX VERSIONS much higher than the other
> families.
> The MAX VERSIONS value should be brought in sync with the other families.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19806) Lower max versions for table column family of hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-19806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19806:
---
Attachment: lower-namespace-max-vers.txt
> Lower max versions for table column family of hbase:meta
>
>
> Key: HBASE-19806
> URL: https://issues.apache.org/jira/browse/HBASE-19806
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 2.0.0-beta-2
>
> Attachments: 19806.v1.txt, lower-namespace-max-vers.txt
>
>
> On an hbase 2 cluster, I got the description of hbase:meta table:
> {code}
> {NAME => 'info', VERSIONS => '3', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
> CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
> 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'NONE', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'true', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '81
> 92'}
> ...
> {NAME => 'table', VERSIONS => '10', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
> CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
> 'FOREVER', MIN_VERSIONS => '0'
> , REPLICATION_SCOPE => '0', BLOOMFILTER => 'NONE', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'true', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '
> 8192'}
> {code}
> You can see that 'table' family has MAX VERSIONS much higher than the other
> families.
> The MAX VERSIONS value should be brought in sync with the other families.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19806) Lower max versions for table column family of hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-19806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338717#comment-16338717
]
Ted Yu commented on HBASE-19806:
namespace entries should not change much.
The current MAX VERSIONS of 10 is high.
I can prepare a patch (in a new issue) where namespace descriptor is created in
TableNamespaceManager#createNamespaceTable() so that we can check config for
'hbase.namespace.versions'
What do you think ?
> Lower max versions for table column family of hbase:meta
>
>
> Key: HBASE-19806
> URL: https://issues.apache.org/jira/browse/HBASE-19806
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 2.0.0-beta-2
>
> Attachments: 19806.v1.txt
>
>
> On an hbase 2 cluster, I got the description of hbase:meta table:
> {code}
> {NAME => 'info', VERSIONS => '3', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
> CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
> 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'NONE', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'true', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '81
> 92'}
> ...
> {NAME => 'table', VERSIONS => '10', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
> CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
> 'FOREVER', MIN_VERSIONS => '0'
> , REPLICATION_SCOPE => '0', BLOOMFILTER => 'NONE', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'true', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '
> 8192'}
> {code}
> You can see that 'table' family has MAX VERSIONS much higher than the other
> families.
> The MAX VERSIONS value should be brought in sync with the other families.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19806) Lower max versions for table column family of hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-19806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338715#comment-16338715
]
Anoop Sam John commented on HBASE-19806:
The config 'hbase.meta.versions' looks to be specific for META. So I dont
think same to be applied to NS table also. But the versions change for NS
table is very relevant as for META? For META entries, it is expected to get
more versions as the time progress and region moves etc. NS entries will get
changed much? WDYT Ted?
> Lower max versions for table column family of hbase:meta
>
>
> Key: HBASE-19806
> URL: https://issues.apache.org/jira/browse/HBASE-19806
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 2.0.0-beta-2
>
> Attachments: 19806.v1.txt
>
>
> On an hbase 2 cluster, I got the description of hbase:meta table:
> {code}
> {NAME => 'info', VERSIONS => '3', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
> CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
> 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'NONE', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'true', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '81
> 92'}
> ...
> {NAME => 'table', VERSIONS => '10', EVICT_BLOCKS_ON_CLOSE => 'false',
> NEW_VERSION_BEHAVIOR => 'false', KEEP_DELETED_CELLS => 'FALSE',
> CACHE_DATA_ON_WRITE => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL =>
> 'FOREVER', MIN_VERSIONS => '0'
> , REPLICATION_SCOPE => '0', BLOOMFILTER => 'NONE', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'true', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '
> 8192'}
> {code}
> You can see that 'table' family has MAX VERSIONS much higher than the other
> families.
> The MAX VERSIONS value should be brought in sync with the other families.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-19860) Run branch-2 nightly job more frequently
[ https://issues.apache.org/jira/browse/HBASE-19860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-19860. -- Resolution: Fixed > Run branch-2 nightly job more frequently > > > Key: HBASE-19860 > URL: https://issues.apache.org/jira/browse/HBASE-19860 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy >Priority: Minor > Attachments: HBASE-19860.branch-2.001.patch > > > For time being, running the job more frequently will give us more data faster > to debug flaky tests . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19860) Run branch-2 nightly job more frequently
[ https://issues.apache.org/jira/browse/HBASE-19860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338699#comment-16338699 ] Appy commented on HBASE-19860: -- fyi [~stack] > Run branch-2 nightly job more frequently > > > Key: HBASE-19860 > URL: https://issues.apache.org/jira/browse/HBASE-19860 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy >Priority: Minor > Attachments: HBASE-19860.branch-2.001.patch > > > For time being, running the job more frequently will give us more data faster > to debug flaky tests . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19860) Run branch-2 nightly job more frequently
[ https://issues.apache.org/jira/browse/HBASE-19860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19860: - Attachment: HBASE-19860.branch-2.001.patch > Run branch-2 nightly job more frequently > > > Key: HBASE-19860 > URL: https://issues.apache.org/jira/browse/HBASE-19860 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy >Priority: Minor > Attachments: HBASE-19860.branch-2.001.patch > > > For time being, running the job more frequently will give us more data faster > to debug flaky tests . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19860) Run branch-2 nightly job more frequently
[ https://issues.apache.org/jira/browse/HBASE-19860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338697#comment-16338697 ] Appy commented on HBASE-19860: -- Syntax ref: https://jenkins.io/doc/book/pipeline/syntax/#triggers > Run branch-2 nightly job more frequently > > > Key: HBASE-19860 > URL: https://issues.apache.org/jira/browse/HBASE-19860 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy >Priority: Minor > Attachments: HBASE-19860.branch-2.001.patch > > > For time being, running the job more frequently will give us more data faster > to debug flaky tests . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19860) Run branch-2 nightly job more frequently
Appy created HBASE-19860: Summary: Run branch-2 nightly job more frequently Key: HBASE-19860 URL: https://issues.apache.org/jira/browse/HBASE-19860 Project: HBase Issue Type: Improvement Reporter: Appy Assignee: Appy For time being, running the job more frequently will give us more data faster to debug flaky tests . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18415) The local timeout may cause Admin to submit duplicate request
[
https://issues.apache.org/jira/browse/HBASE-18415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338662#comment-16338662
]
Hadoop QA commented on HBASE-18415:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 15m
41s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
22s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} branch-1 passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} branch-1 passed with JDK v1.7.0_161 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
25s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
17s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
29s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} branch-1 passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} branch-1 passed with JDK v1.7.0_161 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} the patch passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed with JDK v1.7.0_161 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
49s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 16s{color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.5 2.7.4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} the patch passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} the patch passed with JDK v1.7.0_161 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}120m 3s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}165m 10s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestClientScannerRPCTimeout |
| | hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles |
| | hadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:36a7029 |
| JIRA Issue | HBASE-18415 |
| JIRA Patch URL |
[jira] [Comment Edited] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)
[
https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338635#comment-16338635
]
Vladimir Rodionov edited comment on HBASE-17852 at 1/25/18 2:43 AM:
[~appy] wrote:
{quote}Of the top of my head, I think the main areas to touch upon are:
- Make backups concurrent
- Use procedure framework: Long-standing request. The procv2 framework has
features like locking, queuing operations, etc. Replication is already moving
to it. I don't see a reason why backup can't too.
- Can't use CP hooks for incremental backup. Backup should/will become first
class feature - more important and critical than Coprocessor.
- There should be some basic access control, if only, limiting everything to
ADMIN (like RS group recently did in HBASE-19483){quote}
OK,
h4. Concurrent backups
It is doable, but ...
# Will require transaction management support - it complicates implementations
a lot. We will need to provide full isolation of operations and complex
conflict resolutions on commit. And rollback?
# Complicates testing, as well - a lot. Imagine all different possible
collisions between create, merge, delete sessions
What I suggest is a slightly different approach:
# Make restore operations concurrent
# Implement fair queuing for *create-merge-delete* sessions
# *create-merge-restore* executions will be serialized (one-by-one), but from
user's point of view they will run, kind of, in parallel.
YES/NO
h4. Use procedure framework
Short answer - no. I will wait until procv2 becomes more mature and robust. I
do not want to build new feature on a foundation of a new feature. Too risky in
my opinion. NO
h4. Can't use CP hooks for incremental backup
Currently backup lives in a separate module and we would like to keep it there.
There is no need for the tight integration of a HBase core and backup and
therefore, CP is the only our option here. NO
h4. Access control
Currently, only ADMIN can run backups/restore/delete/merge operations, but we
do not enforce this explicitly, so we should probably, do the access right
check *before* starting critical operation. YES.
[~appy], [~elserj] - comments?
was (Author: vrodionov):
[~appy] wrote:
{quote}
Of the top of my head, I think the main areas to touch upon are:
- Make backups concurrent
- Use procedure framework: Long-standing request. The procv2 framework has
features like locking, queuing operations, etc. Replication is already moving
to it. I don't see a reason why backup can't too.
- Can't use CP hooks for incremental backup. Backup should/will become first
class feature - more important and critical than Coprocessor.
- There should be some basic access control, if only, limiting everything to
ADMIN (like RS group recently did in HBASE-19483)
{quote}
OK,
h4. Concurrent backups
It is doable, but ...
# Will require transaction management support - it complicates implementations
a lot. We will need to provide full isolation of operations and complex
conflict resolutions on commit. And rollback?
# Complicates testing, as well - a lot. Imagine all different possible
collisions between create, merge, delete sessions
What I suggest is a slightly different approach:
# Make restore operations concurrent
# Implement fair queuing for *create-merge-delete* sessions
# *create-merge-restore* executions will be serialized (one-by-one), but from
user's point of view they will run, kind of, in parallel.
YES/NO
h4. Use procedure framework
Short answer - no. I will wait until procv2 becomes more mature and robust. I
do not want to build new feature on a foundation of a new feature. Too risky in
my opinion. NO
h4. Can't use CP hooks for incremental backup
Currently backup lives in a separate module and we would like to keep it there.
There is no need for the tight integration of a HBase core and backup and
therefore, CP is the only our option here. NO
h4. Access control
Currently, only ADMIN can run backups/restore/delete/merge operations, but we
do not enforce this explicitly, so we should probably, do the access right
check *before* starting critical operation. YES.
[~appy], [~elserj] - comments?
> Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental
> backup)
>
>
> Key: HBASE-17852
> URL: https://issues.apache.org/jira/browse/HBASE-17852
> Project: HBase
> Issue Type: Sub-task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-17852-v10.patch, screenshot-1.png
>
>
> Design approach rollback-via-snapshot implemented in this ticket:
> # Before backup create/delete/merge starts we take
[jira] [Commented] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)
[
https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338635#comment-16338635
]
Vladimir Rodionov commented on HBASE-17852:
---
[~appy] wrote:
{quote}
Of the top of my head, I think the main areas to touch upon are:
- Make backups concurrent
- Use procedure framework: Long-standing request. The procv2 framework has
features like locking, queuing operations, etc. Replication is already moving
to it. I don't see a reason why backup can't too.
- Can't use CP hooks for incremental backup. Backup should/will become first
class feature - more important and critical than Coprocessor.
- There should be some basic access control, if only, limiting everything to
ADMIN (like RS group recently did in HBASE-19483)
{quote}
OK,
h4. Concurrent backups
It is doable, but ...
# Will require transaction management support - it complicates implementations
a lot. We will need to provide full isolation of operations and complex
conflict resolutions on commit. And rollback?
# Complicates testing, as well - a lot. Imagine all different possible
collisions between create, merge, delete sessions
What I suggest is a slightly different approach:
# Make restore operations concurrent
# Implement fair queuing for *create-merge-delete* sessions
# *create-merge-restore* executions will be serialized (one-by-one), but from
user's point of view they will run, kind of, in parallel.
YES/NO
h4. Use procedure framework
Short answer - no. I will wait until procv2 becomes more mature and robust. I
do not want to build new feature on a foundation of a new feature. Too risky in
my opinion. NO
h4. Can't use CP hooks for incremental backup
Currently backup lives in a separate module and we would like to keep it there.
There is no need for the tight integration of a HBase core and backup and
therefore, CP is the only our option here. NO
h4. Access control
Currently, only ADMIN can run backups/restore/delete/merge operations, but we
do not enforce this explicitly, so we should probably, do the access right
check *before* starting critical operation. YES.
[~appy], [~elserj] - comments?
> Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental
> backup)
>
>
> Key: HBASE-17852
> URL: https://issues.apache.org/jira/browse/HBASE-17852
> Project: HBase
> Issue Type: Sub-task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-17852-v10.patch, screenshot-1.png
>
>
> Design approach rollback-via-snapshot implemented in this ticket:
> # Before backup create/delete/merge starts we take a snapshot of the backup
> meta-table (backup system table). This procedure is lightweight because meta
> table is small, usually should fit a single region.
> # When operation fails on a server side, we handle this failure by cleaning
> up partial data in backup destination, followed by restoring backup
> meta-table from a snapshot.
> # When operation fails on a client side (abnormal termination, for example),
> next time user will try create/merge/delete he(she) will see error message,
> that system is in inconsistent state and repair is required, he(she) will
> need to run backup repair tool.
> # To avoid multiple writers to the backup system table (backup client and
> BackupObserver's) we introduce small table ONLY to keep listing of bulk
> loaded files. All backup observers will work only with this new tables. The
> reason: in case of a failure during backup create/delete/merge/restore, when
> system performs automatic rollback, some data written by backup observers
> during failed operation may be lost. This is what we try to avoid.
> # Second table keeps only bulk load related references. We do not care about
> consistency of this table, because bulk load is idempotent operation and can
> be repeated after failure. Partially written data in second table does not
> affect on BackupHFileCleaner plugin, because this data (list of bulk loaded
> files) correspond to a files which have not been loaded yet successfully and,
> hence - are not visible to the system
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-17079) HBase build fails on windows, hbase-archetype-builder is reason for failure
[
https://issues.apache.org/jira/browse/HBASE-17079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338629#comment-16338629
]
Hadoop QA commented on HBASE-17079:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1.3 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
42s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
55s{color} | {color:green} branch-1.3 passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} branch-1.3 passed with JDK v1.7.0_161 {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 8m
23s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
35s{color} | {color:green} branch-1.3 passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
24s{color} | {color:green} branch-1.3 passed with JDK v1.7.0_161 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed with JDK v1.7.0_161 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} xml {color} | {color:red} 0m 1s{color} |
{color:red} The patch has 2 ill-formed XML file(s). {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
14s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 53s{color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.5 2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
33s{color} | {color:green} the patch passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
17s{color} | {color:green} the patch passed with JDK v1.7.0_161 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 99m 27s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}134m 25s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestEndToEndSplitTransaction |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:dca6535 |
| JIRA Issue | HBASE-17079 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12896093/HBASE-17079.branch-1.3.002.patch
|
| Optional Tests | asflicense javac javadoc unit shadedjars hadoopcheck
xml compile |
| uname | Linux c0e8b0ec417d 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64
[jira] [Resolved] (HBASE-17883) release 1.4.0
[ https://issues.apache.org/jira/browse/HBASE-17883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-17883. Resolution: Fixed > release 1.4.0 > - > > Key: HBASE-17883 > URL: https://issues.apache.org/jira/browse/HBASE-17883 > Project: HBase > Issue Type: Task > Components: community >Affects Versions: 1.4.0 >Reporter: Sean Busbey >Assignee: Andrew Purtell >Priority: Critical > > Let's start working through doing the needful; it's been almost 3 months sine > 1.3.0. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19803) False positive for the HBASE-Find-Flaky-Tests job
[ https://issues.apache.org/jira/browse/HBASE-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338603#comment-16338603 ] Appy commented on HBASE-19803: -- Arghh, setting -XX:ErrorFile doesn't work with surefire. Apparently coredump uses the same output stream as required by maven, this results in following issue: http://maven.apache.org/components/surefire/maven-failsafe-plugin/faq.html#corruptedstream The results file is [^2018-01-24T17-45-37_000-jvmRun1.dumpstream] (generated it by running a test and manually killing it using kill -9) It's not ideal, but it's better than nothing. Ptal at [^HBASE-19803.master.001.patch] . Let's get it in and wait for more failures? [~Apache9] [~stack] > False positive for the HBASE-Find-Flaky-Tests job > - > > Key: HBASE-19803 > URL: https://issues.apache.org/jira/browse/HBASE-19803 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Priority: Major > Attachments: 2018-01-24T17-45-37_000-jvmRun1.dumpstream, > HBASE-19803.master.001.patch > > > It reports two hangs for TestAsyncTableGetMultiThreaded, but I checked the > surefire output > https://builds.apache.org/job/HBASE-Flaky-Tests/24830/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was likely to be killed in the middle of the run within 20 seconds. > https://builds.apache.org/job/HBASE-Flaky-Tests/24852/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was also killed within about 1 minutes. > The test is declared as LargeTests so the time limit should be 10 minutes. It > seems that the jvm may crash during the mvn test run and then we will kill > all the running tests and then we may mark some of them as hang which leads > to the false positive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19803) False positive for the HBASE-Find-Flaky-Tests job
[ https://issues.apache.org/jira/browse/HBASE-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19803: - Attachment: HBASE-19803.master.001.patch > False positive for the HBASE-Find-Flaky-Tests job > - > > Key: HBASE-19803 > URL: https://issues.apache.org/jira/browse/HBASE-19803 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Priority: Major > Attachments: 2018-01-24T17-45-37_000-jvmRun1.dumpstream, > HBASE-19803.master.001.patch > > > It reports two hangs for TestAsyncTableGetMultiThreaded, but I checked the > surefire output > https://builds.apache.org/job/HBASE-Flaky-Tests/24830/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was likely to be killed in the middle of the run within 20 seconds. > https://builds.apache.org/job/HBASE-Flaky-Tests/24852/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was also killed within about 1 minutes. > The test is declared as LargeTests so the time limit should be 10 minutes. It > seems that the jvm may crash during the mvn test run and then we will kill > all the running tests and then we may mark some of them as hang which leads > to the false positive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19803) False positive for the HBASE-Find-Flaky-Tests job
[ https://issues.apache.org/jira/browse/HBASE-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19803: - Attachment: 2018-01-24T17-45-37_000-jvmRun1.dumpstream > False positive for the HBASE-Find-Flaky-Tests job > - > > Key: HBASE-19803 > URL: https://issues.apache.org/jira/browse/HBASE-19803 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Priority: Major > Attachments: 2018-01-24T17-45-37_000-jvmRun1.dumpstream > > > It reports two hangs for TestAsyncTableGetMultiThreaded, but I checked the > surefire output > https://builds.apache.org/job/HBASE-Flaky-Tests/24830/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was likely to be killed in the middle of the run within 20 seconds. > https://builds.apache.org/job/HBASE-Flaky-Tests/24852/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was also killed within about 1 minutes. > The test is declared as LargeTests so the time limit should be 10 minutes. It > seems that the jvm may crash during the mvn test run and then we will kill > all the running tests and then we may mark some of them as hang which leads > to the false positive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19803) False positive for the HBASE-Find-Flaky-Tests job
[
https://issues.apache.org/jira/browse/HBASE-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338581#comment-16338581
]
Appy commented on HBASE-19803:
--
So at this point, main thing to demystify is - is it system.exit calls that's
screwing us up, or some failed native calls crashing jvm.
Either way, setting jvm option {{-XX:ErrorFile}} and archiving that location is
the way to go because:
- If we don't see that file in artifacts, then it should have been system.exit
call
- If we do see it, then it's likely some native call and we can use the error
file for further debug.
> False positive for the HBASE-Find-Flaky-Tests job
> -
>
> Key: HBASE-19803
> URL: https://issues.apache.org/jira/browse/HBASE-19803
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Priority: Major
>
> It reports two hangs for TestAsyncTableGetMultiThreaded, but I checked the
> surefire output
> https://builds.apache.org/job/HBASE-Flaky-Tests/24830/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt
> This one was likely to be killed in the middle of the run within 20 seconds.
> https://builds.apache.org/job/HBASE-Flaky-Tests/24852/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt
> This one was also killed within about 1 minutes.
> The test is declared as LargeTests so the time limit should be 10 minutes. It
> seems that the jvm may crash during the mvn test run and then we will kill
> all the running tests and then we may mark some of them as hang which leads
> to the false positive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-19803) False positive for the HBASE-Find-Flaky-Tests job
[
https://issues.apache.org/jira/browse/HBASE-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338516#comment-16338516
]
Appy edited comment on HBASE-19803 at 1/25/18 1:24 AM:
---
So that's because we set
{{${surefire.timeout}}}
where surefire.timeout is 900 i.e. 15 min.
I tried setting it to 30 sec, but the tests still keep on running :-/
Edit: So it makes sense, the timeout is to kill rouge forked jvms which don't
shutdown themselves. Whereas in our case, jvm goes away without returned the
result to surefire - totally different thing.
was (Author: appy):
So that's because we set
{{${surefire.timeout}}}
where surefire.timeout is 900 i.e. 15 min.
I tried setting it to 30 sec, but the tests still keep on running :-/
> False positive for the HBASE-Find-Flaky-Tests job
> -
>
> Key: HBASE-19803
> URL: https://issues.apache.org/jira/browse/HBASE-19803
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Priority: Major
>
> It reports two hangs for TestAsyncTableGetMultiThreaded, but I checked the
> surefire output
> https://builds.apache.org/job/HBASE-Flaky-Tests/24830/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt
> This one was likely to be killed in the middle of the run within 20 seconds.
> https://builds.apache.org/job/HBASE-Flaky-Tests/24852/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt
> This one was also killed within about 1 minutes.
> The test is declared as LargeTests so the time limit should be 10 minutes. It
> seems that the jvm may crash during the mvn test run and then we will kill
> all the running tests and then we may mark some of them as hang which leads
> to the false positive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-19803) False positive for the HBASE-Find-Flaky-Tests job
[
https://issues.apache.org/jira/browse/HBASE-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338516#comment-16338516
]
Appy edited comment on HBASE-19803 at 1/25/18 1:24 AM:
---
So that's because we set
{{${surefire.timeout}}}
where surefire.timeout is 900 i.e. 15 min.
I tried setting it to 30 sec, but the tests still keep on running :-/
{color:red}Edit{color}: So it makes sense, the timeout is to kill rouge forked
jvms which don't shutdown themselves. Whereas in our case, jvm goes away
without returned the result to surefire - totally different thing.
was (Author: appy):
So that's because we set
{{${surefire.timeout}}}
where surefire.timeout is 900 i.e. 15 min.
I tried setting it to 30 sec, but the tests still keep on running :-/
Edit: So it makes sense, the timeout is to kill rouge forked jvms which don't
shutdown themselves. Whereas in our case, jvm goes away without returned the
result to surefire - totally different thing.
> False positive for the HBASE-Find-Flaky-Tests job
> -
>
> Key: HBASE-19803
> URL: https://issues.apache.org/jira/browse/HBASE-19803
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Priority: Major
>
> It reports two hangs for TestAsyncTableGetMultiThreaded, but I checked the
> surefire output
> https://builds.apache.org/job/HBASE-Flaky-Tests/24830/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt
> This one was likely to be killed in the middle of the run within 20 seconds.
> https://builds.apache.org/job/HBASE-Flaky-Tests/24852/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt
> This one was also killed within about 1 minutes.
> The test is declared as LargeTests so the time limit should be 10 minutes. It
> seems that the jvm may crash during the mvn test run and then we will kill
> all the running tests and then we may mark some of them as hang which leads
> to the false positive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-19803) False positive for the HBASE-Find-Flaky-Tests job
[
https://issues.apache.org/jira/browse/HBASE-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338521#comment-16338521
]
Appy edited comment on HBASE-19803 at 1/25/18 1:23 AM:
---
So to iterate faster in trying to understand what/when/how this happens, i
switched to hbase-http module and made following two changes:
- Added System.exit(0) at the end of all tests of TestConfServlet (since it
runs first)
- Added 5 sec sleep to TestHttpServer so i can {{kill -9 }}it when it's running
Some important mvn props:
reuseForks=false (that's our default in root pom)
surefire.firstPartForkCount = 1, so next jvm is not started until previous one
ends.
It's interesting that the error is thrown only after all tests finish running,
and not immediately when some test calls System.exit(0) or some test jvm
crashes.
{noformat}
$ mvn test -PrunAllTests -pl hbase-http
---
T E S T S
---
Running org.apache.hadoop.hbase.http.conf.TestConfServlet
Running org.apache.hadoop.hbase.http.jmx.TestJMXJsonServlet
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.63 s - in
org.apache.hadoop.hbase.http.jmx.TestJMXJsonServlet
Running org.apache.hadoop.hbase.http.lib.TestStaticUserWebFilter
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.338 s - in
org.apache.hadoop.hbase.http.lib.TestStaticUserWebFilter
Running org.apache.hadoop.hbase.http.log.TestLogLevel
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.73 s - in
org.apache.hadoop.hbase.http.log.TestLogLevel
Running org.apache.hadoop.hbase.http.TestGlobalFilter
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.724 s - in
org.apache.hadoop.hbase.http.TestGlobalFilter
Running org.apache.hadoop.hbase.http.TestHtmlQuoting
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.143 s - in
org.apache.hadoop.hbase.http.TestHtmlQuoting
Running org.apache.hadoop.hbase.http.TestHttpRequestLog
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.376 s - in
org.apache.hadoop.hbase.http.TestHttpRequestLog
Running org.apache.hadoop.hbase.http.TestHttpRequestLogAppender
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.223 s - in
org.apache.hadoop.hbase.http.TestHttpRequestLogAppender
Running org.apache.hadoop.hbase.http.TestHttpServer
Running org.apache.hadoop.hbase.http.TestHttpServerLifecycle
Tests run: 6, Failures: 0, Errors: 0, Skipped: 6, Time elapsed: 0.002 s - in
org.apache.hadoop.hbase.http.TestHttpServerLifecycle
Running org.apache.hadoop.hbase.http.TestHttpServerWebapps
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.546 s - in
org.apache.hadoop.hbase.http.TestHttpServerWebapps
Running org.apache.hadoop.hbase.http.TestPathFilter
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.766 s - in
org.apache.hadoop.hbase.http.TestPathFilter
Running org.apache.hadoop.hbase.http.TestServletFilter
Tests run: 3, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.757 s - in
org.apache.hadoop.hbase.http.TestServletFilter
Running org.apache.hadoop.hbase.http.TestSpnegoHttpServer
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.651 s - in
org.apache.hadoop.hbase.http.TestSpnegoHttpServer
Running org.apache.hadoop.hbase.http.TestSSLHttpServer
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.389 s - in
org.apache.hadoop.hbase.http.TestSSLHttpServer
Results:
Tests run: 31, Failures: 0, Errors: 0, Skipped: 7
BUILD FAILURE
Total time: 41.684 s
Finished at: 2018-01-24T17:17:36-08:00
Final Memory: 77M/867M
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.20.1:test (default-test) on
project hbase-http: There are test failures.
[ERROR]
[ERROR] Please refer to
/Users/appy/apache/hbase/hbase-http/target/surefire-reports for the individual
test results.
[ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump,
[date].dumpstream and [date]-jvmRun[N].dumpstream.
[ERROR] ExecutionException The forked VM terminated without properly saying
goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd /Users/appy/apache/hbase/hbase-http &&
/Library/Java/JavaVirtualMachines/jdk1.8.0_152.jdk/Contents/Home/jre/bin/java
-enableassertions -Dhbase.build.id=2018-01-25T01:16:55Z -Xmx2800m
-Djava.security.egd=file:/dev/./urandom -Djava.net.preferIPv4Stack=true
-Djava.awt.headless=true
-XX:ErrorFile=/Users/appy/apache/hbase/hbase-http/target/classes/hs_err_pid%p.log
-jar
/Users/appy/apache/hbase/hbase-http/target/surefire/surefirebooter1670234724455998120.jar
[jira] [Commented] (HBASE-19756) Master NPE during completed failed proc eviction
[
https://issues.apache.org/jira/browse/HBASE-19756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338534#comment-16338534
]
Hudson commented on HBASE-19756:
SUCCESS: Integrated in Jenkins build HBase-1.3-IT #341 (See
[https://builds.apache.org/job/HBase-1.3-IT/341/])
HBASE-19756 Master NPE during completed failed proc eviction (apurtell: rev
a02b3d9f7e340863fcfc10c7e1f3880fa05ce839)
* (edit)
hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java
* (add)
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestNonceProcCleanerOnFailure.java
> Master NPE during completed failed proc eviction
>
>
> Key: HBASE-19756
> URL: https://issues.apache.org/jira/browse/HBASE-19756
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.0, 1.3.1
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.3.2, 1.4.1, 1.5.0
>
> Attachments: HBASE-19756.branch-1.4.001.patch,
> HBASE-19756.branch-1.4.002.patch, HBASE-19756.branch-1.4.003.patch,
> HBASE-19756.master.001.patch
>
>
> When procedures like Create table fails due to say AccessDeniedException,
> then a rollback procedure is created. When the rollback is being cleaned up,
> it results in an NPE because those nonce procs aren't persisted
> Stack trace when this happens:
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.procedure2.store.ProcedureStoreTracker.delete(ProcedureStoreTracker.java:385)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.updateStoreTracker(WALProcedureStore.java:547)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.pushData(WALProcedureStore.java:504)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.delete(WALProcedureStore.java:453)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$CompletedProcedureCleaner.periodicExecute(ProcedureExecutor.java:184)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.timeoutLoop(ProcedureExecutor.java:995)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$500(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$3.run(ProcedureExecutor.java:507)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HBASE-19859) Update download page header for 1.1 EOL
[ https://issues.apache.org/jira/browse/HBASE-19859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-19859 started by Mike Drob. - > Update download page header for 1.1 EOL > --- > > Key: HBASE-19859 > URL: https://issues.apache.org/jira/browse/HBASE-19859 > Project: HBase > Issue Type: Task >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Attachments: HBASE-19583.patch > > > See example mirror: http://mirrors.ocf.berkeley.edu/apache/hbase/ > They still claim that 1.1 is under active development. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19859) Update download page header for 1.1 EOL
[ https://issues.apache.org/jira/browse/HBASE-19859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338527#comment-16338527 ] Mike Drob commented on HBASE-19859: --- Not marking as patch available because Yetus QA is going to fail on it anyway. > Update download page header for 1.1 EOL > --- > > Key: HBASE-19859 > URL: https://issues.apache.org/jira/browse/HBASE-19859 > Project: HBase > Issue Type: Task >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Attachments: HBASE-19583.patch > > > See example mirror: http://mirrors.ocf.berkeley.edu/apache/hbase/ > They still claim that 1.1 is under active development. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-19859) Update download page header for 1.1 EOL
[ https://issues.apache.org/jira/browse/HBASE-19859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob reassigned HBASE-19859: - Assignee: Mike Drob > Update download page header for 1.1 EOL > --- > > Key: HBASE-19859 > URL: https://issues.apache.org/jira/browse/HBASE-19859 > Project: HBase > Issue Type: Task >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Attachments: HBASE-19583.patch > > > See example mirror: http://mirrors.ocf.berkeley.edu/apache/hbase/ > They still claim that 1.1 is under active development. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19859) Update download page header for 1.1 EOL
[ https://issues.apache.org/jira/browse/HBASE-19859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-19859: -- Attachment: HBASE-19583.patch > Update download page header for 1.1 EOL > --- > > Key: HBASE-19859 > URL: https://issues.apache.org/jira/browse/HBASE-19859 > Project: HBase > Issue Type: Task >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Attachments: HBASE-19583.patch > > > See example mirror: http://mirrors.ocf.berkeley.edu/apache/hbase/ > They still claim that 1.1 is under active development. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19859) Update download page header for 1.1 EOL
Mike Drob created HBASE-19859: - Summary: Update download page header for 1.1 EOL Key: HBASE-19859 URL: https://issues.apache.org/jira/browse/HBASE-19859 Project: HBase Issue Type: Task Reporter: Mike Drob See example mirror: http://mirrors.ocf.berkeley.edu/apache/hbase/ They still claim that 1.1 is under active development. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-19803) False positive for the HBASE-Find-Flaky-Tests job
[
https://issues.apache.org/jira/browse/HBASE-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338521#comment-16338521
]
Appy edited comment on HBASE-19803 at 1/25/18 1:09 AM:
---
So to iterate faster in trying to understand what/when/how this happens, i
switched to hbase-http module, added System.exit(0) at the end of
--TestGlobalFilter#testServletFilter.-- all tests of TestConfServlet (since it
runs first)
forkedProcessTimeoutInSeconds=10sec (should be fine since no http test takes
more than 4 sec on my machine, see list below)
reuseForks=false (that's our default in root pom)
{noformat}
---
T E S T S
---
Running org.apache.hadoop.hbase.http.conf.TestConfServlet
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.756 s - in
org.apache.hadoop.hbase.http.conf.TestConfServlet
Running org.apache.hadoop.hbase.http.jmx.TestJMXJsonServlet
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.806 s - in
org.apache.hadoop.hbase.http.jmx.TestJMXJsonServlet
Running org.apache.hadoop.hbase.http.lib.TestStaticUserWebFilter
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.274 s - in
org.apache.hadoop.hbase.http.lib.TestStaticUserWebFilter
Running org.apache.hadoop.hbase.http.log.TestLogLevel
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.786 s - in
org.apache.hadoop.hbase.http.log.TestLogLevel
Running org.apache.hadoop.hbase.http.TestGlobalFilter
Running org.apache.hadoop.hbase.http.TestHtmlQuoting
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.291 s - in
org.apache.hadoop.hbase.http.TestHtmlQuoting
Running org.apache.hadoop.hbase.http.TestHttpRequestLog
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.415 s - in
org.apache.hadoop.hbase.http.TestHttpRequestLog
Running org.apache.hadoop.hbase.http.TestHttpRequestLogAppender
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.253 s - in
org.apache.hadoop.hbase.http.TestHttpRequestLogAppender
Running org.apache.hadoop.hbase.http.TestHttpServer
Tests run: 15, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 3.275 s - in
org.apache.hadoop.hbase.http.TestHttpServer
Running org.apache.hadoop.hbase.http.TestHttpServerLifecycle
Tests run: 6, Failures: 0, Errors: 0, Skipped: 6, Time elapsed: 0.002 s - in
org.apache.hadoop.hbase.http.TestHttpServerLifecycle
Running org.apache.hadoop.hbase.http.TestHttpServerWebapps
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.557 s - in
org.apache.hadoop.hbase.http.TestHttpServerWebapps
Running org.apache.hadoop.hbase.http.TestPathFilter
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.71 s - in
org.apache.hadoop.hbase.http.TestPathFilter
Running org.apache.hadoop.hbase.http.TestServletFilter
Tests run: 3, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.82 s - in
org.apache.hadoop.hbase.http.TestServletFilter
Running org.apache.hadoop.hbase.http.TestSpnegoHttpServer
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.881 s - in
org.apache.hadoop.hbase.http.TestSpnegoHttpServer
Running org.apache.hadoop.hbase.http.TestSSLHttpServer
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.33 s - in
org.apache.hadoop.hbase.http.TestSSLHttpServer
Results:
Tests run: 48, Failures: 0, Errors: 0, Skipped: 9
{noformat}
was (Author: appy):
So to iterate faster in trying to understand what/when/how this happens, i
switched to hbase-http module, added System.exit(0) at the end of
TestGlobalFilter#testServletFilter.
forkedProcessTimeoutInSeconds=10sec (should be fine since no http test takes
more than 4 sec on my machine, see list below)
reuseForks=false (that's our default in root pom)
{noformat}
---
T E S T S
---
Running org.apache.hadoop.hbase.http.conf.TestConfServlet
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.756 s - in
org.apache.hadoop.hbase.http.conf.TestConfServlet
Running org.apache.hadoop.hbase.http.jmx.TestJMXJsonServlet
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.806 s - in
org.apache.hadoop.hbase.http.jmx.TestJMXJsonServlet
Running org.apache.hadoop.hbase.http.lib.TestStaticUserWebFilter
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.274 s - in
org.apache.hadoop.hbase.http.lib.TestStaticUserWebFilter
Running org.apache.hadoop.hbase.http.log.TestLogLevel
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.786 s - in
org.apache.hadoop.hbase.http.log.TestLogLevel
Running org.apache.hadoop.hbase.http.TestGlobalFilter
Running org.apache.hadoop.hbase.http.TestHtmlQuoting
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.291 s - in
[jira] [Commented] (HBASE-19803) False positive for the HBASE-Find-Flaky-Tests job
[
https://issues.apache.org/jira/browse/HBASE-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338521#comment-16338521
]
Appy commented on HBASE-19803:
--
So to iterate faster in trying to understand what/when/how this happens, i
switched to hbase-http module, added System.exit(0) at the end of
TestGlobalFilter#testServletFilter.
forkedProcessTimeoutInSeconds=10sec (should be fine since no http test takes
more than 4 sec on my machine, see list below)
reuseForks=false (that's our default in root pom)
{noformat}
---
T E S T S
---
Running org.apache.hadoop.hbase.http.conf.TestConfServlet
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.756 s - in
org.apache.hadoop.hbase.http.conf.TestConfServlet
Running org.apache.hadoop.hbase.http.jmx.TestJMXJsonServlet
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.806 s - in
org.apache.hadoop.hbase.http.jmx.TestJMXJsonServlet
Running org.apache.hadoop.hbase.http.lib.TestStaticUserWebFilter
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.274 s - in
org.apache.hadoop.hbase.http.lib.TestStaticUserWebFilter
Running org.apache.hadoop.hbase.http.log.TestLogLevel
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.786 s - in
org.apache.hadoop.hbase.http.log.TestLogLevel
Running org.apache.hadoop.hbase.http.TestGlobalFilter
Running org.apache.hadoop.hbase.http.TestHtmlQuoting
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.291 s - in
org.apache.hadoop.hbase.http.TestHtmlQuoting
Running org.apache.hadoop.hbase.http.TestHttpRequestLog
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.415 s - in
org.apache.hadoop.hbase.http.TestHttpRequestLog
Running org.apache.hadoop.hbase.http.TestHttpRequestLogAppender
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.253 s - in
org.apache.hadoop.hbase.http.TestHttpRequestLogAppender
Running org.apache.hadoop.hbase.http.TestHttpServer
Tests run: 15, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 3.275 s - in
org.apache.hadoop.hbase.http.TestHttpServer
Running org.apache.hadoop.hbase.http.TestHttpServerLifecycle
Tests run: 6, Failures: 0, Errors: 0, Skipped: 6, Time elapsed: 0.002 s - in
org.apache.hadoop.hbase.http.TestHttpServerLifecycle
Running org.apache.hadoop.hbase.http.TestHttpServerWebapps
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.557 s - in
org.apache.hadoop.hbase.http.TestHttpServerWebapps
Running org.apache.hadoop.hbase.http.TestPathFilter
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.71 s - in
org.apache.hadoop.hbase.http.TestPathFilter
Running org.apache.hadoop.hbase.http.TestServletFilter
Tests run: 3, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.82 s - in
org.apache.hadoop.hbase.http.TestServletFilter
Running org.apache.hadoop.hbase.http.TestSpnegoHttpServer
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.881 s - in
org.apache.hadoop.hbase.http.TestSpnegoHttpServer
Running org.apache.hadoop.hbase.http.TestSSLHttpServer
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.33 s - in
org.apache.hadoop.hbase.http.TestSSLHttpServer
Results:
Tests run: 48, Failures: 0, Errors: 0, Skipped: 9
{noformat}
> False positive for the HBASE-Find-Flaky-Tests job
> -
>
> Key: HBASE-19803
> URL: https://issues.apache.org/jira/browse/HBASE-19803
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Priority: Major
>
> It reports two hangs for TestAsyncTableGetMultiThreaded, but I checked the
> surefire output
> https://builds.apache.org/job/HBASE-Flaky-Tests/24830/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt
> This one was likely to be killed in the middle of the run within 20 seconds.
> https://builds.apache.org/job/HBASE-Flaky-Tests/24852/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt
> This one was also killed within about 1 minutes.
> The test is declared as LargeTests so the time limit should be 10 minutes. It
> seems that the jvm may crash during the mvn test run and then we will kill
> all the running tests and then we may mark some of them as hang which leads
> to the false positive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19803) False positive for the HBASE-Find-Flaky-Tests job
[
https://issues.apache.org/jira/browse/HBASE-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338516#comment-16338516
]
Appy commented on HBASE-19803:
--
So that's because we set
{{${surefire.timeout}}}
where surefire.timeout is 900 i.e. 15 min.
I tried setting it to 30 sec, but the tests still keep on running :-/
> False positive for the HBASE-Find-Flaky-Tests job
> -
>
> Key: HBASE-19803
> URL: https://issues.apache.org/jira/browse/HBASE-19803
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Priority: Major
>
> It reports two hangs for TestAsyncTableGetMultiThreaded, but I checked the
> surefire output
> https://builds.apache.org/job/HBASE-Flaky-Tests/24830/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt
> This one was likely to be killed in the middle of the run within 20 seconds.
> https://builds.apache.org/job/HBASE-Flaky-Tests/24852/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt
> This one was also killed within about 1 minutes.
> The test is declared as LargeTests so the time limit should be 10 minutes. It
> seems that the jvm may crash during the mvn test run and then we will kill
> all the running tests and then we may mark some of them as hang which leads
> to the false positive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-17079) HBase build fails on windows, hbase-archetype-builder is reason for failure
[
https://issues.apache.org/jira/browse/HBASE-17079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338512#comment-16338512
]
Hudson commented on HBASE-17079:
SUCCESS: Integrated in Jenkins build HBase-1.3-IT #340 (See
[https://builds.apache.org/job/HBase-1.3-IT/340/])
HBASE-17079 HBase build fails on windows, hbase-archetype-builder is (apurtell:
rev 40e78e56b93f71b1a55dc7a51431933b96c1168d)
* (edit) pom.xml
* (edit) hbase-archetypes/hbase-archetype-builder/pom.xml
> HBase build fails on windows, hbase-archetype-builder is reason for failure
> ---
>
> Key: HBASE-17079
> URL: https://issues.apache.org/jira/browse/HBASE-17079
> Project: HBase
> Issue Type: Bug
> Components: build
> Environment: windows
>Reporter: Mohammad Arshad
>Assignee: Mohammad Arshad
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.3.2, 1.4.1, 1.5.0
>
> Attachments: HBASE-17079.branch-1.3.002.patch,
> HBASE-17079.branch-1.3.Verification.jpg, HBASE-17079.master.001.patch,
> HBASE-17079.master.002.patch, screenshot-1.png
>
>
> HBase buid fails on windows, hbase-archetype-builder is reason for failure.
> Cygwin is installed so the shell scripts should execute successfully.
> Here is build failure log
> {code}
> [INFO] Apache HBase - Archetype builder ... FAILURE [ 1.014
> s]
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 06:13 min
> [INFO] Finished at: 2016-11-12T18:12:26+05:30
> [INFO] Final Memory: 235M/1012M
> [INFO]
>
> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:exec
> (make-scripts-executable) on project hbase-archetype-builder: Command
> execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :hbase-archetype-builder
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19756) Master NPE during completed failed proc eviction
[
https://issues.apache.org/jira/browse/HBASE-19756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-19756:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 1.5.0
3.0.0
2.0.0
Status: Resolved (was: Patch Available)
> Master NPE during completed failed proc eviction
>
>
> Key: HBASE-19756
> URL: https://issues.apache.org/jira/browse/HBASE-19756
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.0, 1.3.1
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.3.2, 1.4.1, 1.5.0
>
> Attachments: HBASE-19756.branch-1.4.001.patch,
> HBASE-19756.branch-1.4.002.patch, HBASE-19756.branch-1.4.003.patch,
> HBASE-19756.master.001.patch
>
>
> When procedures like Create table fails due to say AccessDeniedException,
> then a rollback procedure is created. When the rollback is being cleaned up,
> it results in an NPE because those nonce procs aren't persisted
> Stack trace when this happens:
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.procedure2.store.ProcedureStoreTracker.delete(ProcedureStoreTracker.java:385)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.updateStoreTracker(WALProcedureStore.java:547)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.pushData(WALProcedureStore.java:504)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.delete(WALProcedureStore.java:453)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$CompletedProcedureCleaner.periodicExecute(ProcedureExecutor.java:184)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.timeoutLoop(ProcedureExecutor.java:995)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$500(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$3.run(ProcedureExecutor.java:507)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19803) False positive for the HBASE-Find-Flaky-Tests job
[
https://issues.apache.org/jira/browse/HBASE-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338498#comment-16338498
]
Appy commented on HBASE-19803:
--
This is weird, trying to see a crash explicitly on my machine, i added
{{System.exit(0)}} to TestEntityLocks#setUp and ran {{mvn test -PrunAllTests
-pl hbase-server}}, but the tests just keep going on for several minutes (at
which point i stop them myself).
{noformat}
---
T E S T S
---
Running org.apache.hadoop.hbase.client.locking.TestEntityLocks
Running org.apache.hadoop.hbase.client.TestAllowPartialScanResultCache
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.293 s - in
org.apache.hadoop.hbase.client.TestAllowPartialScanResultCache
Running org.apache.hadoop.hbase.client.TestBatchScanResultCache
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.285 s - in
org.apache.hadoop.hbase.client.TestBatchScanResultCache
Running org.apache.hadoop.hbase.client.TestCompleteResultScanResultCache
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.183 s - in
org.apache.hadoop.hbase.client.TestCompleteResultScanResultCache
Running org.apache.hadoop.hbase.client.TestConnectionUtils
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.547 s - in
org.apache.hadoop.hbase.client.TestConnectionUtils
Running org.apache.hadoop.hbase.client.TestHBaseAdminNoCluster
Tests run: 2, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 8.153 s - in
org.apache.hadoop.hbase.client.TestHBaseAdminNoCluster
Running org.apache.hadoop.hbase.client.TestIntraRowPagination
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.268 s - in
org.apache.hadoop.hbase.client.TestIntraRowPagination
Running org.apache.hadoop.hbase.client.TestPutDeleteEtcCellIteration
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.273 s - in
org.apache.hadoop.hbase.client.TestPutDeleteEtcCellIteration
Running org.apache.hadoop.hbase.client.TestResult
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.187 s - in
org.apache.hadoop.hbase.client.TestResult
Running org.apache.hadoop.hbase.codec.TestCellMessageCodec
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.654 s - in
org.apache.hadoop.hbase.codec.TestCellMessageCodec
Running org.apache.hadoop.hbase.conf.TestConfigurationManager
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.467 s - in
org.apache.hadoop.hbase.conf.TestConfigurationManager
Running org.apache.hadoop.hbase.constraint.TestConstraints
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.963 s - in
org.apache.hadoop.hbase.constraint.TestConstraints
Running org.apache.hadoop.hbase.coprocessor.TestCoprocessorConfiguration
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.401 s - in
org.apache.hadoop.hbase.coprocessor.TestCoprocessorConfiguration
Running org.apache.hadoop.hbase.coprocessor.TestCoprocessorHost
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.573 s - in
org.apache.hadoop.hbase.coprocessor.TestCoprocessorHost
Running org.apache.hadoop.hbase.coprocessor.TestCoprocessorInterface
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.89 s - in
org.apache.hadoop.hbase.coprocessor.TestCoprocessorInterface
{noformat}
> False positive for the HBASE-Find-Flaky-Tests job
> -
>
> Key: HBASE-19803
> URL: https://issues.apache.org/jira/browse/HBASE-19803
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Priority: Major
>
> It reports two hangs for TestAsyncTableGetMultiThreaded, but I checked the
> surefire output
> https://builds.apache.org/job/HBASE-Flaky-Tests/24830/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt
> This one was likely to be killed in the middle of the run within 20 seconds.
> https://builds.apache.org/job/HBASE-Flaky-Tests/24852/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt
> This one was also killed within about 1 minutes.
> The test is declared as LargeTests so the time limit should be 10 minutes. It
> seems that the jvm may crash during the mvn test run and then we will kill
> all the running tests and then we may mark some of them as hang which leads
> to the false positive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-17079) HBase build fails on windows, hbase-archetype-builder is reason for failure
[
https://issues.apache.org/jira/browse/HBASE-17079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-17079:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
> HBase build fails on windows, hbase-archetype-builder is reason for failure
> ---
>
> Key: HBASE-17079
> URL: https://issues.apache.org/jira/browse/HBASE-17079
> Project: HBase
> Issue Type: Bug
> Components: build
> Environment: windows
>Reporter: Mohammad Arshad
>Assignee: Mohammad Arshad
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.3.2, 1.4.1, 1.5.0
>
> Attachments: HBASE-17079.branch-1.3.002.patch,
> HBASE-17079.branch-1.3.Verification.jpg, HBASE-17079.master.001.patch,
> HBASE-17079.master.002.patch, screenshot-1.png
>
>
> HBase buid fails on windows, hbase-archetype-builder is reason for failure.
> Cygwin is installed so the shell scripts should execute successfully.
> Here is build failure log
> {code}
> [INFO] Apache HBase - Archetype builder ... FAILURE [ 1.014
> s]
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 06:13 min
> [INFO] Finished at: 2016-11-12T18:12:26+05:30
> [INFO] Final Memory: 235M/1012M
> [INFO]
>
> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:exec
> (make-scripts-executable) on project hbase-archetype-builder: Command
> execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :hbase-archetype-builder
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19803) False positive for the HBASE-Find-Flaky-Tests job
[ https://issues.apache.org/jira/browse/HBASE-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338489#comment-16338489 ] Duo Zhang commented on HBASE-19803: --- I mean the only place where we call System.exit for the crashed UTs is from the surefire plugin itself. So it is very strange that why surefire tells us the UT is crashed since it is killed by surefire itself. And for the JNI crash, first as you said, we should have a hs_err file, and still, the surefire plugin will not have the chance to call System.exit if the JVM is crashed in native code. But the result shows that, it does call System.exit before the UT exits. Thanks. > False positive for the HBASE-Find-Flaky-Tests job > - > > Key: HBASE-19803 > URL: https://issues.apache.org/jira/browse/HBASE-19803 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Priority: Major > > It reports two hangs for TestAsyncTableGetMultiThreaded, but I checked the > surefire output > https://builds.apache.org/job/HBASE-Flaky-Tests/24830/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was likely to be killed in the middle of the run within 20 seconds. > https://builds.apache.org/job/HBASE-Flaky-Tests/24852/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was also killed within about 1 minutes. > The test is declared as LargeTests so the time limit should be 10 minutes. It > seems that the jvm may crash during the mvn test run and then we will kill > all the running tests and then we may mark some of them as hang which leads > to the false positive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19770) Add '--return-values' option to Shell to print return values of commands in interactive mode
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338485#comment-16338485
]
Hudson commented on HBASE-19770:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4462 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/4462/])
HBASE-19770 ADDENDUM Replace `if not` with `unless` (josh.elser: rev
8b2b59fdeefba57ee49d225b26fcd37f5cfa55b2)
* (edit) hbase-shell/src/main/ruby/shell.rb
> Add '--return-values' option to Shell to print return values of commands in
> interactive mode
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.000.branch-2.addendum.patch,
> HBASE-19770.001.branch-2.patch, HBASE-19770.002.branch-2.patch,
> HBASE-19770.003.branch-2.patch, HBASE-19770.004.branch-2.patch,
> HBASE-19770.ADDENDUM.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19853) duplicate slf4j declaration in mapreduce pom
[ https://issues.apache.org/jira/browse/HBASE-19853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338484#comment-16338484 ] Hudson commented on HBASE-19853: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4462 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4462/]) HBASE-19853 Remove duplicate slf4j-api dep (mdrob: rev 548426edc03f40619520ee6fc2eadcf231681459) * (edit) hbase-mapreduce/pom.xml > duplicate slf4j declaration in mapreduce pom > > > Key: HBASE-19853 > URL: https://issues.apache.org/jira/browse/HBASE-19853 > Project: HBase > Issue Type: Bug > Components: dependencies >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Minor > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19853.master.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19803) False positive for the HBASE-Find-Flaky-Tests job
[ https://issues.apache.org/jira/browse/HBASE-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338478#comment-16338478 ] Appy commented on HBASE-19803: -- Oh oh, another way to narrow down search space is, run small+medium tests separately from large tests. > False positive for the HBASE-Find-Flaky-Tests job > - > > Key: HBASE-19803 > URL: https://issues.apache.org/jira/browse/HBASE-19803 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Priority: Major > > It reports two hangs for TestAsyncTableGetMultiThreaded, but I checked the > surefire output > https://builds.apache.org/job/HBASE-Flaky-Tests/24830/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was likely to be killed in the middle of the run within 20 seconds. > https://builds.apache.org/job/HBASE-Flaky-Tests/24852/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was also killed within about 1 minutes. > The test is declared as LargeTests so the time limit should be 10 minutes. It > seems that the jvm may crash during the mvn test run and then we will kill > all the running tests and then we may mark some of them as hang which leads > to the false positive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-17079) HBase build fails on windows, hbase-archetype-builder is reason for failure
[
https://issues.apache.org/jira/browse/HBASE-17079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell reassigned HBASE-17079:
--
Assignee: Mohammad Arshad
> HBase build fails on windows, hbase-archetype-builder is reason for failure
> ---
>
> Key: HBASE-17079
> URL: https://issues.apache.org/jira/browse/HBASE-17079
> Project: HBase
> Issue Type: Bug
> Components: build
> Environment: windows
>Reporter: Mohammad Arshad
>Assignee: Mohammad Arshad
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.3.2, 1.4.1, 1.5.0
>
> Attachments: HBASE-17079.branch-1.3.002.patch,
> HBASE-17079.branch-1.3.Verification.jpg, HBASE-17079.master.001.patch,
> HBASE-17079.master.002.patch, screenshot-1.png
>
>
> HBase buid fails on windows, hbase-archetype-builder is reason for failure.
> Cygwin is installed so the shell scripts should execute successfully.
> Here is build failure log
> {code}
> [INFO] Apache HBase - Archetype builder ... FAILURE [ 1.014
> s]
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 06:13 min
> [INFO] Finished at: 2016-11-12T18:12:26+05:30
> [INFO] Final Memory: 235M/1012M
> [INFO]
>
> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:exec
> (make-scripts-executable) on project hbase-archetype-builder: Command
> execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :hbase-archetype-builder
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-19444) RSGroups test units cannot be concurrently executed
[ https://issues.apache.org/jira/browse/HBASE-19444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-19444: --- Fix Version/s: (was: 1.4.1) 1.4.2 > RSGroups test units cannot be concurrently executed > --- > > Key: HBASE-19444 > URL: https://issues.apache.org/jira/browse/HBASE-19444 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0 >Reporter: Andrew Purtell >Priority: Minor > Fix For: 2.0.0, 1.5.0, 1.4.2 > > > TestRSGroups and friends cannot be concurrently executed or they are very > likely to flake, failing with constraint exceptions. If executed serially all > units pass. Fix for concurrent execution. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19466) Rare failure in TestScannerCursor
[ https://issues.apache.org/jira/browse/HBASE-19466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-19466: --- Fix Version/s: (was: 1.5.0) (was: 1.4.1) > Rare failure in TestScannerCursor > - > > Key: HBASE-19466 > URL: https://issues.apache.org/jira/browse/HBASE-19466 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0 >Reporter: Andrew Purtell >Priority: Minor > > I think we just need to increase the timeout interval to deal with occasional > slowdowns on test executors. 1998 ms is a pretty short timeout. > By the way "rpcTimetout" in the exception message is a misspelling. > [ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: > 37.412 s <<< FAILURE! - in > org.apache.hadoop.hbase.regionserver.TestScannerCursor > [ERROR] > testHeartbeatWithSparseFilter(org.apache.hadoop.hbase.regionserver.TestScannerCursor) > Time elapsed: 35.604 s <<< ERROR! > org.apache.hadoop.hbase.client.RetriesExhaustedException: > Failed after attempts=36, exceptions: > Thu Dec 07 22:27:16 UTC 2017, null, java.net.SocketTimeoutException: > callTimeout=4000, callDuration=4108: Call to > ip-172-31-47-35.us-west-2.compute.internal/172.31.47.35:35690 failed on local > exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=52, > waitTime=2002, rpcTimetout=1998 row '' on table 'TestScannerCursor' at > region=TestScannerCursor,,1512685598567.1d4e59215a881d6ccbd0b5b5bdec5587., > hostname=ip-172-31-47-35.us-west-2.compute.internal,35690,1512685593244, > seqNum=2 > at > org.apache.hadoop.hbase.regionserver.TestScannerCursor.testHeartbeatWithSparseFilter(TestScannerCursor.java:154) > Caused by: java.net.SocketTimeoutException: callTimeout=4000, > callDuration=4108: Call to > ip-172-31-47-35.us-west-2.compute.internal/172.31.47.35:35690 failed on local > exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=52, > waitTime=2002, rpcTimetout=1998 row '' on table 'TestScannerCursor' at > region=TestScannerCursor,,1512685598567.1d4e59215a881d6ccbd0b5b5bdec5587., > hostname=ip-172-31-47-35.us-west-2.compute.internal,35690,1512685593244, > seqNum=2 > Caused by: java.io.IOException: Call to > ip-172-31-47-35.us-west-2.compute.internal/172.31.47.35:35690 failed on local > exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=52, > waitTime=2002, rpcTimetout=1998 > Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=52, > waitTime=2002, rpcTimetout=1998 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-19466) Rare failure in TestScannerCursor
[ https://issues.apache.org/jira/browse/HBASE-19466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-19466. Resolution: Cannot Reproduce > Rare failure in TestScannerCursor > - > > Key: HBASE-19466 > URL: https://issues.apache.org/jira/browse/HBASE-19466 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0 >Reporter: Andrew Purtell >Priority: Minor > > I think we just need to increase the timeout interval to deal with occasional > slowdowns on test executors. 1998 ms is a pretty short timeout. > By the way "rpcTimetout" in the exception message is a misspelling. > [ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: > 37.412 s <<< FAILURE! - in > org.apache.hadoop.hbase.regionserver.TestScannerCursor > [ERROR] > testHeartbeatWithSparseFilter(org.apache.hadoop.hbase.regionserver.TestScannerCursor) > Time elapsed: 35.604 s <<< ERROR! > org.apache.hadoop.hbase.client.RetriesExhaustedException: > Failed after attempts=36, exceptions: > Thu Dec 07 22:27:16 UTC 2017, null, java.net.SocketTimeoutException: > callTimeout=4000, callDuration=4108: Call to > ip-172-31-47-35.us-west-2.compute.internal/172.31.47.35:35690 failed on local > exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=52, > waitTime=2002, rpcTimetout=1998 row '' on table 'TestScannerCursor' at > region=TestScannerCursor,,1512685598567.1d4e59215a881d6ccbd0b5b5bdec5587., > hostname=ip-172-31-47-35.us-west-2.compute.internal,35690,1512685593244, > seqNum=2 > at > org.apache.hadoop.hbase.regionserver.TestScannerCursor.testHeartbeatWithSparseFilter(TestScannerCursor.java:154) > Caused by: java.net.SocketTimeoutException: callTimeout=4000, > callDuration=4108: Call to > ip-172-31-47-35.us-west-2.compute.internal/172.31.47.35:35690 failed on local > exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=52, > waitTime=2002, rpcTimetout=1998 row '' on table 'TestScannerCursor' at > region=TestScannerCursor,,1512685598567.1d4e59215a881d6ccbd0b5b5bdec5587., > hostname=ip-172-31-47-35.us-west-2.compute.internal,35690,1512685593244, > seqNum=2 > Caused by: java.io.IOException: Call to > ip-172-31-47-35.us-west-2.compute.internal/172.31.47.35:35690 failed on local > exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=52, > waitTime=2002, rpcTimetout=1998 > Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=52, > waitTime=2002, rpcTimetout=1998 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18985) Fix the building warning of missing version in hbase-shaded-check-invariants module
[
https://issues.apache.org/jira/browse/HBASE-18985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-18985:
---
Fix Version/s: (was: 1.4.1)
1.4.2
> Fix the building warning of missing version in hbase-shaded-check-invariants
> module
> ---
>
> Key: HBASE-18985
> URL: https://issues.apache.org/jira/browse/HBASE-18985
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Priority: Minor
> Labels: beginner
> Fix For: 1.3.2, 1.5.0, 1.2.8, 1.4.2
>
>
> {quote}
> 19:20:38 [WARNING]
> 19:20:38 [WARNING] Some problems were encountered while building the
> effective model for
> org.apache.hbase:hbase-shaded-check-invariants:pom:1.2.7-SNAPSHOT
> 19:20:38 [WARNING] 'build.plugins.plugin.version' for
> org.codehaus.mojo:exec-maven-plugin is missing. @
> org.apache.hbase:hbase-shaded-check-invariants:[unknown-version],
> /var/lib/jenkins/workspace/PreCommit-HBASE-BRANCH-1.2/hbase-shaded/hbase-shaded-check-invariants/pom.xml,
> line 161, column 15
> 19:20:38 [WARNING]
> 19:20:38 [WARNING] It is highly recommended to fix these problems because
> they threaten the stability of your build.
> 19:20:38 [WARNING]
> 19:20:38 [WARNING] For this reason, future Maven versions might no longer
> support building such malformed projects.
> 19:20:38 [WARNING]
> {quote}
> This warning doesn't occur on master because it had been fixed by HBASE-18723.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18862) backport HBASE-15109 to branch-1.1,branch-1.2,branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-18862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-18862: --- Fix Version/s: (was: 1.4.1) > backport HBASE-15109 to branch-1.1,branch-1.2,branch-1.3 > > > Key: HBASE-18862 > URL: https://issues.apache.org/jira/browse/HBASE-18862 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 1.3.1, 1.2.6, 1.1.12 >Reporter: Yechao Chen >Assignee: Yechao Chen >Priority: Critical > Fix For: 1.3.2, 1.2.8 > > Attachments: HBASE-18862-branch-1.1-v1.patch, > HBASE-18862-branch-1.1.patch, HBASE-18862-branch-1.2-v1.patch, > HBASE-18862-branch-1.2.patch, HBASE-18862-branch-1.3-v1.patch, > HBASE-18862-branch-1.3.patch, HBASE-18862-branch-1.patch > > > HBASE-15109 should apply to branch-1.1,branch-1.2,branch-1.3 also. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18648) Update release checksum generation instructions
[
https://issues.apache.org/jira/browse/HBASE-18648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-18648:
---
Fix Version/s: (was: 1.2.8)
(was: 1.5.0)
(was: 1.4.1)
(was: 1.3.2)
(was: 3.0.0)
(was: 2.0.0)
> Update release checksum generation instructions
> ---
>
> Key: HBASE-18648
> URL: https://issues.apache.org/jira/browse/HBASE-18648
> Project: HBase
> Issue Type: Task
> Components: build
>Reporter: Nick Dimiduk
>Priority: Minor
>
> [Apache policy on release
> checksums|http://www.apache.org/dev/release-distribution#sigs-and-sums] has
> been updated. Adapt our existing documentation and {{make_rc.sh}} script to
> conform to the new guidelines.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18182) Setting HBASE_REGIONSERVER_OPTS with UseG1GC will cause regionserver start error
[
https://issues.apache.org/jira/browse/HBASE-18182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338474#comment-16338474
]
Hadoop QA commented on HBASE-18182:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
3s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
1s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 0m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-18182 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12871795/HBASE-18182.0.patch |
| Optional Tests | asflicense shellcheck shelldocs |
| uname | Linux 79dabe4ecd80 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 8b2b59fdee |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| shellcheck | v0.4.4 |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11186/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> Setting HBASE_REGIONSERVER_OPTS with UseG1GC will cause regionserver start
> error
>
>
> Key: HBASE-18182
> URL: https://issues.apache.org/jira/browse/HBASE-18182
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.1.6, 1.3.1, 1.2.6
>Reporter: Fangyuan Deng
>Assignee: Fangyuan Deng
>Priority: Major
> Fix For: 2.1.0, 1.3.2, 1.2.8, 1.4.2
>
> Attachments: HBASE-18182.0.patch
>
>
> when we set in hbase-env.sh to use G1GC
> HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -XX:+UseG1GC -Xms96g
> -Xmx96g -XX:MaxGCPauseMillis=100"
> then run ./hbase-daemon.sh start regionserver
> Conflicting collector combinations in option list; please refer to the
> release notes for the combinations allowed
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> because the HBASE_OPTS default use CMS and is conflicted with
> HBASE_REGIONSERVER_OPTS
> so I add a patch to modify bin/hbase like this
> elif [ "$COMMAND" = "regionserver" ] ; then
> CLASS='org.apache.hadoop.hbase.regionserver.HRegionServer'
> if [ "$1" != "stop" ] ; then
> HBASE_OPTS="$HBASE_OPTS $HBASE_REGIONSERVER_OPTS"
> if [[ $HBASE_REGIONSERVER_OPTS == *UseG1GC* ]] ; then
> HBASE_OPTS=${HBASE_OPTS/'-XX:+UseConcMarkSweepGC'/''}
> fi
> fi
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18415) The local timeout may cause Admin to submit duplicate request
[ https://issues.apache.org/jira/browse/HBASE-18415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-18415: --- Fix Version/s: (was: 1.4.1) 1.4.2 > The local timeout may cause Admin to submit duplicate request > - > > Key: HBASE-18415 > URL: https://issues.apache.org/jira/browse/HBASE-18415 > Project: HBase > Issue Type: Bug >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Major > Fix For: 2.0.0, 1.3.2, 1.5.0, 1.2.8, 1.4.2 > > Attachments: HBASE-18415.branch-1.ut.patch, > HBASE-18415.branch-1.v0.patch, HBASE-18415.branch-1.v1.patch, > HBASE-18415.branch-1.v2.patch, HBASE-18415.branch-1.v3.patch, > HBASE-18415.branch-1.v3.patch, HBASE-18415.branch-1.v3.patch, > HBASE-18415.branch-1.v4.patch, HBASE-18415.branch-1.v4.patch, > HBASE-18415.branch-1.v4.patch > > > After a timeout occurs on first request, client will retry the request with > distinct group/nonce. The second request may bring the TableXXXException back > if the first request have changed the table state. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18549) Unclaimed replication queues can go undetected
[ https://issues.apache.org/jira/browse/HBASE-18549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-18549: --- Fix Version/s: (was: 1.4.1) 1.4.2 > Unclaimed replication queues can go undetected > -- > > Key: HBASE-18549 > URL: https://issues.apache.org/jira/browse/HBASE-18549 > Project: HBase > Issue Type: Bug > Components: Replication >Reporter: Ashu Pachauri >Priority: Critical > Fix For: 1.3.2, 1.5.0, 1.4.2 > > > We have come across this situation multiple times where a zookeeper issues > can cause NodeFailoverWorker to fail picking up replication queue for a dead > region server silently. One example is when the znode size for a particular > queue exceed jute.maxBuffer value. > There can be other situations that may lead to this and just go undetected. > We need to have a metric for number of unclaimed replication queues. This > will help in mitigating the problem through alerting on the metric and > identifying underlying issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18639) nightly jobs that need to do jdk7 + jdk8 need more than 6 hours
[ https://issues.apache.org/jira/browse/HBASE-18639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-18639: --- Fix Version/s: (was: 1.4.1) 1.4.2 > nightly jobs that need to do jdk7 + jdk8 need more than 6 hours > --- > > Key: HBASE-18639 > URL: https://issues.apache.org/jira/browse/HBASE-18639 > Project: HBase > Issue Type: Task > Components: test >Reporter: Sean Busbey >Priority: Major > Labels: beginner > Fix For: 1.3.2, 1.5.0, 1.2.8, 1.4.2 > > > looks like the branches that need to do 2 jdks will need more than 6 hours, > at least until we can parallelize things. > e.g. > * > [branch-1.2|https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/branch-1.2/buildTimeTrend] > * > [branch-1.3|https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/branch-1.3/buildTimeTrend] > branch-1.4 and branch-1 are failing in jdk7 checks, so they don't go that > long yet, but once we get those tests fixed presumably they'll need the time > as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18228) HBCK improvements
[ https://issues.apache.org/jira/browse/HBASE-18228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-18228: --- Fix Version/s: (was: 1.4.1) 1.4.2 Status: Open (was: Patch Available) > HBCK improvements > - > > Key: HBASE-18228 > URL: https://issues.apache.org/jira/browse/HBASE-18228 > Project: HBase > Issue Type: Improvement > Components: hbck >Reporter: Lars Hofhansl >Assignee: Karan Mehta >Priority: Critical > Fix For: 2.0.0, 3.0.0, 1.5.0, 1.4.2 > > Attachments: HBASE-18228.branch-1.3.patch > > > We just had a prod issue and running HBCK the way we did actually causes more > problems. > In part HBCK did stuff we did not expect, in part we had little visibility > into what HBCK was doing, and in part the logging was confusing. > I'm proposing 2 improvements: > 1. A dry-run mode. Run, and just list what would have been done. > 2. An interactive mode. Run, and for each action request Y/N user input. So > that a user can opt-out of stuff. > [~jmhsieh], FYI -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-18020: --- Fix Version/s: (was: 1.4.2) (was: 1.5.0) 1.4.0 > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang >Priority: Major > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.2.8 > > Attachments: HBASE-18020-addendum.001.patch, HBASE-18020.0.patch, > HBASE-18020.branch-1.2.001.patch, HBASE-18020.branch-1.2.002.patch, > HBASE-18020.branch-1.2.003.patch, HBASE-18020.branch-1.2.004.patch, > HBASE-18020.branch-1.2.005.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18182) Setting HBASE_REGIONSERVER_OPTS with UseG1GC will cause regionserver start error
[
https://issues.apache.org/jira/browse/HBASE-18182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-18182:
---
Fix Version/s: (was: 1.4.1)
1.4.2
> Setting HBASE_REGIONSERVER_OPTS with UseG1GC will cause regionserver start
> error
>
>
> Key: HBASE-18182
> URL: https://issues.apache.org/jira/browse/HBASE-18182
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.1.6, 1.3.1, 1.2.6
>Reporter: Fangyuan Deng
>Assignee: Fangyuan Deng
>Priority: Major
> Fix For: 2.1.0, 1.3.2, 1.2.8, 1.4.2
>
> Attachments: HBASE-18182.0.patch
>
>
> when we set in hbase-env.sh to use G1GC
> HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -XX:+UseG1GC -Xms96g
> -Xmx96g -XX:MaxGCPauseMillis=100"
> then run ./hbase-daemon.sh start regionserver
> Conflicting collector combinations in option list; please refer to the
> release notes for the combinations allowed
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> because the HBASE_OPTS default use CMS and is conflicted with
> HBASE_REGIONSERVER_OPTS
> so I add a patch to modify bin/hbase like this
> elif [ "$COMMAND" = "regionserver" ] ; then
> CLASS='org.apache.hadoop.hbase.regionserver.HRegionServer'
> if [ "$1" != "stop" ] ; then
> HBASE_OPTS="$HBASE_OPTS $HBASE_REGIONSERVER_OPTS"
> if [[ $HBASE_REGIONSERVER_OPTS == *UseG1GC* ]] ; then
> HBASE_OPTS=${HBASE_OPTS/'-XX:+UseConcMarkSweepGC'/''}
> fi
> fi
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-17960) IntegrationTestReplication fails in successive runs due to lack of appropriate cleanup
[ https://issues.apache.org/jira/browse/HBASE-17960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-17960: --- Fix Version/s: (was: 1.4.1) 1.4.2 > IntegrationTestReplication fails in successive runs due to lack of > appropriate cleanup > -- > > Key: HBASE-17960 > URL: https://issues.apache.org/jira/browse/HBASE-17960 > Project: HBase > Issue Type: Bug > Components: integration tests >Reporter: Ashu Pachauri >Assignee: Ashu Pachauri >Priority: Major > Fix For: 2.0.0, 3.0.0, 1.3.2, 1.5.0, 1.4.2 > > > The way ITR works right now is that it adds a peer named 'TestPeer' for the > replication destination cluster. The name of the peer is same across runs. > Also, it removes the peer in the beginning of each run. However, it does not > wait for the queues corresponding to the peer to get cleaned up (which is an > asynchronous operation and can take 10s of seconds). This causes the next run > to fail and so on. > The test setup should wait for a non-trivial amount of time to cleanup the > queues corresponding to the peer. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-17890) FuzzyRowFilter fail if unaligned support is false
[
https://issues.apache.org/jira/browse/HBASE-17890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-17890:
---
Fix Version/s: (was: 1.4.1)
1.4.2
> FuzzyRowFilter fail if unaligned support is false
> -
>
> Key: HBASE-17890
> URL: https://issues.apache.org/jira/browse/HBASE-17890
> Project: HBase
> Issue Type: Sub-task
> Components: util
>Affects Versions: 2.0.0, 1.2.5
>Reporter: Jerry He
>Assignee: Chia-Ping Tsai
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.3.2, 1.2.8, 1.4.2
>
> Attachments: HBASE-17890.v0.branch-1.patch, HBASE-17890.v0.patch,
> HBASE-17890.v1.branch-1.patch, HBASE-17890.v1.patch, HBASE-17890.v2.patch,
> HBASE-17890.v3.patch, HBASE-17890.v3.patch, HBASE-17890.v3.patch,
> HBASE-17890.v3.patch, HBASE-17890.v3.patch
>
>
> When unaligned support is false, FuzzyRow tests fail:
> {noformat}
> Failed tests:
> TestFuzzyRowAndColumnRangeFilter.Test:134->runTest:157->runScanner:186
> expected:<10> but was:<0>
> TestFuzzyRowFilter.testSatisfiesForward:81 expected: but was:
> TestFuzzyRowFilter.testSatisfiesReverse:121 expected: but
> was:
> TestFuzzyRowFilterEndToEnd.testEndToEnd:247->runTest1:278->runScanner:343
> expected:<6250> but was:<0>
> TestFuzzyRowFilterEndToEnd.testFilterList:385->runTest:417->runScanner:445
> expected:<5> but was:<0>
> TestFuzzyRowFilterEndToEnd.testHBASE14782:204 expected:<6> but was:<0>
> {noformat}
> This can be reproduced in the case described in HBASE-17869. Or on a platform
> really without unaligned support.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-18020: --- Fix Version/s: (was: 1.4.1) 1.4.2 > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang >Priority: Major > Fix For: 2.0.0, 3.0.0, 1.3.2, 1.5.0, 1.2.8, 1.4.2 > > Attachments: HBASE-18020-addendum.001.patch, HBASE-18020.0.patch, > HBASE-18020.branch-1.2.001.patch, HBASE-18020.branch-1.2.002.patch, > HBASE-18020.branch-1.2.003.patch, HBASE-18020.branch-1.2.004.patch, > HBASE-18020.branch-1.2.005.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-17229) Backport of purge ThreadLocals
[ https://issues.apache.org/jira/browse/HBASE-17229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-17229: --- Fix Version/s: (was: 1.4.1) (was: 1.3.2) > Backport of purge ThreadLocals > -- > > Key: HBASE-17229 > URL: https://issues.apache.org/jira/browse/HBASE-17229 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Critical > Fix For: 1.2.8 > > > Backport HBASE-17072 and HBASE-16146. The former needs to be backported to > 1.3 ([~mantonov]) and 1.2 ([~busbey]). The latter is already in 1.3. Needs > to be backported to 1.2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-17885) Backport HBASE-15871 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-17885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-17885: --- Fix Version/s: (was: 1.4.1) 1.4.2 > Backport HBASE-15871 to branch-1 > > > Key: HBASE-17885 > URL: https://issues.apache.org/jira/browse/HBASE-17885 > Project: HBase > Issue Type: Bug > Components: Scanners >Affects Versions: 1.3.1, 1.2.5, 1.1.8 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Fix For: 1.3.2, 1.2.8, 1.4.2 > > > Will try to rebase the branch-1 patch at the earliest. Hope the fix versions > are correct. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-17079) HBase build fails on windows, hbase-archetype-builder is reason for failure
[
https://issues.apache.org/jira/browse/HBASE-17079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-17079:
---
Fix Version/s: (was: 1.4.2)
1.4.1
> HBase build fails on windows, hbase-archetype-builder is reason for failure
> ---
>
> Key: HBASE-17079
> URL: https://issues.apache.org/jira/browse/HBASE-17079
> Project: HBase
> Issue Type: Bug
> Components: build
> Environment: windows
>Reporter: Mohammad Arshad
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.3.2, 1.4.1, 1.5.0
>
> Attachments: HBASE-17079.branch-1.3.002.patch,
> HBASE-17079.branch-1.3.Verification.jpg, HBASE-17079.master.001.patch,
> HBASE-17079.master.002.patch, screenshot-1.png
>
>
> HBase buid fails on windows, hbase-archetype-builder is reason for failure.
> Cygwin is installed so the shell scripts should execute successfully.
> Here is build failure log
> {code}
> [INFO] Apache HBase - Archetype builder ... FAILURE [ 1.014
> s]
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 06:13 min
> [INFO] Finished at: 2016-11-12T18:12:26+05:30
> [INFO] Final Memory: 235M/1012M
> [INFO]
>
> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:exec
> (make-scripts-executable) on project hbase-archetype-builder: Command
> execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :hbase-archetype-builder
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-17079) HBase build fails on windows, hbase-archetype-builder is reason for failure
[
https://issues.apache.org/jira/browse/HBASE-17079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338467#comment-16338467
]
Andrew Purtell commented on HBASE-17079:
Let me try to commit this to flush out pending commits for 1.4.1
> HBase build fails on windows, hbase-archetype-builder is reason for failure
> ---
>
> Key: HBASE-17079
> URL: https://issues.apache.org/jira/browse/HBASE-17079
> Project: HBase
> Issue Type: Bug
> Components: build
> Environment: windows
>Reporter: Mohammad Arshad
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.3.2, 1.4.1, 1.5.0
>
> Attachments: HBASE-17079.branch-1.3.002.patch,
> HBASE-17079.branch-1.3.Verification.jpg, HBASE-17079.master.001.patch,
> HBASE-17079.master.002.patch, screenshot-1.png
>
>
> HBase buid fails on windows, hbase-archetype-builder is reason for failure.
> Cygwin is installed so the shell scripts should execute successfully.
> Here is build failure log
> {code}
> [INFO] Apache HBase - Archetype builder ... FAILURE [ 1.014
> s]
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 06:13 min
> [INFO] Finished at: 2016-11-12T18:12:26+05:30
> [INFO] Final Memory: 235M/1012M
> [INFO]
>
> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:exec
> (make-scripts-executable) on project hbase-archetype-builder: Command
> execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :hbase-archetype-builder
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-15291) FileSystem not closed in secure bulkLoad
[ https://issues.apache.org/jira/browse/HBASE-15291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15291: --- Fix Version/s: (was: 1.4.1) 1.4.2 > FileSystem not closed in secure bulkLoad > > > Key: HBASE-15291 > URL: https://issues.apache.org/jira/browse/HBASE-15291 > Project: HBase > Issue Type: Bug >Affects Versions: 1.0.2, 0.98.16.1 >Reporter: Yong Zhang >Assignee: Yong Zhang >Priority: Major > Fix For: 2.0.0, 1.3.2, 1.5.0, 1.2.7, 1.4.2 > > Attachments: HBASE-15291-revert-master.patch, HBASE-15291.001.patch, > HBASE-15291.002.patch, HBASE-15291.003.patch, HBASE-15291.004.patch, > HBASE-15291.addendum, patch > > > FileSystem not closed in secure bulkLoad after bulkLoad finish, it will > cause memory used more and more if too many bulkLoad . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-16071) The VisibilityLabelFilter and AccessControlFilter should not count the "delete cell"
[
https://issues.apache.org/jira/browse/HBASE-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-16071:
---
Fix Version/s: (was: 1.4.1)
1.4.2
> The VisibilityLabelFilter and AccessControlFilter should not count the
> "delete cell"
>
>
> Key: HBASE-16071
> URL: https://issues.apache.org/jira/browse/HBASE-16071
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 2.0.0, 1.3.2, 1.5.0, 1.4.2
>
> Attachments: HBASE-16071-v1.patch, HBASE-16071-v2.patch,
> HBASE-16071-v3.patch
>
>
> The VisibilityLabelFilter will see and count the "delete cell" if the
> scan.isRaw() returns true, so the (put) cell will be skipped if it has lower
> version than "delete cell"
> The critical code is shown below:
> {code:title=VisibilityLabelFilter.java|borderStyle=solid}
> public ReturnCode filterKeyValue(Cell cell) throws IOException {
> if (curFamily.getBytes() == null
> || !(CellUtil.matchingFamily(cell, curFamily.getBytes(),
> curFamily.getOffset(),
> curFamily.getLength( {
> curFamily.set(cell.getFamilyArray(), cell.getFamilyOffset(),
> cell.getFamilyLength());
> // For this family, all the columns can have max of
> curFamilyMaxVersions versions. No need to
> // consider the older versions for visibility label check.
> // Ideally this should have been done at a lower layer by HBase (?)
> curFamilyMaxVersions = cfVsMaxVersions.get(curFamily);
> // Family is changed. Just unset curQualifier.
> curQualifier.unset();
> }
> if (curQualifier.getBytes() == null
> || !(CellUtil.matchingQualifier(cell, curQualifier.getBytes(),
> curQualifier.getOffset(),
> curQualifier.getLength( {
> curQualifier.set(cell.getQualifierArray(), cell.getQualifierOffset(),
> cell.getQualifierLength());
> curQualMetVersions = 0;
> }
> curQualMetVersions++;
> if (curQualMetVersions > curFamilyMaxVersions) {
> return ReturnCode.SKIP;
> }
> return this.expEvaluator.evaluate(cell) ? ReturnCode.INCLUDE :
> ReturnCode.SKIP;
> }
> {code}
> [VisibilityLabelFilter.java|https://github.com/apache/hbase/blob/d7a4499dfc8b3936a0eca867589fc2b23b597866/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityLabelFilter.java]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-17079) HBase build fails on windows, hbase-archetype-builder is reason for failure
[
https://issues.apache.org/jira/browse/HBASE-17079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-17079:
---
Fix Version/s: (was: 1.4.1)
1.4.2
> HBase build fails on windows, hbase-archetype-builder is reason for failure
> ---
>
> Key: HBASE-17079
> URL: https://issues.apache.org/jira/browse/HBASE-17079
> Project: HBase
> Issue Type: Bug
> Components: build
> Environment: windows
>Reporter: Mohammad Arshad
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.3.2, 1.5.0, 1.4.2
>
> Attachments: HBASE-17079.branch-1.3.002.patch,
> HBASE-17079.branch-1.3.Verification.jpg, HBASE-17079.master.001.patch,
> HBASE-17079.master.002.patch, screenshot-1.png
>
>
> HBase buid fails on windows, hbase-archetype-builder is reason for failure.
> Cygwin is installed so the shell scripts should execute successfully.
> Here is build failure log
> {code}
> [INFO] Apache HBase - Archetype builder ... FAILURE [ 1.014
> s]
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 06:13 min
> [INFO] Finished at: 2016-11-12T18:12:26+05:30
> [INFO] Final Memory: 235M/1012M
> [INFO]
>
> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:exec
> (make-scripts-executable) on project hbase-archetype-builder: Command
> execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :hbase-archetype-builder
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-16071) The VisibilityLabelFilter and AccessControlFilter should not count the "delete cell"
[
https://issues.apache.org/jira/browse/HBASE-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-16071:
---
Fix Version/s: 1.5.0
> The VisibilityLabelFilter and AccessControlFilter should not count the
> "delete cell"
>
>
> Key: HBASE-16071
> URL: https://issues.apache.org/jira/browse/HBASE-16071
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 2.0.0, 1.3.2, 1.5.0, 1.4.2
>
> Attachments: HBASE-16071-v1.patch, HBASE-16071-v2.patch,
> HBASE-16071-v3.patch
>
>
> The VisibilityLabelFilter will see and count the "delete cell" if the
> scan.isRaw() returns true, so the (put) cell will be skipped if it has lower
> version than "delete cell"
> The critical code is shown below:
> {code:title=VisibilityLabelFilter.java|borderStyle=solid}
> public ReturnCode filterKeyValue(Cell cell) throws IOException {
> if (curFamily.getBytes() == null
> || !(CellUtil.matchingFamily(cell, curFamily.getBytes(),
> curFamily.getOffset(),
> curFamily.getLength( {
> curFamily.set(cell.getFamilyArray(), cell.getFamilyOffset(),
> cell.getFamilyLength());
> // For this family, all the columns can have max of
> curFamilyMaxVersions versions. No need to
> // consider the older versions for visibility label check.
> // Ideally this should have been done at a lower layer by HBase (?)
> curFamilyMaxVersions = cfVsMaxVersions.get(curFamily);
> // Family is changed. Just unset curQualifier.
> curQualifier.unset();
> }
> if (curQualifier.getBytes() == null
> || !(CellUtil.matchingQualifier(cell, curQualifier.getBytes(),
> curQualifier.getOffset(),
> curQualifier.getLength( {
> curQualifier.set(cell.getQualifierArray(), cell.getQualifierOffset(),
> cell.getQualifierLength());
> curQualMetVersions = 0;
> }
> curQualMetVersions++;
> if (curQualMetVersions > curFamilyMaxVersions) {
> return ReturnCode.SKIP;
> }
> return this.expEvaluator.evaluate(cell) ? ReturnCode.INCLUDE :
> ReturnCode.SKIP;
> }
> {code}
> [VisibilityLabelFilter.java|https://github.com/apache/hbase/blob/d7a4499dfc8b3936a0eca867589fc2b23b597866/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityLabelFilter.java]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable
[
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-14498:
---
Fix Version/s: (was: 1.4.1)
1.4.2
> Master stuck in infinite loop when all Zookeeper servers are unreachable
>
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.0.0, 1.3.2, 1.5.0, 1.2.8, 1.4.2
>
> Attachments: HBASE-14498-V2.patch, HBASE-14498-V3.patch,
> HBASE-14498-V4.patch, HBASE-14498-V5.patch, HBASE-14498-V6.patch,
> HBASE-14498-V6.patch, HBASE-14498-addendum.patch,
> HBASE-14498-branch-1.2.patch, HBASE-14498-branch-1.3-V2.patch,
> HBASE-14498-branch-1.3.patch, HBASE-14498-branch-1.4.patch,
> HBASE-14498-branch-1.patch, HBASE-14498.master.001.patch,
> HBASE-14498.master.002.patch, HBASE-14498.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000]
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)]
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181.
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an
> > active master.
> > HM2 is keep on waiting for region server to report him as part of active
> > master intialization.
> {noformat}
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 0 ms,
> expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval
> of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> ---
> ---
> 2015-09-26 15:32:50,841 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 483913
> ms, expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms,
> interval of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> {noformat}
> > At other end, region servers are reporting to HM1 on 3 sec interval. Here
> > region server retrieve master
[jira] [Updated] (HBASE-14847) Add FIFO compaction section to HBase book
[ https://issues.apache.org/jira/browse/HBASE-14847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-14847: --- Fix Version/s: (was: 1.4.1) 1.4.2 > Add FIFO compaction section to HBase book > - > > Key: HBASE-14847 > URL: https://issues.apache.org/jira/browse/HBASE-14847 > Project: HBase > Issue Type: Task > Components: documentation >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Priority: Major > Fix For: 2.0.0, 1.3.2, 1.4.2 > > > HBASE-14468 introduced new compaction policy. Book needs to be updated. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-14610) IntegrationTestRpcClient from HBASE-14535 is failing with Async RPC client
[ https://issues.apache.org/jira/browse/HBASE-14610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-14610: --- Fix Version/s: (was: 1.4.1) 1.4.2 > IntegrationTestRpcClient from HBASE-14535 is failing with Async RPC client > -- > > Key: HBASE-14610 > URL: https://issues.apache.org/jira/browse/HBASE-14610 > Project: HBase > Issue Type: Bug > Components: IPC/RPC >Reporter: Enis Soztutar >Priority: Major > Fix For: 2.0.0, 3.0.0, 1.3.2, 1.5.0, 1.2.8, 1.4.2 > > Attachments: output > > > HBASE-14535 introduces an IT to simulate a running cluster with RPC servers > and RPC clients doing requests against the servers. > It passes with the sync client, but fails with async client. Probably we need > to take a look. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-15127) NPE in RpcServer$Call.wrapWithSasl()
[
https://issues.apache.org/jira/browse/HBASE-15127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-15127:
---
Fix Version/s: (was: 1.4.1)
1.4.2
> NPE in RpcServer$Call.wrapWithSasl()
>
>
> Key: HBASE-15127
> URL: https://issues.apache.org/jira/browse/HBASE-15127
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Priority: Major
> Fix For: 2.0.0, 1.3.2, 1.4.2
>
>
> I saw this in a log file, not sure whether it is important to fix or not:
> {code}
> 2016-01-09 01:30:58,905 WARN [FifoRpcScheduler.handler1-thread-25]
> ipc.RpcServer: FifoRpcScheduler.handler1-thread-25: caught:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Call.wrapWithSasl(RpcServer.java:412)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Call.setResponse(RpcServer.java:395)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:128)
> at
> org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:74)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> This is 0.98 based code base corresponding to:
> {code}
> synchronized (connection.saslServer) {
> token = connection.saslServer.wrap(responseBytes, 0,
> responseBytes.length);
> }
> {code}
> I believe the saslserver was set to null earlier by {{disposeSasl()}} because
> Connection is closing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-14223) Meta WALs are not cleared if meta region was closed and RS aborts
[
https://issues.apache.org/jira/browse/HBASE-14223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-14223:
---
Fix Version/s: (was: 1.4.1)
1.4.2
> Meta WALs are not cleared if meta region was closed and RS aborts
> -
>
> Key: HBASE-14223
> URL: https://issues.apache.org/jira/browse/HBASE-14223
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.3.2, 1.5.0, 1.2.8, 1.4.2
>
> Attachments: HBASE-14223logs, hbase-14223_v0.patch,
> hbase-14223_v1-branch-1.patch, hbase-14223_v2-branch-1.patch,
> hbase-14223_v3-branch-1.patch, hbase-14223_v3-branch-1.patch,
> hbase-14223_v3-master.patch
>
>
> When an RS opens meta, and later closes it, the WAL(FSHlog) is not closed.
> The last WAL file just sits there in the RS WAL directory. If RS stops
> gracefully, the WAL file for meta is deleted. Otherwise if RS aborts, WAL for
> meta is not cleaned. It is also not split (which is correct) since master
> determines that the RS no longer hosts meta at the time of RS abort.
> From a cluster after running ITBLL with CM, I see a lot of {{-splitting}}
> directories left uncleaned:
> {code}
> [root@os-enis-dal-test-jun-4-7 cluster-os]# sudo -u hdfs hadoop fs -ls
> /apps/hbase/data/WALs
> Found 31 items
> drwxr-xr-x - hbase hadoop 0 2015-06-05 01:14
> /apps/hbase/data/WALs/hregion-58203265
> drwxr-xr-x - hbase hadoop 0 2015-06-05 07:54
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-1.openstacklocal,16020,1433489308745-splitting
> drwxr-xr-x - hbase hadoop 0 2015-06-05 09:28
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-1.openstacklocal,16020,1433494382959-splitting
> drwxr-xr-x - hbase hadoop 0 2015-06-05 10:01
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-1.openstacklocal,16020,1433498252205-splitting
> ...
> {code}
> The directories contain WALs from meta:
> {code}
> [root@os-enis-dal-test-jun-4-7 cluster-os]# sudo -u hdfs hadoop fs -ls
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting
> Found 2 items
> -rw-r--r-- 3 hbase hadoop 201608 2015-06-05 03:15
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433470511501.meta
> -rw-r--r-- 3 hbase hadoop 44420 2015-06-05 04:36
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433474111645.meta
> {code}
> The RS hosted the meta region for some time:
> {code}
> 2015-06-05 03:14:28,692 INFO [PostOpenDeployTasks:1588230740]
> zookeeper.MetaTableLocator: Setting hbase:meta region location in ZooKeeper
> as os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285
> ...
> 2015-06-05 03:15:17,302 INFO
> [RS_CLOSE_META-os-enis-dal-test-jun-4-5:16020-0] regionserver.HRegion: Closed
> hbase:meta,,1.1588230740
> {code}
> In between, a WAL is created:
> {code}
> 2015-06-05 03:15:11,707 INFO
> [RS_OPEN_META-os-enis-dal-test-jun-4-5:16020-0-MetaLogRoller] wal.FSHLog:
> Rolled WAL
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433470511501.meta
> with entries=385, filesize=196.88 KB; new WAL
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433474111645.meta
> {code}
> When CM killed the region server later master did not see these WAL files:
> {code}
> ./hbase-hbase-master-os-enis-dal-test-jun-4-3.log:2015-06-05 03:36:46,075
> INFO [MASTER_SERVER_OPERATIONS-os-enis-dal-test-jun-4-3:16000-0]
> master.SplitLogManager: started splitting 2 logs in
> [hdfs://os-enis-dal-test-jun-4-1.openstacklocal:8020/apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting]
> for [os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285]
> ./hbase-hbase-master-os-enis-dal-test-jun-4-3.log:2015-06-05 03:36:47,300
> INFO [main-EventThread] wal.WALSplitter: Archived processed log
> hdfs://os-enis-dal-test-jun-4-1.openstacklocal:8020/apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285.default.1433475074436
> to
> hdfs://os-enis-dal-test-jun-4-1.openstacklocal:8020/apps/hbase/data/oldWALs/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285.default.1433475074436
> ./hbase-hbase-master-os-enis-dal-test-jun-4-3.log:2015-06-05 03:36:50,497
[jira] [Updated] (HBASE-13841) Master UI: Region server table compactions percentage less useful over time
[ https://issues.apache.org/jira/browse/HBASE-13841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-13841: --- Fix Version/s: (was: 1.4.1) 1.4.2 > Master UI: Region server table compactions percentage less useful over time > --- > > Key: HBASE-13841 > URL: https://issues.apache.org/jira/browse/HBASE-13841 > Project: HBase > Issue Type: Bug > Components: master, UI >Affects Versions: 1.1.0 >Reporter: Lars George >Priority: Major > Labels: beginner > Fix For: 2.0.0, 1.3.2, 1.4.2 > > > The problem is that the percentage is computed as "current KVs to > compact"/"total KVs to compact", and the template accumulating the counts > over its lifetime. Initially it is OK (for the very first run), but after a > short amount of time the percentage idles around 99-100% always. > Not sure how to fix, since we do not have the previews value (unless we save > it somewhere in the page, which is messy during reloads etc.). We may have to > simply drop the percentage? Or better would be not to accumulate the counts, > but keep the current/total number as well, which makes the percentage > computation always correct. I vote for the latter. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-14583) Enabled client-side metrics by default
[ https://issues.apache.org/jira/browse/HBASE-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-14583: --- Fix Version/s: (was: 1.4.1) 1.4.2 > Enabled client-side metrics by default > -- > > Key: HBASE-14583 > URL: https://issues.apache.org/jira/browse/HBASE-14583 > Project: HBase > Issue Type: Task > Components: Client, Operability, Performance >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Critical > Fix For: 2.0.0, 1.3.2, 1.4.2 > > Attachments: 14583.00.branch-1.patch, 14583.00.patch > > > Enabling this feature by default for master and branch-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-14391) Empty regionserver WAL will never be deleted although the coresponding regionserver has been stale
[
https://issues.apache.org/jira/browse/HBASE-14391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-14391:
---
Fix Version/s: (was: 1.4.1)
1.4.2
> Empty regionserver WAL will never be deleted although the coresponding
> regionserver has been stale
> --
>
> Key: HBASE-14391
> URL: https://issues.apache.org/jira/browse/HBASE-14391
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 1.0.2
>Reporter: Qianxi Zhang
>Assignee: Qianxi Zhang
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.3.2, 1.5.0, 1.2.8, 1.4.2
>
> Attachments: HBASE-14391-master-v3.patch,
> HBASE_14391_master_v4.patch, HBASE_14391_trunk_v1.patch,
> HBASE_14391_trunk_v2.patch, WALs-leftover-dir.txt
>
>
> When I restarted the hbase cluster in which there was few data, I found there
> are two directories for one host with different timestamp which indicates
> that the old regionserver wal directory is not deleted.
> FHLog#989
> {code}
> @Override
> public void close() throws IOException {
> shutdown();
> final FileStatus[] files = getFiles();
> if (null != files && 0 != files.length) {
> for (FileStatus file : files) {
> Path p = getWALArchivePath(this.fullPathArchiveDir, file.getPath());
> // Tell our listeners that a log is going to be archived.
> if (!this.listeners.isEmpty()) {
> for (WALActionsListener i : this.listeners) {
> i.preLogArchive(file.getPath(), p);
> }
> }
> if (!FSUtils.renameAndSetModifyTime(fs, file.getPath(), p)) {
> throw new IOException("Unable to rename " + file.getPath() + " to "
> + p);
> }
> // Tell our listeners that a log was archived.
> if (!this.listeners.isEmpty()) {
> for (WALActionsListener i : this.listeners) {
> i.postLogArchive(file.getPath(), p);
> }
> }
> }
> LOG.debug("Moved " + files.length + " WAL file(s) to " +
> FSUtils.getPath(this.fullPathArchiveDir));
> }
> LOG.info("Closed WAL: " + toString());
> }
> {code}
> When regionserver is stopped, the hlog will be archived, so wal/regionserver
> is empty in hdfs.
> MasterFileSystem#252
> {code}
> if (curLogFiles == null || curLogFiles.length == 0) {
> // Empty log folder. No recovery needed
> continue;
> }
> {code}
> The regionserver directory will be not splitted, it makes sense. But it will
> be not deleted.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-13840) Server UIs should rename column labels from KVs to Cell
[ https://issues.apache.org/jira/browse/HBASE-13840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-13840: --- Fix Version/s: (was: 1.4.1) 1.4.2 > Server UIs should rename column labels from KVs to Cell > --- > > Key: HBASE-13840 > URL: https://issues.apache.org/jira/browse/HBASE-13840 > Project: HBase > Issue Type: Bug > Components: master, regionserver, UI >Affects Versions: 1.1.0 >Reporter: Lars George >Priority: Major > Fix For: 2.0.0, 1.3.2, 1.4.2 > > > Currently the master UI still refers to KVs in some of the tables. We should > do a sweep and rename to Cell. > Also do for RS templates. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-13838) Fix shared TaskStatusTmpl.jamon issues (coloring, content, etc.)
[ https://issues.apache.org/jira/browse/HBASE-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-13838: --- Fix Version/s: (was: 1.4.1) 1.4.2 > Fix shared TaskStatusTmpl.jamon issues (coloring, content, etc.) > > > Key: HBASE-13838 > URL: https://issues.apache.org/jira/browse/HBASE-13838 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 1.1.0 >Reporter: Lars George >Assignee: Matt Warhaftig >Priority: Major > Labels: beginner > Fix For: 2.0.0, 1.3.2, 1.4.2 > > Attachments: hbase-13838-v1.patch, hbase-13838_post.tiff, > hbase-13838_post_put_command.txt, hbase-13838_pre.tiff > > > There are a few issues with the shared TaskStatusTmpl: > - "Client operations" tab is always empty > For Master this is expected, but for RegionServers there is never anything > listed either. Fix for RS status page (probably caused by params not > containing Operation subclass anymore, but some PB generated classes?) > - Hide “Client Operations” tab for master UI > Since operations are RS only. Or we fix this and make other calls show here. > - The "alert-error" for aborted tasks is not set in CSS at all > When a task was aborted it should be amber or red, but the assigned style is > not in any of the linked stylesheets (abort-error). Add. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-13603) Write test asserting desired priority of RS->Master RPCs
[
https://issues.apache.org/jira/browse/HBASE-13603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-13603:
---
Fix Version/s: (was: 1.4.1)
(was: 1.1.7)
1.4.2
> Write test asserting desired priority of RS->Master RPCs
>
>
> Key: HBASE-13603
> URL: https://issues.apache.org/jira/browse/HBASE-13603
> Project: HBase
> Issue Type: Test
> Components: IPC/RPC, test
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Minor
> Fix For: 2.0.0, 1.3.2, 1.2.8, 1.4.2
>
>
> From HBASE-13351:
> {quote}
> Any way we can write a FT test to assert that the RS->Master APIs are treated
> with higher priority. I see your UT for asserting the annotation.
> {quote}
> Write a test that verifies expected RPCs are run on the correct pools in as
> real of an environment possible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-12943) Set sun.net.inetaddr.ttl in HBase
[ https://issues.apache.org/jira/browse/HBASE-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-12943: --- Fix Version/s: 1.4.2 > Set sun.net.inetaddr.ttl in HBase > - > > Key: HBASE-12943 > URL: https://issues.apache.org/jira/browse/HBASE-12943 > Project: HBase > Issue Type: Bug >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Major > Fix For: 2.0.0, 1.3.2, 1.4.2 > > Attachments: 12943-1-master.txt > > > The default value of config: sun.net.inetaddr.ttl is -1 and the java > processes will cache the mapping of hostname to ip address forever, See: > http://docs.oracle.com/javase/7/docs/technotes/guides/net/properties.html > But things go wrong when a regionserver with same hostname and different ip > address rejoins the hbase cluster. The HMaster will get wrong ip address of > the regionserver from this cache and every region assignment to this > regionserver will be blocked for a time because the HMaster can't communicate > with the regionserver. > A tradeoff is to set the sun.net.inetaddr.ttl to 10m or 1h and make the wrong > cache expired. > Suggestions are welcomed. Thanks~ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-13160) SplitLogWorker does not pick up the task immediately
[
https://issues.apache.org/jira/browse/HBASE-13160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-13160:
---
Fix Version/s: (was: 1.4.1)
1.4.2
> SplitLogWorker does not pick up the task immediately
>
>
> Key: HBASE-13160
> URL: https://issues.apache.org/jira/browse/HBASE-13160
> Project: HBase
> Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Major
> Fix For: 2.0.0, 1.3.2, 1.4.2
>
> Attachments: hbase-13160_v1.patch
>
>
> We were reading some code with Jeffrey, and we realized that the
> SplitLogWorker's internal task loop is weird. It does {{ls}} every second and
> sleeps, but have another mechanism to learn about new tasks, but does not
> make affective use of the zk notification.
> I have a simple patch which might improve this area.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-12943) Set sun.net.inetaddr.ttl in HBase
[ https://issues.apache.org/jira/browse/HBASE-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-12943: --- Fix Version/s: (was: 1.4.1) > Set sun.net.inetaddr.ttl in HBase > - > > Key: HBASE-12943 > URL: https://issues.apache.org/jira/browse/HBASE-12943 > Project: HBase > Issue Type: Bug >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Major > Fix For: 2.0.0, 1.3.2, 1.4.2 > > Attachments: 12943-1-master.txt > > > The default value of config: sun.net.inetaddr.ttl is -1 and the java > processes will cache the mapping of hostname to ip address forever, See: > http://docs.oracle.com/javase/7/docs/technotes/guides/net/properties.html > But things go wrong when a regionserver with same hostname and different ip > address rejoins the hbase cluster. The HMaster will get wrong ip address of > the regionserver from this cache and every region assignment to this > regionserver will be blocked for a time because the HMaster can't communicate > with the regionserver. > A tradeoff is to set the sun.net.inetaddr.ttl to 10m or 1h and make the wrong > cache expired. > Suggestions are welcomed. Thanks~ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-13354) Add documentation and tests for external block cache.
[ https://issues.apache.org/jira/browse/HBASE-13354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-13354: --- Fix Version/s: (was: 1.4.1) 1.4.2 > Add documentation and tests for external block cache. > - > > Key: HBASE-13354 > URL: https://issues.apache.org/jira/browse/HBASE-13354 > Project: HBase > Issue Type: Bug > Components: BlockCache, documentation, test >Affects Versions: 2.0.0, 1.1.0 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Major > Fix For: 2.0.0, 1.3.2, 1.4.2 > > > The new memcached integration needs some documentation and some testing > showing how it works and what can go wrong. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19756) Master NPE during completed failed proc eviction
[
https://issues.apache.org/jira/browse/HBASE-19756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338458#comment-16338458
]
Andrew Purtell commented on HBASE-19756:
Committing to flush out pending commits for 1.4.1
> Master NPE during completed failed proc eviction
>
>
> Key: HBASE-19756
> URL: https://issues.apache.org/jira/browse/HBASE-19756
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.0, 1.3.1
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
>Priority: Major
> Fix For: 1.3.2, 1.4.1
>
> Attachments: HBASE-19756.branch-1.4.001.patch,
> HBASE-19756.branch-1.4.002.patch, HBASE-19756.branch-1.4.003.patch,
> HBASE-19756.master.001.patch
>
>
> When procedures like Create table fails due to say AccessDeniedException,
> then a rollback procedure is created. When the rollback is being cleaned up,
> it results in an NPE because those nonce procs aren't persisted
> Stack trace when this happens:
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.procedure2.store.ProcedureStoreTracker.delete(ProcedureStoreTracker.java:385)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.updateStoreTracker(WALProcedureStore.java:547)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.pushData(WALProcedureStore.java:504)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.delete(WALProcedureStore.java:453)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$CompletedProcedureCleaner.periodicExecute(ProcedureExecutor.java:184)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.timeoutLoop(ProcedureExecutor.java:995)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$500(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$3.run(ProcedureExecutor.java:507)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
