[jira] [Commented] (HBASE-20404) Ugly cleanerchore complaint that dir is not empty
[
https://issues.apache.org/jira/browse/HBASE-20404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438244#comment-16438244
]
Reid Chan commented on HBASE-20404:
---
Remove is better since it is chatty...
> Ugly cleanerchore complaint that dir is not empty
> -
>
> Key: HBASE-20404
> URL: https://issues.apache.org/jira/browse/HBASE-20404
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-20404.0.patch, HBASE-20404.1.patch
>
>
> I see these big dirty exceptions in my master log during a long-run Lets
> clean them up (Are they exceptions I as an operator can actually do something
> about? Are they 'problems'? Should they be LOG.warn?)
> {code}
> 2018-04-12 16:02:09,911 WARN [ForkJoinPool-1-worker-15]
> cleaner.CleanerChore: Could not delete dir under
> hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta;
> {}
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException):
>
> `/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta
> is non empty': Directory is not empty
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2848)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1048)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:641)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:847)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:790)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2486)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1489)
> at org.apache.hadoop.ipc.Client.call(Client.java:1435)
> at org.apache.hadoop.ipc.Client.call(Client.java:1345)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy26.delete(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:568)
> at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346)
> at com.sun.proxy.$Proxy27.delete(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)
> ...
> {code}
> Looks like log format is off too...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20404) Ugly cleanerchore complaint that dir is not empty
[
https://issues.apache.org/jira/browse/HBASE-20404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438242#comment-16438242
]
Reid Chan commented on HBASE-20404:
---
I can answer this question, because i wrote this.
It's normal phenomenon. Two cases,
1. There're late added files under that dir. An intended test case in
{{TestCleanerChore}} method
{{testCleanerDoesNotDeleteDirectoryWithLateAddedFiles}}
2. There is file under dir, whose ttl doesn't reach zero, or other reasons
depending on
{code:title=FileCleanerDelegate#getDeletableFiles}
DEBUG or removing this warning is fine.
> Ugly cleanerchore complaint that dir is not empty
> -
>
> Key: HBASE-20404
> URL: https://issues.apache.org/jira/browse/HBASE-20404
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-20404.0.patch, HBASE-20404.1.patch
>
>
> I see these big dirty exceptions in my master log during a long-run Lets
> clean them up (Are they exceptions I as an operator can actually do something
> about? Are they 'problems'? Should they be LOG.warn?)
> {code}
> 2018-04-12 16:02:09,911 WARN [ForkJoinPool-1-worker-15]
> cleaner.CleanerChore: Could not delete dir under
> hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta;
> {}
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException):
>
> `/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta
> is non empty': Directory is not empty
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2848)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1048)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:641)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:847)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:790)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2486)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1489)
> at org.apache.hadoop.ipc.Client.call(Client.java:1435)
> at org.apache.hadoop.ipc.Client.call(Client.java:1345)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy26.delete(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:568)
> at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346)
> at com.sun.proxy.$Proxy27.delete(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)
> ...
> {code}
> Looks like log format is off too...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20415) branches-2 don't need maven-scala-plugin
[
https://issues.apache.org/jira/browse/HBASE-20415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-20415:
Status: Patch Available (was: In Progress)
-v0
- just rip out all the scala references in our pom.
> branches-2 don't need maven-scala-plugin
>
>
> Key: HBASE-20415
> URL: https://issues.apache.org/jira/browse/HBASE-20415
> Project: HBase
> Issue Type: Task
> Components: build
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-20415-branch-2.v0.patch
>
>
> the spark support revert missed a couple of plugins in the top level pom for
> making sure we get scaladocs.
> verified there is no scala in the source tree, so these plugins aren't needed.
> {code}
> hbase $ find . -name '*.scala'
> hbase $
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20415) branches-2 don't need maven-scala-plugin
[
https://issues.apache.org/jira/browse/HBASE-20415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-20415:
Attachment: HBASE-20415-branch-2.v0.patch
> branches-2 don't need maven-scala-plugin
>
>
> Key: HBASE-20415
> URL: https://issues.apache.org/jira/browse/HBASE-20415
> Project: HBase
> Issue Type: Task
> Components: build
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-20415-branch-2.v0.patch
>
>
> the spark support revert missed a couple of plugins in the top level pom for
> making sure we get scaladocs.
> verified there is no scala in the source tree, so these plugins aren't needed.
> {code}
> hbase $ find . -name '*.scala'
> hbase $
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20335) nightly jobs no longer contain machine information
[ https://issues.apache.org/jira/browse/HBASE-20335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438231#comment-16438231 ] Hudson commented on HBASE-20335: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1103 (See [https://builds.apache.org/job/HBase-1.2-IT/1103/]) HBASE-20335 ensure each stage of the nightly job gathers machine (busbey: rev 00a70c2c827e6717f3c2d73e387d05fc4adc3ba6) * (edit) dev-support/hbase_nightly_yetus.sh * (edit) dev-support/Jenkinsfile > nightly jobs no longer contain machine information > -- > > Key: HBASE-20335 > URL: https://issues.apache.org/jira/browse/HBASE-20335 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.4, 2.0.1 > > Attachments: HBASE-20335.0.patch, HBASE-20335.1.patch > > > something is up with nightly jobs. they no longer have the machine > information from HBASE-19228. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20335) nightly jobs no longer contain machine information
[ https://issues.apache.org/jira/browse/HBASE-20335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438229#comment-16438229 ] Hudson commented on HBASE-20335: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #395 (See [https://builds.apache.org/job/HBase-1.3-IT/395/]) HBASE-20335 ensure each stage of the nightly job gathers machine (busbey: rev 1f2dbe14e3b16363ea63351f308260529f7fb8d0) * (edit) dev-support/hbase_nightly_yetus.sh * (edit) dev-support/Jenkinsfile > nightly jobs no longer contain machine information > -- > > Key: HBASE-20335 > URL: https://issues.apache.org/jira/browse/HBASE-20335 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.4, 2.0.1 > > Attachments: HBASE-20335.0.patch, HBASE-20335.1.patch > > > something is up with nightly jobs. they no longer have the machine > information from HBASE-19228. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HBASE-20415) branches-2 don't need maven-scala-plugin
[
https://issues.apache.org/jira/browse/HBASE-20415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-20415 started by Sean Busbey.
---
> branches-2 don't need maven-scala-plugin
>
>
> Key: HBASE-20415
> URL: https://issues.apache.org/jira/browse/HBASE-20415
> Project: HBase
> Issue Type: Task
> Components: build
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
>
> the spark support revert missed a couple of plugins in the top level pom for
> making sure we get scaladocs.
> verified there is no scala in the source tree, so these plugins aren't needed.
> {code}
> hbase $ find . -name '*.scala'
> hbase $
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20389) Move website building flags into a profile
[
https://issues.apache.org/jira/browse/HBASE-20389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-20389:
Resolution: Fixed
Fix Version/s: 2.0.1
2.1.0
3.0.0
Status: Resolved (was: Patch Available)
> Move website building flags into a profile
> --
>
> Key: HBASE-20389
> URL: https://issues.apache.org/jira/browse/HBASE-20389
> Project: HBase
> Issue Type: Improvement
> Components: build, website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20389.0.patch, HBASE-20389.1.patch
>
>
> we have some "magic" in our website building right now. The script that's
> used bout our automated website build + publish mechanism manually sets a
> bunch of stuff on the maven command line.
> It'd be better to reflect those settings in a maven profile, so that folks
> are less likely to be surprised e.g. when trying to debug a failure in the
> {{site}} goal happens.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20335) nightly jobs no longer contain machine information
[ https://issues.apache.org/jira/browse/HBASE-20335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20335: Resolution: Fixed Fix Version/s: 2.0.1 1.4.4 1.3.3 1.2.7 1.5.0 2.1.0 3.0.0 Status: Resolved (was: Patch Available) > nightly jobs no longer contain machine information > -- > > Key: HBASE-20335 > URL: https://issues.apache.org/jira/browse/HBASE-20335 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.4, 2.0.1 > > Attachments: HBASE-20335.0.patch, HBASE-20335.1.patch > > > something is up with nightly jobs. they no longer have the machine > information from HBASE-19228. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20415) branches-2 don't need maven-scala-plugin
Sean Busbey created HBASE-20415:
---
Summary: branches-2 don't need maven-scala-plugin
Key: HBASE-20415
URL: https://issues.apache.org/jira/browse/HBASE-20415
Project: HBase
Issue Type: Task
Components: build
Affects Versions: 2.0.0
Reporter: Sean Busbey
Assignee: Sean Busbey
the spark support revert missed a couple of plugins in the top level pom for
making sure we get scaladocs.
verified there is no scala in the source tree, so these plugins aren't needed.
{code}
hbase $ find . -name '*.scala'
hbase $
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20364) nightly job gives old results or no results for stages that timeout on SCM
[
https://issues.apache.org/jira/browse/HBASE-20364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438220#comment-16438220
]
Sean Busbey commented on HBASE-20364:
-
above explanation sound good [~mdrob]?
> nightly job gives old results or no results for stages that timeout on SCM
> --
>
> Key: HBASE-20364
> URL: https://issues.apache.org/jira/browse/HBASE-20364
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Attachments: HBASE-20364.0.patch
>
>
> seen in the branch-2.0 nightly report for HBASE-18828:
>
> {quote}
> Results for branch branch-2.0
> [build #143 on
> builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/143/]:
> (x) *\{color:red}-1 overall\{color}*
>
> details (if available):
> (/) \{color:green}+1 general checks\{color}
> -- For more information [see general
> report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/140//General_Nightly_Build_Report/]
>
> (/) \{color:green}+1 jdk8 hadoop2 checks\{color}
> -- For more information [see jdk8 (hadoop2)
> report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/143//JDK8_Nightly_Build_Report_(Hadoop2)/]
> (/) \{color:green}+1 jdk8 hadoop3 checks\{color}
> -- For more information [see jdk8 (hadoop3)
> report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/143//JDK8_Nightly_Build_Report_(Hadoop3)/]
>
> {quote}
>
> -1 for the overall build was correct. build #143 failed both the general
> check and the source tarball check.
>
> but in the posted comment, we get a false "passing" that links to the general
> result from build #140. and we get no result for the source tarball at all.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20389) Move website building flags into a profile
[
https://issues.apache.org/jira/browse/HBASE-20389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438219#comment-16438219
]
Sean Busbey commented on HBASE-20389:
-
I'm assuming you meant the second patch, give a shout if not.
> Move website building flags into a profile
> --
>
> Key: HBASE-20389
> URL: https://issues.apache.org/jira/browse/HBASE-20389
> Project: HBase
> Issue Type: Improvement
> Components: build, website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Attachments: HBASE-20389.0.patch, HBASE-20389.1.patch
>
>
> we have some "magic" in our website building right now. The script that's
> used bout our automated website build + publish mechanism manually sets a
> bunch of stuff on the maven command line.
> It'd be better to reflect those settings in a maven profile, so that folks
> are less likely to be surprised e.g. when trying to debug a failure in the
> {{site}} goal happens.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20412) Update our compliance-checker from 2.1 to 2.4
[ https://issues.apache.org/jira/browse/HBASE-20412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438218#comment-16438218 ] Sean Busbey commented on HBASE-20412: - +1 > Update our compliance-checker from 2.1 to 2.4 > - > > Key: HBASE-20412 > URL: https://issues.apache.org/jira/browse/HBASE-20412 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: update.txt > > > I thought we had an issue to do this already but I can't find it. > The newer compatibility-checker has the filtering on annotation added by one > of us (or at least asked-for by one-of-us). > i tried it yesterday. Seems to work nicely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20403) Prefetch sometimes doesn't work with encrypted file system
[
https://issues.apache.org/jira/browse/HBASE-20403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438210#comment-16438210
]
Andrew Purtell commented on HBASE-20403:
I came to also suggest it's an HDFS bug, and edge case, and wouldn't this also
affect other reads besides the prefetch case? I don't follow what you mean
about header sizes. We see our stream of bytes at the requested offset
regardless of HDFS metadata or internals. I think it may be a rarely occurring
seek bug when HDFS is using its CryptoInputStream
> Prefetch sometimes doesn't work with encrypted file system
> --
>
> Key: HBASE-20403
> URL: https://issues.apache.org/jira/browse/HBASE-20403
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-beta-2
>Reporter: Umesh Agashe
>Priority: Major
> Fix For: 2.0.0
>
>
> Log from long running test has following stack trace a few times:
> {code}
> 2018-04-09 18:33:21,523 WARN
> org.apache.hadoop.hbase.io.hfile.HFileReaderImpl: Prefetch
> path=hdfs://ns1/hbase/data/default/IntegrationTestBigLinkedList_20180409172704/35f1a7ef13b9d327665228abdbcdffae/meta/9089d98b2a6b4847b3fcf6aceb124988,
> offset=36884200, end=231005989
> java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Buffer.java:275)
> at
> org.apache.hadoop.hdfs.ByteBufferStrategy.readFromBlock(ReaderStrategy.java:183)
> at org.apache.hadoop.hdfs.DFSInputStream.readBuffer(DFSInputStream.java:705)
> at
> org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:766)
> at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:831)
> at
> org.apache.hadoop.crypto.CryptoInputStream.read(CryptoInputStream.java:197)
> at java.io.DataInputStream.read(DataInputStream.java:149)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock.readWithExtra(HFileBlock.java:762)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readAtOffset(HFileBlock.java:1559)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockDataInternal(HFileBlock.java:1771)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockData(HFileBlock.java:1594)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderImpl.readBlock(HFileReaderImpl.java:1488)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$1.run(HFileReaderImpl.java:278)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> Size on disk calculations seem to get messed up due to encryption. Possible
> fixes can be:
> * if file is encrypted with FileStatus#isEncrypted() and do not prefetch.
> * document that hbase.rs.prefetchblocksonopen cannot be true if file is
> encrypted.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20233) [metrics] Ill-formatted numRegions metric in "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
[
https://issues.apache.org/jira/browse/HBASE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438202#comment-16438202
]
Hadoop QA commented on HBASE-20233:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
51s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
47s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
14m 24s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
25s{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
29s{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 8s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20233 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12919042/HBASE-20233-v1.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 37f3f7710d00 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 73275f1774 |
| maven |
[jira] [Updated] (HBASE-20233) [metrics] Ill-formatted numRegions metric in "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
[
https://issues.apache.org/jira/browse/HBASE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xu Cang updated HBASE-20233:
Status: Patch Available (was: Open)
No new unit tests added because this change is minimal, just removed one
redundant metric.
The manual test has been done. I verified the numsRegion metrics is removed
from JMX.
> [metrics] Ill-formatted numRegions metric in
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
> ---
>
> Key: HBASE-20233
> URL: https://issues.apache.org/jira/browse/HBASE-20233
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Reporter: stack
>Assignee: Xu Cang
>Priority: Trivial
> Labels: beginner
> Attachments: HBASE-20233-v1.patch, HBASE-20233.patch
>
>
> Noticed this poking at metrics. The MBean
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" carries per region
> metrics. They are all formatted like so:
> {code}
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_98th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99.9th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_num_ops:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_min: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_max: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_mean:
> 0,
> {code}
> In the middle of them all is a metric named...
> {code}
> numRegions: 15,
> {code}
> It has a different format and it is out of scope when compared to the other
> metrics in this mbean; it belongs better elsewhere, perhaps in the Server
> bean.
> I noticed it because it breaks the parse done by tcollector scraping our
> metrics from /jmx
> It was added long ago in HBASE-14166
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20233) [metrics] Ill-formatted numRegions metric in "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
[
https://issues.apache.org/jira/browse/HBASE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438179#comment-16438179
]
Xu Cang edited comment on HBASE-20233 at 4/14/18 1:34 AM:
--
No new unit tests added because this change is minimal, just removed one
redundant metric.
The manual test has been done. I verified the numsRegions metric is removed
from JMX.
was (Author: xucang):
No new unit tests added because this change is minimal, just removed one
redundant metric.
The manual test has been done. I verified the numsRegion metrics is removed
from JMX.
> [metrics] Ill-formatted numRegions metric in
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
> ---
>
> Key: HBASE-20233
> URL: https://issues.apache.org/jira/browse/HBASE-20233
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Reporter: stack
>Assignee: Xu Cang
>Priority: Trivial
> Labels: beginner
> Attachments: HBASE-20233-v1.patch, HBASE-20233.patch
>
>
> Noticed this poking at metrics. The MBean
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" carries per region
> metrics. They are all formatted like so:
> {code}
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_98th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99.9th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_num_ops:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_min: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_max: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_mean:
> 0,
> {code}
> In the middle of them all is a metric named...
> {code}
> numRegions: 15,
> {code}
> It has a different format and it is out of scope when compared to the other
> metrics in this mbean; it belongs better elsewhere, perhaps in the Server
> bean.
> I noticed it because it breaks the parse done by tcollector scraping our
> metrics from /jmx
> It was added long ago in HBASE-14166
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20233) [metrics] Ill-formatted numRegions metric in "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
[
https://issues.apache.org/jira/browse/HBASE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xu Cang updated HBASE-20233:
Status: Open (was: Patch Available)
> [metrics] Ill-formatted numRegions metric in
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
> ---
>
> Key: HBASE-20233
> URL: https://issues.apache.org/jira/browse/HBASE-20233
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Reporter: stack
>Assignee: Xu Cang
>Priority: Trivial
> Labels: beginner
> Attachments: HBASE-20233-v1.patch, HBASE-20233.patch
>
>
> Noticed this poking at metrics. The MBean
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" carries per region
> metrics. They are all formatted like so:
> {code}
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_98th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99.9th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_num_ops:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_min: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_max: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_mean:
> 0,
> {code}
> In the middle of them all is a metric named...
> {code}
> numRegions: 15,
> {code}
> It has a different format and it is out of scope when compared to the other
> metrics in this mbean; it belongs better elsewhere, perhaps in the Server
> bean.
> I noticed it because it breaks the parse done by tcollector scraping our
> metrics from /jmx
> It was added long ago in HBASE-14166
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20233) [metrics] Ill-formatted numRegions metric in "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
[
https://issues.apache.org/jira/browse/HBASE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xu Cang updated HBASE-20233:
Status: Patch Available (was: Open)
[^HBASE-20233-v1.patch]
> [metrics] Ill-formatted numRegions metric in
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
> ---
>
> Key: HBASE-20233
> URL: https://issues.apache.org/jira/browse/HBASE-20233
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Reporter: stack
>Assignee: Xu Cang
>Priority: Trivial
> Labels: beginner
> Attachments: HBASE-20233-v1.patch, HBASE-20233.patch
>
>
> Noticed this poking at metrics. The MBean
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" carries per region
> metrics. They are all formatted like so:
> {code}
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_98th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99.9th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_num_ops:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_min: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_max: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_mean:
> 0,
> {code}
> In the middle of them all is a metric named...
> {code}
> numRegions: 15,
> {code}
> It has a different format and it is out of scope when compared to the other
> metrics in this mbean; it belongs better elsewhere, perhaps in the Server
> bean.
> I noticed it because it breaks the parse done by tcollector scraping our
> metrics from /jmx
> It was added long ago in HBASE-14166
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20233) [metrics] Ill-formatted numRegions metric in "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
[
https://issues.apache.org/jira/browse/HBASE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xu Cang updated HBASE-20233:
Attachment: HBASE-20233-v1.patch
> [metrics] Ill-formatted numRegions metric in
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
> ---
>
> Key: HBASE-20233
> URL: https://issues.apache.org/jira/browse/HBASE-20233
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Reporter: stack
>Assignee: Xu Cang
>Priority: Trivial
> Labels: beginner
> Attachments: HBASE-20233-v1.patch, HBASE-20233.patch
>
>
> Noticed this poking at metrics. The MBean
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" carries per region
> metrics. They are all formatted like so:
> {code}
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_98th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99.9th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_num_ops:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_min: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_max: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_mean:
> 0,
> {code}
> In the middle of them all is a metric named...
> {code}
> numRegions: 15,
> {code}
> It has a different format and it is out of scope when compared to the other
> metrics in this mbean; it belongs better elsewhere, perhaps in the Server
> bean.
> I noticed it because it breaks the parse done by tcollector scraping our
> metrics from /jmx
> It was added long ago in HBASE-14166
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20233) [metrics] Ill-formatted numRegions metric in "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
[
https://issues.apache.org/jira/browse/HBASE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xu Cang updated HBASE-20233:
Status: Open (was: Patch Available)
> [metrics] Ill-formatted numRegions metric in
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
> ---
>
> Key: HBASE-20233
> URL: https://issues.apache.org/jira/browse/HBASE-20233
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Reporter: stack
>Assignee: Xu Cang
>Priority: Trivial
> Labels: beginner
> Attachments: HBASE-20233-v1.patch, HBASE-20233.patch
>
>
> Noticed this poking at metrics. The MBean
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" carries per region
> metrics. They are all formatted like so:
> {code}
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_98th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99.9th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_num_ops:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_min: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_max: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_mean:
> 0,
> {code}
> In the middle of them all is a metric named...
> {code}
> numRegions: 15,
> {code}
> It has a different format and it is out of scope when compared to the other
> metrics in this mbean; it belongs better elsewhere, perhaps in the Server
> bean.
> I noticed it because it breaks the parse done by tcollector scraping our
> metrics from /jmx
> It was added long ago in HBASE-14166
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-20309) [hbck] bin/hbase hbck -boundaries throws exception on HBase 2
[
https://issues.apache.org/jira/browse/HBASE-20309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang reassigned HBASE-20309:
---
Assignee: Wei-Chiu Chuang
> [hbck] bin/hbase hbck -boundaries throws exception on HBase 2
> -
>
> Key: HBASE-20309
> URL: https://issues.apache.org/jira/browse/HBASE-20309
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Affects Versions: 2.0.0-beta-2
>Reporter: Umesh Agashe
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Fix For: 2.0.0
>
>
> hbck2 on HBase 2 fails with exception:
> {code:java}
> $ bin/hbase hbck -boundaries
> Exception in thread "main" java.util.NoSuchElementException: No value present
> at java.util.Optional.get(Optional.java:135)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.checkRegionBoundaries(HBaseFsck.java:865)
> at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:777)
> at org.apache.hadoop.hbase.util.HBaseFsck.exec(HBaseFsck.java:5059)
> at
> org.apache.hadoop.hbase.util.HBaseFsck$HBaseFsckTool.run(HBaseFsck.java:4860)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:4848){code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20309) [hbck] bin/hbase hbck -boundaries throws exception on HBase 2
[
https://issues.apache.org/jira/browse/HBASE-20309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438174#comment-16438174
]
Wei-Chiu Chuang commented on HBASE-20309:
-
I can add a try block around each file in HBaseFsck#checkRegionBoundaries(),
and probably add a test for this specific hbck parameter.
> [hbck] bin/hbase hbck -boundaries throws exception on HBase 2
> -
>
> Key: HBASE-20309
> URL: https://issues.apache.org/jira/browse/HBASE-20309
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Affects Versions: 2.0.0-beta-2
>Reporter: Umesh Agashe
>Priority: Major
> Fix For: 2.0.0
>
>
> hbck2 on HBase 2 fails with exception:
> {code:java}
> $ bin/hbase hbck -boundaries
> Exception in thread "main" java.util.NoSuchElementException: No value present
> at java.util.Optional.get(Optional.java:135)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.checkRegionBoundaries(HBaseFsck.java:865)
> at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:777)
> at org.apache.hadoop.hbase.util.HBaseFsck.exec(HBaseFsck.java:5059)
> at
> org.apache.hadoop.hbase.util.HBaseFsck$HBaseFsckTool.run(HBaseFsck.java:4860)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:4848){code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20233) [metrics] Ill-formatted numRegions metric in "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
[
https://issues.apache.org/jira/browse/HBASE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438160#comment-16438160
]
Hadoop QA commented on HBASE-20233:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
55s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} hbase-hadoop2-compat: The patch generated 1 new + 1
unchanged - 0 fixed = 2 total (was 1) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
52s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 10s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
27s{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
31s{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 42m 9s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20233 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12919033/HBASE-20233.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 1c71c91542a9 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HBASE-20145) HMaster start fails with IllegalStateException when HADOOP_HOME is set
[
https://issues.apache.org/jira/browse/HBASE-20145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438158#comment-16438158
]
Hadoop QA commented on HBASE-20145:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
17s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
22s{color} | {color:red} hbase-common: The patch generated 1 new + 1 unchanged
- 0 fixed = 2 total (was 1) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
13s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
13m 2s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
27s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20145 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12918983/HBASE-20145.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux def60d3fb016 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 73275f1774 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_162 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/12446/artifact/patchprocess/diff-checkstyle-hbase-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12446/testReport/ |
| Max.
[jira] [Commented] (HBASE-20389) Move website building flags into a profile
[
https://issues.apache.org/jira/browse/HBASE-20389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438157#comment-16438157
]
Hadoop QA commented on HBASE-20389:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
32s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
3s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
52s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
1s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
58s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 47s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}208m
17s{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
49s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}274m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20389 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12919003/HBASE-20389.1.patch |
| Optional Tests | asflicense shellcheck shelldocs javac javadoc unit
shadedjars hadoopcheck xml compile |
| uname | Linux f77278cb570b 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4
19:09:19 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / da7776d428 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_162 |
| shellcheck | v0.4.4 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12442/testReport/ |
| Max. process+thread count | 4775 (vs. ulimit of 1) |
| modules | C: hbase-spark . U: . |
| Console output |
[jira] [Commented] (HBASE-20414) TestLockProcedure#testMultipleLocks may fail on slow machine
[
https://issues.apache.org/jira/browse/HBASE-20414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438153#comment-16438153
]
Hadoop QA commented on HBASE-20414:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
56s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
53s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 12s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}111m
36s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}158m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20414 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12919013/20414.v1.txt |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux fe040100d141 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 73275f1774 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_162 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12444/testReport/ |
| Max. process+thread count | 4405 (vs. ulimit of 1) |
| modules |
[jira] [Commented] (HBASE-20233) [metrics] Ill-formatted numRegions metric in "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
[
https://issues.apache.org/jira/browse/HBASE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438144#comment-16438144
]
Xu Cang commented on HBASE-20233:
-
Thanks, [~yuzhih...@gmail.com]
After this change, I don't see the redundant metric from jmx anymore:
> [metrics] Ill-formatted numRegions metric in
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
> ---
>
> Key: HBASE-20233
> URL: https://issues.apache.org/jira/browse/HBASE-20233
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Reporter: stack
>Assignee: Xu Cang
>Priority: Trivial
> Labels: beginner
> Attachments: HBASE-20233.patch
>
>
> Noticed this poking at metrics. The MBean
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" carries per region
> metrics. They are all formatted like so:
> {code}
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_98th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99.9th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_num_ops:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_min: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_max: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_mean:
> 0,
> {code}
> In the middle of them all is a metric named...
> {code}
> numRegions: 15,
> {code}
> It has a different format and it is out of scope when compared to the other
> metrics in this mbean; it belongs better elsewhere, perhaps in the Server
> bean.
> I noticed it because it breaks the parse done by tcollector scraping our
> metrics from /jmx
> It was added long ago in HBASE-14166
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-18116) Replication source in-memory accounting should not include bulk transfer hfiles
[ https://issues.apache.org/jira/browse/HBASE-18116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell reassigned HBASE-18116: -- Assignee: (was: Geoffrey Jacoby) > Replication source in-memory accounting should not include bulk transfer > hfiles > --- > > Key: HBASE-18116 > URL: https://issues.apache.org/jira/browse/HBASE-18116 > Project: HBase > Issue Type: Bug > Components: Replication >Reporter: Andrew Purtell >Priority: Major > > In ReplicationSourceWALReaderThread we maintain a global quota on enqueued > replication work for preventing OOM by queuing up too many edits into queues > on heap. When calculating the size of a given replication queue entry, if it > has associated hfiles (is a bulk load to be replicated as a batch of hfiles), > we get the file sizes and include the sum. We then apply that result to the > quota. This isn't quite right. Those hfiles will be pulled by the sink as a > file copy, not pushed by the source. The cells in those files are not queued > in memory at the source and therefore shouldn't be counted against the quota. > Related, the sum of the hfile sizes are also included when checking if queued > work exceeds the configured replication queue capacity, which is by default > 64 MB. HFiles are commonly much larger than this. > So what happens is when we encounter a bulk load replication entry typically > both the quota and capacity limits are exceeded, we break out of loops, and > send right away. What is transferred on the wire via HBase RPC though has > only a partial relationship to the calculation. > Depending how you look at it, it makes sense to factor hfile file sizes > against replication queue capacity limits. The sink will be occupied > transferring those files at the HDFS level. Anyway, this is how we have been > doing it and it is too late to change now. I do not however think it is > correct to apply hfile file sizes against a quota for in memory state on the > source. The source doesn't queue or even transfer those bytes. > Something I noticed while working on HBASE-18027. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20233) [metrics] Ill-formatted numRegions metric in "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
[
https://issues.apache.org/jira/browse/HBASE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xu Cang updated HBASE-20233:
Status: Patch Available (was: In Progress)
> [metrics] Ill-formatted numRegions metric in
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
> ---
>
> Key: HBASE-20233
> URL: https://issues.apache.org/jira/browse/HBASE-20233
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Reporter: stack
>Assignee: Xu Cang
>Priority: Trivial
> Labels: beginner
> Attachments: HBASE-20233.patch
>
>
> Noticed this poking at metrics. The MBean
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" carries per region
> metrics. They are all formatted like so:
> {code}
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_98th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99.9th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_num_ops:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_min: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_max: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_mean:
> 0,
> {code}
> In the middle of them all is a metric named...
> {code}
> numRegions: 15,
> {code}
> It has a different format and it is out of scope when compared to the other
> metrics in this mbean; it belongs better elsewhere, perhaps in the Server
> bean.
> I noticed it because it breaks the parse done by tcollector scraping our
> metrics from /jmx
> It was added long ago in HBASE-14166
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20145) HMaster start fails with IllegalStateException when HADOOP_HOME is set
[
https://issues.apache.org/jira/browse/HBASE-20145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HBASE-20145:
Affects Version/s: 2.0.0-beta-1
Priority: Major (was: Critical)
Component/s: wal
> HMaster start fails with IllegalStateException when HADOOP_HOME is set
> --
>
> Key: HBASE-20145
> URL: https://issues.apache.org/jira/browse/HBASE-20145
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 2.0.0-beta-1
> Environment: HBase-2.0-beta1.
> Hadoop trunk version.
> java version "1.8.0_144"
>Reporter: Rohith Sharma K S
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HBASE-20145.master.001.patch
>
>
> It is observed that HMaster start is failed when HADOOP_HOME is set as env
> while starting HMaster. HADOOP_HOME is pointing to Hadoop trunk version.
> {noformat}
> 2018-03-07 16:59:52,654 ERROR [master//10.200.4.200:16000] master.HMaster:
> Failed to become active master
> java.lang.IllegalStateException: The procedure WAL relies on the ability to
> hsync for proper operation during component failures, but the underlying
> filesystem does not support doing so. Please check the config value of
> 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness
> and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount
> that can provide it.
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1036)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:374)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.start(ProcedureExecutor.java:532)
> at
> org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1232)
> at
> org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1145)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:837)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2026)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:547)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> The same configs is working in HBase-1.2.6 build properly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20145) HMaster start fails with IllegalStateException when HADOOP_HOME is set
[
https://issues.apache.org/jira/browse/HBASE-20145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HBASE-20145:
Status: Patch Available (was: Open)
> HMaster start fails with IllegalStateException when HADOOP_HOME is set
> --
>
> Key: HBASE-20145
> URL: https://issues.apache.org/jira/browse/HBASE-20145
> Project: HBase
> Issue Type: Bug
> Environment: HBase-2.0-beta1.
> Hadoop trunk version.
> java version "1.8.0_144"
>Reporter: Rohith Sharma K S
>Assignee: Wei-Chiu Chuang
>Priority: Critical
> Attachments: HBASE-20145.master.001.patch
>
>
> It is observed that HMaster start is failed when HADOOP_HOME is set as env
> while starting HMaster. HADOOP_HOME is pointing to Hadoop trunk version.
> {noformat}
> 2018-03-07 16:59:52,654 ERROR [master//10.200.4.200:16000] master.HMaster:
> Failed to become active master
> java.lang.IllegalStateException: The procedure WAL relies on the ability to
> hsync for proper operation during component failures, but the underlying
> filesystem does not support doing so. Please check the config value of
> 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness
> and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount
> that can provide it.
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1036)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:374)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.start(ProcedureExecutor.java:532)
> at
> org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1232)
> at
> org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1145)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:837)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2026)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:547)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> The same configs is working in HBase-1.2.6 build properly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20233) [metrics] Ill-formatted numRegions metric in "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
[
https://issues.apache.org/jira/browse/HBASE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438112#comment-16438112
]
Ted Yu commented on HBASE-20233:
If you're done with composing the patch, please put the JIRA in 'Patch
Available' state.
> [metrics] Ill-formatted numRegions metric in
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
> ---
>
> Key: HBASE-20233
> URL: https://issues.apache.org/jira/browse/HBASE-20233
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Reporter: stack
>Assignee: Xu Cang
>Priority: Trivial
> Labels: beginner
> Attachments: HBASE-20233.patch
>
>
> Noticed this poking at metrics. The MBean
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" carries per region
> metrics. They are all formatted like so:
> {code}
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_98th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99.9th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_num_ops:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_min: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_max: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_mean:
> 0,
> {code}
> In the middle of them all is a metric named...
> {code}
> numRegions: 15,
> {code}
> It has a different format and it is out of scope when compared to the other
> metrics in this mbean; it belongs better elsewhere, perhaps in the Server
> bean.
> I noticed it because it breaks the parse done by tcollector scraping our
> metrics from /jmx
> It was added long ago in HBASE-14166
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20233) [metrics] Ill-formatted numRegions metric in "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
[
https://issues.apache.org/jira/browse/HBASE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xu Cang updated HBASE-20233:
Attachment: HBASE-20233.patch
> [metrics] Ill-formatted numRegions metric in
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
> ---
>
> Key: HBASE-20233
> URL: https://issues.apache.org/jira/browse/HBASE-20233
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Reporter: stack
>Assignee: Xu Cang
>Priority: Trivial
> Labels: beginner
> Attachments: HBASE-20233.patch
>
>
> Noticed this poking at metrics. The MBean
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" carries per region
> metrics. They are all formatted like so:
> {code}
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_98th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99.9th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_num_ops:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_min: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_max: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_mean:
> 0,
> {code}
> In the middle of them all is a metric named...
> {code}
> numRegions: 15,
> {code}
> It has a different format and it is out of scope when compared to the other
> metrics in this mbean; it belongs better elsewhere, perhaps in the Server
> bean.
> I noticed it because it breaks the parse done by tcollector scraping our
> metrics from /jmx
> It was added long ago in HBASE-14166
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HBASE-20233) [metrics] Ill-formatted numRegions metric in "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
[
https://issues.apache.org/jira/browse/HBASE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-20233 started by Xu Cang.
---
> [metrics] Ill-formatted numRegions metric in
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
> ---
>
> Key: HBASE-20233
> URL: https://issues.apache.org/jira/browse/HBASE-20233
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Reporter: stack
>Assignee: Xu Cang
>Priority: Trivial
> Labels: beginner
>
> Noticed this poking at metrics. The MBean
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" carries per region
> metrics. They are all formatted like so:
> {code}
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_98th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99.9th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_num_ops:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_min: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_max: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_mean:
> 0,
> {code}
> In the middle of them all is a metric named...
> {code}
> numRegions: 15,
> {code}
> It has a different format and it is out of scope when compared to the other
> metrics in this mbean; it belongs better elsewhere, perhaps in the Server
> bean.
> I noticed it because it breaks the parse done by tcollector scraping our
> metrics from /jmx
> It was added long ago in HBASE-14166
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20377) Deal with table in enabling and disabling state when modifying serial replication peer
[
https://issues.apache.org/jira/browse/HBASE-20377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438101#comment-16438101
]
Hudson commented on HBASE-20377:
Results for branch branch-2
[build #610 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/610/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/610//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/610//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/610//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Deal with table in enabling and disabling state when modifying serial
> replication peer
> --
>
> Key: HBASE-20377
> URL: https://issues.apache.org/jira/browse/HBASE-20377
> Project: HBase
> Issue Type: Sub-task
> Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20377-v1.patch, HBASE-20377.patch
>
>
> There could be race between reopening regions and enabling table, and also
> between disabling table and write last pushed sequence id for disabled table.
> Maybe we need to wait for the table state to become enabled or disabled.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20379) shadedjars yetus plugin should add a footer link
[
https://issues.apache.org/jira/browse/HBASE-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438103#comment-16438103
]
Hudson commented on HBASE-20379:
Results for branch branch-2
[build #610 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/610/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/610//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/610//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/610//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> shadedjars yetus plugin should add a footer link
>
>
> Key: HBASE-20379
> URL: https://issues.apache.org/jira/browse/HBASE-20379
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.4, 2.0.1
>
> Attachments: HBASE-20379.0.patch
>
>
> investigating the failure on HBASE-20219, it would be nice if we posted a
> footer link to what failed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20410) upgrade protoc compiler to 3.5.1-1
[
https://issues.apache.org/jira/browse/HBASE-20410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438104#comment-16438104
]
Hudson commented on HBASE-20410:
Results for branch branch-2
[build #610 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/610/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/610//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/610//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/610//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> upgrade protoc compiler to 3.5.1-1
> --
>
> Key: HBASE-20410
> URL: https://issues.apache.org/jira/browse/HBASE-20410
> Project: HBase
> Issue Type: Bug
> Components: build, dependencies, Protobufs
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Critical
> Fix For: 3.0.0, 2.1.0, 2.0.0
>
> Attachments: HBASE-20410.patch
>
>
> See HBASE-20356
> After doing the cleanup there, I was informed that there's a 3.5.1-1 version
> of the compiler binaries that work on rhel6, so let's just go to that. Wish I
> knew about it beforehand.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20112) Include test results from nightly hadoop3 tests in jenkins test results
[
https://issues.apache.org/jira/browse/HBASE-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438102#comment-16438102
]
Hudson commented on HBASE-20112:
Results for branch branch-2
[build #610 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/610/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/610//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/610//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/610//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Include test results from nightly hadoop3 tests in jenkins test results
> ---
>
> Key: HBASE-20112
> URL: https://issues.apache.org/jira/browse/HBASE-20112
> Project: HBase
> Issue Type: Task
> Components: test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.4, 2.0.1
>
> Attachments: HBASE-20112.0.patch
>
>
> right now our nightly tests that run atop hadoop 3 are reported on pass/fail
> but aren't recorded via the jenkins reporting mechanism.
> we should add them.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-20233) [metrics] Ill-formatted numRegions metric in "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
[
https://issues.apache.org/jira/browse/HBASE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu reassigned HBASE-20233:
--
Assignee: Xu Cang
> [metrics] Ill-formatted numRegions metric in
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
> ---
>
> Key: HBASE-20233
> URL: https://issues.apache.org/jira/browse/HBASE-20233
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Reporter: stack
>Assignee: Xu Cang
>Priority: Trivial
> Labels: beginner
>
> Noticed this poking at metrics. The MBean
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" carries per region
> metrics. They are all formatted like so:
> {code}
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_98th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99.9th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_num_ops:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_min: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_max: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_mean:
> 0,
> {code}
> In the middle of them all is a metric named...
> {code}
> numRegions: 15,
> {code}
> It has a different format and it is out of scope when compared to the other
> metrics in this mbean; it belongs better elsewhere, perhaps in the Server
> bean.
> I noticed it because it breaks the parse done by tcollector scraping our
> metrics from /jmx
> It was added long ago in HBASE-14166
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20112) Include test results from nightly hadoop3 tests in jenkins test results
[
https://issues.apache.org/jira/browse/HBASE-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438080#comment-16438080
]
Hudson commented on HBASE-20112:
Results for branch branch-1.3
[build #295 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/295/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/295//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/295//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/295//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Include test results from nightly hadoop3 tests in jenkins test results
> ---
>
> Key: HBASE-20112
> URL: https://issues.apache.org/jira/browse/HBASE-20112
> Project: HBase
> Issue Type: Task
> Components: test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.4, 2.0.1
>
> Attachments: HBASE-20112.0.patch
>
>
> right now our nightly tests that run atop hadoop 3 are reported on pass/fail
> but aren't recorded via the jenkins reporting mechanism.
> we should add them.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20379) shadedjars yetus plugin should add a footer link
[
https://issues.apache.org/jira/browse/HBASE-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438081#comment-16438081
]
Hudson commented on HBASE-20379:
Results for branch branch-1.3
[build #295 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/295/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/295//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/295//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/295//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> shadedjars yetus plugin should add a footer link
>
>
> Key: HBASE-20379
> URL: https://issues.apache.org/jira/browse/HBASE-20379
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.4, 2.0.1
>
> Attachments: HBASE-20379.0.patch
>
>
> investigating the failure on HBASE-20219, it would be nice if we posted a
> footer link to what failed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20233) [metrics] Ill-formatted numRegions metric in "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
[
https://issues.apache.org/jira/browse/HBASE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438075#comment-16438075
]
Xu Cang edited comment on HBASE-20233 at 4/13/18 11:18 PM:
---
Hi,
Please see my patch for this issue. I just simply removed it since it's
redundant.
Update: not sure why I cannot upload the patch file. Should I send an email
instead of upload patch here? thanks.
was (Author: xucang):
Hi,
Please see my patch for this issue. I just simply removed it since it's
redundant.
> [metrics] Ill-formatted numRegions metric in
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
> ---
>
> Key: HBASE-20233
> URL: https://issues.apache.org/jira/browse/HBASE-20233
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Reporter: stack
>Priority: Trivial
> Labels: beginner
>
> Noticed this poking at metrics. The MBean
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" carries per region
> metrics. They are all formatted like so:
> {code}
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_98th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99.9th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_num_ops:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_min: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_max: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_mean:
> 0,
> {code}
> In the middle of them all is a metric named...
> {code}
> numRegions: 15,
> {code}
> It has a different format and it is out of scope when compared to the other
> metrics in this mbean; it belongs better elsewhere, perhaps in the Server
> bean.
> I noticed it because it breaks the parse done by tcollector scraping our
> metrics from /jmx
> It was added long ago in HBASE-14166
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20233) [metrics] Ill-formatted numRegions metric in "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
[
https://issues.apache.org/jira/browse/HBASE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438075#comment-16438075
]
Xu Cang commented on HBASE-20233:
-
Hi,
Please see my patch for this issue. I just simply removed it since it's
redundant.
> [metrics] Ill-formatted numRegions metric in
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" mbean
> ---
>
> Key: HBASE-20233
> URL: https://issues.apache.org/jira/browse/HBASE-20233
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Reporter: stack
>Priority: Trivial
> Labels: beginner
>
> Noticed this poking at metrics. The MBean
> "Hadoop:service=HBase,name=RegionServer,sub=Regions" carries per region
> metrics. They are all formatted like so:
> {code}
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_98th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_incrementTime_99.9th_percentile:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_num_ops:
> 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_min: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_max: 0,
> Namespace_default_table_IntegrationTestBigLinkedList_metric_appendTime_mean:
> 0,
> {code}
> In the middle of them all is a metric named...
> {code}
> numRegions: 15,
> {code}
> It has a different format and it is out of scope when compared to the other
> metrics in this mbean; it belongs better elsewhere, perhaps in the Server
> bean.
> I noticed it because it breaks the parse done by tcollector scraping our
> metrics from /jmx
> It was added long ago in HBASE-14166
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20410) upgrade protoc compiler to 3.5.1-1
[
https://issues.apache.org/jira/browse/HBASE-20410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438064#comment-16438064
]
Hudson commented on HBASE-20410:
Results for branch branch-2.0
[build #173 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/173/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/173//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/173//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/173//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> upgrade protoc compiler to 3.5.1-1
> --
>
> Key: HBASE-20410
> URL: https://issues.apache.org/jira/browse/HBASE-20410
> Project: HBase
> Issue Type: Bug
> Components: build, dependencies, Protobufs
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Critical
> Fix For: 3.0.0, 2.1.0, 2.0.0
>
> Attachments: HBASE-20410.patch
>
>
> See HBASE-20356
> After doing the cleanup there, I was informed that there's a 3.5.1-1 version
> of the compiler binaries that work on rhel6, so let's just go to that. Wish I
> knew about it beforehand.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20394) HBase over rides the value of HBASE_OPTS (if any) set by client
[
https://issues.apache.org/jira/browse/HBASE-20394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438063#comment-16438063
]
Hudson commented on HBASE-20394:
Results for branch branch-2.0
[build #173 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/173/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/173//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/173//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/173//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> HBase over rides the value of HBASE_OPTS (if any) set by client
> ---
>
> Key: HBASE-20394
> URL: https://issues.apache.org/jira/browse/HBASE-20394
> Project: HBase
> Issue Type: Bug
> Components: Operability
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-20394.master.001.patch
>
>
> Currently HBase will over ride the value of HBASE_OPTS (if any) set by user
> {code:java}
> export HBASE_OPTS="-XX:+UseConcMarkSweepGC" {code}
> [See
> hbase-env.sh|https://github.com/apache/hbase/blob/68726b0ee3ef3eb52d32481444e64236c5a9e733/conf/hbase-env.sh#L43]
>
> But, a user may have the following set in his environment:
> {code:java}
> HBASE_OPTS="-Xmn512m"{code}
> While starting the processes, HBase will internally over-ride the existing
> HBASE_OPTS value with the one set in hbase-env.sh
>
> Instead of over-riding we can have the following in hbase-env.sh:
> {code:java}
> export HBASE_OPTS="$HBASE_OPTS -XX:+UseConcMarkSweepGC"{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20411) Ameliorate MutableSegment synchronize
[
https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438041#comment-16438041
]
huaxiang sun commented on HBASE-20411:
--
It seems that we can use LongAdder for
{code:java}
protected volatile long dataSize;
/** 'heapSize' tracks all Cell's heap size occupancy. This will include Cell
POJO heap overhead.
* When Cells in on heap area, this will include the cells data size as well.
*/
protected volatile long heapSize;
/** off-heap size: the aggregated size of all data that is allocated off-heap
including all
* key-values that reside off-heap and the metadata that resides off-heap
*/
protected volatile long offHeapSize;
{code}
to avoid lock contention. The sum() wont return the accurate result when there
are concurrent updates. The argument is that the current way of implementation
wont return accurate result as well. The cells have been added to the internal
CellSet(concurrently), updating counters are synced, so reading the counters
wont reflect the accurate cellSet size.
> Ameliorate MutableSegment synchronize
> -
>
> Key: HBASE-20411
> URL: https://issues.apache.org/jira/browse/HBASE-20411
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Priority: Major
> Attachments: 41901.lock.svg
>
>
> This item is migrated from HBASE-20236 so it gets dedicated issue.
> Let me upload evidence that has this synchronize as a stake in our write-time
> perf. I'll migrate the patch I posted with updates that come of comments
> posted by [~mdrob] on the HBASE-20236 issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20112) Include test results from nightly hadoop3 tests in jenkins test results
[
https://issues.apache.org/jira/browse/HBASE-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438023#comment-16438023
]
Hudson commented on HBASE-20112:
Results for branch branch-1.2
[build #299 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/299/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/299//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/299//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/299//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Include test results from nightly hadoop3 tests in jenkins test results
> ---
>
> Key: HBASE-20112
> URL: https://issues.apache.org/jira/browse/HBASE-20112
> Project: HBase
> Issue Type: Task
> Components: test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.4, 2.0.1
>
> Attachments: HBASE-20112.0.patch
>
>
> right now our nightly tests that run atop hadoop 3 are reported on pass/fail
> but aren't recorded via the jenkins reporting mechanism.
> we should add them.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20379) shadedjars yetus plugin should add a footer link
[
https://issues.apache.org/jira/browse/HBASE-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438024#comment-16438024
]
Hudson commented on HBASE-20379:
Results for branch branch-1.2
[build #299 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/299/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/299//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/299//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/299//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> shadedjars yetus plugin should add a footer link
>
>
> Key: HBASE-20379
> URL: https://issues.apache.org/jira/browse/HBASE-20379
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.4, 2.0.1
>
> Attachments: HBASE-20379.0.patch
>
>
> investigating the failure on HBASE-20219, it would be nice if we posted a
> footer link to what failed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20379) shadedjars yetus plugin should add a footer link
[
https://issues.apache.org/jira/browse/HBASE-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437995#comment-16437995
]
Hudson commented on HBASE-20379:
Results for branch branch-1.4
[build #288 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/288/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/288//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/288//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/288//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> shadedjars yetus plugin should add a footer link
>
>
> Key: HBASE-20379
> URL: https://issues.apache.org/jira/browse/HBASE-20379
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.4, 2.0.1
>
> Attachments: HBASE-20379.0.patch
>
>
> investigating the failure on HBASE-20219, it would be nice if we posted a
> footer link to what failed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20112) Include test results from nightly hadoop3 tests in jenkins test results
[
https://issues.apache.org/jira/browse/HBASE-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437994#comment-16437994
]
Hudson commented on HBASE-20112:
Results for branch branch-1.4
[build #288 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/288/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/288//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/288//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/288//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Include test results from nightly hadoop3 tests in jenkins test results
> ---
>
> Key: HBASE-20112
> URL: https://issues.apache.org/jira/browse/HBASE-20112
> Project: HBase
> Issue Type: Task
> Components: test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.4, 2.0.1
>
> Attachments: HBASE-20112.0.patch
>
>
> right now our nightly tests that run atop hadoop 3 are reported on pass/fail
> but aren't recorded via the jenkins reporting mechanism.
> we should add them.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20389) Move website building flags into a profile
[
https://issues.apache.org/jira/browse/HBASE-20389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437992#comment-16437992
]
Mike Drob commented on HBASE-20389:
---
+1 on v2
> Move website building flags into a profile
> --
>
> Key: HBASE-20389
> URL: https://issues.apache.org/jira/browse/HBASE-20389
> Project: HBase
> Issue Type: Improvement
> Components: build, website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Attachments: HBASE-20389.0.patch, HBASE-20389.1.patch
>
>
> we have some "magic" in our website building right now. The script that's
> used bout our automated website build + publish mechanism manually sets a
> bunch of stuff on the maven command line.
> It'd be better to reflect those settings in a maven profile, so that folks
> are less likely to be surprised e.g. when trying to debug a failure in the
> {{site}} goal happens.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-7129) Need documentation for REST atomic operations (HBASE-4720)
[
https://issues.apache.org/jira/browse/HBASE-7129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437991#comment-16437991
]
Hadoop QA commented on HBASE-7129:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
6s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 3m
18s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 2m
53s{color} | {color:blue} patch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 10m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-7129 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12917030/HBASE-7129.0002.patch
|
| Optional Tests | asflicense refguide |
| uname | Linux 7c94db017701 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 73275f1774 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| refguide |
https://builds.apache.org/job/PreCommit-HBASE-Build/12443/artifact/patchprocess/branch-site/book.html
|
| refguide |
https://builds.apache.org/job/PreCommit-HBASE-Build/12443/artifact/patchprocess/patch-site/book.html
|
| Max. process+thread count | 93 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/12443/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Need documentation for REST atomic operations (HBASE-4720)
> --
>
> Key: HBASE-7129
> URL: https://issues.apache.org/jira/browse/HBASE-7129
> Project: HBase
> Issue Type: Improvement
> Components: documentation, REST
>Reporter: Joe Pallas
>Assignee: Dequan Chen
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-7129.0001.patch, HBASE-7129.0002.patch,
> HBASE-7129.patch
>
>
> HBASE-4720 added checkAndPut/checkAndDelete capability to the REST interface,
> but the REST documentation (in the package summary) needs to be updated so
> people know that this feature exists and how to use it.
> http://wiki.apache.org/hadoop/Hbase/Stargate
> http://hbase.apache.org/book/rest.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20414) TestLockProcedure#testMultipleLocks may fail on slow machine
[
https://issues.apache.org/jira/browse/HBASE-20414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437982#comment-16437982
]
Ted Yu commented on HBASE-20414:
In patch v1, I calculate the actual elapsed duration from the success of nsLock
and the queueing of regionsLock2.
The sleep is the shorter of the remaining time and 1 second (the previous sleep
duration).
> TestLockProcedure#testMultipleLocks may fail on slow machine
>
>
> Key: HBASE-20414
> URL: https://issues.apache.org/jira/browse/HBASE-20414
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20414.v1.txt
>
>
> Here was recent failure :
> https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/172/testReport/junit/org.apache.hadoop.hbase.master.locking/TestLockProcedure/health_checks___yetus_jdk8_hadoop2_checks___testMultipleLocks/
> {code}
> java.lang.AssertionError: expected: but was:
> at
> org.apache.hadoop.hbase.master.locking.TestLockProcedure.sendHeartbeatAndCheckLocked(TestLockProcedure.java:221)
> at
> org.apache.hadoop.hbase.master.locking.TestLockProcedure.testMultipleLocks(TestLockProcedure.java:311)
> {code}
> In the test output, we can see this:
> {code}
> 2018-04-13 20:19:54,230 DEBUG [Time-limited test]
> locking.TestLockProcedure(225): Proc id 22 : LOCKED.
> ...
> 2018-04-13 20:19:55,529 DEBUG [Time-limited test]
> procedure2.ProcedureExecutor(865): Stored pid=26, state=RUNNABLE;
> org.apache.hadoop.hbase.master.locking.LockProcedure
> regions=a7f9adfd047350eabb199a39564ba4db,c1e297609590b707233a2f9c8bb51fa1,
> type=EXCLUSIVE
> 2018-04-13 20:19:56,230 DEBUG [ProcExecTimeout] locking.LockProcedure(207):
> Timeout failure ProcedureEvent for pid=22, state=WAITING_TIMEOUT;
> org.apache.hadoop.hbase.master.locking.LockProcedure, namespace=namespace,
> type=EXCLUSIVE, ready=false, [pid=22, state=WAITING_TIMEOUT;
> org.apache.hadoop.hbase.master.locking.LockProcedure, namespace=namespace,
> type=EXCLUSIVE]
> {code}
> After the pid=26 log, the code does this (1 second wait):
> {code}
> // Assert tables & region locks are waiting because of namespace lock.
> Thread.sleep(HEARTBEAT_TIMEOUT / 2);
> {code}
> On a slow machine (in the case above), there was only 730 msec between the
> queueing of regionsLock2 and the WAITING_TIMEOUT state of the nsLock. The 1
> second wait was too long, leading to assertion failure.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20414) TestLockProcedure#testMultipleLocks may fail on slow machine
[
https://issues.apache.org/jira/browse/HBASE-20414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-20414:
---
Assignee: Ted Yu
Status: Patch Available (was: Open)
> TestLockProcedure#testMultipleLocks may fail on slow machine
>
>
> Key: HBASE-20414
> URL: https://issues.apache.org/jira/browse/HBASE-20414
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20414.v1.txt
>
>
> Here was recent failure :
> https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/172/testReport/junit/org.apache.hadoop.hbase.master.locking/TestLockProcedure/health_checks___yetus_jdk8_hadoop2_checks___testMultipleLocks/
> {code}
> java.lang.AssertionError: expected: but was:
> at
> org.apache.hadoop.hbase.master.locking.TestLockProcedure.sendHeartbeatAndCheckLocked(TestLockProcedure.java:221)
> at
> org.apache.hadoop.hbase.master.locking.TestLockProcedure.testMultipleLocks(TestLockProcedure.java:311)
> {code}
> In the test output, we can see this:
> {code}
> 2018-04-13 20:19:54,230 DEBUG [Time-limited test]
> locking.TestLockProcedure(225): Proc id 22 : LOCKED.
> ...
> 2018-04-13 20:19:55,529 DEBUG [Time-limited test]
> procedure2.ProcedureExecutor(865): Stored pid=26, state=RUNNABLE;
> org.apache.hadoop.hbase.master.locking.LockProcedure
> regions=a7f9adfd047350eabb199a39564ba4db,c1e297609590b707233a2f9c8bb51fa1,
> type=EXCLUSIVE
> 2018-04-13 20:19:56,230 DEBUG [ProcExecTimeout] locking.LockProcedure(207):
> Timeout failure ProcedureEvent for pid=22, state=WAITING_TIMEOUT;
> org.apache.hadoop.hbase.master.locking.LockProcedure, namespace=namespace,
> type=EXCLUSIVE, ready=false, [pid=22, state=WAITING_TIMEOUT;
> org.apache.hadoop.hbase.master.locking.LockProcedure, namespace=namespace,
> type=EXCLUSIVE]
> {code}
> After the pid=26 log, the code does this (1 second wait):
> {code}
> // Assert tables & region locks are waiting because of namespace lock.
> Thread.sleep(HEARTBEAT_TIMEOUT / 2);
> {code}
> On a slow machine (in the case above), there was only 730 msec between the
> queueing of regionsLock2 and the WAITING_TIMEOUT state of the nsLock. The 1
> second wait was too long, leading to assertion failure.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20414) TestLockProcedure#testMultipleLocks may fail on slow machine
[
https://issues.apache.org/jira/browse/HBASE-20414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-20414:
---
Attachment: 20414.v1.txt
> TestLockProcedure#testMultipleLocks may fail on slow machine
>
>
> Key: HBASE-20414
> URL: https://issues.apache.org/jira/browse/HBASE-20414
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Priority: Major
> Attachments: 20414.v1.txt
>
>
> Here was recent failure :
> https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/172/testReport/junit/org.apache.hadoop.hbase.master.locking/TestLockProcedure/health_checks___yetus_jdk8_hadoop2_checks___testMultipleLocks/
> {code}
> java.lang.AssertionError: expected: but was:
> at
> org.apache.hadoop.hbase.master.locking.TestLockProcedure.sendHeartbeatAndCheckLocked(TestLockProcedure.java:221)
> at
> org.apache.hadoop.hbase.master.locking.TestLockProcedure.testMultipleLocks(TestLockProcedure.java:311)
> {code}
> In the test output, we can see this:
> {code}
> 2018-04-13 20:19:54,230 DEBUG [Time-limited test]
> locking.TestLockProcedure(225): Proc id 22 : LOCKED.
> ...
> 2018-04-13 20:19:55,529 DEBUG [Time-limited test]
> procedure2.ProcedureExecutor(865): Stored pid=26, state=RUNNABLE;
> org.apache.hadoop.hbase.master.locking.LockProcedure
> regions=a7f9adfd047350eabb199a39564ba4db,c1e297609590b707233a2f9c8bb51fa1,
> type=EXCLUSIVE
> 2018-04-13 20:19:56,230 DEBUG [ProcExecTimeout] locking.LockProcedure(207):
> Timeout failure ProcedureEvent for pid=22, state=WAITING_TIMEOUT;
> org.apache.hadoop.hbase.master.locking.LockProcedure, namespace=namespace,
> type=EXCLUSIVE, ready=false, [pid=22, state=WAITING_TIMEOUT;
> org.apache.hadoop.hbase.master.locking.LockProcedure, namespace=namespace,
> type=EXCLUSIVE]
> {code}
> After the pid=26 log, the code does this (1 second wait):
> {code}
> // Assert tables & region locks are waiting because of namespace lock.
> Thread.sleep(HEARTBEAT_TIMEOUT / 2);
> {code}
> On a slow machine (in the case above), there was only 730 msec between the
> queueing of regionsLock2 and the WAITING_TIMEOUT state of the nsLock. The 1
> second wait was too long, leading to assertion failure.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20414) TestLockProcedure#testMultipleLocks may fail on slow machine
[
https://issues.apache.org/jira/browse/HBASE-20414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437975#comment-16437975
]
Ted Yu commented on HBASE-20414:
I marked this as Major since TestLockProcedure is medium test, the failure of
which would prevent large tests from running.
> TestLockProcedure#testMultipleLocks may fail on slow machine
>
>
> Key: HBASE-20414
> URL: https://issues.apache.org/jira/browse/HBASE-20414
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Priority: Major
>
> Here was recent failure :
> https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/172/testReport/junit/org.apache.hadoop.hbase.master.locking/TestLockProcedure/health_checks___yetus_jdk8_hadoop2_checks___testMultipleLocks/
> {code}
> java.lang.AssertionError: expected: but was:
> at
> org.apache.hadoop.hbase.master.locking.TestLockProcedure.sendHeartbeatAndCheckLocked(TestLockProcedure.java:221)
> at
> org.apache.hadoop.hbase.master.locking.TestLockProcedure.testMultipleLocks(TestLockProcedure.java:311)
> {code}
> In the test output, we can see this:
> {code}
> 2018-04-13 20:19:54,230 DEBUG [Time-limited test]
> locking.TestLockProcedure(225): Proc id 22 : LOCKED.
> ...
> 2018-04-13 20:19:55,529 DEBUG [Time-limited test]
> procedure2.ProcedureExecutor(865): Stored pid=26, state=RUNNABLE;
> org.apache.hadoop.hbase.master.locking.LockProcedure
> regions=a7f9adfd047350eabb199a39564ba4db,c1e297609590b707233a2f9c8bb51fa1,
> type=EXCLUSIVE
> 2018-04-13 20:19:56,230 DEBUG [ProcExecTimeout] locking.LockProcedure(207):
> Timeout failure ProcedureEvent for pid=22, state=WAITING_TIMEOUT;
> org.apache.hadoop.hbase.master.locking.LockProcedure, namespace=namespace,
> type=EXCLUSIVE, ready=false, [pid=22, state=WAITING_TIMEOUT;
> org.apache.hadoop.hbase.master.locking.LockProcedure, namespace=namespace,
> type=EXCLUSIVE]
> {code}
> After the pid=26 log, the code does this (1 second wait):
> {code}
> // Assert tables & region locks are waiting because of namespace lock.
> Thread.sleep(HEARTBEAT_TIMEOUT / 2);
> {code}
> On a slow machine (in the case above), there was only 730 msec between the
> queueing of regionsLock2 and the WAITING_TIMEOUT state of the nsLock. The 1
> second wait was too long, leading to assertion failure.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20414) TestLockProcedure#testMultipleLocks may fail on slow machine
Ted Yu created HBASE-20414:
--
Summary: TestLockProcedure#testMultipleLocks may fail on slow
machine
Key: HBASE-20414
URL: https://issues.apache.org/jira/browse/HBASE-20414
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Here was recent failure :
https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/172/testReport/junit/org.apache.hadoop.hbase.master.locking/TestLockProcedure/health_checks___yetus_jdk8_hadoop2_checks___testMultipleLocks/
{code}
java.lang.AssertionError: expected: but was:
at
org.apache.hadoop.hbase.master.locking.TestLockProcedure.sendHeartbeatAndCheckLocked(TestLockProcedure.java:221)
at
org.apache.hadoop.hbase.master.locking.TestLockProcedure.testMultipleLocks(TestLockProcedure.java:311)
{code}
In the test output, we can see this:
{code}
2018-04-13 20:19:54,230 DEBUG [Time-limited test]
locking.TestLockProcedure(225): Proc id 22 : LOCKED.
...
2018-04-13 20:19:55,529 DEBUG [Time-limited test]
procedure2.ProcedureExecutor(865): Stored pid=26, state=RUNNABLE;
org.apache.hadoop.hbase.master.locking.LockProcedure
regions=a7f9adfd047350eabb199a39564ba4db,c1e297609590b707233a2f9c8bb51fa1,
type=EXCLUSIVE
2018-04-13 20:19:56,230 DEBUG [ProcExecTimeout] locking.LockProcedure(207):
Timeout failure ProcedureEvent for pid=22, state=WAITING_TIMEOUT;
org.apache.hadoop.hbase.master.locking.LockProcedure, namespace=namespace,
type=EXCLUSIVE, ready=false, [pid=22, state=WAITING_TIMEOUT;
org.apache.hadoop.hbase.master.locking.LockProcedure, namespace=namespace,
type=EXCLUSIVE]
{code}
After the pid=26 log, the code does this (1 second wait):
{code}
// Assert tables & region locks are waiting because of namespace lock.
Thread.sleep(HEARTBEAT_TIMEOUT / 2);
{code}
On a slow machine (in the case above), there was only 730 msec between the
queueing of regionsLock2 and the WAITING_TIMEOUT state of the nsLock. The 1
second wait was too long, leading to assertion failure.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20257) hbase-spark should not depend on com.google.code.findbugs.jsr305
[
https://issues.apache.org/jira/browse/HBASE-20257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437942#comment-16437942
]
Sean Busbey commented on HBASE-20257:
-
I haven't gotten to close out HBASE-20336 yet. I'm unlikely to prioritize any
non-2.0.0 targeting fix prior to a 2.0.0 RC passing.
> hbase-spark should not depend on com.google.code.findbugs.jsr305
>
>
> Key: HBASE-20257
> URL: https://issues.apache.org/jira/browse/HBASE-20257
> Project: HBase
> Issue Type: Task
> Components: build
>Affects Versions: 3.0.0
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-20257.v01.patch, HBASE-20257.v02.patch,
> HBASE-20257.v03.patch, HBASE-20257.v04.patch
>
>
> The following can be observed in the build output of master branch:
> {code}
> [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies failed
> with message:
> We don't allow the JSR305 jar from the Findbugs project, see HBASE-16321.
> Found Banned Dependency: com.google.code.findbugs:jsr305:jar:1.3.9
> Use 'mvn dependency:tree' to locate the source of the banned dependencies.
> {code}
> Here is related snippet from hbase-spark/pom.xml:
> {code}
>
> com.google.code.findbugs
> jsr305
> {code}
> Dependency on jsr305 should be dropped.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20364) nightly job gives old results or no results for stages that timeout on SCM
[
https://issues.apache.org/jira/browse/HBASE-20364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437939#comment-16437939
]
Sean Busbey commented on HBASE-20364:
-
{quote}
to make sure i understand... we write a default failure and then when we finish
we write the real result. but if we don't finish and time out then we still
have the assumed failure message written?
{quote}
If we don't finish in a way that still allows post actions, but yes that's
essentially correct.
{quote}
do we know that this will only happen in case of timeout? will sometime like
OOM also cause the stage to fail without writing a message or will that get a
known failure state? trying to understand if we can write a more specific
initial fail message
{quote}
No it's not just timeout. we could, for example, transiently run into a
resource limit while trying to use the "gather machine stats" script. That
would fail the stage but still run post, which would have us use the default
message.
We could get into updating the message incrementally as we go, but I'm
skeptical of incremental benefit beyond "we hit an unexpected failure of the
test infra."
> nightly job gives old results or no results for stages that timeout on SCM
> --
>
> Key: HBASE-20364
> URL: https://issues.apache.org/jira/browse/HBASE-20364
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Attachments: HBASE-20364.0.patch
>
>
> seen in the branch-2.0 nightly report for HBASE-18828:
>
> {quote}
> Results for branch branch-2.0
> [build #143 on
> builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/143/]:
> (x) *\{color:red}-1 overall\{color}*
>
> details (if available):
> (/) \{color:green}+1 general checks\{color}
> -- For more information [see general
> report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/140//General_Nightly_Build_Report/]
>
> (/) \{color:green}+1 jdk8 hadoop2 checks\{color}
> -- For more information [see jdk8 (hadoop2)
> report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/143//JDK8_Nightly_Build_Report_(Hadoop2)/]
> (/) \{color:green}+1 jdk8 hadoop3 checks\{color}
> -- For more information [see jdk8 (hadoop3)
> report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/143//JDK8_Nightly_Build_Report_(Hadoop3)/]
>
> {quote}
>
> -1 for the overall build was correct. build #143 failed both the general
> check and the source tarball check.
>
> but in the posted comment, we get a false "passing" that links to the general
> result from build #140. and we get no result for the source tarball at all.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20257) hbase-spark should not depend on com.google.code.findbugs.jsr305
[
https://issues.apache.org/jira/browse/HBASE-20257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437899#comment-16437899
]
Ted Yu commented on HBASE-20257:
[~busbey]:
Mind taking another look when you have some cycles ?
> hbase-spark should not depend on com.google.code.findbugs.jsr305
>
>
> Key: HBASE-20257
> URL: https://issues.apache.org/jira/browse/HBASE-20257
> Project: HBase
> Issue Type: Task
> Components: build
>Affects Versions: 3.0.0
>Reporter: Ted Yu
>Assignee: Artem Ervits
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-20257.v01.patch, HBASE-20257.v02.patch,
> HBASE-20257.v03.patch, HBASE-20257.v04.patch
>
>
> The following can be observed in the build output of master branch:
> {code}
> [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies failed
> with message:
> We don't allow the JSR305 jar from the Findbugs project, see HBASE-16321.
> Found Banned Dependency: com.google.code.findbugs:jsr305:jar:1.3.9
> Use 'mvn dependency:tree' to locate the source of the banned dependencies.
> {code}
> Here is related snippet from hbase-spark/pom.xml:
> {code}
>
> com.google.code.findbugs
> jsr305
> {code}
> Dependency on jsr305 should be dropped.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20214) Review of RegionLocationFinder Class
[
https://issues.apache.org/jira/browse/HBASE-20214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437895#comment-16437895
]
Sean Busbey commented on HBASE-20214:
-
don't use commons-collections v3. You should use the commons-collection v4 API
that's been relocated in the hbase-thirdparty library.
FWIW, the description is here: [http://hbase.apache.org/book.html#thirdparty]
Basically, any class found in commons-collections4 should be present if you
prefix the package with {{org.apache.hbase.thirdparty}}, e.g.
{{org.apache.hbase.thirdparty.org.apache.commons.collections4.CollectionUtils}}.
> Review of RegionLocationFinder Class
>
>
> Key: HBASE-20214
> URL: https://issues.apache.org/jira/browse/HBASE-20214
> Project: HBase
> Issue Type: Improvement
> Components: Balancer, master
>Affects Versions: 2.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HBASE-20214.1.patch, HBASE-20214.2.patch
>
>
> # Use SLF4J parameter logging
> # Remove superfluous code
> # Replace code with re-usable libraries where possible
> # Use different data structure
> # Small perf improvements
> # Fix some checkstyle
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20112) Include test results from nightly hadoop3 tests in jenkins test results
[
https://issues.apache.org/jira/browse/HBASE-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437886#comment-16437886
]
Hudson commented on HBASE-20112:
Results for branch branch-2.0
[build #172 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/172/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/172//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/172//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/172//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Include test results from nightly hadoop3 tests in jenkins test results
> ---
>
> Key: HBASE-20112
> URL: https://issues.apache.org/jira/browse/HBASE-20112
> Project: HBase
> Issue Type: Task
> Components: test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.4, 2.0.1
>
> Attachments: HBASE-20112.0.patch
>
>
> right now our nightly tests that run atop hadoop 3 are reported on pass/fail
> but aren't recorded via the jenkins reporting mechanism.
> we should add them.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20379) shadedjars yetus plugin should add a footer link
[
https://issues.apache.org/jira/browse/HBASE-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437887#comment-16437887
]
Hudson commented on HBASE-20379:
Results for branch branch-2.0
[build #172 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/172/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/172//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/172//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/172//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> shadedjars yetus plugin should add a footer link
>
>
> Key: HBASE-20379
> URL: https://issues.apache.org/jira/browse/HBASE-20379
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.4, 2.0.1
>
> Attachments: HBASE-20379.0.patch
>
>
> investigating the failure on HBASE-20219, it would be nice if we posted a
> footer link to what failed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20224) Web UI is broken in standalone mode
[
https://issues.apache.org/jira/browse/HBASE-20224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437885#comment-16437885
]
Hudson commented on HBASE-20224:
Results for branch branch-2.0
[build #172 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/172/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/172//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/172//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/172//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Web UI is broken in standalone mode
> ---
>
> Key: HBASE-20224
> URL: https://issues.apache.org/jira/browse/HBASE-20224
> Project: HBase
> Issue Type: Bug
> Components: UI, Usability
>Affects Versions: 2.0.0-beta-2
>Reporter: Umesh Agashe
>Assignee: Umesh Agashe
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments:
> 0001-HBASE-20224-Web-UI-is-broken-in-standalone-mode-ADDE.ADDENDUM.patch,
> 20224-addendum.3.txt, 20224.addendum.4, 20224.addendum.5, 20224.addendum.6,
> hbase-20224.master.001.patch, hbase-20224.master.002.patch,
> hbase-20224.master.003.patch, hbase-20224.master.addendum.patch
>
>
> Web UI doesn't show up in standalone mode on default port. This can be seen
> on master and branch-2.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20291) Fix for The POM for net.minidev:json-smart:jar:2.3-SNAPSHOT is missing, no dependency information available with hadoop.profile=3.0
[
https://issues.apache.org/jira/browse/HBASE-20291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437878#comment-16437878
]
Ted Yu commented on HBASE-20291:
I reverted the previous commit considering the resolution for this JIRA.
> Fix for The POM for net.minidev:json-smart:jar:2.3-SNAPSHOT is missing, no
> dependency information available with hadoop.profile=3.0
> ---
>
> Key: HBASE-20291
> URL: https://issues.apache.org/jira/browse/HBASE-20291
> Project: HBase
> Issue Type: Bug
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-20291.v01.patch, HBASE-20291.v02.patch
>
>
> receiving message
> {code:java}
> The POM for net.minidev:json-smart:jar:2.3-SNAPSHOT is missing, no dependency
> information available{code}
> when running with
> {code:java}
> mvn clean install -DHBasePatchProcess -Dhadoop-three.version=3.0.0
> -Dhadoop.profile=3.0 -DskipTests{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20389) Move website building flags into a profile
[
https://issues.apache.org/jira/browse/HBASE-20389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-20389:
Status: Patch Available (was: In Progress)
-v1
- workaround scala-maven-plugin not having a skip by using a profile
- update website script to activate the workaround
- skip the warbucks plugin both on when building site (both install and site
goals)
I suspect there are still more things we could pull out, but it's all small
incremental gains at this point and I'd rather leave additional things for
follow-on work.
> Move website building flags into a profile
> --
>
> Key: HBASE-20389
> URL: https://issues.apache.org/jira/browse/HBASE-20389
> Project: HBase
> Issue Type: Improvement
> Components: build, website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Attachments: HBASE-20389.0.patch, HBASE-20389.1.patch
>
>
> we have some "magic" in our website building right now. The script that's
> used bout our automated website build + publish mechanism manually sets a
> bunch of stuff on the maven command line.
> It'd be better to reflect those settings in a maven profile, so that folks
> are less likely to be surprised e.g. when trying to debug a failure in the
> {{site}} goal happens.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20404) Ugly cleanerchore complaint that dir is not empty
[
https://issues.apache.org/jira/browse/HBASE-20404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437866#comment-16437866
]
Andrew Purtell commented on HBASE-20404:
Seeing this in the branch-1 backport too, was just going to come file an issue.
Please fix. Definitely don't do WARN if not something an operator has to be
concerned about.
> Ugly cleanerchore complaint that dir is not empty
> -
>
> Key: HBASE-20404
> URL: https://issues.apache.org/jira/browse/HBASE-20404
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-20404.0.patch, HBASE-20404.1.patch
>
>
> I see these big dirty exceptions in my master log during a long-run Lets
> clean them up (Are they exceptions I as an operator can actually do something
> about? Are they 'problems'? Should they be LOG.warn?)
> {code}
> 2018-04-12 16:02:09,911 WARN [ForkJoinPool-1-worker-15]
> cleaner.CleanerChore: Could not delete dir under
> hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta;
> {}
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException):
>
> `/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta
> is non empty': Directory is not empty
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2848)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1048)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:641)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:847)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:790)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2486)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1489)
> at org.apache.hadoop.ipc.Client.call(Client.java:1435)
> at org.apache.hadoop.ipc.Client.call(Client.java:1345)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy26.delete(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:568)
> at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346)
> at com.sun.proxy.$Proxy27.delete(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)
> ...
> {code}
> Looks like log format is off too...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20389) Move website building flags into a profile
[
https://issues.apache.org/jira/browse/HBASE-20389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-20389:
Attachment: HBASE-20389.1.patch
> Move website building flags into a profile
> --
>
> Key: HBASE-20389
> URL: https://issues.apache.org/jira/browse/HBASE-20389
> Project: HBase
> Issue Type: Improvement
> Components: build, website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Attachments: HBASE-20389.0.patch, HBASE-20389.1.patch
>
>
> we have some "magic" in our website building right now. The script that's
> used bout our automated website build + publish mechanism manually sets a
> bunch of stuff on the maven command line.
> It'd be better to reflect those settings in a maven profile, so that folks
> are less likely to be surprised e.g. when trying to debug a failure in the
> {{site}} goal happens.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20236) [locking] Write-time worst offenders
[
https://issues.apache.org/jira/browse/HBASE-20236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437860#comment-16437860
]
stack commented on HBASE-20236:
---
I moved the MutableSegment locking to its own issue, HBASE-20411
> [locking] Write-time worst offenders
>
>
> Key: HBASE-20236
> URL: https://issues.apache.org/jira/browse/HBASE-20236
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
>Affects Versions: 2.0.0-beta-2
>Reporter: stack
>Priority: Major
> Attachments: HBASE-20236.master.001.patch,
> no_semaphore_vs_semaphore.png
>
>
> Messing w/ my new toy, here are worst offenders locking; they must be bad if
> they show up in this sampling profiler:
> {code}
> 7 Total: 769321884622 (99.24%) samples: 2965
> 8 [ 0] java.util.concurrent.Semaphore$NonfairSync
> 9 [ 1] sun.misc.Unsafe.park
> 10 [ 2] java.util.concurrent.locks.LockSupport.park
> 11 [ 3]
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt
> 12 [ 4]
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly
> 13 [ 5]
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly
> 14 [ 6] java.util.concurrent.Semaphore.acquire
> 15 [ 7]
> org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner
> 16 [ 8] org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run
> 17
> 18 Total: 4284274263 (0.55%) samples: 23543
> 19 [ 0] org.apache.hadoop.hbase.regionserver.MutableSegment
> 20 [ 1] org.apache.hadoop.hbase.ByteBufferKeyValue.getSequenceId
> 21 [ 2] org.apache.hadoop.hbase.regionserver.Segment.updateMetaInfo
> 22 [ 3] org.apache.hadoop.hbase.regionserver.Segment.internalAdd
> 23 [ 4] org.apache.hadoop.hbase.regionserver.MutableSegment.add
> 24 [ 5] org.apache.hadoop.hbase.regionserver.AbstractMemStore.internalAdd
> 25 [ 6] org.apache.hadoop.hbase.regionserver.AbstractMemStore.add
> 26 [ 7] org.apache.hadoop.hbase.regionserver.AbstractMemStore.add
> 27 [ 8] org.apache.hadoop.hbase.regionserver.HStore.add
> 28 [ 9] org.apache.hadoop.hbase.regionserver.HRegion.applyToMemStore
> 29 [10] org.apache.hadoop.hbase.regionserver.HRegion.access$600
> 30 [11]
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.applyFamilyMapToMemStore
> 31 [12]
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.lambda$writeMiniBatchOperationsToMemStore$0
> 32 [13]
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation$$Lambda$442.1445825895.visit
> 33 [14]
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.visitBatchOperations
> 34 [15]
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.writeMiniBatchOperationsToMemStore
> 35 [16]
> org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation.writeMiniBatchOperationsToMemStore
> 36 [17] org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate
> 37 [18] org.apache.hadoop.hbase.regionserver.HRegion.batchMutate
> 38 [19] org.apache.hadoop.hbase.regionserver.HRegion.batchMutate
> 39 [20] org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp
> 40 [21]
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicBatchOp
> 41 [22]
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation
> 42 [23] org.apache.hadoop.hbase.regionserver.RSRpcServices.multi
> 43 [24]
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod
> 44 [25] org.apache.hadoop.hbase.ipc.RpcServer.call
> 45 [26] org.apache.hadoop.hbase.ipc.CallRunner.run
> 46 [27] org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run
> 47 [28] org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run
> 48
> 49 Total: 717708856 (0.09%) samples: 214
> 50 [ 0] java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync
> 51 [ 1] sun.misc.Unsafe.park
> 52 [ 2] java.util.concurrent.locks.LockSupport.park
> 53 [ 3]
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt
> 54 [ 4] java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued
> 55 [ 5] java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire
> 56 [ 6] java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock
> 57 [ 7] org.apache.hadoop.hbase.regionserver.HRegion.blockUpdates
> 58 [ 8]
> org.apache.hadoop.hbase.regionserver.RegionServicesForStores.blockUpdates
> 59 [ 9]
> org.apache.hadoop.hbase.regionserver.CompactingMemStore.flushInMemory
> 60 [10]
> org.apache.hadoop.hbase.regionserver.CompactingMemStore$InMemoryFlushRunnable.run
> 61 [11] java.util.concurrent.ThreadPoolExecutor.runWorker
> 62 [12] java.util.concurrent.ThreadPoolExecutor$Worker.run
> 63 [13]
[jira] [Created] (HBASE-20413) IntegrationTestRegionReplicaReplication fails
stack created HBASE-20413: - Summary: IntegrationTestRegionReplicaReplication fails Key: HBASE-20413 URL: https://issues.apache.org/jira/browse/HBASE-20413 Project: HBase Issue Type: Bug Components: test Reporter: stack Fix For: 2.0.0 Following [~psomogyi]'s inspiration, I've been running unit tests on a GCE instance. IntegrationTestRegionReplicaReplication fails always for branch-2.0. It fails for branch-1 too but for a different looking reason. On branch-2.0, its OOME'ing. Downing the size of ingested data, it seems to hang unable to update because it is unable to flush. Marking this a blocker for now. Fix or disable it for branch-2.0. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18152) [AMv2] Corrupt Procedure WAL file; procedure data stored out of order
[ https://issues.apache.org/jira/browse/HBASE-18152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437836#comment-16437836 ] Josh Elser commented on HBASE-18152: Oh, exciting. I have the pv2 wal still in oldWALs too. Let me look at that. > [AMv2] Corrupt Procedure WAL file; procedure data stored out of order > - > > Key: HBASE-18152 > URL: https://issues.apache.org/jira/browse/HBASE-18152 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-18152.master.001.patch, > hbase-hbase-master-ctr-e138-1518143905142-221855-01-02.hwx.site.log.gz, > pv2-0036.log, pv2-0047.log, > reading_bad_wal.patch > > > I've seen corruption from time-to-time testing. Its rare enough. Often we > can get over it but sometimes we can't. It took me a while to capture an > instance of corruption. Turns out we are write to the WAL out-of-order which > undoes a basic tenet; that WAL content is ordered in line w/ execution. > Below I'll post a corrupt WAL. > Looking at the write-side, there is a lot going on. I'm not clear on how we > could write out of order. Will try and get more insight. Meantime parking > this issue here to fill data into. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18152) [AMv2] Corrupt Procedure WAL file; procedure data stored out of order
[ https://issues.apache.org/jira/browse/HBASE-18152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-18152: --- Attachment: hbase-hbase-master-ctr-e138-1518143905142-221855-01-02.hwx.site.log.gz > [AMv2] Corrupt Procedure WAL file; procedure data stored out of order > - > > Key: HBASE-18152 > URL: https://issues.apache.org/jira/browse/HBASE-18152 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-18152.master.001.patch, > hbase-hbase-master-ctr-e138-1518143905142-221855-01-02.hwx.site.log.gz, > pv2-0036.log, pv2-0047.log, > reading_bad_wal.patch > > > I've seen corruption from time-to-time testing. Its rare enough. Often we > can get over it but sometimes we can't. It took me a while to capture an > instance of corruption. Turns out we are write to the WAL out-of-order which > undoes a basic tenet; that WAL content is ordered in line w/ execution. > Below I'll post a corrupt WAL. > Looking at the write-side, there is a lot going on. I'm not clear on how we > could write out of order. Will try and get more insight. Meantime parking > this issue here to fill data into. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18152) [AMv2] Corrupt Procedure WAL file; procedure data stored out of order
[
https://issues.apache.org/jira/browse/HBASE-18152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437835#comment-16437835
]
Josh Elser commented on HBASE-18152:
I think I saw something similar to this, but it may be less of the bug Stack
outlines and more self-inflicted. I was trying out 2.0.0-rc0 on Hadoop 3.
The version of Hadoop3 I had installed doesn't work with the asyncwal, which I
noticed after tryign to start up the first time. Then, I realized I needed to
set the meta-wal provider configuration option in addition to the normal wal
provider option. IIRC, {{stop-hbase.sh}} hung so I ended up {{kill}}'ing the
process. This caused a bunch of WAL split failures in the Master to get logged
around {{2018-04-12 21:43:01}}.
Interestingly, the Master eventually started complaining around the region
transition that the connection was null (because the server was stopping, I
assume)
{code:java}
2018-04-12 21:43:05,781 WARN [PEWorker-8]
assignment.RegionTransitionProcedure: Retryable error trying to transition:
pid=3586, ppid=3534, state=RUNNABLE:REGION_TRANSITION_QUEUE; AssignProcedure
table=TestTable, region=0e5c6da974e1d3753ba0cb6a8f0db5b1; rit=OPENING,
location=ctr-e138-1518143905142-221855-01-09.hwx.site,16020,1523564755147
java.io.InterruptedIOException: Interrupted after 7 tries while maxAttempts=31
at
org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:173)
at org.apache.hadoop.hbase.client.HTable.get(HTable.java:386)
at org.apache.hadoop.hbase.client.HTable.get(HTable.java:360)
at
org.apache.hadoop.hbase.MetaTableAccessor.getTableState(MetaTableAccessor.java:1076)
at
org.apache.hadoop.hbase.master.TableStateManager.readMetaState(TableStateManager.java:213)
at
org.apache.hadoop.hbase.master.TableStateManager.getTableState(TableStateManager.java:191)
at
org.apache.hadoop.hbase.master.assignment.AssignProcedure.startTransition(AssignProcedure.java:163)
at
org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:295)
at
org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:85)
at
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1453)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1221)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:75)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1741)
{code}
After lots of these, I think I eventually kill'ed the Master and restarted it.
When it came back, I got all of the corrupt pv2 wal errors:
{noformat}
2018-04-12 22:05:24,050 INFO
[master/ctr-e138-1518143905142-221855-01-02:16000] util.FSHDFSUtils:
Recover lease on dfs file
hdfs://ctr-e138-1518143905142-221855-01-02.hwx.site:8020/user/hbase/data/MasterProcWALs/pv2-0008.log
2018-04-12 22:05:24,054 INFO
[master/ctr-e138-1518143905142-221855-01-02:16000] util.FSHDFSUtils:
Recovered lease, attempt=0 on
file=hdfs://ctr-e138-1518143905142-221855-01-02.hwx.site:8020/user/hbase/data/MasterProcWALs/pv2-0008.log
after 3ms
2018-04-12 22:05:24,103 INFO
[master/ctr-e138-1518143905142-221855-01-02:16000]
procedure2.ProcedureExecutor: Recovered WALProcedureStore lease in 55msec
2018-04-12 22:05:24,431 ERROR
[master/ctr-e138-1518143905142-221855-01-02:16000]
procedure2.ProcedureExecutor: Corrupt pid=3580, ppid=3534,
state=RUNNABLE:REGION_TRANSITION_QUEUE; AssignProcedure table=TestTable,
region=71fefffe6b5b3cf1cb6d3328a5a58690
2018-04-12 22:05:24,431 ERROR
[master/ctr-e138-1518143905142-221855-01-02:16000]
procedure2.ProcedureExecutor: Corrupt pid=3586, ppid=3534,
state=RUNNABLE:REGION_TRANSITION_QUEUE; AssignProcedure table=TestTable,
region=0e5c6da974e1d3753ba0cb6a8f0db5b1
2018-04-12 22:05:24,431 ERROR
[master/ctr-e138-1518143905142-221855-01-02:16000]
procedure2.ProcedureExecutor: Corrupt pid=3593, ppid=3534,
state=RUNNABLE:REGION_TRANSITION_DISPATCH; AssignProcedure table=TestTable,
region=f19fbb2563464bcafd528eeba863c10f
2018-04-12 22:05:24,431 ERROR
[master/ctr-e138-1518143905142-221855-01-02:16000]
procedure2.ProcedureExecutor: Corrupt pid=3592, ppid=3534,
state=RUNNABLE:REGION_TRANSITION_DISPATCH; AssignProcedure table=TestTable,
region=7df42ee12cc6ff354144e7915e6f0c5a
{noformat}
Will attach the relevant master log. I have ITBLL running now, but will try to
reproduce these steps and see if I can get something reliably happening.
> [AMv2] Corrupt Procedure WAL file; procedure data stored out of order
> -
[jira] [Commented] (HBASE-20404) Ugly cleanerchore complaint that dir is not empty
[
https://issues.apache.org/jira/browse/HBASE-20404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437810#comment-16437810
]
stack commented on HBASE-20404:
---
If you do top-10 DEBUG I'll keep an eye on it and make a new issue w/
findings. (If TRACE, it'll never be enabled). Thanks.
> Ugly cleanerchore complaint that dir is not empty
> -
>
> Key: HBASE-20404
> URL: https://issues.apache.org/jira/browse/HBASE-20404
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-20404.0.patch, HBASE-20404.1.patch
>
>
> I see these big dirty exceptions in my master log during a long-run Lets
> clean them up (Are they exceptions I as an operator can actually do something
> about? Are they 'problems'? Should they be LOG.warn?)
> {code}
> 2018-04-12 16:02:09,911 WARN [ForkJoinPool-1-worker-15]
> cleaner.CleanerChore: Could not delete dir under
> hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta;
> {}
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException):
>
> `/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta
> is non empty': Directory is not empty
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2848)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1048)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:641)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:847)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:790)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2486)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1489)
> at org.apache.hadoop.ipc.Client.call(Client.java:1435)
> at org.apache.hadoop.ipc.Client.call(Client.java:1345)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy26.delete(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:568)
> at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346)
> at com.sun.proxy.$Proxy27.delete(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)
> ...
> {code}
> Looks like log format is off too...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20404) Ugly cleanerchore complaint that dir is not empty
[
https://issues.apache.org/jira/browse/HBASE-20404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437808#comment-16437808
]
Sean Busbey commented on HBASE-20404:
-
okay, maybe I should add more log information so we have a chance to figure out
what's going on?
I could add a TRACE that prints out the list in our chosen sort order? maybe
DEBUG for the top 10?
> Ugly cleanerchore complaint that dir is not empty
> -
>
> Key: HBASE-20404
> URL: https://issues.apache.org/jira/browse/HBASE-20404
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-20404.0.patch, HBASE-20404.1.patch
>
>
> I see these big dirty exceptions in my master log during a long-run Lets
> clean them up (Are they exceptions I as an operator can actually do something
> about? Are they 'problems'? Should they be LOG.warn?)
> {code}
> 2018-04-12 16:02:09,911 WARN [ForkJoinPool-1-worker-15]
> cleaner.CleanerChore: Could not delete dir under
> hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta;
> {}
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException):
>
> `/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta
> is non empty': Directory is not empty
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2848)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1048)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:641)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:847)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:790)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2486)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1489)
> at org.apache.hadoop.ipc.Client.call(Client.java:1435)
> at org.apache.hadoop.ipc.Client.call(Client.java:1345)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy26.delete(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:568)
> at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346)
> at com.sun.proxy.$Proxy27.delete(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)
> ...
> {code}
> Looks like log format is off too...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437794#comment-16437794 ] Eshcar Hillel commented on HBASE-20188: --- [~anoop.hbase] can you post the command line or script + settings you are using so we can re-produce these results? > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, > HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB(1).pdf, HBase 2.0 > performance evaluation - 8GB.pdf, HBase 2.0 performance evaluation - Basic vs > None_ system settings.pdf, ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > hbase-site.xml, lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, run_ycsb.sh, tree.txt, workloadx, > workloadx > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20412) Update our compliance-checker from 2.1 to 2.4
[ https://issues.apache.org/jira/browse/HBASE-20412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20412: -- Attachment: update.txt > Update our compliance-checker from 2.1 to 2.4 > - > > Key: HBASE-20412 > URL: https://issues.apache.org/jira/browse/HBASE-20412 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: update.txt > > > I thought we had an issue to do this already but I can't find it. > The newer compatibility-checker has the filtering on annotation added by one > of us (or at least asked-for by one-of-us). > i tried it yesterday. Seems to work nicely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20412) Update our compliance-checker from 2.1 to 2.4
stack created HBASE-20412: - Summary: Update our compliance-checker from 2.1 to 2.4 Key: HBASE-20412 URL: https://issues.apache.org/jira/browse/HBASE-20412 Project: HBase Issue Type: Bug Reporter: stack I thought we had an issue to do this already but I can't find it. The newer compatibility-checker has the filtering on annotation added by one of us (or at least asked-for by one-of-us). i tried it yesterday. Seems to work nicely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20410) upgrade protoc compiler to 3.5.1-1
[
https://issues.apache.org/jira/browse/HBASE-20410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437768#comment-16437768
]
Hadoop QA commented on HBASE-20410:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
57s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 3m
59s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
50s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 3m
38s{color} | {color:blue} patch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
17s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 19s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}144m 1s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
45s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}216m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20410 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12918964/HBASE-20410.patch |
| Optional Tests | asflicense javac javadoc unit shadedjars hadoopcheck
xml compile refguide |
| uname | Linux c8e986ee0f16 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4
19:09:19 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 5a633adffe |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_162 |
| refguide |
https://builds.apache.org/job/PreCommit-HBASE-Build/12440/artifact/patchprocess/branch-site/book.html
|
| refguide |
[jira] [Updated] (HBASE-20291) Fix for The POM for net.minidev:json-smart:jar:2.3-SNAPSHOT is missing, no dependency information available with hadoop.profile=3.0
[
https://issues.apache.org/jira/browse/HBASE-20291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-20291:
-
Resolution: Invalid
Status: Resolved (was: Patch Available)
> Fix for The POM for net.minidev:json-smart:jar:2.3-SNAPSHOT is missing, no
> dependency information available with hadoop.profile=3.0
> ---
>
> Key: HBASE-20291
> URL: https://issues.apache.org/jira/browse/HBASE-20291
> Project: HBase
> Issue Type: Bug
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-20291.v01.patch, HBASE-20291.v02.patch
>
>
> receiving message
> {code:java}
> The POM for net.minidev:json-smart:jar:2.3-SNAPSHOT is missing, no dependency
> information available{code}
> when running with
> {code:java}
> mvn clean install -DHBasePatchProcess -Dhadoop-three.version=3.0.0
> -Dhadoop.profile=3.0 -DskipTests{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20291) Fix for The POM for net.minidev:json-smart:jar:2.3-SNAPSHOT is missing, no dependency information available with hadoop.profile=3.0
[
https://issues.apache.org/jira/browse/HBASE-20291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437744#comment-16437744
]
Artem Ervits commented on HBASE-20291:
--
this issue is only applicable to hadoop-3.0.0. I do not see the issue with
hadoop.version=3.1.0. Please see the Hadoop jiras earlier in the thread for
reference.
> Fix for The POM for net.minidev:json-smart:jar:2.3-SNAPSHOT is missing, no
> dependency information available with hadoop.profile=3.0
> ---
>
> Key: HBASE-20291
> URL: https://issues.apache.org/jira/browse/HBASE-20291
> Project: HBase
> Issue Type: Bug
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-20291.v01.patch, HBASE-20291.v02.patch
>
>
> receiving message
> {code:java}
> The POM for net.minidev:json-smart:jar:2.3-SNAPSHOT is missing, no dependency
> information available{code}
> when running with
> {code:java}
> mvn clean install -DHBasePatchProcess -Dhadoop-three.version=3.0.0
> -Dhadoop.profile=3.0 -DskipTests{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437723#comment-16437723 ] stack commented on HBASE-20411: --- Attached svg is from ycsb load phase. Shows that its worth spending time on this synchronize. In this sample, it looks like we show as stuck here 62% of the time. The remainder is mostly given over to the Semaphore that we have in RPC scheduling. > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 41901.lock.svg > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20411: -- Attachment: 41901.lock.svg > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 41901.lock.svg > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20411) Ameliorate MutableSegment synchronize
stack created HBASE-20411: - Summary: Ameliorate MutableSegment synchronize Key: HBASE-20411 URL: https://issues.apache.org/jira/browse/HBASE-20411 Project: HBase Issue Type: Bug Reporter: stack This item is migrated from HBASE-20236 so it gets dedicated issue. Let me upload evidence that has this synchronize as a stake in our write-time perf. I'll migrate the patch I posted with updates that come of comments posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20364) nightly job gives old results or no results for stages that timeout on SCM
[
https://issues.apache.org/jira/browse/HBASE-20364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437692#comment-16437692
]
Mike Drob commented on HBASE-20364:
---
to make sure i understand... we write a default failure and then when we finish
we write the real result. but if we don't finish and time out then we still
have the assumed failure message written?
do we know that this will only happen in case of timeout? will sometime like
OOM also cause the stage to fail without writing a message or will that get a
known failure state? trying to understand if we can write a more specific
initial fail message
> nightly job gives old results or no results for stages that timeout on SCM
> --
>
> Key: HBASE-20364
> URL: https://issues.apache.org/jira/browse/HBASE-20364
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Attachments: HBASE-20364.0.patch
>
>
> seen in the branch-2.0 nightly report for HBASE-18828:
>
> {quote}
> Results for branch branch-2.0
> [build #143 on
> builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/143/]:
> (x) *\{color:red}-1 overall\{color}*
>
> details (if available):
> (/) \{color:green}+1 general checks\{color}
> -- For more information [see general
> report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/140//General_Nightly_Build_Report/]
>
> (/) \{color:green}+1 jdk8 hadoop2 checks\{color}
> -- For more information [see jdk8 (hadoop2)
> report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/143//JDK8_Nightly_Build_Report_(Hadoop2)/]
> (/) \{color:green}+1 jdk8 hadoop3 checks\{color}
> -- For more information [see jdk8 (hadoop3)
> report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/143//JDK8_Nightly_Build_Report_(Hadoop3)/]
>
> {quote}
>
> -1 for the overall build was correct. build #143 failed both the general
> check and the source tarball check.
>
> but in the posted comment, we get a false "passing" that links to the general
> result from build #140. and we get no result for the source tarball at all.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20410) upgrade protoc compiler to 3.5.1-1
[ https://issues.apache.org/jira/browse/HBASE-20410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437691#comment-16437691 ] stack commented on HBASE-20410: --- Thanks [~mdrob] I'm planning on a new RC Monday or so but don't tell anyone else they'll stop trying current RC. (I changed the fix version). > upgrade protoc compiler to 3.5.1-1 > -- > > Key: HBASE-20410 > URL: https://issues.apache.org/jira/browse/HBASE-20410 > Project: HBase > Issue Type: Bug > Components: build, dependencies, Protobufs >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Critical > Fix For: 3.0.0, 2.1.0, 2.0.0 > > Attachments: HBASE-20410.patch > > > See HBASE-20356 > After doing the cleanup there, I was informed that there's a 3.5.1-1 version > of the compiler binaries that work on rhel6, so let's just go to that. Wish I > knew about it beforehand. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20410) upgrade protoc compiler to 3.5.1-1
[ https://issues.apache.org/jira/browse/HBASE-20410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20410: -- Fix Version/s: (was: 2.0.1) 2.0.0 > upgrade protoc compiler to 3.5.1-1 > -- > > Key: HBASE-20410 > URL: https://issues.apache.org/jira/browse/HBASE-20410 > Project: HBase > Issue Type: Bug > Components: build, dependencies, Protobufs >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Critical > Fix For: 3.0.0, 2.1.0, 2.0.0 > > Attachments: HBASE-20410.patch > > > See HBASE-20356 > After doing the cleanup there, I was informed that there's a 3.5.1-1 version > of the compiler binaries that work on rhel6, so let's just go to that. Wish I > knew about it beforehand. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20410) upgrade protoc compiler to 3.5.1-1
[ https://issues.apache.org/jira/browse/HBASE-20410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-20410: -- Resolution: Fixed Fix Version/s: 2.0.1 2.1.0 3.0.0 Status: Resolved (was: Patch Available) thanks, updated comment on commit, pushed to branch-2.0+ [~stack] FYI, if we have another RC please update fix version accordingly. > upgrade protoc compiler to 3.5.1-1 > -- > > Key: HBASE-20410 > URL: https://issues.apache.org/jira/browse/HBASE-20410 > Project: HBase > Issue Type: Bug > Components: build, dependencies, Protobufs >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Critical > Fix For: 3.0.0, 2.1.0, 2.0.1 > > Attachments: HBASE-20410.patch > > > See HBASE-20356 > After doing the cleanup there, I was informed that there's a 3.5.1-1 version > of the compiler binaries that work on rhel6, so let's just go to that. Wish I > knew about it beforehand. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20145) HMaster start fails with IllegalStateException when HADOOP_HOME is set
[
https://issues.apache.org/jira/browse/HBASE-20145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437675#comment-16437675
]
Wei-Chiu Chuang commented on HBASE-20145:
-
Posted a patch to log additional messages when
1. the stream class does not implement StreamCapabilities,
2. the result of StreamCapabilities test. Log the output stream class and the
associated wrapped stream class.
The 2nd item will be useful when troubleshooting hbase problem on a HDFS
erasure coded cluster.
> HMaster start fails with IllegalStateException when HADOOP_HOME is set
> --
>
> Key: HBASE-20145
> URL: https://issues.apache.org/jira/browse/HBASE-20145
> Project: HBase
> Issue Type: Bug
> Environment: HBase-2.0-beta1.
> Hadoop trunk version.
> java version "1.8.0_144"
>Reporter: Rohith Sharma K S
>Assignee: Wei-Chiu Chuang
>Priority: Critical
> Attachments: HBASE-20145.master.001.patch
>
>
> It is observed that HMaster start is failed when HADOOP_HOME is set as env
> while starting HMaster. HADOOP_HOME is pointing to Hadoop trunk version.
> {noformat}
> 2018-03-07 16:59:52,654 ERROR [master//10.200.4.200:16000] master.HMaster:
> Failed to become active master
> java.lang.IllegalStateException: The procedure WAL relies on the ability to
> hsync for proper operation during component failures, but the underlying
> filesystem does not support doing so. Please check the config value of
> 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness
> and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount
> that can provide it.
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1036)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:374)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.start(ProcedureExecutor.java:532)
> at
> org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1232)
> at
> org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1145)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:837)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2026)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:547)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> The same configs is working in HBase-1.2.6 build properly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20398) Redirect doesn't work on web UI
[
https://issues.apache.org/jira/browse/HBASE-20398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437665#comment-16437665
]
stack commented on HBASE-20398:
---
Good one [~balazs.meszaros] Removal looks like it was unintentional?
> Redirect doesn't work on web UI
> ---
>
> Key: HBASE-20398
> URL: https://issues.apache.org/jira/browse/HBASE-20398
> Project: HBase
> Issue Type: Bug
> Components: UI
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Balazs Meszaros
>Priority: Major
>
> {{table.jsp}} contains _"Go Back, or wait for the redirect."_ string after we
> invoke compaction, split, etc. Previously it redirected to the previous page
> after 5 seconds. The string is there currently, but nothing happens.
> After digging into the code, a found the following:
> - {{ecce7c2}} (HBASE-3948) it was introduced,
> - {{3bd9191}} (HBASE-9850) it was refactored,
> - {{3b2b22b}} (HBASE-19291) it was removed.
> This string appears on {{rsgroup.jsp}}, {{snapshot.jsp}} and {{table.jsp}}.
> We should fix them by removing the string, or by adding redirection again.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20364) nightly job gives old results or no results for stages that timeout on SCM
[
https://issues.apache.org/jira/browse/HBASE-20364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437657#comment-16437657
]
stack commented on HBASE-20364:
---
+1 Looks helpful. Pity about the duplication but I think I know the why.
> nightly job gives old results or no results for stages that timeout on SCM
> --
>
> Key: HBASE-20364
> URL: https://issues.apache.org/jira/browse/HBASE-20364
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Attachments: HBASE-20364.0.patch
>
>
> seen in the branch-2.0 nightly report for HBASE-18828:
>
> {quote}
> Results for branch branch-2.0
> [build #143 on
> builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/143/]:
> (x) *\{color:red}-1 overall\{color}*
>
> details (if available):
> (/) \{color:green}+1 general checks\{color}
> -- For more information [see general
> report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/140//General_Nightly_Build_Report/]
>
> (/) \{color:green}+1 jdk8 hadoop2 checks\{color}
> -- For more information [see jdk8 (hadoop2)
> report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/143//JDK8_Nightly_Build_Report_(Hadoop2)/]
> (/) \{color:green}+1 jdk8 hadoop3 checks\{color}
> -- For more information [see jdk8 (hadoop3)
> report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/143//JDK8_Nightly_Build_Report_(Hadoop3)/]
>
> {quote}
>
> -1 for the overall build was correct. build #143 failed both the general
> check and the source tarball check.
>
> but in the posted comment, we get a false "passing" that links to the general
> result from build #140. and we get no result for the source tarball at all.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20394) HBase over rides the value of HBASE_OPTS (if any) set by client
[
https://issues.apache.org/jira/browse/HBASE-20394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437654#comment-16437654
]
stack commented on HBASE-20394:
---
Picked to branch-2.0 for the operators.
> HBase over rides the value of HBASE_OPTS (if any) set by client
> ---
>
> Key: HBASE-20394
> URL: https://issues.apache.org/jira/browse/HBASE-20394
> Project: HBase
> Issue Type: Bug
> Components: Operability
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-20394.master.001.patch
>
>
> Currently HBase will over ride the value of HBASE_OPTS (if any) set by user
> {code:java}
> export HBASE_OPTS="-XX:+UseConcMarkSweepGC" {code}
> [See
> hbase-env.sh|https://github.com/apache/hbase/blob/68726b0ee3ef3eb52d32481444e64236c5a9e733/conf/hbase-env.sh#L43]
>
> But, a user may have the following set in his environment:
> {code:java}
> HBASE_OPTS="-Xmn512m"{code}
> While starting the processes, HBase will internally over-ride the existing
> HBASE_OPTS value with the one set in hbase-env.sh
>
> Instead of over-riding we can have the following in hbase-env.sh:
> {code:java}
> export HBASE_OPTS="$HBASE_OPTS -XX:+UseConcMarkSweepGC"{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20394) HBase over rides the value of HBASE_OPTS (if any) set by client
[
https://issues.apache.org/jira/browse/HBASE-20394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20394:
--
Fix Version/s: (was: 2.1.0)
(was: 3.0.0)
2.0.0
> HBase over rides the value of HBASE_OPTS (if any) set by client
> ---
>
> Key: HBASE-20394
> URL: https://issues.apache.org/jira/browse/HBASE-20394
> Project: HBase
> Issue Type: Bug
> Components: Operability
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-20394.master.001.patch
>
>
> Currently HBase will over ride the value of HBASE_OPTS (if any) set by user
> {code:java}
> export HBASE_OPTS="-XX:+UseConcMarkSweepGC" {code}
> [See
> hbase-env.sh|https://github.com/apache/hbase/blob/68726b0ee3ef3eb52d32481444e64236c5a9e733/conf/hbase-env.sh#L43]
>
> But, a user may have the following set in his environment:
> {code:java}
> HBASE_OPTS="-Xmn512m"{code}
> While starting the processes, HBase will internally over-ride the existing
> HBASE_OPTS value with the one set in hbase-env.sh
>
> Instead of over-riding we can have the following in hbase-env.sh:
> {code:java}
> export HBASE_OPTS="$HBASE_OPTS -XX:+UseConcMarkSweepGC"{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20394) HBase over rides the value of HBASE_OPTS (if any) set by client
[
https://issues.apache.org/jira/browse/HBASE-20394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20394:
--
Component/s: Operability
> HBase over rides the value of HBASE_OPTS (if any) set by client
> ---
>
> Key: HBASE-20394
> URL: https://issues.apache.org/jira/browse/HBASE-20394
> Project: HBase
> Issue Type: Bug
> Components: Operability
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-20394.master.001.patch
>
>
> Currently HBase will over ride the value of HBASE_OPTS (if any) set by user
> {code:java}
> export HBASE_OPTS="-XX:+UseConcMarkSweepGC" {code}
> [See
> hbase-env.sh|https://github.com/apache/hbase/blob/68726b0ee3ef3eb52d32481444e64236c5a9e733/conf/hbase-env.sh#L43]
>
> But, a user may have the following set in his environment:
> {code:java}
> HBASE_OPTS="-Xmn512m"{code}
> While starting the processes, HBase will internally over-ride the existing
> HBASE_OPTS value with the one set in hbase-env.sh
>
> Instead of over-riding we can have the following in hbase-env.sh:
> {code:java}
> export HBASE_OPTS="$HBASE_OPTS -XX:+UseConcMarkSweepGC"{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20145) HMaster start fails with IllegalStateException when HADOOP_HOME is set
[
https://issues.apache.org/jira/browse/HBASE-20145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HBASE-20145:
Attachment: HBASE-20145.master.001.patch
> HMaster start fails with IllegalStateException when HADOOP_HOME is set
> --
>
> Key: HBASE-20145
> URL: https://issues.apache.org/jira/browse/HBASE-20145
> Project: HBase
> Issue Type: Bug
> Environment: HBase-2.0-beta1.
> Hadoop trunk version.
> java version "1.8.0_144"
>Reporter: Rohith Sharma K S
>Assignee: Wei-Chiu Chuang
>Priority: Critical
> Attachments: HBASE-20145.master.001.patch
>
>
> It is observed that HMaster start is failed when HADOOP_HOME is set as env
> while starting HMaster. HADOOP_HOME is pointing to Hadoop trunk version.
> {noformat}
> 2018-03-07 16:59:52,654 ERROR [master//10.200.4.200:16000] master.HMaster:
> Failed to become active master
> java.lang.IllegalStateException: The procedure WAL relies on the ability to
> hsync for proper operation during component failures, but the underlying
> filesystem does not support doing so. Please check the config value of
> 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness
> and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount
> that can provide it.
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1036)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:374)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.start(ProcedureExecutor.java:532)
> at
> org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1232)
> at
> org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1145)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:837)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2026)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:547)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> The same configs is working in HBase-1.2.6 build properly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20404) Ugly cleanerchore complaint that dir is not empty
[
https://issues.apache.org/jira/browse/HBASE-20404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437616#comment-16437616
]
stack commented on HBASE-20404:
---
bq. We end up in that block if we hit literally any kind of IOException. I
think we need to print it, because the problem could be e.g. HDFS in safe mode
or some other thing that an operator could do something about.
Fair enough.
bq. Are we seeing the specific IOException of "you're trying to delete a
directory before the contents are deleted" a bunch of times because of a bad
sort order? i.e. does a directory sort ahead of the stuff in the directory
because of the "prioritize by size" check?
Zero study done by me. Doesn't happen that frequently... Just ugly when it does
happen... Only exception in my log.
{code}
2018-04-12 04:42:06,543 WARN [ForkJoinPool-1-worker-7] cleaner.CleanerChore:
Could not delete dir under
hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/hbase/meta/1588230740;
{}
2018-04-12 06:52:10,365 WARN [ForkJoinPool-1-worker-22] cleaner.CleanerChore:
Could not delete dir under
hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/03a723d5e024f4db900a185ccf946910/big;
{}
2018-04-12 08:22:07,378 WARN [ForkJoinPool-1-worker-22] cleaner.CleanerChore:
Could not delete dir under
hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/b10528e8b4ee831ec7db083823570f09/meta;
{}
2018-04-12 10:32:10,324 WARN [ForkJoinPool-1-worker-26] cleaner.CleanerChore:
Could not delete dir under
hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/adf743126026818a92350a53d5176897/meta;
{}
2018-04-12 14:02:10,534 WARN [ForkJoinPool-1-worker-14] cleaner.CleanerChore:
Could not delete dir under
hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553;
{}
2018-04-12 15:42:10,314 WARN [ForkJoinPool-1-worker-14] cleaner.CleanerChore:
Could not delete dir under
hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/639945dfda2ed3c7e783b02ef40d91fd/tiny;
{}
2018-04-12 15:42:10,329 WARN [ForkJoinPool-1-worker-0] cleaner.CleanerChore:
Could not delete dir under
hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/614d71f722cb9f8865b4eafdf180/tiny;
{}
2018-04-12 15:52:15,532 WARN [ForkJoinPool-1-worker-13] cleaner.CleanerChore:
Could not delete dir under
hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/tsdb/82e18bf2a034476b116226b0f7071738/t;
{}
2018-04-12 16:02:09,911 WARN [ForkJoinPool-1-worker-15] cleaner.CleanerChore:
Could not delete dir under
hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta;
{}
2018-04-12 16:02:09,926 WARN [ForkJoinPool-1-worker-12] cleaner.CleanerChore:
Could not delete dir under
hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/45e381c40cd3ce0e81947ad40369adfd/meta;
{}
2018-04-12 16:02:09,951 WARN [ForkJoinPool-1-worker-29] cleaner.CleanerChore:
Could not delete dir under
hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/45e381c40cd3ce0e81947ad40369adfd/big;
{}
{code}
> Ugly cleanerchore complaint that dir is not empty
> -
>
> Key: HBASE-20404
> URL: https://issues.apache.org/jira/browse/HBASE-20404
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-20404.0.patch, HBASE-20404.1.patch
>
>
> I see these big dirty exceptions in my master log during a long-run Lets
> clean them up (Are they exceptions I as an operator can actually do something
> about? Are they 'problems'? Should they be LOG.warn?)
> {code}
> 2018-04-12 16:02:09,911 WARN [ForkJoinPool-1-worker-15]
> cleaner.CleanerChore: Could not delete dir under
> hdfs://ve0524.halxg.cloudera.com:8020/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta;
> {}
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException):
>
> `/hbase/archive/data/default/IntegrationTestBigLinkedList/1e24549061df3adc4858fbcaf1929553/meta
> is non empty': Directory is not empty
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2848)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1048)
> at
>
[jira] [Commented] (HBASE-15557) document SyncTable in ref guide
[
https://issues.apache.org/jira/browse/HBASE-15557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437596#comment-16437596
]
Roland Teague commented on HBASE-15557:
---
[~davelatham] Can you add documentation for this MR tool to the HBase Ref Guide
on how to use the tool? This have been open for 2 years now.
> document SyncTable in ref guide
> ---
>
> Key: HBASE-15557
> URL: https://issues.apache.org/jira/browse/HBASE-15557
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 1.2.0
>Reporter: Sean Busbey
>Priority: Critical
>
> The docs for SyncTable are insufficient. Brief description from [~davelatham]
> HBASE-13639 comment:
> {quote}
> Sorry for the lack of better documentation, Abhishek Soni. Thanks for
> bringing it up. I'll try to provide a better explanation. You may have
> already seen it, but if not, the design doc linked in the description above
> may also give you some better clues as to how it should be used.
> Briefly, the feature is intended to start with a pair of tables in remote
> clusters that are already substantially similar and make them identical by
> comparing hashes of the data and copying only the diffs instead of having to
> copy the entire table. So it is targeted at a very specific use case (with
> some work it could generalize to cover things like CopyTable and
> VerifyRepliaction but it's not there yet). To use it, you choose one table to
> be the "source", and the other table is the "target". After the process is
> complete the target table should end up being identical to the source table.
> In the source table's cluster, run
> org.apache.hadoop.hbase.mapreduce.HashTable and pass it the name of the
> source table and an output directory in HDFS. HashTable will scan the source
> table, break the data up into row key ranges (default of 8kB per range) and
> produce a hash of the data for each range.
> Make the hashes available to the target cluster - I'd recommend using DistCp
> to copy it across.
> In the target table's cluster, run
> org.apache.hadoop.hbase.mapreduce.SyncTable and pass it the directory where
> you put the hashes, and the names of the source and destination tables. You
> will likely also need to specify the source table's ZK quorum via the
> --sourcezkcluster option. SyncTable will then read the hash information, and
> compute the hashes of the same row ranges for the target table. For any row
> range where the hash fails to match, it will open a remote scanner to the
> source table, read the data for that range, and do Puts and Deletes to the
> target table to update it to match the source.
> I hope that clarifies it a bit. Let me know if you need a hand. If anyone
> wants to work on getting some documentation into the book, I can try to write
> some more but would love a hand on turning it into an actual book patch.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20145) HMaster start fails with IllegalStateException when HADOOP_HOME is set
[
https://issues.apache.org/jira/browse/HBASE-20145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437565#comment-16437565
]
Artem Ervits edited comment on HBASE-20145 at 4/13/18 5:10 PM:
---
I just got the same issue and can confirm that it's mixing jars on the classpath
{code:java}
/opt/hbase/hbase-2.0.0/lib/hadoop-annotations-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-auth-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-client-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-common-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-common-2.7.4-tests.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-distcp-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-hdfs-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-hdfs-2.7.4-tests.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-mapreduce-client-app-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-mapreduce-client-common-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-mapreduce-client-core-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-mapreduce-client-hs-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-mapreduce-client-jobclient-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-mapreduce-client-shuffle-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-minicluster-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-yarn-api-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-yarn-client-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-yarn-common-2.7.4.jar{code}
the key issue for me was omission of -Dhadoop.profile=3.0 as the stackoverflow
article by [~rohithsharma] specifies. I only used -Dhadoop.version=3.1.0.
was (Author: dbist13):
I just got the same issue and can confirm that it's mixing jars on the classpath
{code:java}
/opt/hbase/hbase-2.0.0/lib/hadoop-annotations-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-auth-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-client-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-common-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-common-2.7.4-tests.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-distcp-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-hdfs-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-hdfs-2.7.4-tests.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-mapreduce-client-app-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-mapreduce-client-common-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-mapreduce-client-core-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-mapreduce-client-hs-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-mapreduce-client-jobclient-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-mapreduce-client-shuffle-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-minicluster-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-yarn-api-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-yarn-client-2.7.4.jar:/opt/hbase/hbase-2.0.0/lib/hadoop-yarn-common-2.7.4.jar{code}
> HMaster start fails with IllegalStateException when HADOOP_HOME is set
> --
>
> Key: HBASE-20145
> URL: https://issues.apache.org/jira/browse/HBASE-20145
> Project: HBase
> Issue Type: Bug
> Environment: HBase-2.0-beta1.
> Hadoop trunk version.
> java version "1.8.0_144"
>Reporter: Rohith Sharma K S
>Assignee: Wei-Chiu Chuang
>Priority: Critical
>
> It is observed that HMaster start is failed when HADOOP_HOME is set as env
> while starting HMaster. HADOOP_HOME is pointing to Hadoop trunk version.
> {noformat}
> 2018-03-07 16:59:52,654 ERROR [master//10.200.4.200:16000] master.HMaster:
> Failed to become active master
> java.lang.IllegalStateException: The procedure WAL relies on the ability to
> hsync for proper operation during component failures, but the underlying
> filesystem does not support doing so. Please check the config value of
> 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness
> and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount
> that can provide it.
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1036)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:374)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.start(ProcedureExecutor.java:532)
> at
> org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1232)
> at
> org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1145)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:837)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2026)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:547)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> The same configs is working in HBase-1.2.6 build properly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20145) HMaster start fails with IllegalStateException when HADOOP_HOME is set
[
https://issues.apache.org/jira/browse/HBASE-20145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437475#comment-16437475
]
Wei-Chiu Chuang edited comment on HBASE-20145 at 4/13/18 4:53 PM:
--
Thanks [~rohithsharma] now I understand why:
Turns out that if you point env HADOOP_HOME to a Hadoop3 path with hbase2.0
convenience binary, the classpath will contain both Hadoop 2.7.4 and Hadoop 3
jar files. Effectively causing class conflicts.
Since Hadoop 3 jars are available, HBase is able to query StreamCapabilities of
the output stream. However, because Hadoop 2.7.4 jars is loaded before Hadoop 3
jars in classpath, it loaded the HdfsDataOutputStream class implemented in
Hadoop 2.7.4, which does not inherit from StreamCapabilities.
{code:java}
if (StreamCapabilities.CLASS.isAssignableFrom(stream.getClass())) {
try {
result = ((Boolean)StreamCapabilities.METHOD.invoke(stream,
capability)).booleanValue();
} catch
(IllegalAccessException|IllegalArgumentException|InvocationTargetException
exception) {
LOG.warn("Your Hadoop installation's StreamCapabilities
implementation doesn't match " +
"our understanding of how it's supposed to work. Please file a
JIRA and include " +
"the following stack trace. In the mean time we're interpreting
this behavior " +
"difference as a lack of capability support, which will probably
cause a failure.",
exception);
}
}
{code}
This is not a problem in the code itself. But the code can be improved to log a
warning message when output stream class does not inherit from
StreamCapabilities.
was (Author: jojochuang):
Thanks [~rohithsharma] now I understand why:
Turns out that if you point env HADOOP_HOME to a Hadoop3 path with hbase2.0
convenience binary, the classpath will contain both Hadoop 2.7.4 and Hadoop 3
jar files. Effectively causing class conflicts.
Since Hadoop 3 jars are available, HBase is able to query StreamCapabilities of
the output stream. However, because Hadoop 2.7.4 jars is loaded before Hadoop 3
jars in classpath, it loaded the HdfsDataOutputStream class implemented in
Hadoop 2.7.4, which does not inherit from StreamCapabilities.
{code:java}
if (StreamCapabilities.CLASS.isAssignableFrom(stream.getClass())) {
try {
LOG.warn("stream type = " +
stream.getWrappedStream().getClass().getCanonicalName());
result = ((Boolean)StreamCapabilities.METHOD.invoke(stream,
capability)).booleanValue();
} catch
(IllegalAccessException|IllegalArgumentException|InvocationTargetException
exception) {
LOG.warn("Your Hadoop installation's StreamCapabilities
implementation doesn't match " +
"our understanding of how it's supposed to work. Please file a
JIRA and include " +
"the following stack trace. In the mean time we're interpreting
this behavior " +
"difference as a lack of capability support, which will probably
cause a failure.",
exception);
}
}
{code}
This is not a problem in the code itself. But the code can be improved to log
warning message when output stream class does not inherit from
StreamCapabilities.
> HMaster start fails with IllegalStateException when HADOOP_HOME is set
> --
>
> Key: HBASE-20145
> URL: https://issues.apache.org/jira/browse/HBASE-20145
> Project: HBase
> Issue Type: Bug
> Environment: HBase-2.0-beta1.
> Hadoop trunk version.
> java version "1.8.0_144"
>Reporter: Rohith Sharma K S
>Assignee: Wei-Chiu Chuang
>Priority: Critical
>
> It is observed that HMaster start is failed when HADOOP_HOME is set as env
> while starting HMaster. HADOOP_HOME is pointing to Hadoop trunk version.
> {noformat}
> 2018-03-07 16:59:52,654 ERROR [master//10.200.4.200:16000] master.HMaster:
> Failed to become active master
> java.lang.IllegalStateException: The procedure WAL relies on the ability to
> hsync for proper operation during component failures, but the underlying
> filesystem does not support doing so. Please check the config value of
> 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness
> and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount
> that can provide it.
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1036)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:374)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.start(ProcedureExecutor.java:532)
> at
> org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1232)
>
