[jira] [Updated] (HBASE-20881) Introduce a region transition procedure to handle all the state transition for a region
[ https://issues.apache.org/jira/browse/HBASE-20881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20881: -- Attachment: HBASE-20881-v11.patch > Introduce a region transition procedure to handle all the state transition > for a region > --- > > Key: HBASE-20881 > URL: https://issues.apache.org/jira/browse/HBASE-20881 > Project: HBase > Issue Type: Sub-task > Components: amv2, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-20881-v1.patch, HBASE-20881-v10.patch, > HBASE-20881-v11.patch, HBASE-20881-v2.patch, HBASE-20881-v3.patch, > HBASE-20881-v4.patch, HBASE-20881-v4.patch, HBASE-20881-v5.patch, > HBASE-20881-v6.patch, HBASE-20881-v7.patch, HBASE-20881-v7.patch, > HBASE-20881-v8.patch, HBASE-20881-v9.patch, HBASE-20881.patch > > > Now have an AssignProcedure, an UnssignProcedure, and also a > MoveRegionProcedure which schedules an AssignProcedure and an > UnssignProcedure to move a region. This makes the logic a bit complicated, as > MRP is not a RIT, so when SCP can not interrupt it directly... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20881) Introduce a region transition procedure to handle all the state transition for a region
[ https://issues.apache.org/jira/browse/HBASE-20881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580788#comment-16580788 ] Duo Zhang commented on HBASE-20881: --- Added a TestCloseRegionWhileRSCrash for testing the backoff mechanism. > Introduce a region transition procedure to handle all the state transition > for a region > --- > > Key: HBASE-20881 > URL: https://issues.apache.org/jira/browse/HBASE-20881 > Project: HBase > Issue Type: Sub-task > Components: amv2, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-20881-v1.patch, HBASE-20881-v10.patch, > HBASE-20881-v11.patch, HBASE-20881-v2.patch, HBASE-20881-v3.patch, > HBASE-20881-v4.patch, HBASE-20881-v4.patch, HBASE-20881-v5.patch, > HBASE-20881-v6.patch, HBASE-20881-v7.patch, HBASE-20881-v7.patch, > HBASE-20881-v8.patch, HBASE-20881-v9.patch, HBASE-20881.patch > > > Now have an AssignProcedure, an UnssignProcedure, and also a > MoveRegionProcedure which schedules an AssignProcedure and an > UnssignProcedure to move a region. This makes the logic a bit complicated, as > MRP is not a RIT, so when SCP can not interrupt it directly... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20881) Introduce a region transition procedure to handle all the state transition for a region
[
https://issues.apache.org/jira/browse/HBASE-20881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580797#comment-16580797
]
Hadoop QA commented on HBASE-20881:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} HBASE-20881 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-20881 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12935660/HBASE-20881-v11.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/14045/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Introduce a region transition procedure to handle all the state transition
> for a region
> ---
>
> Key: HBASE-20881
> URL: https://issues.apache.org/jira/browse/HBASE-20881
> Project: HBase
> Issue Type: Sub-task
> Components: amv2, proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-20881-v1.patch, HBASE-20881-v10.patch,
> HBASE-20881-v11.patch, HBASE-20881-v2.patch, HBASE-20881-v3.patch,
> HBASE-20881-v4.patch, HBASE-20881-v4.patch, HBASE-20881-v5.patch,
> HBASE-20881-v6.patch, HBASE-20881-v7.patch, HBASE-20881-v7.patch,
> HBASE-20881-v8.patch, HBASE-20881-v9.patch, HBASE-20881.patch
>
>
> Now have an AssignProcedure, an UnssignProcedure, and also a
> MoveRegionProcedure which schedules an AssignProcedure and an
> UnssignProcedure to move a region. This makes the logic a bit complicated, as
> MRP is not a RIT, so when SCP can not interrupt it directly...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21040) Replace call to printStackTrace() with proper logger call
[
https://issues.apache.org/jira/browse/HBASE-21040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580898#comment-16580898
]
Hadoop QA commented on HBASE-21040:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
12s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} The patch hbase-client passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch hbase-zookeeper passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} The patch hbase-procedure passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} The patch hbase-server passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} hbase-mapreduce: The patch generated 0 new + 85
unchanged - 4 fixed = 85 total (was 89) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} The patch hbase-thrift passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} The patch hbase-backup passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} hbase-examples: The patch generated 0 new + 248
unchanged - 8 fixed = 248 total (was 256) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
20s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 29s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {c
[jira] [Updated] (HBASE-20881) Introduce a region transition procedure to handle all the state transition for a region
[ https://issues.apache.org/jira/browse/HBASE-20881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20881: -- Attachment: (was: HBASE-20881-v11.patch) > Introduce a region transition procedure to handle all the state transition > for a region > --- > > Key: HBASE-20881 > URL: https://issues.apache.org/jira/browse/HBASE-20881 > Project: HBase > Issue Type: Sub-task > Components: amv2, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-20881-v1.patch, HBASE-20881-v10.patch, > HBASE-20881-v2.patch, HBASE-20881-v3.patch, HBASE-20881-v4.patch, > HBASE-20881-v4.patch, HBASE-20881-v5.patch, HBASE-20881-v6.patch, > HBASE-20881-v7.patch, HBASE-20881-v7.patch, HBASE-20881-v8.patch, > HBASE-20881-v9.patch, HBASE-20881.patch > > > Now have an AssignProcedure, an UnssignProcedure, and also a > MoveRegionProcedure which schedules an AssignProcedure and an > UnssignProcedure to move a region. This makes the logic a bit complicated, as > MRP is not a RIT, so when SCP can not interrupt it directly... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20993) [Auth] IPC client fallback to simple auth allowed doesn't work
[
https://issues.apache.org/jira/browse/HBASE-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580947#comment-16580947
]
Reid Chan commented on HBASE-20993:
---
Let's talk in engineering way, i show my code in patch(not have enough cycles
to test), it may work or not, but it does speak out my thought.
> [Auth] IPC client fallback to simple auth allowed doesn't work
> --
>
> Key: HBASE-20993
> URL: https://issues.apache.org/jira/browse/HBASE-20993
> Project: HBase
> Issue Type: Bug
> Components: Client, security
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Jack Bearden
>Priority: Critical
> Attachments: HBASE-20993.001.patch, HBASE-20993.branch-1.002.patch,
> HBASE-20993.branch-1.2.001.patch
>
>
> It is easily reproducible.
> client's hbase-site.xml: hadoop.security.authentication:kerberos,
> hbase.security.authentication:kerberos,
> hbase.ipc.client.fallback-to-simple-auth-allowed:true, keytab and principal
> are right set
> A simple auth hbase cluster, a kerberized hbase client application.
> application trying to r/w/c/d table will have following exception:
> {code}
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:617)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:162)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:743)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:740)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:740)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:58383)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1592)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1530)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1552)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1581)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1738)
> at
> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4297)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4289)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsyncV2(HBaseAdmin.java:753)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:674)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:607)
> at
> org.playground.hbase.KerberizedClientFallback.main(KerberizedClientFallback.java:55)
> Caused by: GSSException: No valid credentials provided (Mechanism level:
> Failed to find any Kerberos tgt)
> at
> sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
> at
> sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
> at
> sun.security.j
[jira] [Updated] (HBASE-20993) [Auth] IPC client fallback to simple auth allowed doesn't work
[
https://issues.apache.org/jira/browse/HBASE-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-20993:
--
Attachment: HBASE-20993.branch-1.wip.patch
> [Auth] IPC client fallback to simple auth allowed doesn't work
> --
>
> Key: HBASE-20993
> URL: https://issues.apache.org/jira/browse/HBASE-20993
> Project: HBase
> Issue Type: Bug
> Components: Client, security
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Jack Bearden
>Priority: Critical
> Attachments: HBASE-20993.001.patch, HBASE-20993.branch-1.002.patch,
> HBASE-20993.branch-1.2.001.patch, HBASE-20993.branch-1.wip.patch
>
>
> It is easily reproducible.
> client's hbase-site.xml: hadoop.security.authentication:kerberos,
> hbase.security.authentication:kerberos,
> hbase.ipc.client.fallback-to-simple-auth-allowed:true, keytab and principal
> are right set
> A simple auth hbase cluster, a kerberized hbase client application.
> application trying to r/w/c/d table will have following exception:
> {code}
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:617)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:162)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:743)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:740)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:740)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:58383)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1592)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1530)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1552)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1581)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1738)
> at
> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4297)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4289)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsyncV2(HBaseAdmin.java:753)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:674)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:607)
> at
> org.playground.hbase.KerberizedClientFallback.main(KerberizedClientFallback.java:55)
> Caused by: GSSException: No valid credentials provided (Mechanism level:
> Failed to find any Kerberos tgt)
> at
> sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
> at
> sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
> at
> sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
> at
> sun.security.jgss.GSSManagerImpl.getM
[jira] [Updated] (HBASE-20993) [Auth] IPC client fallback to simple auth allowed doesn't work
[
https://issues.apache.org/jira/browse/HBASE-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-20993:
--
Attachment: (was: HBASE-20993.branch-1.wip.patch)
> [Auth] IPC client fallback to simple auth allowed doesn't work
> --
>
> Key: HBASE-20993
> URL: https://issues.apache.org/jira/browse/HBASE-20993
> Project: HBase
> Issue Type: Bug
> Components: Client, security
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Jack Bearden
>Priority: Critical
> Attachments: HBASE-20993.001.patch, HBASE-20993.branch-1.002.patch,
> HBASE-20993.branch-1.2.001.patch
>
>
> It is easily reproducible.
> client's hbase-site.xml: hadoop.security.authentication:kerberos,
> hbase.security.authentication:kerberos,
> hbase.ipc.client.fallback-to-simple-auth-allowed:true, keytab and principal
> are right set
> A simple auth hbase cluster, a kerberized hbase client application.
> application trying to r/w/c/d table will have following exception:
> {code}
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:617)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:162)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:743)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:740)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:740)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:58383)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1592)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1530)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1552)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1581)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1738)
> at
> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4297)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4289)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsyncV2(HBaseAdmin.java:753)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:674)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:607)
> at
> org.playground.hbase.KerberizedClientFallback.main(KerberizedClientFallback.java:55)
> Caused by: GSSException: No valid credentials provided (Mechanism level:
> Failed to find any Kerberos tgt)
> at
> sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
> at
> sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
> at
> sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
> at
> sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSMa
[jira] [Updated] (HBASE-20993) [Auth] IPC client fallback to simple auth allowed doesn't work
[
https://issues.apache.org/jira/browse/HBASE-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-20993:
--
Attachment: HBASE-20993.branch-1.wip.patch
> [Auth] IPC client fallback to simple auth allowed doesn't work
> --
>
> Key: HBASE-20993
> URL: https://issues.apache.org/jira/browse/HBASE-20993
> Project: HBase
> Issue Type: Bug
> Components: Client, security
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Jack Bearden
>Priority: Critical
> Attachments: HBASE-20993.001.patch, HBASE-20993.branch-1.002.patch,
> HBASE-20993.branch-1.2.001.patch, HBASE-20993.branch-1.wip.patch
>
>
> It is easily reproducible.
> client's hbase-site.xml: hadoop.security.authentication:kerberos,
> hbase.security.authentication:kerberos,
> hbase.ipc.client.fallback-to-simple-auth-allowed:true, keytab and principal
> are right set
> A simple auth hbase cluster, a kerberized hbase client application.
> application trying to r/w/c/d table will have following exception:
> {code}
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:617)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:162)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:743)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:740)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:740)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:58383)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1592)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1530)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1552)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1581)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1738)
> at
> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4297)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4289)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsyncV2(HBaseAdmin.java:753)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:674)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:607)
> at
> org.playground.hbase.KerberizedClientFallback.main(KerberizedClientFallback.java:55)
> Caused by: GSSException: No valid credentials provided (Mechanism level:
> Failed to find any Kerberos tgt)
> at
> sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
> at
> sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
> at
> sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
> at
> sun.security.jgss.GSSManagerImpl.getM
[jira] [Commented] (HBASE-20978) [amv2] Worker terminating UNNATURALLY during MoveRegionProcedure
[
https://issues.apache.org/jira/browse/HBASE-20978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580999#comment-16580999
]
Hudson commented on HBASE-20978:
Results for branch master
[build #434 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/434/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/434//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/434//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/434//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [amv2] Worker terminating UNNATURALLY during MoveRegionProcedure
>
>
> Key: HBASE-20978
> URL: https://issues.apache.org/jira/browse/HBASE-20978
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 2.0.1
>Reporter: stack
>Assignee: Allan Yang
>Priority: Critical
> Fix For: 3.0.0, 2.0.2, 2.1.1
>
> Attachments: HBASE-20978.branch-2.0.001.patch,
> HBASE-20978.branch-2.0.002.patch
>
>
> Testing tip of branch-2.0, ran into this:
> {code}
> 2018-07-29 01:45:33,002 INFO [master/ve0524:16000] master.HMaster: Master
> has completed initialization 13.854sec
>2018-07-29
> 01:45:33,003 INFO [PEWorker-4] procedure.MasterProcedureScheduler: pid=1820,
> state=WAITING:MOVE_REGION_ASSIGN; MoveRegionProcedure
> hri=533fb79ba23b27e9e0715b51daeb30c1,
> source=ve0538.halxg.cloudera.com,16020,1532847421672,
> destination=ve0540.halxg.cloudera.com,16020,1532853151031 checking lock on
> 533fb79ba23b27e9e0715b51daeb30c1
> 2018-07-29 01:45:33,003
> WARN [PEWorker-4] procedure2.ProcedureExecutor: Worker terminating
> UNNATURALLY null
> java.lang.IllegalArgumentException: pid=1820,
> state=WAITING:MOVE_REGION_ASSIGN; MoveRegionProcedure
> hri=533fb79ba23b27e9e0715b51daeb30c1,
> source=ve0538.halxg.cloudera.com,16020,1532847421672,
> destination=ve0540.halxg.cloudera.com,16020,1532853151031
> at
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:134)
>
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1458)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1249)
>
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:76)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1763)
> {code}
> It then shows as the below in the UI:
> {code}
> IdParent State Owner TypeStart Time Last Update Errors
> Parameters
> 1820 WAITING stack MoveRegionProcedure
> hri=533fb79ba23b27e9e0715b51daeb30c1,
> source=ve0538.halxg.cloudera.com,16020,1532847421672,
> destination=ve0540.halxg.cloudera.com,16020,1532853151031 Sun Jul 29
> 01:33:37 PDT 2018Sun Jul 29 01:33:38 PDT 2018[ { state => [
> '1', '2' ] }, { regionId => '1532851768240', tableName => { namespace =>
> 'ZGVmYXVsdA==', qualifier => 'SW50ZWdyYXRpb25UZXN0QmlnTGlua2VkTGlzdA==' },
> startKey => 'VttDLvXHdcmzwqNdrNoUFg==', endKey => 'WGFV8k+hFqhcIJGiKZ8L4Q==',
> offline => 'false', split => 'false', replicaId => '0' }, { sourceServer => {
> hostName => 've0538.halxg.cloudera.com', port => '16020', startCode =>
> '1532847421672' }, destinationServer => { hostName =>
> 've0540.halxg.cloudera.com', port => '16020', startCode => '1532853151031' }
> } ]
> {code}
> This is what we'd just read from hbase:meta:
> {code}
> 2018-07-29 01:45:32,802 INFO [master/ve0524:16000]
> assignment.RegionStateStore: Load hbase:meta entry
> region=533fb79ba23b27e9e0715b51daeb30c1, regionState=CLOSED,
> lastHost=ve0538.halxg.cloudera.com,16020,1532847421672,
> regionLocation=ve0538.halxg.cloudera.com,16020,1532847421672,
[jira] [Commented] (HBASE-20979) Flaky test reporting should specify what JSON it needs and handle HTTP errors
[
https://issues.apache.org/jira/browse/HBASE-20979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580998#comment-16580998
]
Hudson commented on HBASE-20979:
Results for branch master
[build #434 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/434/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/434//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/434//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/434//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Flaky test reporting should specify what JSON it needs and handle HTTP errors
> -
>
> Key: HBASE-20979
> URL: https://issues.apache.org/jira/browse/HBASE-20979
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-20979.0.txt, HBASE-20979.1.patch,
> HBASE-20979.2.patch
>
>
> Current flaky test report should be including the {{tree=}} parameter in its
> Jenkins API calls (see
> https://support.cloudbees.com/hc/en-us/articles/217911388-Best-Practice-For-Using-Jenkins-REST-API).
> Also should provide some info on failure so that when jobs change or go away
> we don't get blank failures.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20387) flaky infrastructure should work for all branches
[
https://issues.apache.org/jira/browse/HBASE-20387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581034#comment-16581034
]
Hudson commented on HBASE-20387:
Results for branch HBASE-20387
[build #12 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20387/12/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20387/12//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20387/12//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20387/12//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> flaky infrastructure should work for all branches
> -
>
> Key: HBASE-20387
> URL: https://issues.apache.org/jira/browse/HBASE-20387
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Attachments: HBASE-20387.0.patch, HBASE-20387.1.patch
>
>
> We need a flaky list per-branch, since what does/does not work reliably on
> master isn't really relevant to our older maintenance release lines.
> We should just make the invocation a step in the current per-branch nightly
> jobs, prior to when we need the list in the stages that run unit tests. We
> can publish it in the nightly job as well so that precommit can still get it.
> (and can fetch it per-branch if needed)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20993) [Auth] IPC client fallback to simple auth allowed doesn't work
[
https://issues.apache.org/jira/browse/HBASE-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581042#comment-16581042
]
Hadoop QA commented on HBASE-20993:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
37s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} branch-1 passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} branch-1 passed with JDK v1.7.0_181 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
46s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
36s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} branch-1 passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} branch-1 passed with JDK v1.7.0_181 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} the patch passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed with JDK v1.7.0_181 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
48s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
1m 36s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} the patch passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed with JDK v1.7.0_181 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
23s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 24m 1s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 49m 2s{color
[jira] [Updated] (HBASE-20993) [Auth] IPC client fallback to simple auth allowed doesn't work
[
https://issues.apache.org/jira/browse/HBASE-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-20993:
--
Attachment: HBASE-20993.branch-1.wip.002.patch
> [Auth] IPC client fallback to simple auth allowed doesn't work
> --
>
> Key: HBASE-20993
> URL: https://issues.apache.org/jira/browse/HBASE-20993
> Project: HBase
> Issue Type: Bug
> Components: Client, security
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Jack Bearden
>Priority: Critical
> Attachments: HBASE-20993.001.patch, HBASE-20993.branch-1.002.patch,
> HBASE-20993.branch-1.2.001.patch, HBASE-20993.branch-1.wip.002.patch,
> HBASE-20993.branch-1.wip.patch
>
>
> It is easily reproducible.
> client's hbase-site.xml: hadoop.security.authentication:kerberos,
> hbase.security.authentication:kerberos,
> hbase.ipc.client.fallback-to-simple-auth-allowed:true, keytab and principal
> are right set
> A simple auth hbase cluster, a kerberized hbase client application.
> application trying to r/w/c/d table will have following exception:
> {code}
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:617)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:162)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:743)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:740)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:740)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:58383)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1592)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1530)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1552)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1581)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1738)
> at
> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4297)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4289)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsyncV2(HBaseAdmin.java:753)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:674)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:607)
> at
> org.playground.hbase.KerberizedClientFallback.main(KerberizedClientFallback.java:55)
> Caused by: GSSException: No valid credentials provided (Mechanism level:
> Failed to find any Kerberos tgt)
> at
> sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
> at
> sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
> at
> sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
>
[jira] [Created] (HBASE-21056) BucketCache.persistToFile may fail to clean up java.io.OutputStream
Sean Busbey created HBASE-21056:
---
Summary: BucketCache.persistToFile may fail to clean up
java.io.OutputStream
Key: HBASE-21056
URL: https://issues.apache.org/jira/browse/HBASE-21056
Project: HBase
Issue Type: Bug
Components: BucketCache
Affects Versions: 3.0.0
Reporter: Sean Busbey
Found by the nightly job via FindBugs:
{code}
FindBugsmodule:hbase-server
org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.persistToFile() may fail to
clean up java.io.OutputStream Obligation to clean up resource created at
BucketCache.java:up java.io.OutputStream Obligation to clean up resource
created at BucketCache.java:[line 1089] is not discharged
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21056) BucketCache.persistToFile may fail to clean up java.io.OutputStream
[
https://issues.apache.org/jira/browse/HBASE-21056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581112#comment-16581112
]
Sean Busbey commented on HBASE-21056:
-
the complained about section uses a try-with-resources
{code}
private void persistToFile() throws IOException {
assert !cacheEnabled;
if (!ioEngine.isPersistent()) {
throw new IOException("Attempt to persist non-persistent cache
mappings!");
}
try (FileOutputStream fos = new FileOutputStream(persistencePath, false)) {
fos.write(ProtobufMagic.PB_MAGIC);
BucketProtoUtils.toPB(this).writeDelimitedTo(fos);
}
}
{code}
lemme see if the full report has more details
> BucketCache.persistToFile may fail to clean up java.io.OutputStream
>
>
> Key: HBASE-21056
> URL: https://issues.apache.org/jira/browse/HBASE-21056
> Project: HBase
> Issue Type: Bug
> Components: BucketCache
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Priority: Critical
>
> Found by the nightly job via FindBugs:
> {code}
> FindBugs module:hbase-server
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.persistToFile() may fail
> to clean up java.io.OutputStream Obligation to clean up resource created at
> BucketCache.java:up java.io.OutputStream Obligation to clean up resource
> created at BucketCache.java:[line 1089] is not discharged
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21056) BucketCache.persistToFile may fail to clean up java.io.OutputStream
[
https://issues.apache.org/jira/browse/HBASE-21056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581124#comment-16581124
]
Sean Busbey commented on HBASE-21056:
-
this is a false positive. the code came in via HBASE-20894. In that issue,
precommit said findbugs was okay. the nightly run after the checkin failed to
run the jdk8-hadoop2 step, which is where we do findbugs. :( a couple of
evenings later the nightly run started doing that check again and this failure
started showing up (build #418).
> BucketCache.persistToFile may fail to clean up java.io.OutputStream
>
>
> Key: HBASE-21056
> URL: https://issues.apache.org/jira/browse/HBASE-21056
> Project: HBase
> Issue Type: Bug
> Components: BucketCache
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Priority: Critical
>
> Found by the nightly job via FindBugs:
> {code}
> FindBugs module:hbase-server
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.persistToFile() may fail
> to clean up java.io.OutputStream Obligation to clean up resource created at
> BucketCache.java:up java.io.OutputStream Obligation to clean up resource
> created at BucketCache.java:[line 1089] is not discharged
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20993) [Auth] IPC client fallback to simple auth allowed doesn't work
[
https://issues.apache.org/jira/browse/HBASE-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581129#comment-16581129
]
Hadoop QA commented on HBASE-20993:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} branch-1 passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} branch-1 passed with JDK v1.7.0_181 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
46s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
37s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} branch-1 passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} branch-1 passed with JDK v1.7.0_181 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} the patch passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed with JDK v1.7.0_181 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
38s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
1m 35s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} the patch passed with JDK v1.8.0_172 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed with JDK v1.7.0_181 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
23s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 23m 46s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 48m 1s{color
[jira] [Commented] (HBASE-21056) BucketCache.persistToFile may fail to clean up java.io.OutputStream
[
https://issues.apache.org/jira/browse/HBASE-21056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581133#comment-16581133
]
Sean Busbey commented on HBASE-21056:
-
looking at the precommit run for HBASE-20894 (
[ref|https://builds.apache.org/job/PreCommit-HBASE-Build/13887/] ) and the
findbugs artifacts line up with findbugs not having an issue there.
the findbugs versions are the same, as are the spotbugs definitions. Looks
like maybe it's the difference between running just in the hbase-server module
and running at the top level? maybe a bug in findbugs?
> BucketCache.persistToFile may fail to clean up java.io.OutputStream
>
>
> Key: HBASE-21056
> URL: https://issues.apache.org/jira/browse/HBASE-21056
> Project: HBase
> Issue Type: Bug
> Components: BucketCache
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Priority: Critical
>
> Found by the nightly job via FindBugs:
> {code}
> FindBugs module:hbase-server
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.persistToFile() may fail
> to clean up java.io.OutputStream Obligation to clean up resource created at
> BucketCache.java:up java.io.OutputStream Obligation to clean up resource
> created at BucketCache.java:[line 1089] is not discharged
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-21056) BucketCache.persistToFile may fail to clean up java.io.OutputStream
[
https://issues.apache.org/jira/browse/HBASE-21056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey reassigned HBASE-21056:
---
Assignee: Sean Busbey
> BucketCache.persistToFile may fail to clean up java.io.OutputStream
>
>
> Key: HBASE-21056
> URL: https://issues.apache.org/jira/browse/HBASE-21056
> Project: HBase
> Issue Type: Bug
> Components: BucketCache
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
>
> Found by the nightly job via FindBugs:
> {code}
> FindBugs module:hbase-server
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.persistToFile() may fail
> to clean up java.io.OutputStream Obligation to clean up resource created at
> BucketCache.java:up java.io.OutputStream Obligation to clean up resource
> created at BucketCache.java:[line 1089] is not discharged
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HBASE-21056) BucketCache.persistToFile may fail to clean up java.io.OutputStream
[
https://issues.apache.org/jira/browse/HBASE-21056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-21056 started by Sean Busbey.
---
> BucketCache.persistToFile may fail to clean up java.io.OutputStream
>
>
> Key: HBASE-21056
> URL: https://issues.apache.org/jira/browse/HBASE-21056
> Project: HBase
> Issue Type: Bug
> Components: BucketCache
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
>
> Found by the nightly job via FindBugs:
> {code}
> FindBugs module:hbase-server
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.persistToFile() may fail
> to clean up java.io.OutputStream Obligation to clean up resource created at
> BucketCache.java:up java.io.OutputStream Obligation to clean up resource
> created at BucketCache.java:[line 1089] is not discharged
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21056) BucketCache.persistToFile may fail to clean up java.io.OutputStream
[
https://issues.apache.org/jira/browse/HBASE-21056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21056:
Priority: Minor (was: Critical)
> BucketCache.persistToFile may fail to clean up java.io.OutputStream
>
>
> Key: HBASE-21056
> URL: https://issues.apache.org/jira/browse/HBASE-21056
> Project: HBase
> Issue Type: Bug
> Components: BucketCache
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
>
> Found by the nightly job via FindBugs:
> {code}
> FindBugs module:hbase-server
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.persistToFile() may fail
> to clean up java.io.OutputStream Obligation to clean up resource created at
> BucketCache.java:up java.io.OutputStream Obligation to clean up resource
> created at BucketCache.java:[line 1089] is not discharged
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21056) Findbugs false positive: BucketCache.persistToFile may fail to clean up java.io.OutputStream
[
https://issues.apache.org/jira/browse/HBASE-21056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21056:
Summary: Findbugs false positive: BucketCache.persistToFile may fail to
clean up java.io.OutputStream (was: BucketCache.persistToFile may fail to
clean up java.io.OutputStream )
> Findbugs false positive: BucketCache.persistToFile may fail to clean up
> java.io.OutputStream
> -
>
> Key: HBASE-21056
> URL: https://issues.apache.org/jira/browse/HBASE-21056
> Project: HBase
> Issue Type: Bug
> Components: BucketCache
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
>
> Found by the nightly job via FindBugs:
> {code}
> FindBugs module:hbase-server
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.persistToFile() may fail
> to clean up java.io.OutputStream Obligation to clean up resource created at
> BucketCache.java:up java.io.OutputStream Obligation to clean up resource
> created at BucketCache.java:[line 1089] is not discharged
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21056) Findbugs false positive: BucketCache.persistToFile may fail to clean up java.io.OutputStream
[
https://issues.apache.org/jira/browse/HBASE-21056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21056:
Status: Patch Available (was: In Progress)
-v1 suppress the obligation warning.
tested locally with maven findbugs plugin on just the hbase-server module and
on the top level.
> Findbugs false positive: BucketCache.persistToFile may fail to clean up
> java.io.OutputStream
> -
>
> Key: HBASE-21056
> URL: https://issues.apache.org/jira/browse/HBASE-21056
> Project: HBase
> Issue Type: Bug
> Components: BucketCache
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Attachments: HBASE-21056.0.patch
>
>
> Found by the nightly job via FindBugs:
> {code}
> FindBugs module:hbase-server
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.persistToFile() may fail
> to clean up java.io.OutputStream Obligation to clean up resource created at
> BucketCache.java:up java.io.OutputStream Obligation to clean up resource
> created at BucketCache.java:[line 1089] is not discharged
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21056) Findbugs false positive: BucketCache.persistToFile may fail to clean up java.io.OutputStream
[
https://issues.apache.org/jira/browse/HBASE-21056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21056:
Attachment: HBASE-21056.0.patch
> Findbugs false positive: BucketCache.persistToFile may fail to clean up
> java.io.OutputStream
> -
>
> Key: HBASE-21056
> URL: https://issues.apache.org/jira/browse/HBASE-21056
> Project: HBase
> Issue Type: Bug
> Components: BucketCache
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Attachments: HBASE-21056.0.patch
>
>
> Found by the nightly job via FindBugs:
> {code}
> FindBugs module:hbase-server
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.persistToFile() may fail
> to clean up java.io.OutputStream Obligation to clean up resource created at
> BucketCache.java:up java.io.OutputStream Obligation to clean up resource
> created at BucketCache.java:[line 1089] is not discharged
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21056) Findbugs false positive: BucketCache.persistToFile may fail to clean up java.io.OutputStream
[
https://issues.apache.org/jira/browse/HBASE-21056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581148#comment-16581148
]
Sean Busbey commented on HBASE-21056:
-
related, maybe time to update our spotbugs version?
> Findbugs false positive: BucketCache.persistToFile may fail to clean up
> java.io.OutputStream
> -
>
> Key: HBASE-21056
> URL: https://issues.apache.org/jira/browse/HBASE-21056
> Project: HBase
> Issue Type: Bug
> Components: BucketCache
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Attachments: HBASE-21056.0.patch
>
>
> Found by the nightly job via FindBugs:
> {code}
> FindBugs module:hbase-server
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.persistToFile() may fail
> to clean up java.io.OutputStream Obligation to clean up resource created at
> BucketCache.java:up java.io.OutputStream Obligation to clean up resource
> created at BucketCache.java:[line 1089] is not discharged
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21057) upgrade to latest spotbugs
Sean Busbey created HBASE-21057: --- Summary: upgrade to latest spotbugs Key: HBASE-21057 URL: https://issues.apache.org/jira/browse/HBASE-21057 Project: HBase Issue Type: Task Components: community, test Reporter: Sean Busbey we currently rely on [spotbugs definitions from 3.1.0-RC3|https://github.com/spotbugs/spotbugs/releases/tag/3.1.0_RC3], which was a pre-release candidate from Jun 2017. [spotbugs version 3.1.6|https://github.com/spotbugs/spotbugs/releases/tag/3.1.6] came out about a month ago. We should update to the latest. they also have their own maven plugin now. as a stretch goal we could switch over to that, if it works with yetus. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19008) Add missing equals or hashCode method(s) to stock Filter implementations
[
https://issues.apache.org/jira/browse/HBASE-19008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liubangchen updated HBASE-19008:
Attachment: HBASE-19008-12.patch
> Add missing equals or hashCode method(s) to stock Filter implementations
>
>
> Key: HBASE-19008
> URL: https://issues.apache.org/jira/browse/HBASE-19008
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: liubangchen
>Priority: Major
> Labels: filter
> Attachments: Filters.png, HBASE-19008-1.patch, HBASE-19008-10.patch,
> HBASE-19008-11.patch, HBASE-19008-12.patch, HBASE-19008-2.patch,
> HBASE-19008-3.patch, HBASE-19008-4.patch, HBASE-19008-5.patch,
> HBASE-19008-6.patch, HBASE-19008-7.patch, HBASE-19008-8.patch,
> HBASE-19008-9.patch, HBASE-19008.patch
>
>
> In HBASE-15410, [~mdrob] reminded me that Filter implementations may not
> write {{equals}} or {{hashCode}} method(s).
> This issue is to add missing {{equals}} or {{hashCode}} method(s) to stock
> Filter implementations such as KeyOnlyFilter.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21040) Replace call to printStackTrace() with proper logger call
[
https://issues.apache.org/jira/browse/HBASE-21040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-21040:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Thanks for the patch, Subrat.
I modified the subject of the patch to match the JIRA title.
> Replace call to printStackTrace() with proper logger call
> -
>
> Key: HBASE-21040
> URL: https://issues.apache.org/jira/browse/HBASE-21040
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Subrat Mishra
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-21040.master.001.patch,
> HBASE-21040.master.002.patch
>
>
> Here is related code:
> {code}
> hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/RestoreDriver.java:
> e.printStackTrace();
> hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientAsyncPrefetchScanner.java:
> first.printStackTrace();
> hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java:
> t.printStackTrace();
> hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ParseFilter.java:
>e.printStackTrace();
> hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ParseFilter.java:
>e.printStackTrace();
> hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ParseFilter.java:
>e.printStackTrace();
> hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ParseFilter.java:
>e.printStackTrace();
> hbase-examples/src/main/java/org/apache/hadoop/hbase/mapreduce/SampleUploader.java:
> e.printStackTrace();
> hbase-examples/src/main/java/org/apache/hadoop/hbase/thrift/DemoClient.java:
> e.printStackTrace();
> hbase-examples/src/main/java/org/apache/hadoop/hbase/thrift/HttpDoAsClient.java:
> e.printStackTrace();
> hbase-examples/src/main/java/org/apache/hadoop/hbase/thrift/HttpDoAsClient.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapred/TableMapReduceUtil.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapred/TableMapReduceUtil.java:
> ioe.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapred/TableMapReduceUtil.java:
> ie.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/CellCounter.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/CopyTable.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/HashTable.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/Import.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/Import.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/Import.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/SyncTable.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TsvImporterMapper.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TsvImporterTextMapper.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALPlayer.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALPlayer.java:
> e.printStackTrace();
> hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java:
> e.printStackTrace();
> hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/ProcedureWALPrettyPrinter.java:
> e.printStackTrace();
> hbase-server/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java:
> e.printStackTrace();
> hbase-server/src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java:
> e.printStackTrace();
> hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java:
> e.printStackTrace();
> hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALPrettyPrinter.java:
> e.printStackTrace();
> hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServer.java:
> ex.printStackTrace();
> hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/HQuorumPeer.java:
> e.printStackTrace();
> {code}
> The correct way of logging stack trace is to use the Logger instance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20892) [UI] Start / End keys are empty on table.jsp
[ https://issues.apache.org/jira/browse/HBASE-20892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-20892: --- Labels: web-ui (was: ) > [UI] Start / End keys are empty on table.jsp > > > Key: HBASE-20892 > URL: https://issues.apache.org/jira/browse/HBASE-20892 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: Ted Yu >Priority: Major > Labels: web-ui > > When viewing table.jsp?name=TestTable , I found that the Start / End keys for > all the regions were simply dashes without real value. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20676) Give .hbase-snapshot proper ownership upon directory creation
[ https://issues.apache.org/jira/browse/HBASE-20676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-20676: --- Labels: security snapshot (was: snapshot) > Give .hbase-snapshot proper ownership upon directory creation > - > > Key: HBASE-20676 > URL: https://issues.apache.org/jira/browse/HBASE-20676 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Priority: Major > Labels: security, snapshot > > This is continuation of the discussion over HBASE-20668. > Tthe .hbase-snapshot directory is not created at cluster startup. Normally it > is created when snapshot operation is initiated. > However, if before any snapshot operation is performed, some non-super user > from another cluster conducts ExportSnapshot to this cluster, the > .hbase-snapshot directory would be created as that user. > (This is just one scenario that can lead to wrong ownership) > This JIRA is to seek proper way(s) to ensure that .hbase-snapshot directory > would always carry proper onwership and permission upon creation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21047) Object creation of StoreFileScanner thru constructor and close may leave refCount to -1
[ https://issues.apache.org/jira/browse/HBASE-21047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581332#comment-16581332 ] Andrew Purtell commented on HBASE-21047: FWIW tests with the branch-1 patch all pass > Object creation of StoreFileScanner thru constructor and close may leave > refCount to -1 > --- > > Key: HBASE-21047 > URL: https://issues.apache.org/jira/browse/HBASE-21047 > Project: HBase > Issue Type: Bug >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21047.branch-1.v1.patch > > > During object creation "*StoreFileScanner*", it does not increase the > refCount whereas while close it decrements the reader refCount. This will > cause refCount to -1 and isReadReference method was returning true > (refCount.get() != 0 This is causing store file not to be deleted. This may > also cause issue in situation when some thread is holding a scanner but it > may actually be closed due to above bug. Impact of this would be really high. > Attatching patch for the fix which makes sure if reference is held either > thru getScanner method or constructor, ref is always updated. Patch also > contains a test which validates the issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21056) Findbugs false positive: BucketCache.persistToFile may fail to clean up java.io.OutputStream
[
https://issues.apache.org/jira/browse/HBASE-21056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581353#comment-16581353
]
Hadoop QA commented on HBASE-21056:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
18s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
23s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 53s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}117m
28s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}154m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21056 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12935707/HBASE-21056.0.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux c833a596334d 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 1114a1a65e |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/14048/testReport/ |
| Max. process+thread count | 4351 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/14048/cons
[jira] [Commented] (HBASE-19008) Add missing equals or hashCode method(s) to stock Filter implementations
[
https://issues.apache.org/jira/browse/HBASE-19008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581424#comment-16581424
]
Hadoop QA commented on HBASE-19008:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
39s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
40s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} hbase-client: The patch generated 17 new + 325
unchanged - 96 fixed = 342 total (was 421) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
19s{color} | {color:red} hbase-server: The patch generated 2 new + 392
unchanged - 14 fixed = 394 total (was 406) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} hbase-spark: The patch generated 4 new + 0 unchanged -
0 fixed = 4 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
21s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 55s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
2s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}112m
31s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
45s{color} | {color:green} hbase-spark in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
56s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}173m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-19008 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attac
[jira] [Created] (HBASE-21058) Nightly tests for branches 1 fail to build ref guide
Sean Busbey created HBASE-21058:
---
Summary: Nightly tests for branches 1 fail to build ref guide
Key: HBASE-21058
URL: https://issues.apache.org/jira/browse/HBASE-21058
Project: HBase
Issue Type: Bug
Components: documentation
Affects Versions: 1.5.0, 1.2.7, 1.3.3, 1.4.7
Reporter: Sean Busbey
Assignee: Sean Busbey
Nightly on all branches 1 reports failure to get a PDF version of the ref guide
{code}
-1 refguide2m 14s patch failed to produce the pdf version of the
reference guide.
{code}
Actual build log looks clean
{code}
[INFO] --- asciidoctor-maven-plugin:1.5.2.1:process-asciidoc (output-pdf) @
hbase ---
asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
asciidoctor: WARNING: conversion missing in backend pdf for pass
asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
asciidoctor: WARNING: conversion missing in backend pdf for inline_image
asciidoctor: WARNING: conversion missing in backend pdf for inline_image
[INFO] Rendered /testptch/hbase/src/main/asciidoc/book.adoc
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HBASE-21058) Nightly tests for branches 1 fail to build ref guide
[
https://issues.apache.org/jira/browse/HBASE-21058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-21058 started by Sean Busbey.
---
> Nightly tests for branches 1 fail to build ref guide
>
>
> Key: HBASE-21058
> URL: https://issues.apache.org/jira/browse/HBASE-21058
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 1.5.0, 1.2.7, 1.3.3, 1.4.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
>
> Nightly on all branches 1 reports failure to get a PDF version of the ref
> guide
> {code}
> -1refguide2m 14s patch failed to produce the pdf version of the
> reference guide.
> {code}
> Actual build log looks clean
> {code}
> [INFO] --- asciidoctor-maven-plugin:1.5.2.1:process-asciidoc (output-pdf) @
> hbase ---
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for pass
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_image
> asciidoctor: WARNING: conversion missing in backend pdf for inline_image
> [INFO] Rendered /testptch/hbase/src/main/asciidoc/book.adoc
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21058) Nightly tests for branches 1 fail to build ref guide
[
https://issues.apache.org/jira/browse/HBASE-21058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581432#comment-16581432
]
Sean Busbey commented on HBASE-21058:
-
okay here's the issue, from our check for "did the PDF work":
{code}
if [[ ! -f "${PATCH_DIR}/${repostatus}-site/apache_hbase_reference_guide.pdf"
]]; then
add_vote_table -1 refguide "${repostatus} failed to produce the pdf version
of the reference guide."
add_footer_table refguide "@@BASE@@/${repostatus}-refguide.log"
return 1
fi
{code}
in branches 1, the output file is named {{book.pdf}}.
> Nightly tests for branches 1 fail to build ref guide
>
>
> Key: HBASE-21058
> URL: https://issues.apache.org/jira/browse/HBASE-21058
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 1.5.0, 1.2.7, 1.3.3, 1.4.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
>
> Nightly on all branches 1 reports failure to get a PDF version of the ref
> guide
> {code}
> -1refguide2m 14s patch failed to produce the pdf version of the
> reference guide.
> {code}
> Actual build log looks clean
> {code}
> [INFO] --- asciidoctor-maven-plugin:1.5.2.1:process-asciidoc (output-pdf) @
> hbase ---
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for pass
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_image
> asciidoctor: WARNING: conversion missing in backend pdf for inline_image
> [INFO] Rendered /testptch/hbase/src/main/asciidoc/book.adoc
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21058) Nightly tests for branches 1 fail to build ref guide
[
https://issues.apache.org/jira/browse/HBASE-21058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21058:
Attachment: HBASE-21058.0.patch
> Nightly tests for branches 1 fail to build ref guide
>
>
> Key: HBASE-21058
> URL: https://issues.apache.org/jira/browse/HBASE-21058
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 1.5.0, 1.2.7, 1.3.3, 1.4.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-21058.0.patch
>
>
> Nightly on all branches 1 reports failure to get a PDF version of the ref
> guide
> {code}
> -1refguide2m 14s patch failed to produce the pdf version of the
> reference guide.
> {code}
> Actual build log looks clean
> {code}
> [INFO] --- asciidoctor-maven-plugin:1.5.2.1:process-asciidoc (output-pdf) @
> hbase ---
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for pass
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_image
> asciidoctor: WARNING: conversion missing in backend pdf for inline_image
> [INFO] Rendered /testptch/hbase/src/main/asciidoc/book.adoc
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21058) Nightly tests for branches 1 fail to build ref guide
[
https://issues.apache.org/jira/browse/HBASE-21058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21058:
Status: Patch Available (was: In Progress)
> Nightly tests for branches 1 fail to build ref guide
>
>
> Key: HBASE-21058
> URL: https://issues.apache.org/jira/browse/HBASE-21058
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 1.5.0, 1.2.7, 1.3.3, 1.4.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-21058.0.patch
>
>
> Nightly on all branches 1 reports failure to get a PDF version of the ref
> guide
> {code}
> -1refguide2m 14s patch failed to produce the pdf version of the
> reference guide.
> {code}
> Actual build log looks clean
> {code}
> [INFO] --- asciidoctor-maven-plugin:1.5.2.1:process-asciidoc (output-pdf) @
> hbase ---
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for pass
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_image
> asciidoctor: WARNING: conversion missing in backend pdf for inline_image
> [INFO] Rendered /testptch/hbase/src/main/asciidoc/book.adoc
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21058) Nightly tests for branches 1 fail to build ref guide
[
https://issues.apache.org/jira/browse/HBASE-21058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581444#comment-16581444
]
Hadoop QA commented on HBASE-21058:
---
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at
https://builds.apache.org/job/PreCommit-HBASE-Build/14050/console in case of
problems.
> Nightly tests for branches 1 fail to build ref guide
>
>
> Key: HBASE-21058
> URL: https://issues.apache.org/jira/browse/HBASE-21058
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 1.5.0, 1.2.7, 1.3.3, 1.4.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-21058.0.patch
>
>
> Nightly on all branches 1 reports failure to get a PDF version of the ref
> guide
> {code}
> -1refguide2m 14s patch failed to produce the pdf version of the
> reference guide.
> {code}
> Actual build log looks clean
> {code}
> [INFO] --- asciidoctor-maven-plugin:1.5.2.1:process-asciidoc (output-pdf) @
> hbase ---
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for pass
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_image
> asciidoctor: WARNING: conversion missing in backend pdf for inline_image
> [INFO] Rendered /testptch/hbase/src/main/asciidoc/book.adoc
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21058) Nightly tests for branches 1 fail to build ref guide
[
https://issues.apache.org/jira/browse/HBASE-21058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581447#comment-16581447
]
Hadoop QA commented on HBASE-21058:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
3s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
1s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 0m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21058 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12935730/HBASE-21058.0.patch |
| Optional Tests | asflicense shellcheck shelldocs |
| uname | Linux 4dc84da13f5b 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 49ae8549cf |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| shellcheck | v0.4.4 |
| Max. process+thread count | 43 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/14050/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Nightly tests for branches 1 fail to build ref guide
>
>
> Key: HBASE-21058
> URL: https://issues.apache.org/jira/browse/HBASE-21058
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 1.5.0, 1.2.7, 1.3.3, 1.4.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-21058.0.patch
>
>
> Nightly on all branches 1 reports failure to get a PDF version of the ref
> guide
> {code}
> -1refguide2m 14s patch failed to produce the pdf version of the
> reference guide.
> {code}
> Actual build log looks clean
> {code}
> [INFO] --- asciidoctor-maven-plugin:1.5.2.1:process-asciidoc (output-pdf) @
> hbase ---
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for pass
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_indexterm
> asciidoctor: WARNING: conversion missing in backend pdf for inline_image
> asciidoctor: WARNING: conversion missing in backend pdf for inline_image
> [INFO] Rendered /testptch/hbase/src/main/asciidoc/book.adoc
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21059) Findbugs failures on branch-1.2
Sean Busbey created HBASE-21059:
---
Summary: Findbugs failures on branch-1.2
Key: HBASE-21059
URL: https://issues.apache.org/jira/browse/HBASE-21059
Project: HBase
Issue Type: Bug
Components: Coprocessors, rpc
Affects Versions: 1.2.7
Reporter: Sean Busbey
Assignee: Sean Busbey
Fix For: 1.2.7
nightly shows two failures for branch-1.2 findbugs check:
{code}
FindBugsmodule:hbase-server
Inconsistent synchronization of
org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap; locked 75% of time
Unsynchronized access at RpcServer.java:75% of time Unsynchronized access at
RpcServer.java:[line 458]
Dead store to fsSet in
org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(CoprocessorEnvironment)
At
SecureBulkLoadEndpoint.java:org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(CoprocessorEnvironment)
At SecureBulkLoadEndpoint.java:[line 145]
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21059) Findbugs failures on branch-1.2
[
https://issues.apache.org/jira/browse/HBASE-21059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581450#comment-16581450
]
Sean Busbey commented on HBASE-21059:
-
bq. Inconsistent synchronization of
org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap; locked 75% of time
Unsynchronized access at RpcServer.java:75% of time Unsynchronized access at
RpcServer.java:[line 458]
This is in all branches-1
> Findbugs failures on branch-1.2
> ---
>
> Key: HBASE-21059
> URL: https://issues.apache.org/jira/browse/HBASE-21059
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors, rpc
>Affects Versions: 1.2.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 1.2.7
>
>
> nightly shows two failures for branch-1.2 findbugs check:
> {code}
> FindBugs module:hbase-server
> Inconsistent synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap; locked 75% of time
> Unsynchronized access at RpcServer.java:75% of time Unsynchronized access at
> RpcServer.java:[line 458]
> Dead store to fsSet in
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(CoprocessorEnvironment)
> At
> SecureBulkLoadEndpoint.java:org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(CoprocessorEnvironment)
> At SecureBulkLoadEndpoint.java:[line 145]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21059) Findbugs failures on branch-1.2
[
https://issues.apache.org/jira/browse/HBASE-21059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581453#comment-16581453
]
Sean Busbey commented on HBASE-21059:
-
bq. Dead store to fsSet in
org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(CoprocessorEnvironment)
At
SecureBulkLoadEndpoint.java:org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(CoprocessorEnvironment)
At SecureBulkLoadEndpoint.java:[line 145]
this is in branch-1.3 and branch-1.2
> Findbugs failures on branch-1.2
> ---
>
> Key: HBASE-21059
> URL: https://issues.apache.org/jira/browse/HBASE-21059
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors, rpc
>Affects Versions: 1.2.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 1.2.7
>
>
> nightly shows two failures for branch-1.2 findbugs check:
> {code}
> FindBugs module:hbase-server
> Inconsistent synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap; locked 75% of time
> Unsynchronized access at RpcServer.java:75% of time Unsynchronized access at
> RpcServer.java:[line 458]
> Dead store to fsSet in
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(CoprocessorEnvironment)
> At
> SecureBulkLoadEndpoint.java:org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(CoprocessorEnvironment)
> At SecureBulkLoadEndpoint.java:[line 145]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21060) fix dead store in SecureBulkLoadEndpoint
Sean Busbey created HBASE-21060: --- Summary: fix dead store in SecureBulkLoadEndpoint Key: HBASE-21060 URL: https://issues.apache.org/jira/browse/HBASE-21060 Project: HBase Issue Type: Sub-task Components: Coprocessors Affects Versions: 1.2.7, 1.3.3 Reporter: Sean Busbey Assignee: Sean Busbey Fix For: 1.2.7, 1.3.3 Dead store to fsSet in org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(CoprocessorEnvironment) At SecureBulkLoadEndpoint.java:org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(CoprocessorEnvironment) At SecureBulkLoadEndpoint.java:[line 145] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HBASE-21059) Findbugs failures on branch-1.2
[
https://issues.apache.org/jira/browse/HBASE-21059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-21059 started by Sean Busbey.
---
> Findbugs failures on branch-1.2
> ---
>
> Key: HBASE-21059
> URL: https://issues.apache.org/jira/browse/HBASE-21059
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors, rpc
>Affects Versions: 1.2.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 1.2.7
>
>
> nightly shows two failures for branch-1.2 findbugs check:
> {code}
> FindBugs module:hbase-server
> Inconsistent synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap; locked 75% of time
> Unsynchronized access at RpcServer.java:75% of time Unsynchronized access at
> RpcServer.java:[line 458]
> Dead store to fsSet in
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(CoprocessorEnvironment)
> At
> SecureBulkLoadEndpoint.java:org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(CoprocessorEnvironment)
> At SecureBulkLoadEndpoint.java:[line 145]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HBASE-21060) fix dead store in SecureBulkLoadEndpoint
[ https://issues.apache.org/jira/browse/HBASE-21060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-21060 started by Sean Busbey. --- > fix dead store in SecureBulkLoadEndpoint > > > Key: HBASE-21060 > URL: https://issues.apache.org/jira/browse/HBASE-21060 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Affects Versions: 1.2.7, 1.3.3 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > Fix For: 1.2.7, 1.3.3 > > > Dead store to fsSet in > org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(CoprocessorEnvironment) > At > SecureBulkLoadEndpoint.java:org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(CoprocessorEnvironment) > At SecureBulkLoadEndpoint.java:[line 145] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21060) fix dead store in SecureBulkLoadEndpoint
[ https://issues.apache.org/jira/browse/HBASE-21060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581466#comment-16581466 ] Sean Busbey commented on HBASE-21060: - traced this back to a bad backport of HBASE-20605 to branch-1.3 and branch-1.2. neither of those branches got HBASE-17861, and the backports of HBASE-20605 act as though they did. Anyone have an opinion on backporting HBASE-17861 vs reverting HBASE-20605? ping interested parties from the above two jiras: [~elserj], [~yuzhih...@gmail.com], [~jinghe], [~yiliang], [~zyork] > fix dead store in SecureBulkLoadEndpoint > > > Key: HBASE-21060 > URL: https://issues.apache.org/jira/browse/HBASE-21060 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Affects Versions: 1.2.7, 1.3.3 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > Fix For: 1.2.7, 1.3.3 > > > Dead store to fsSet in > org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(CoprocessorEnvironment) > At > SecureBulkLoadEndpoint.java:org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(CoprocessorEnvironment) > At SecureBulkLoadEndpoint.java:[line 145] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21061) fix synchronization of org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
Sean Busbey created HBASE-21061:
---
Summary: fix synchronization of
org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
Key: HBASE-21061
URL: https://issues.apache.org/jira/browse/HBASE-21061
Project: HBase
Issue Type: Sub-task
Components: rpc
Affects Versions: 1.5.0, 1.2.7, 1.3.3, 1.4.7
Reporter: Sean Busbey
Assignee: Sean Busbey
fix findbugs highlighted issue
{code}
Inconsistent synchronization of
org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap; locked 75% of time
Unsynchronized access at RpcServer.java:75% of time Unsynchronized access at
RpcServer.java
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HBASE-21061) fix synchronization of org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
[
https://issues.apache.org/jira/browse/HBASE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-21061 started by Sean Busbey.
---
> fix synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
> ---
>
> Key: HBASE-21061
> URL: https://issues.apache.org/jira/browse/HBASE-21061
> Project: HBase
> Issue Type: Sub-task
> Components: rpc
>Affects Versions: 1.5.0, 1.2.7, 1.3.3, 1.4.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
>
> fix findbugs highlighted issue
> {code}
> Inconsistent synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap; locked 75% of time
> Unsynchronized access at RpcServer.java:75% of time Unsynchronized access at
> RpcServer.java
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18477) Umbrella JIRA for HBase Read Replica clusters
[ https://issues.apache.org/jira/browse/HBASE-18477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581481#comment-16581481 ] Zach York commented on HBASE-18477: --- [~busbey] ping - Are you still interested or should I try to look for another reviewer? Thanks! > Umbrella JIRA for HBase Read Replica clusters > - > > Key: HBASE-18477 > URL: https://issues.apache.org/jira/browse/HBASE-18477 > Project: HBase > Issue Type: New Feature >Reporter: Zach York >Assignee: Zach York >Priority: Major > Attachments: HBase Read-Replica Clusters Scope doc.docx, HBase > Read-Replica Clusters Scope doc.pdf, HBase Read-Replica Clusters Scope > doc_v2.docx, HBase Read-Replica Clusters Scope doc_v2.pdf > > > Recently, changes (such as HBASE-17437) have unblocked HBase to run with a > root directory external to the cluster (such as in Amazon S3). This means > that the data is stored outside of the cluster and can be accessible after > the cluster has been terminated. One use case that is often asked about is > pointing multiple clusters to one root directory (sharing the data) to have > read resiliency in the case of a cluster failure. > > This JIRA is an umbrella JIRA to contain all the tasks necessary to create a > read-replica HBase cluster that is pointed at the same root directory. > > This requires making the Read-Replica cluster Read-Only (no metadata > operation or data operations). > Separating the hbase:meta table for each cluster (Otherwise HBase gets > confused with multiple clusters trying to update the meta table with their ip > addresses) > Adding refresh functionality for the meta table to ensure new metadata is > picked up on the read replica cluster. > Adding refresh functionality for HFiles for a given table to ensure new data > is picked up on the read replica cluster. > > This can be used with any existing cluster that is backed by an external > filesystem. > > Please note that this feature is still quite manual (with the potential for > automation later). > > More information on this particular feature can be found here: > https://aws.amazon.com/blogs/big-data/setting-up-read-replica-clusters-with-hbase-on-amazon-s3/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21060) fix dead store in SecureBulkLoadEndpoint
[ https://issues.apache.org/jira/browse/HBASE-21060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581486#comment-16581486 ] Zach York commented on HBASE-21060: --- I personally don't need HBASE-17861 in branch-1.3 or branch-1.2, but I think it would be better to backport since it is a bug and others might hit it in those lines. (although I think they are close to EOM anyways). > fix dead store in SecureBulkLoadEndpoint > > > Key: HBASE-21060 > URL: https://issues.apache.org/jira/browse/HBASE-21060 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Affects Versions: 1.2.7, 1.3.3 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > Fix For: 1.2.7, 1.3.3 > > > Dead store to fsSet in > org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(CoprocessorEnvironment) > At > SecureBulkLoadEndpoint.java:org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(CoprocessorEnvironment) > At SecureBulkLoadEndpoint.java:[line 145] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21062) WALFactory has misleading notion of "default"
Josh Elser created HBASE-21062:
--
Summary: WALFactory has misleading notion of "default"
Key: HBASE-21062
URL: https://issues.apache.org/jira/browse/HBASE-21062
Project: HBase
Issue Type: Bug
Components: wal
Reporter: Josh Elser
Assignee: Josh Elser
Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1
In WALFactory, there is an enum {{Providers}} which has a list of supported
WALProvider implementations. In addition to list this, there is also a
{{defaultProvider}} (which the Configuration defaults to), that is meant to be
our "advertised" default WALProvider.
However, the implementation of {{getProviderClass}} in WALFactory doesn't
actually adhere to the value of this enum, instead *always* returning
AsyncFSWal if it can be loaded.
Having the default value in the enum but then overriding it in the
implementation of {{getProviderClass}} is silly and misleading.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21031) Memory leak if replay edits failed during region opening
[
https://issues.apache.org/jira/browse/HBASE-21031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581499#comment-16581499
]
Mike Drob commented on HBASE-21031:
---
bq. +LOG.error("replay failed, that's expected", t);
I think the test is better without this since we already fail the test if the
replay doesn't fail, so this is unneeded lines in the log. Especially stack
trace at error will make it stick out and harder to diagnose other failures.
> Memory leak if replay edits failed during region opening
>
>
> Key: HBASE-21031
> URL: https://issues.apache.org/jira/browse/HBASE-21031
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.1.0, 2.0.1
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Attachments: HBASE-21031.branch-2.0.001.patch,
> HBASE-21031.branch-2.0.002.patch, HBASE-21031.branch-2.0.003.patch,
> HBASE-21031.branch-2.0.004.patch, HBASE-21031.branch-2.0.005.patch,
> memoryleak.png
>
>
> Due to HBASE-21029, when replaying edits with a lot of same cells, the
> memstore won't flush, a exception will throw when all heap space was used:
> {code}
> 2018-08-06 15:52:27,590 ERROR
> [RS_OPEN_REGION-regionserver/hb-bp10cw4ejoy0a2f3f-009:16020-2]
> handler.OpenRegionHandler(302): Failed open of
> region=hbase_test,dffa78,1531227033378.cbf9a2daf3aaa0c7e931e9c9a7b53f41.,
> starting to roll back the global memstore size.
> java.lang.OutOfMemoryError: Java heap space
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
> at
> org.apache.hadoop.hbase.regionserver.OnheapChunk.allocateDataBuffer(OnheapChunk.java:41)
> at org.apache.hadoop.hbase.regionserver.Chunk.init(Chunk.java:104)
> at
> org.apache.hadoop.hbase.regionserver.ChunkCreator.getChunk(ChunkCreator.java:226)
> at
> org.apache.hadoop.hbase.regionserver.ChunkCreator.getChunk(ChunkCreator.java:180)
> at
> org.apache.hadoop.hbase.regionserver.ChunkCreator.getChunk(ChunkCreator.java:163)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.getOrMakeChunk(MemStoreLABImpl.java:273)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.copyCellInto(MemStoreLABImpl.java:148)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.copyCellInto(MemStoreLABImpl.java:111)
> at
> org.apache.hadoop.hbase.regionserver.Segment.maybeCloneWithAllocator(Segment.java:178)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.maybeCloneWithAllocator(AbstractMemStore.java:287)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.add(AbstractMemStore.java:107)
> at org.apache.hadoop.hbase.regionserver.HStore.add(HStore.java:706)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.restoreEdit(HRegion.java:5494)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:4608)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:4404)
> {code}
> After this exception, the memstore did not roll back, and since MSLAB is
> used, all the chunk allocated won't release for ever. Those memory is leak
> forever...
> We need to rollback the memory if open region fails(For now, only global
> memstore size is decreased after failure).
> Another problem is that we use replayEditsPerRegion in RegionServerAccounting
> to record how many memory used during replaying. And decrease the global
> memstore size if replay fails. This is not right, since during replaying, we
> may also flush the memstore, the size in the map of replayEditsPerRegion is
> not accurate at all!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21031) Memory leak if replay edits failed during region opening
[
https://issues.apache.org/jira/browse/HBASE-21031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581501#comment-16581501
]
Mike Drob commented on HBASE-21031:
---
{quote}
+ LOG.warn("Failed drop memstore of region= {}, s"
+ + "ome chunks may not released forever since MSLAB is
enabled",
{quote}
please try to put line breaks in between words instead of in the middle of
words.
Also, I responded to your other question about my comments for the
try/catch/finally directly on RB. Let me know if that doesn't make sense or if
you think it's better to leave them as it is.
> Memory leak if replay edits failed during region opening
>
>
> Key: HBASE-21031
> URL: https://issues.apache.org/jira/browse/HBASE-21031
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.1.0, 2.0.1
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Attachments: HBASE-21031.branch-2.0.001.patch,
> HBASE-21031.branch-2.0.002.patch, HBASE-21031.branch-2.0.003.patch,
> HBASE-21031.branch-2.0.004.patch, HBASE-21031.branch-2.0.005.patch,
> memoryleak.png
>
>
> Due to HBASE-21029, when replaying edits with a lot of same cells, the
> memstore won't flush, a exception will throw when all heap space was used:
> {code}
> 2018-08-06 15:52:27,590 ERROR
> [RS_OPEN_REGION-regionserver/hb-bp10cw4ejoy0a2f3f-009:16020-2]
> handler.OpenRegionHandler(302): Failed open of
> region=hbase_test,dffa78,1531227033378.cbf9a2daf3aaa0c7e931e9c9a7b53f41.,
> starting to roll back the global memstore size.
> java.lang.OutOfMemoryError: Java heap space
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
> at
> org.apache.hadoop.hbase.regionserver.OnheapChunk.allocateDataBuffer(OnheapChunk.java:41)
> at org.apache.hadoop.hbase.regionserver.Chunk.init(Chunk.java:104)
> at
> org.apache.hadoop.hbase.regionserver.ChunkCreator.getChunk(ChunkCreator.java:226)
> at
> org.apache.hadoop.hbase.regionserver.ChunkCreator.getChunk(ChunkCreator.java:180)
> at
> org.apache.hadoop.hbase.regionserver.ChunkCreator.getChunk(ChunkCreator.java:163)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.getOrMakeChunk(MemStoreLABImpl.java:273)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.copyCellInto(MemStoreLABImpl.java:148)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.copyCellInto(MemStoreLABImpl.java:111)
> at
> org.apache.hadoop.hbase.regionserver.Segment.maybeCloneWithAllocator(Segment.java:178)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.maybeCloneWithAllocator(AbstractMemStore.java:287)
> at
> org.apache.hadoop.hbase.regionserver.AbstractMemStore.add(AbstractMemStore.java:107)
> at org.apache.hadoop.hbase.regionserver.HStore.add(HStore.java:706)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.restoreEdit(HRegion.java:5494)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:4608)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:4404)
> {code}
> After this exception, the memstore did not roll back, and since MSLAB is
> used, all the chunk allocated won't release for ever. Those memory is leak
> forever...
> We need to rollback the memory if open region fails(For now, only global
> memstore size is decreased after failure).
> Another problem is that we use replayEditsPerRegion in RegionServerAccounting
> to record how many memory used during replaying. And decrease the global
> memstore size if replay fails. This is not right, since during replaying, we
> may also flush the memstore, the size in the map of replayEditsPerRegion is
> not accurate at all!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20175) hbase-spark needs scala dependency convergance
[
https://issues.apache.org/jira/browse/HBASE-20175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581504#comment-16581504
]
Mike Drob commented on HBASE-20175:
---
Tried building this locally, got the following error:
{noformat}
[ERROR] Failed to execute goal
net.alchim31.maven:scala-maven-plugin:3.4.1:add-source (scala-compile-first) on
project hbase-spark: The plugin net.alchim31.maven:scala-maven-plugin:3.4.1
requires Maven version 3.5.3 -> [Help 1]
{noformat}
Let me try to figure out what our current maven version requirements are.
> hbase-spark needs scala dependency convergance
> --
>
> Key: HBASE-20175
> URL: https://issues.apache.org/jira/browse/HBASE-20175
> Project: HBase
> Issue Type: Bug
> Components: dependencies, spark
>Reporter: Mike Drob
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-20175.v01.patch, HBASE-20175.v04.patch,
> HBASE-20175.v05.patch
>
>
> This is a follow-on to HBASE-16179 - I think we might need to specify an
> exclude in the dependency management.
> {noformat}
> [INFO] --- scala-maven-plugin:3.2.0:compile (scala-compile-first) @
> hbase-spark ---
> [WARNING] Expected all dependencies to require Scala version: 2.11.8
> [WARNING] org.apache.hbase:hbase-spark:3.0.0-SNAPSHOT requires scala
> version: 2.11.8
> [WARNING] org.apache.spark:spark-streaming_2.11:2.1.1 requires scala
> version: 2.11.8
> [WARNING] org.apache.spark:spark-streaming_2.11:2.1.1 requires scala
> version: 2.11.8
> [WARNING] org.scalatest:scalatest_2.11:2.2.4 requires scala version: 2.11.2
> {noformat}
> [~tedyu] - since you're already fiddling in this area, do you want to take a
> look?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21061) fix synchronization of org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
[
https://issues.apache.org/jira/browse/HBASE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581511#comment-16581511
]
Sean Busbey commented on HBASE-21061:
-
This was broken by HBASE-20895, as well as several other "inconsistent
synchronization" warnings on some of branches-1, essentially because all
variables referenced both in readAndProcess and somewhere else became covered
by the instance synchronization added there.
> fix synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
> ---
>
> Key: HBASE-21061
> URL: https://issues.apache.org/jira/browse/HBASE-21061
> Project: HBase
> Issue Type: Sub-task
> Components: rpc
>Affects Versions: 1.5.0, 1.2.7, 1.3.3, 1.4.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
>
> fix findbugs highlighted issue
> {code}
> Inconsistent synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap; locked 75% of time
> Unsynchronized access at RpcServer.java:75% of time Unsynchronized access at
> RpcServer.java
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20175) hbase-spark needs scala dependency convergance
[
https://issues.apache.org/jira/browse/HBASE-20175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581512#comment-16581512
]
Mike Drob commented on HBASE-20175:
---
Our current enforced minimum maven version is 3.0.4 according to both the pom
and the ref guide. We'd need to update both of those places if we're doing this
change. Probably need to give folks a heads-up on the dev@ list and see if
there are any concerns. Do you want to send that email out or would you prefer
I take care of it?
> hbase-spark needs scala dependency convergance
> --
>
> Key: HBASE-20175
> URL: https://issues.apache.org/jira/browse/HBASE-20175
> Project: HBase
> Issue Type: Bug
> Components: dependencies, spark
>Reporter: Mike Drob
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-20175.v01.patch, HBASE-20175.v04.patch,
> HBASE-20175.v05.patch
>
>
> This is a follow-on to HBASE-16179 - I think we might need to specify an
> exclude in the dependency management.
> {noformat}
> [INFO] --- scala-maven-plugin:3.2.0:compile (scala-compile-first) @
> hbase-spark ---
> [WARNING] Expected all dependencies to require Scala version: 2.11.8
> [WARNING] org.apache.hbase:hbase-spark:3.0.0-SNAPSHOT requires scala
> version: 2.11.8
> [WARNING] org.apache.spark:spark-streaming_2.11:2.1.1 requires scala
> version: 2.11.8
> [WARNING] org.apache.spark:spark-streaming_2.11:2.1.1 requires scala
> version: 2.11.8
> [WARNING] org.scalatest:scalatest_2.11:2.2.4 requires scala version: 2.11.2
> {noformat}
> [~tedyu] - since you're already fiddling in this area, do you want to take a
> look?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21062) WALFactory has misleading notion of "default"
[
https://issues.apache.org/jira/browse/HBASE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-21062:
---
Status: Patch Available (was: Open)
> WALFactory has misleading notion of "default"
> -
>
> Key: HBASE-21062
> URL: https://issues.apache.org/jira/browse/HBASE-21062
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-21062.001.branch-2.0.patch
>
>
> In WALFactory, there is an enum {{Providers}} which has a list of supported
> WALProvider implementations. In addition to list this, there is also a
> {{defaultProvider}} (which the Configuration defaults to), that is meant to
> be our "advertised" default WALProvider.
> However, the implementation of {{getProviderClass}} in WALFactory doesn't
> actually adhere to the value of this enum, instead *always* returning
> AsyncFSWal if it can be loaded.
> Having the default value in the enum but then overriding it in the
> implementation of {{getProviderClass}} is silly and misleading.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21062) WALFactory has misleading notion of "default"
[
https://issues.apache.org/jira/browse/HBASE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-21062:
---
Attachment: HBASE-21062.001.branch-2.0.patch
> WALFactory has misleading notion of "default"
> -
>
> Key: HBASE-21062
> URL: https://issues.apache.org/jira/browse/HBASE-21062
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-21062.001.branch-2.0.patch
>
>
> In WALFactory, there is an enum {{Providers}} which has a list of supported
> WALProvider implementations. In addition to list this, there is also a
> {{defaultProvider}} (which the Configuration defaults to), that is meant to
> be our "advertised" default WALProvider.
> However, the implementation of {{getProviderClass}} in WALFactory doesn't
> actually adhere to the value of this enum, instead *always* returning
> AsyncFSWal if it can be loaded.
> Having the default value in the enum but then overriding it in the
> implementation of {{getProviderClass}} is silly and misleading.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21061) fix synchronization of org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
[
https://issues.apache.org/jira/browse/HBASE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581523#comment-16581523
]
Sean Busbey commented on HBASE-21061:
-
precommit on HBASE-20895 didn't run findbugs:
{code}
00:00:30.499 Step 20/46 : RUN mkdir -p /opt/findbugs && curl -L -s -S
https://sourceforge.net/projects/findbugs/files/findbugs/3.0.1/findbugs-noUpdateChecks-3.0.1.tar.gz/download
-o /opt/findbugs.tar.gz && tar xzf /opt/findbugs.tar.gz
--strip-components 1 -C /opt/findbugs
00:00:30.500 ---> Using cache
00:00:30.500 ---> 9fd52b187bc9
00:00:30.500 Step 21/46 : ENV FINDBUGS_HOME /opt/findbugs
00:00:30.500 ---> Using cache
00:00:30.500 ---> 302e9c623019
{code}
{code}
00:00:43.956 executable '/usr/bin/findbugs' for 'findbugs' does not exist.
00:00:43.956 executable '/usr/bin/computeBugHistory' for 'computeBugHistory'
does not exist.
00:00:43.957 executable '/usr/bin/convertXmlToText' for 'convertXmlToText' does
not exist.
00:00:43.958 executable '/usr/bin/filterBugs' for 'filterBugs' does not exist.
00:00:43.959 executable '/usr/bin/setBugDatabaseInfo' for 'setBugDatabaseInfo'
does not exist.
{code}
{code}
0 findbugs0m 0s Findbugs executables are not available.
{code}
the nightly build on branch-1 that posted back included a findbugs failure that
called the inconsistent use out ([results for build #395 if still up on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/395/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/].
compared to no findbugs failure on [results for build #394 if still up on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/394/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]).
But literally every sub-check on every branch that went up was red, as were
all of the top-level build histories for branches-1, so I'd guess it was just
noise.
Here's the first complaint on branch-1 in build #395
{code}
FindBugsmodule:hbase-server
Inconsistent synchronization of
org.apache.hadoop.hbase.ipc.RpcServer$Connection.connectionHeader; locked 81%
of time Unsynchronized access at RpcServer.java:81% of time Unsynchronized
access at RpcServer.java:[line 1371]
Inconsistent synchronization of
org.apache.hadoop.hbase.ipc.RpcServer$Connection.service; locked 57% of time
Unsynchronized access at RpcServer.java:57% of time Unsynchronized access at
RpcServer.java:[line 422]
Inconsistent synchronization of
org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap; locked 75% of time
Unsynchronized access at RpcServer.java:75% of time Unsynchronized access at
RpcServer.java:[line 477]
{code}
> fix synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
> ---
>
> Key: HBASE-21061
> URL: https://issues.apache.org/jira/browse/HBASE-21061
> Project: HBase
> Issue Type: Sub-task
> Components: rpc
>Affects Versions: 1.5.0, 1.2.7, 1.3.3, 1.4.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
>
> fix findbugs highlighted issue
> {code}
> Inconsistent synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap; locked 75% of time
> Unsynchronized access at RpcServer.java:75% of time Unsynchronized access at
> RpcServer.java
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18477) Umbrella JIRA for HBase Read Replica clusters
[
https://issues.apache.org/jira/browse/HBASE-18477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581559#comment-16581559
]
Hudson commented on HBASE-18477:
Results for branch HBASE-18477
[build #296 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18477/296/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18477/296//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18477/296//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18477/296//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18477/296//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Umbrella JIRA for HBase Read Replica clusters
> -
>
> Key: HBASE-18477
> URL: https://issues.apache.org/jira/browse/HBASE-18477
> Project: HBase
> Issue Type: New Feature
>Reporter: Zach York
>Assignee: Zach York
>Priority: Major
> Attachments: HBase Read-Replica Clusters Scope doc.docx, HBase
> Read-Replica Clusters Scope doc.pdf, HBase Read-Replica Clusters Scope
> doc_v2.docx, HBase Read-Replica Clusters Scope doc_v2.pdf
>
>
> Recently, changes (such as HBASE-17437) have unblocked HBase to run with a
> root directory external to the cluster (such as in Amazon S3). This means
> that the data is stored outside of the cluster and can be accessible after
> the cluster has been terminated. One use case that is often asked about is
> pointing multiple clusters to one root directory (sharing the data) to have
> read resiliency in the case of a cluster failure.
>
> This JIRA is an umbrella JIRA to contain all the tasks necessary to create a
> read-replica HBase cluster that is pointed at the same root directory.
>
> This requires making the Read-Replica cluster Read-Only (no metadata
> operation or data operations).
> Separating the hbase:meta table for each cluster (Otherwise HBase gets
> confused with multiple clusters trying to update the meta table with their ip
> addresses)
> Adding refresh functionality for the meta table to ensure new metadata is
> picked up on the read replica cluster.
> Adding refresh functionality for HFiles for a given table to ensure new data
> is picked up on the read replica cluster.
>
> This can be used with any existing cluster that is backed by an external
> filesystem.
>
> Please note that this feature is still quite manual (with the potential for
> automation later).
>
> More information on this particular feature can be found here:
> https://aws.amazon.com/blogs/big-data/setting-up-read-replica-clusters-with-hbase-on-amazon-s3/
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19008) Add missing equals or hashCode method(s) to stock Filter implementations
[
https://issues.apache.org/jira/browse/HBASE-19008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581561#comment-16581561
]
Ted Yu commented on HBASE-19008:
Please refer to
https://builds.apache.org/job/PreCommit-HBASE-Build/14049/artifact/patchprocess/diff-checkstyle-hbase-spark.txt
and fix the warnings there.
Thanks for your patience, Liubang.
> Add missing equals or hashCode method(s) to stock Filter implementations
>
>
> Key: HBASE-19008
> URL: https://issues.apache.org/jira/browse/HBASE-19008
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: liubangchen
>Priority: Major
> Labels: filter
> Attachments: Filters.png, HBASE-19008-1.patch, HBASE-19008-10.patch,
> HBASE-19008-11.patch, HBASE-19008-12.patch, HBASE-19008-2.patch,
> HBASE-19008-3.patch, HBASE-19008-4.patch, HBASE-19008-5.patch,
> HBASE-19008-6.patch, HBASE-19008-7.patch, HBASE-19008-8.patch,
> HBASE-19008-9.patch, HBASE-19008.patch
>
>
> In HBASE-15410, [~mdrob] reminded me that Filter implementations may not
> write {{equals}} or {{hashCode}} method(s).
> This issue is to add missing {{equals}} or {{hashCode}} method(s) to stock
> Filter implementations such as KeyOnlyFilter.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20387) flaky infrastructure should work for all branches
[
https://issues.apache.org/jira/browse/HBASE-20387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581585#comment-16581585
]
Hudson commented on HBASE-20387:
Results for branch HBASE-20387
[build #13 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20387/13/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20387/13//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20387/13//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20387/13//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> flaky infrastructure should work for all branches
> -
>
> Key: HBASE-20387
> URL: https://issues.apache.org/jira/browse/HBASE-20387
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Attachments: HBASE-20387.0.patch, HBASE-20387.1.patch
>
>
> We need a flaky list per-branch, since what does/does not work reliably on
> master isn't really relevant to our older maintenance release lines.
> We should just make the invocation a step in the current per-branch nightly
> jobs, prior to when we need the list in the stages that run unit tests. We
> can publish it in the nightly job as well so that precommit can still get it.
> (and can fetch it per-branch if needed)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21062) WALFactory has misleading notion of "default"
[
https://issues.apache.org/jira/browse/HBASE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581607#comment-16581607
]
Mingliang Liu commented on HBASE-21062:
---
The existing code implicitly employs the fact that {{defaultProvider}} is
having {{AsyncFSWALProvider}} class value. It makes sense as the enum
{{defaultProvider}} can not have a different value at runtime unless we change
code. This implementation pattern is not ideal to me either - because when we
change the {{defaultProvider}} we may forget to change {{getProviderClass}} -
but it's tolerable. Meanwhile, the newly added test in the patch does not
actually override the {{defaultProvider}} value and {{getDefaultProvider}} is
not used anywhere. So the test will pass anyway: w/ or w/o the fix in
{{getProviderClass}}, w/ or w/o the {{getDefaultProvider}}. Correct me if I'm
wrong.
Another (orthogonal?) problem in {{getProviderClass}} is that, the try-catch
scope is not implemented well. The reason is that, in the
{{catch(IllegalArgumentException exception)\{...\}}} clause, we end up with
returning the {{AsyncFSWALProvider}}. However, this time we would skip the
process that load {{AsyncFSWALProvider}} and then falls back to
{{FSHLogProvider}}.
Last, for any error case, it merits to have logging message.
Overall, a fix in my mind is like - NOT TESTED:
{code:java}
@VisibleForTesting
public Class getProviderClass(String key, String
defaultValue) {
Providers provider;
try {
provider = Providers.valueOf(conf.get(key, defaultValue));
} catch (IllegalArgumentException exception) {
// Fall back to them specifying a class name
// Note that the passed default class shouldn't actually be used, since
the above only fails
// when there is a config value present.
LOG.warn("Unrecognized WALProvider class. Falling back to default
AsyncFSWALProvider.");
provider = Providers.defaultProviderdr;
}
if (provider.clazz == AsyncFSWALProvider.class &&
!AsyncFSWALProvider.load()) {
// AsyncFSWAL has better performance in most cases, and also uses less
resources, we will try
// to use it if possible. But it deeply hacks into the internal of
DFSClient so will be easily
// broken when upgrading hadoop. If it is broken, then we fall back to
use FSHLog.
LOG.warn("Failed to load AsyncFSWALProvider. Falling back to
FSHLogProvider.");
return FSHLogProvider.class;
}
return provider.clazz;
}
{code}
Thanks!
> WALFactory has misleading notion of "default"
> -
>
> Key: HBASE-21062
> URL: https://issues.apache.org/jira/browse/HBASE-21062
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-21062.001.branch-2.0.patch
>
>
> In WALFactory, there is an enum {{Providers}} which has a list of supported
> WALProvider implementations. In addition to list this, there is also a
> {{defaultProvider}} (which the Configuration defaults to), that is meant to
> be our "advertised" default WALProvider.
> However, the implementation of {{getProviderClass}} in WALFactory doesn't
> actually adhere to the value of this enum, instead *always* returning
> AsyncFSWal if it can be loaded.
> Having the default value in the enum but then overriding it in the
> implementation of {{getProviderClass}} is silly and misleading.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21062) WALFactory has misleading notion of "default"
[
https://issues.apache.org/jira/browse/HBASE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581614#comment-16581614
]
Josh Elser commented on HBASE-21062:
{quote}It makes sense as the enum {{defaultProvider}} can not have a different
value at runtime unless we change code
{quote}
My mindset is that the defaultProvider _will_ change at some point in time –
inevitably. I think the change I've already put forward makes this much more
clear for someone new to come along and change what the default is without
having to know what about the impls of a different WALProvider.
{quote}So the test will pass anyway: w/ or w/o the fix in {{getProviderClass}},
w/ or w/o the {{getDefaultProvider}}. Correct me if I'm wrong.
{quote}
The test fails without the fix to WALFactory.
{quote}Another (orthogonal?) problem in {{getProviderClass}} is that, the
try-catch scope is not implemented well
{quote}
I noticed that as well. Without having a clear reason to know why this happened
(nor, it likely being something that we'd be OK with hitting all the time), I
ignored that.
{quote}Last, for any error case, it merits to have logging message.
{quote}
That's a good idea.
> WALFactory has misleading notion of "default"
> -
>
> Key: HBASE-21062
> URL: https://issues.apache.org/jira/browse/HBASE-21062
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-21062.001.branch-2.0.patch
>
>
> In WALFactory, there is an enum {{Providers}} which has a list of supported
> WALProvider implementations. In addition to list this, there is also a
> {{defaultProvider}} (which the Configuration defaults to), that is meant to
> be our "advertised" default WALProvider.
> However, the implementation of {{getProviderClass}} in WALFactory doesn't
> actually adhere to the value of this enum, instead *always* returning
> AsyncFSWal if it can be loaded.
> Having the default value in the enum but then overriding it in the
> implementation of {{getProviderClass}} is silly and misleading.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20387) flaky infrastructure should work for all branches
[ https://issues.apache.org/jira/browse/HBASE-20387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581631#comment-16581631 ] Mike Drob commented on HBASE-20387: --- Can we add a comment to the dockerfile to denote that it is used by the flaky test reporting jobs? With multiple dockerfiles in our repo, it is hard for me to keep track of what belongs where. With that change I'm +1 on the patch. > flaky infrastructure should work for all branches > - > > Key: HBASE-20387 > URL: https://issues.apache.org/jira/browse/HBASE-20387 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Attachments: HBASE-20387.0.patch, HBASE-20387.1.patch > > > We need a flaky list per-branch, since what does/does not work reliably on > master isn't really relevant to our older maintenance release lines. > We should just make the invocation a step in the current per-branch nightly > jobs, prior to when we need the list in the stages that run unit tests. We > can publish it in the nightly job as well so that precommit can still get it. > (and can fetch it per-branch if needed) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21062) WALFactory has misleading notion of "default"
[
https://issues.apache.org/jira/browse/HBASE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581633#comment-16581633
]
Mingliang Liu edited comment on HBASE-21062 at 8/15/18 9:24 PM:
Thanks [~elserj].
It makes sense to me to keep the minimum change in this patch, so fixing the
try-catch can be a separate effort (do you mind if I file one?). I like the
mindset of making the "default" the real "default" being used and not worrying
about future change. I'm +1 (non-binding) on the fix in {{getProviderClass}}.
However, as to the test - I think it's hard to test - I found if I revert the
change in {{getProviderClass}} in you patch, the test will pass; if I revert
the not used {{getDefaultProvider}} class (and the override in test), it passes
as well. Again I'm fine if we don't add a new test here.
was (Author: liuml07):
Thanks [~elserj].
It makes sense to me to keep the minimum change in this patch, so fixing the
try-catch can be a separate effort (do you mind if I file one?). I like the
mindset of making the "default" the real "default" being used and not worrying
about future change. I'm +1 (overall) on the fix in {{getProviderClass}}.
However, as to the test - I think it's hard to test - I found if I revert the
change in {{getProviderClass}} in you patch, the test will pass; if I revert
the not used {{getDefaultProvider}} class (and the override in test), it passes
as well. Again I'm fine if we don't add a new test here.
> WALFactory has misleading notion of "default"
> -
>
> Key: HBASE-21062
> URL: https://issues.apache.org/jira/browse/HBASE-21062
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-21062.001.branch-2.0.patch
>
>
> In WALFactory, there is an enum {{Providers}} which has a list of supported
> WALProvider implementations. In addition to list this, there is also a
> {{defaultProvider}} (which the Configuration defaults to), that is meant to
> be our "advertised" default WALProvider.
> However, the implementation of {{getProviderClass}} in WALFactory doesn't
> actually adhere to the value of this enum, instead *always* returning
> AsyncFSWal if it can be loaded.
> Having the default value in the enum but then overriding it in the
> implementation of {{getProviderClass}} is silly and misleading.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21062) WALFactory has misleading notion of "default"
[
https://issues.apache.org/jira/browse/HBASE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581633#comment-16581633
]
Mingliang Liu commented on HBASE-21062:
---
Thanks [~elserj].
It makes sense to me to keep the minimum change in this patch, so fixing the
try-catch can be a separate effort (do you mind if I file one?). I like the
mindset of making the "default" the real "default" being used and not worrying
about future change. I'm +1 (overall) on the fix in {{getProviderClass}}.
However, as to the test - I think it's hard to test - I found if I revert the
change in {{getProviderClass}} in you patch, the test will pass; if I revert
the not used {{getDefaultProvider}} class (and the override in test), it passes
as well. Again I'm fine if we don't add a new test here.
> WALFactory has misleading notion of "default"
> -
>
> Key: HBASE-21062
> URL: https://issues.apache.org/jira/browse/HBASE-21062
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-21062.001.branch-2.0.patch
>
>
> In WALFactory, there is an enum {{Providers}} which has a list of supported
> WALProvider implementations. In addition to list this, there is also a
> {{defaultProvider}} (which the Configuration defaults to), that is meant to
> be our "advertised" default WALProvider.
> However, the implementation of {{getProviderClass}} in WALFactory doesn't
> actually adhere to the value of this enum, instead *always* returning
> AsyncFSWal if it can be loaded.
> Having the default value in the enum but then overriding it in the
> implementation of {{getProviderClass}} is silly and misleading.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21062) WALFactory has misleading notion of "default"
[
https://issues.apache.org/jira/browse/HBASE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581634#comment-16581634
]
Josh Elser commented on HBASE-21062:
{quote}if I revert the not used {{getDefaultProvider}} class (and the override
in test)
{quote}
Oh! My bad. Let me re-look at this. I had this working locally and must have
botched this before making the patch.
> WALFactory has misleading notion of "default"
> -
>
> Key: HBASE-21062
> URL: https://issues.apache.org/jira/browse/HBASE-21062
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-21062.001.branch-2.0.patch
>
>
> In WALFactory, there is an enum {{Providers}} which has a list of supported
> WALProvider implementations. In addition to list this, there is also a
> {{defaultProvider}} (which the Configuration defaults to), that is meant to
> be our "advertised" default WALProvider.
> However, the implementation of {{getProviderClass}} in WALFactory doesn't
> actually adhere to the value of this enum, instead *always* returning
> AsyncFSWal if it can be loaded.
> Having the default value in the enum but then overriding it in the
> implementation of {{getProviderClass}} is silly and misleading.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21062) WALFactory has misleading notion of "default"
[
https://issues.apache.org/jira/browse/HBASE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581637#comment-16581637
]
Josh Elser commented on HBASE-21062:
{quote}I had this working locally and must have botched this before making the
patch.
{quote}
Oh, I see. When I had that method working earlier, it was only doing this check
when AsyncFSWALProvider when it was the defaultProvider. I think I accidentally
undid that work – if AsyncFSWAL is specified and we can't load it, we should
fail.
> WALFactory has misleading notion of "default"
> -
>
> Key: HBASE-21062
> URL: https://issues.apache.org/jira/browse/HBASE-21062
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-21062.001.branch-2.0.patch
>
>
> In WALFactory, there is an enum {{Providers}} which has a list of supported
> WALProvider implementations. In addition to list this, there is also a
> {{defaultProvider}} (which the Configuration defaults to), that is meant to
> be our "advertised" default WALProvider.
> However, the implementation of {{getProviderClass}} in WALFactory doesn't
> actually adhere to the value of this enum, instead *always* returning
> AsyncFSWal if it can be loaded.
> Having the default value in the enum but then overriding it in the
> implementation of {{getProviderClass}} is silly and misleading.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20925) Canary test to expose table availability rate
[ https://issues.apache.org/jira/browse/HBASE-20925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581652#comment-16581652 ] Xu Cang commented on HBASE-20925: - Addressed [~ckulkarni]'s comment. Now the summary looks like this: 2018-08-15 14:43:24,445 INFO [CanaryMonitor-1534369399255] tool.Canary: 2018-08-15 14:43:24,445 INFO [CanaryMonitor-1534369399255] tool.Canary: === Summary: === 2018-08-15 14:43:24,445 INFO [CanaryMonitor-1534369399255] tool.Canary: Read success rate for table : MyTable is: 1.0 success count: 1 failure count: 0 2018-08-15 14:43:24,446 INFO [CanaryMonitor-1534369399255] tool.Canary: Read success rate for table : mytable3 is: 1.0 success count: 1 failure count: 0 2018-08-15 14:43:24,446 INFO [CanaryMonitor-1534369399255] tool.Canary: Read success rate for table : mytable2 is: 1.0 success count: 1 failure count: 0 2018-08-15 14:43:24,446 INFO [CanaryMonitor-1534369399255] tool.Canary: Read success rate for table : mytable4 is: 1.0 success count: 2 failure count: 0 2018-08-15 14:43:24,446 INFO [CanaryMonitor-1534369399255] tool.Canary: ===END== > Canary test to expose table availability rate > -- > > Key: HBASE-20925 > URL: https://issues.apache.org/jira/browse/HBASE-20925 > Project: HBase > Issue Type: Improvement > Components: canary >Affects Versions: 3.0.0, 2.0.0, 1.4.6 >Reporter: Xu Cang >Assignee: Xu Cang >Priority: Major > Labels: Canary > Attachments: HBASE-20925.master.001.patch, > HBASE-20925.master.002.patch, HBASE-20925.master.003.patch, > HBASE-20925.master.004.patch > > > Canary test to expose table availability rate. > > It will print table availability rate such as below. > > > *2018-07-27 17:11:06,823 INFO [CanaryMonitor-1532736665083] tool.Canary: > * > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: > === Summary: ===* > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: Read > success rate for table : MyTable is: 1.0 .* > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: Read > success rate for table : mytable3 is: 0.9* > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: Read > success rate for table : mytable2 is: 0.8* > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: Read > success rate for table : mytable4 is: 1.0* > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: > ===END==* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20925) Canary test to expose table availability rate
[ https://issues.apache.org/jira/browse/HBASE-20925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Cang updated HBASE-20925: Attachment: HBASE-20925.master.005.patch > Canary test to expose table availability rate > -- > > Key: HBASE-20925 > URL: https://issues.apache.org/jira/browse/HBASE-20925 > Project: HBase > Issue Type: Improvement > Components: canary >Affects Versions: 3.0.0, 2.0.0, 1.4.6 >Reporter: Xu Cang >Assignee: Xu Cang >Priority: Major > Labels: Canary > Attachments: HBASE-20925.master.001.patch, > HBASE-20925.master.002.patch, HBASE-20925.master.003.patch, > HBASE-20925.master.004.patch, HBASE-20925.master.005.patch > > > Canary test to expose table availability rate. > > It will print table availability rate such as below. > > > *2018-07-27 17:11:06,823 INFO [CanaryMonitor-1532736665083] tool.Canary: > * > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: > === Summary: ===* > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: Read > success rate for table : MyTable is: 1.0 .* > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: Read > success rate for table : mytable3 is: 0.9* > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: Read > success rate for table : mytable2 is: 0.8* > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: Read > success rate for table : mytable4 is: 1.0* > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: > ===END==* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20925) Canary test to expose table availability rate
[ https://issues.apache.org/jira/browse/HBASE-20925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581655#comment-16581655 ] Xu Cang commented on HBASE-20925: - [~apurtell] > Canary test to expose table availability rate > -- > > Key: HBASE-20925 > URL: https://issues.apache.org/jira/browse/HBASE-20925 > Project: HBase > Issue Type: Improvement > Components: canary >Affects Versions: 3.0.0, 2.0.0, 1.4.6 >Reporter: Xu Cang >Assignee: Xu Cang >Priority: Major > Labels: Canary > Attachments: HBASE-20925.master.001.patch, > HBASE-20925.master.002.patch, HBASE-20925.master.003.patch, > HBASE-20925.master.004.patch, HBASE-20925.master.005.patch > > > Canary test to expose table availability rate. > > It will print table availability rate such as below. > > > *2018-07-27 17:11:06,823 INFO [CanaryMonitor-1532736665083] tool.Canary: > * > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: > === Summary: ===* > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: Read > success rate for table : MyTable is: 1.0 .* > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: Read > success rate for table : mytable3 is: 0.9* > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: Read > success rate for table : mytable2 is: 0.8* > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: Read > success rate for table : mytable4 is: 1.0* > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: > ===END==* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21062) WALFactory has misleading notion of "default"
[
https://issues.apache.org/jira/browse/HBASE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581657#comment-16581657
]
Josh Elser commented on HBASE-21062:
[~liuml07], thanks for taking a look at v1. What's your take on v2?
> WALFactory has misleading notion of "default"
> -
>
> Key: HBASE-21062
> URL: https://issues.apache.org/jira/browse/HBASE-21062
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-21062.001.branch-2.0.patch,
> HBASE-21062.002.branch-2.0.patch
>
>
> In WALFactory, there is an enum {{Providers}} which has a list of supported
> WALProvider implementations. In addition to list this, there is also a
> {{defaultProvider}} (which the Configuration defaults to), that is meant to
> be our "advertised" default WALProvider.
> However, the implementation of {{getProviderClass}} in WALFactory doesn't
> actually adhere to the value of this enum, instead *always* returning
> AsyncFSWal if it can be loaded.
> Having the default value in the enum but then overriding it in the
> implementation of {{getProviderClass}} is silly and misleading.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21062) WALFactory has misleading notion of "default"
[
https://issues.apache.org/jira/browse/HBASE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-21062:
---
Attachment: HBASE-21062.002.branch-2.0.patch
> WALFactory has misleading notion of "default"
> -
>
> Key: HBASE-21062
> URL: https://issues.apache.org/jira/browse/HBASE-21062
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-21062.001.branch-2.0.patch,
> HBASE-21062.002.branch-2.0.patch
>
>
> In WALFactory, there is an enum {{Providers}} which has a list of supported
> WALProvider implementations. In addition to list this, there is also a
> {{defaultProvider}} (which the Configuration defaults to), that is meant to
> be our "advertised" default WALProvider.
> However, the implementation of {{getProviderClass}} in WALFactory doesn't
> actually adhere to the value of this enum, instead *always* returning
> AsyncFSWal if it can be loaded.
> Having the default value in the enum but then overriding it in the
> implementation of {{getProviderClass}} is silly and misleading.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21033) Separate the StoreHeap from StoreFileHeap
[ https://issues.apache.org/jira/browse/HBASE-21033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581661#comment-16581661 ] Ted Yu commented on HBASE-21033: I looked at 21033-branch-1-minimal.txt and think it makes sense. Probably master patch should be attached (and get a QA run). > Separate the StoreHeap from StoreFileHeap > - > > Key: HBASE-21033 > URL: https://issues.apache.org/jira/browse/HBASE-21033 > Project: HBase > Issue Type: Improvement >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Attachments: 21033-branch-1-minimal.txt, 21033-branch-1-v0.txt > > > Currently KeyValueHeap is used for both, heaps of StoreScanners at the Region > level as well as heaps of StoreFileScanners (and a MemstoreScanner) at the > Store level. > This is various problems: > # Some incorrect method usage can only be deduced at runtime via runtime > exception. > # In profiling sessions it's hard to distinguish the two. > # It's just not clean :) > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20925) Canary test to expose table availability rate
[ https://issues.apache.org/jira/browse/HBASE-20925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Cang updated HBASE-20925: Flags: Patch External issue URL: https://reviews.apache.org/r/68097/ > Canary test to expose table availability rate > -- > > Key: HBASE-20925 > URL: https://issues.apache.org/jira/browse/HBASE-20925 > Project: HBase > Issue Type: Improvement > Components: canary >Affects Versions: 3.0.0, 2.0.0, 1.4.6 >Reporter: Xu Cang >Assignee: Xu Cang >Priority: Major > Labels: Canary > Attachments: HBASE-20925.master.001.patch, > HBASE-20925.master.002.patch, HBASE-20925.master.003.patch, > HBASE-20925.master.004.patch, HBASE-20925.master.005.patch > > > Canary test to expose table availability rate. > > It will print table availability rate such as below. > > > *2018-07-27 17:11:06,823 INFO [CanaryMonitor-1532736665083] tool.Canary: > * > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: > === Summary: ===* > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: Read > success rate for table : MyTable is: 1.0 .* > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: Read > success rate for table : mytable3 is: 0.9* > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: Read > success rate for table : mytable2 is: 0.8* > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: Read > success rate for table : mytable4 is: 1.0* > *2018-07-27 17:11:06,824 INFO [CanaryMonitor-1532736665083] tool.Canary: > ===END==* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21062) WALFactory has misleading notion of "default"
[
https://issues.apache.org/jira/browse/HBASE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581678#comment-16581678
]
Hadoop QA commented on HBASE-21062:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.0 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
31s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
40s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
49s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
14s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} branch-2.0 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
9s{color} | {color:red} hbase-server: The patch generated 2 new + 19 unchanged
- 0 fixed = 21 total (was 19) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
51s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 16s{color} | {color:green} Patch does not cause any errors with Hadoop 2.6.5
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}109m
44s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}143m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:6f01af0 |
| JIRA Issue | HBASE-21062 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12935742/HBASE-21062.001.branch-2.0.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 6d75256da050 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2.0 / 5e06b80e75 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/14051/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/14051/testReport/ |
| Max. process+thread count | 3566 (vs. ulimit of 1) |
| modules | C: hbase-server U:
[jira] [Commented] (HBASE-21062) WALFactory has misleading notion of "default"
[
https://issues.apache.org/jira/browse/HBASE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581677#comment-16581677
]
Mingliang Liu commented on HBASE-21062:
---
+1 (non-binding)
{quote}
if AsyncFSWAL is specified and we can't load it, we should fail.
{quote}
Sorry I missed this point previously. It is right to keep the original behavior.
> WALFactory has misleading notion of "default"
> -
>
> Key: HBASE-21062
> URL: https://issues.apache.org/jira/browse/HBASE-21062
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-21062.001.branch-2.0.patch,
> HBASE-21062.002.branch-2.0.patch
>
>
> In WALFactory, there is an enum {{Providers}} which has a list of supported
> WALProvider implementations. In addition to list this, there is also a
> {{defaultProvider}} (which the Configuration defaults to), that is meant to
> be our "advertised" default WALProvider.
> However, the implementation of {{getProviderClass}} in WALFactory doesn't
> actually adhere to the value of this enum, instead *always* returning
> AsyncFSWal if it can be loaded.
> Having the default value in the enum but then overriding it in the
> implementation of {{getProviderClass}} is silly and misleading.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20881) Introduce a region transition procedure to handle all the state transition for a region
[ https://issues.apache.org/jira/browse/HBASE-20881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20881: -- Attachment: HBASE-20881-v11.patch > Introduce a region transition procedure to handle all the state transition > for a region > --- > > Key: HBASE-20881 > URL: https://issues.apache.org/jira/browse/HBASE-20881 > Project: HBase > Issue Type: Sub-task > Components: amv2, proc-v2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-20881-v1.patch, HBASE-20881-v10.patch, > HBASE-20881-v11.patch, HBASE-20881-v2.patch, HBASE-20881-v3.patch, > HBASE-20881-v4.patch, HBASE-20881-v4.patch, HBASE-20881-v5.patch, > HBASE-20881-v6.patch, HBASE-20881-v7.patch, HBASE-20881-v7.patch, > HBASE-20881-v8.patch, HBASE-20881-v9.patch, HBASE-20881.patch > > > Now have an AssignProcedure, an UnssignProcedure, and also a > MoveRegionProcedure which schedules an AssignProcedure and an > UnssignProcedure to move a region. This makes the logic a bit complicated, as > MRP is not a RIT, so when SCP can not interrupt it directly... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21062) WALFactory has misleading notion of "default"
[
https://issues.apache.org/jira/browse/HBASE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581681#comment-16581681
]
Ted Yu commented on HBASE-21062:
lgtm
{code}
+ // we can't us it, we want to fall back to FSHLog which we know works on
all versions.
{code}
'us it' -> "use it"
> WALFactory has misleading notion of "default"
> -
>
> Key: HBASE-21062
> URL: https://issues.apache.org/jira/browse/HBASE-21062
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.0.2, 2.2.0, 2.1.1
>
> Attachments: HBASE-21062.001.branch-2.0.patch,
> HBASE-21062.002.branch-2.0.patch
>
>
> In WALFactory, there is an enum {{Providers}} which has a list of supported
> WALProvider implementations. In addition to list this, there is also a
> {{defaultProvider}} (which the Configuration defaults to), that is meant to
> be our "advertised" default WALProvider.
> However, the implementation of {{getProviderClass}} in WALFactory doesn't
> actually adhere to the value of this enum, instead *always* returning
> AsyncFSWal if it can be loaded.
> Having the default value in the enum but then overriding it in the
> implementation of {{getProviderClass}} is silly and misleading.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21050) Exclusive lock may be held by a SUCCESS state procedure forever
[
https://issues.apache.org/jira/browse/HBASE-21050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581702#comment-16581702
]
stack commented on HBASE-21050:
---
Ok. Test is hard. All is happening down inside load procedures. The lock is
getting restored post crash and the child is being marked completed because it
is 'finished' ... so it is not being rescheduled. Messing, trying to test this,
the child procedure evaporates before I can get a hold on it. I had various
attempts at an 'entity' lock that had a lifecycle independent of Procedure but
what is wanted is exercising the locking we do inside the
MasterProcedureScheduler where it, an independent entity, has special mechanism
for keeping up region locks. Building up a test case that has
MasterProcedureScheduler at its core with Region entities would be a good bit
of work. I'm passing on it for now.
Let me commit this patch. At the very least, after being in here a while, patch
makes even more sense.
> Exclusive lock may be held by a SUCCESS state procedure forever
> ---
>
> Key: HBASE-21050
> URL: https://issues.apache.org/jira/browse/HBASE-21050
> Project: HBase
> Issue Type: Sub-task
> Components: amv2
>Affects Versions: 2.1.0, 2.0.1
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Attachments: HBASE-21050.branch-2.0.001.patch
>
>
> After HBASE-20846, we restore lock info for procedures. But, there is a case
> that the lock and be held by a already success procedure. Since the procedure
> won't execute again, the lock will held by the procedure forever.
> 1. All children for pid=1208 had been finished, but before procedure 1208
> awake, the master was killed
> {code}
> 2018-08-05 02:20:14,465 INFO [PEWorker-8]
> procedure2.ProcedureExecutor(1659): Finished subprocedure(s) of pid=1208,
> ppid=1206, state=RUNNABLE, hasLock=true; MoveRegionProcedure
> hri=c2a23a735f16df57299
> dba6fd4599f2f, source=e010125050127.bja,60020,1533403109034,
> destination=e010125050127.bja,60020,1533403109034; resume parent processing.
> 2018-08-05 02:20:14,466 INFO [PEWorker-8]
> procedure2.ProcedureExecutor(1296): Finished pid=1232, ppid=1208,
> state=SUCCESS, hasLock=false; AssignProcedure
> table=IntegrationTestBigLinkedList, region=c2a
> 23a735f16df57299dba6fd4599f2f, target=e010125050127.bja,60020,1533403109034
> in 1.5060sec
> {code}
> 2. Master restarts, since procedure 1208 held the lock before restart, so the
> lock was resotore for it
> {code}
> 2018-08-05 02:20:30,803 DEBUG [Thread-15] procedure2.ProcedureExecutor(456):
> Loading pid=1208, ppid=1206, state=SUCCESS, hasLock=false;
> MoveRegionProcedure hri=c2a23a735f16df57299dba6fd4599f2f, source=
> e010125050127.bja,60020,1533403109034,
> destination=e010125050127.bja,60020,1533403109034
> 2018-08-05 02:20:30,818 DEBUG [Thread-15] procedure2.Procedure(898):
> pid=1208, ppid=1206, state=SUCCESS, hasLock=false; MoveRegionProcedure
> hri=c2a23a735f16df57299dba6fd4599f2f, source=e010125050127.bj
> a,60020,1533403109034, destination=e010125050127.bja,60020,1533403109034 held
> the lock before restarting, call acquireLock to restore it.
> 2018-08-05 02:20:30,818 INFO [Thread-15]
> procedure.MasterProcedureScheduler(631): pid=1208, ppid=1206, state=SUCCESS,
> hasLock=false; MoveRegionProcedure hri=c2a23a735f16df57299dba6fd4599f2f,
> source=e0
> 10125050127.bja,60020,1533403109034,
> destination=e010125050127.bja,60020,1533403109034 checking lock on
> c2a23a735f16df57299dba6fd4599f2f
> {code}
> 3. Since procedure 1208 is success, it won't execute later, so the lock will
> be held by it forever
> We need to check the state of the procedure before restoring locks, if the
> procedure is already finished (success or rollback), we do not need to
> acquire lock for it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21050) Exclusive lock may be held by a SUCCESS state procedure forever
[
https://issues.apache.org/jira/browse/HBASE-21050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-21050:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.1.1
2.0.2
3.0.0
Status: Resolved (was: Patch Available)
Pushed to branch-2.0+.
> Exclusive lock may be held by a SUCCESS state procedure forever
> ---
>
> Key: HBASE-21050
> URL: https://issues.apache.org/jira/browse/HBASE-21050
> Project: HBase
> Issue Type: Sub-task
> Components: amv2
>Affects Versions: 2.1.0, 2.0.1
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Fix For: 3.0.0, 2.0.2, 2.1.1
>
> Attachments: HBASE-21050.branch-2.0.001.patch
>
>
> After HBASE-20846, we restore lock info for procedures. But, there is a case
> that the lock and be held by a already success procedure. Since the procedure
> won't execute again, the lock will held by the procedure forever.
> 1. All children for pid=1208 had been finished, but before procedure 1208
> awake, the master was killed
> {code}
> 2018-08-05 02:20:14,465 INFO [PEWorker-8]
> procedure2.ProcedureExecutor(1659): Finished subprocedure(s) of pid=1208,
> ppid=1206, state=RUNNABLE, hasLock=true; MoveRegionProcedure
> hri=c2a23a735f16df57299
> dba6fd4599f2f, source=e010125050127.bja,60020,1533403109034,
> destination=e010125050127.bja,60020,1533403109034; resume parent processing.
> 2018-08-05 02:20:14,466 INFO [PEWorker-8]
> procedure2.ProcedureExecutor(1296): Finished pid=1232, ppid=1208,
> state=SUCCESS, hasLock=false; AssignProcedure
> table=IntegrationTestBigLinkedList, region=c2a
> 23a735f16df57299dba6fd4599f2f, target=e010125050127.bja,60020,1533403109034
> in 1.5060sec
> {code}
> 2. Master restarts, since procedure 1208 held the lock before restart, so the
> lock was resotore for it
> {code}
> 2018-08-05 02:20:30,803 DEBUG [Thread-15] procedure2.ProcedureExecutor(456):
> Loading pid=1208, ppid=1206, state=SUCCESS, hasLock=false;
> MoveRegionProcedure hri=c2a23a735f16df57299dba6fd4599f2f, source=
> e010125050127.bja,60020,1533403109034,
> destination=e010125050127.bja,60020,1533403109034
> 2018-08-05 02:20:30,818 DEBUG [Thread-15] procedure2.Procedure(898):
> pid=1208, ppid=1206, state=SUCCESS, hasLock=false; MoveRegionProcedure
> hri=c2a23a735f16df57299dba6fd4599f2f, source=e010125050127.bj
> a,60020,1533403109034, destination=e010125050127.bja,60020,1533403109034 held
> the lock before restarting, call acquireLock to restore it.
> 2018-08-05 02:20:30,818 INFO [Thread-15]
> procedure.MasterProcedureScheduler(631): pid=1208, ppid=1206, state=SUCCESS,
> hasLock=false; MoveRegionProcedure hri=c2a23a735f16df57299dba6fd4599f2f,
> source=e0
> 10125050127.bja,60020,1533403109034,
> destination=e010125050127.bja,60020,1533403109034 checking lock on
> c2a23a735f16df57299dba6fd4599f2f
> {code}
> 3. Since procedure 1208 is success, it won't execute later, so the lock will
> be held by it forever
> We need to check the state of the procedure before restoring locks, if the
> procedure is already finished (success or rollback), we do not need to
> acquire lock for it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21062) WALFactory has misleading notion of "default"
[
https://issues.apache.org/jira/browse/HBASE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581607#comment-16581607
]
Mingliang Liu edited comment on HBASE-21062 at 8/15/18 10:42 PM:
-
The existing code implicitly employs the fact that {{defaultProvider}} is
having {{AsyncFSWALProvider}} class value. It makes sense as the enum
{{defaultProvider}} can not have a different value at runtime unless we change
code. This implementation pattern is not ideal to me either - because when we
change the {{defaultProvider}} we may forget to change {{getProviderClass}} -
but it's tolerable. Meanwhile, the newly added test in the patch does not
actually override the {{defaultProvider}} value and {{getDefaultProvider}} is
not used anywhere. So the test will pass anyway: w/ or w/o the fix in
{{getProviderClass}}, w/ or w/o the {{getDefaultProvider}}. Correct me if I'm
wrong.
Another (orthogonal?) problem in {{getProviderClass}} is that, the try-catch
scope is not implemented well. The reason is that, in the
{{catch(IllegalArgumentException exception)\{...\}}} clause, we end up with
returning the {{AsyncFSWALProvider}}. However, this time we would skip the
process that load {{AsyncFSWALProvider}} and then falls back to
{{FSHLogProvider}}.
Last, for any error case, it merits to have logging message.
Overall, a fix in my mind is like - NOT TESTED:
{code:java}
@VisibleForTesting
public Class getProviderClass(String key, String
defaultValue) {
Providers provider;
try {
provider = Providers.valueOf(conf.get(key, defaultValue));
} catch (IllegalArgumentException exception) {
// Fall back to them specifying a class name
// Note that the passed default class shouldn't actually be used, since
the above only fails
// when there is a config value present.
LOG.warn("Unrecognized WALProvider class. Falling back to default
AsyncFSWALProvider.");
provider = conf.getClass(key, Providers.defaultProvider.clazz,
WALProvider.class)
}
if (provider.clazz == AsyncFSWALProvider.class &&
!AsyncFSWALProvider.load()) {
// AsyncFSWAL has better performance in most cases, and also uses less
resources, we will try
// to use it if possible. But it deeply hacks into the internal of
DFSClient so will be easily
// broken when upgrading hadoop. If it is broken, then we fall back to
use FSHLog.
LOG.warn("Failed to load AsyncFSWALProvider. Falling back to
FSHLogProvider.");
return FSHLogProvider.class;
}
return provider.clazz;
}
{code}
Thanks!
was (Author: liuml07):
The existing code implicitly employs the fact that {{defaultProvider}} is
having {{AsyncFSWALProvider}} class value. It makes sense as the enum
{{defaultProvider}} can not have a different value at runtime unless we change
code. This implementation pattern is not ideal to me either - because when we
change the {{defaultProvider}} we may forget to change {{getProviderClass}} -
but it's tolerable. Meanwhile, the newly added test in the patch does not
actually override the {{defaultProvider}} value and {{getDefaultProvider}} is
not used anywhere. So the test will pass anyway: w/ or w/o the fix in
{{getProviderClass}}, w/ or w/o the {{getDefaultProvider}}. Correct me if I'm
wrong.
Another (orthogonal?) problem in {{getProviderClass}} is that, the try-catch
scope is not implemented well. The reason is that, in the
{{catch(IllegalArgumentException exception)\{...\}}} clause, we end up with
returning the {{AsyncFSWALProvider}}. However, this time we would skip the
process that load {{AsyncFSWALProvider}} and then falls back to
{{FSHLogProvider}}.
Last, for any error case, it merits to have logging message.
Overall, a fix in my mind is like - NOT TESTED:
{code:java}
@VisibleForTesting
public Class getProviderClass(String key, String
defaultValue) {
Providers provider;
try {
provider = Providers.valueOf(conf.get(key, defaultValue));
} catch (IllegalArgumentException exception) {
// Fall back to them specifying a class name
// Note that the passed default class shouldn't actually be used, since
the above only fails
// when there is a config value present.
LOG.warn("Unrecognized WALProvider class. Falling back to default
AsyncFSWALProvider.");
provider = Providers.defaultProviderdr;
}
if (provider.clazz == AsyncFSWALProvider.class &&
!AsyncFSWALProvider.load()) {
// AsyncFSWAL has better performance in most cases, and also uses less
resources, we will try
// to use it if possible. But it deeply hacks into the internal of
DFSClient so will be easily
// broken when upgrading hadoop. If it is broken, then we fall back to
use FSHLog.
LOG.warn("Failed to load AsyncFSWALProvider. Falling back to
FSHLogProvider.");
return FSHLogProvider.class;
}
[jira] [Updated] (HBASE-20408) Retry policy for admin APIs
[ https://issues.apache.org/jira/browse/HBASE-20408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20408: --- Summary: Retry policy for admin APIs (was: Plug-gable retry policy in all admin API's) > Retry policy for admin APIs > --- > > Key: HBASE-20408 > URL: https://issues.apache.org/jira/browse/HBASE-20408 > Project: HBase > Issue Type: New Feature >Reporter: Divesh Jain >Assignee: Divesh Jain >Priority: Major > > All admin API's that are retry able should support custom retry policy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20408) Retry policy for admin APIs
[ https://issues.apache.org/jira/browse/HBASE-20408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20408: --- Description: Admin APIs, but perhaps not all, should automatically retry to ride over master failover scenarios. This should be implemented as closely as possible to existing retry strategy for client APIs, using common code, perhaps refactored. (was: All admin API's that are retry able should support custom retry policy.) > Retry policy for admin APIs > --- > > Key: HBASE-20408 > URL: https://issues.apache.org/jira/browse/HBASE-20408 > Project: HBase > Issue Type: New Feature >Reporter: Divesh Jain >Assignee: Divesh Jain >Priority: Major > > Admin APIs, but perhaps not all, should automatically retry to ride over > master failover scenarios. This should be implemented as closely as possible > to existing retry strategy for client APIs, using common code, perhaps > refactored. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-20407) Retry HBase admin API if master failover is in progress
[ https://issues.apache.org/jira/browse/HBASE-20407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-20407. Resolution: Duplicate Duping this out in favor of HBASE-20408 > Retry HBase admin API if master failover is in progress > --- > > Key: HBASE-20407 > URL: https://issues.apache.org/jira/browse/HBASE-20407 > Project: HBase > Issue Type: Improvement > Components: Admin >Reporter: Divesh Jain >Assignee: Divesh Jain >Priority: Minor > > When a master switch over is in progress and an admin API is called, perform > a retry operation before throwing an exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20408) Retry policy for admin APIs
[ https://issues.apache.org/jira/browse/HBASE-20408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20408: --- Description: Admin APIs, but not all, should automatically retry to ride over master failover scenarios. This should be implemented as closely as possible to existing retry strategy for client APIs, using common code, perhaps refactored. Note we are not retrying if the API call returns a failure indication. We are retrying if communication between the admin client and HMaster has been interrupted, or the HMaster has failed over. We are only retrying until we know the result status of the request. Not all admin APIs are safe or can be made idempotent. was:Admin APIs, but perhaps not all, should automatically retry to ride over master failover scenarios. This should be implemented as closely as possible to existing retry strategy for client APIs, using common code, perhaps refactored. > Retry policy for admin APIs > --- > > Key: HBASE-20408 > URL: https://issues.apache.org/jira/browse/HBASE-20408 > Project: HBase > Issue Type: New Feature >Reporter: Divesh Jain >Assignee: Divesh Jain >Priority: Major > > Admin APIs, but not all, should automatically retry to ride over master > failover scenarios. This should be implemented as closely as possible to > existing retry strategy for client APIs, using common code, perhaps > refactored. > Note we are not retrying if the API call returns a failure indication. We are > retrying if communication between the admin client and HMaster has been > interrupted, or the HMaster has failed over. We are only retrying until we > know the result status of the request. > Not all admin APIs are safe or can be made idempotent. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20469) Directory used for sidelining old recovered edits files should be made configurable
[
https://issues.apache.org/jira/browse/HBASE-20469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581749#comment-16581749
]
Andrew Purtell commented on HBASE-20469:
Sorry for the long delay.
+1, this looks fine. Will commit shortly.
> Directory used for sidelining old recovered edits files should be made
> configurable
> ---
>
> Key: HBASE-20469
> URL: https://issues.apache.org/jira/browse/HBASE-20469
> Project: HBase
> Issue Type: Improvement
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-20469.master.001.patch
>
>
> Currently the directory used for sidelining of old recovered edit files is
> hardcoded to be "/tmp"
> {code:java}
> Path tmp = new Path("/tmp");
> {code}
> [See L484
> WALSplittter.java|https://github.com/apache/hbase/blob/273d252838e96c4b4af2401743d84e482c4ec565/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java#L484]
> Instead, we can use some configurable directory in the following manner:
>
> {code:java}
> String tmpDirName = conf.get(HConstants.TEMPORARY_FS_DIRECTORY_KEY,
> HConstants.DEFAULT_TEMPORARY_HDFS_DIRECTORY);
> .
> .
> Path tmp = new Path(tmpDirName);
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20874) Sending compaction descriptions from all regionservers to master.
[ https://issues.apache.org/jira/browse/HBASE-20874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohit Goel updated HBASE-20874: --- Attachment: HBASE-20874.master.005.patch > Sending compaction descriptions from all regionservers to master. > - > > Key: HBASE-20874 > URL: https://issues.apache.org/jira/browse/HBASE-20874 > Project: HBase > Issue Type: Sub-task >Reporter: Mohit Goel >Assignee: Mohit Goel >Priority: Minor > Attachments: HBASE-20874.master.004.patch, > HBASE-20874.master.005.patch > > > Need to send the compaction description from region servers to Master , to > let master know of the entire compaction state of the cluster. Further need > to change the implementation of client Side API than like getCompactionState, > which will consult master for the result instead of sending individual > request to regionservers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20469) Directory used for sidelining old recovered edits files should be made configurable
[
https://issues.apache.org/jira/browse/HBASE-20469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581751#comment-16581751
]
Hadoop QA commented on HBASE-20469:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color}
| {color:red} HBASE-20469 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.7.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-20469 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12920152/HBASE-20469.master.001.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/14056/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Directory used for sidelining old recovered edits files should be made
> configurable
> ---
>
> Key: HBASE-20469
> URL: https://issues.apache.org/jira/browse/HBASE-20469
> Project: HBase
> Issue Type: Improvement
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-20469.master.001.patch
>
>
> Currently the directory used for sidelining of old recovered edit files is
> hardcoded to be "/tmp"
> {code:java}
> Path tmp = new Path("/tmp");
> {code}
> [See L484
> WALSplittter.java|https://github.com/apache/hbase/blob/273d252838e96c4b4af2401743d84e482c4ec565/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java#L484]
> Instead, we can use some configurable directory in the following manner:
>
> {code:java}
> String tmpDirName = conf.get(HConstants.TEMPORARY_FS_DIRECTORY_KEY,
> HConstants.DEFAULT_TEMPORARY_HDFS_DIRECTORY);
> .
> .
> Path tmp = new Path(tmpDirName);
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20874) Sending compaction descriptions from all regionservers to master.
[ https://issues.apache.org/jira/browse/HBASE-20874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581753#comment-16581753 ] Mohit Goel commented on HBASE-20874: Addressed review comments from stack. Checkstyle issues. Added Overloaded version of getCompactions(). > Sending compaction descriptions from all regionservers to master. > - > > Key: HBASE-20874 > URL: https://issues.apache.org/jira/browse/HBASE-20874 > Project: HBase > Issue Type: Sub-task >Reporter: Mohit Goel >Assignee: Mohit Goel >Priority: Minor > Attachments: HBASE-20874.master.004.patch, > HBASE-20874.master.005.patch > > > Need to send the compaction description from region servers to Master , to > let master know of the entire compaction state of the cluster. Further need > to change the implementation of client Side API than like getCompactionState, > which will consult master for the result instead of sending individual > request to regionservers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21047) Object creation of StoreFileScanner thru constructor and close may leave refCount to -1
[
https://issues.apache.org/jira/browse/HBASE-21047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581778#comment-16581778
]
Mingliang Liu commented on HBASE-21047:
---
I don't see direct call of the constructor other than
{{StoreFile::getStoreFileScanner()}} method. But I like the idea of increase /
decrease safely in the same class. +1 (non-binding)
Nit: in test {{HColumnDescriptor hcd = mock(HColumnDescriptor.class);}} seems
not useful. We can delete it.
p.s. Maintaining the reference count manually is a miracle to me. Besides this
JIRA, another problem is that one's forgetting to {{close}} might have
"leakage" behavior. I would consider {{WeakHashMap}} if possible, though I
don't have too much context here.
> Object creation of StoreFileScanner thru constructor and close may leave
> refCount to -1
> ---
>
> Key: HBASE-21047
> URL: https://issues.apache.org/jira/browse/HBASE-21047
> Project: HBase
> Issue Type: Bug
>Reporter: Vishal Khandelwal
>Assignee: Vishal Khandelwal
>Priority: Major
> Fix For: 1.5.0
>
> Attachments: HBASE-21047.branch-1.v1.patch
>
>
> During object creation "*StoreFileScanner*", it does not increase the
> refCount whereas while close it decrements the reader refCount. This will
> cause refCount to -1 and isReadReference method was returning true
> (refCount.get() != 0 This is causing store file not to be deleted. This may
> also cause issue in situation when some thread is holding a scanner but it
> may actually be closed due to above bug. Impact of this would be really high.
> Attatching patch for the fix which makes sure if reference is held either
> thru getScanner method or constructor, ref is always updated. Patch also
> contains a test which validates the issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21062) WALFactory has misleading notion of "default"
[
https://issues.apache.org/jira/browse/HBASE-21062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581797#comment-16581797
]
Hadoop QA commented on HBASE-21062:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.0 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
13s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
59s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
25s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
36s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
19s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} branch-2.0 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
8s{color} | {color:red} hbase-server: The patch generated 5 new + 19 unchanged
- 0 fixed = 24 total (was 19) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
49s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 59s{color} | {color:green} Patch does not cause any errors with Hadoop 2.6.5
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}139m 41s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}174m 39s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaHighReplication
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:6f01af0 |
| JIRA Issue | HBASE-21062 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12935758/HBASE-21062.002.branch-2.0.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux b26e215d5cf5 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | branch-2.0 / 5e06b80e75 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/14052/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| unit |
https://builds.apache.org/job/PreComm
[jira] [Commented] (HBASE-21021) Result returned by Append operation should be ordered
[ https://issues.apache.org/jira/browse/HBASE-21021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581799#comment-16581799 ] Andrew Purtell commented on HBASE-21021: bq. The test passes on master after considering the change and asserting for returned result isEmpty(). Hence, the ordering issue does not exist in master. Ok, but we still need a patch for master, please, just the unit test, just to let us know the issue isn't there now and to catch it as a regression going forward. Can't have coverage in branch-1 that doesn't extend forward for an expectation that should also carry forward. Once done +1 for branch-1 for me. I think this is a bug. Increment results are ordered (in effect) so this violates the principle of least surprise. Let's make sure Javadoc for append specifies the results are returned in order. Therefore I would like to further apply this to branch-1.2 and branch-1.3 even though there is a behavioral change. I guess ordering of the return results have not been an issue to date, because this issue has not been filed previously to my knowledge, so a further reordering won't matter, but going forward semantics will better match expectations. Let me know if you have a concern [~busbey] [~toffer] > Result returned by Append operation should be ordered > - > > Key: HBASE-21021 > URL: https://issues.apache.org/jira/browse/HBASE-21021 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.0, 1.5.0 >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21021.branch-1.001.patch > > > *Problem:* > The result returned by the append operation should be ordered. Currently, it > returns an unordered list, which may cause problems like if the user tries to > perform Result.getValue(byte[] family, byte[] qualifier), even if the > returned result has a value corresponding to (family, qualifier), the method > may return null as it performs a binary search over the unsorted result > (which should have been sorted actually). > > The result is enumerated by iterating over each entry of tempMemstore hashmap > (which will never be ordered) and adding the values (see > [HRegion.java#L7882|https://github.com/apache/hbase/blob/1b50fe53724aa62a242b7f64adf7845048df/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L7882]). > > *Actual:* The returned result is unordered > *Expected:* Similar to increment op, the returned result should be ordered. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20469) Directory used for sidelining old recovered edits files should be made configurable
[
https://issues.apache.org/jira/browse/HBASE-20469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-20469:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 1.4.7
2.1.1
2.2.0
1.5.0
Status: Resolved (was: Patch Available)
> Directory used for sidelining old recovered edits files should be made
> configurable
> ---
>
> Key: HBASE-20469
> URL: https://issues.apache.org/jira/browse/HBASE-20469
> Project: HBase
> Issue Type: Improvement
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 3.0.0, 1.5.0, 2.2.0, 2.1.1, 1.4.7
>
> Attachments: HBASE-20469.master.001.patch
>
>
> Currently the directory used for sidelining of old recovered edit files is
> hardcoded to be "/tmp"
> {code:java}
> Path tmp = new Path("/tmp");
> {code}
> [See L484
> WALSplittter.java|https://github.com/apache/hbase/blob/273d252838e96c4b4af2401743d84e482c4ec565/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java#L484]
> Instead, we can use some configurable directory in the following manner:
>
> {code:java}
> String tmpDirName = conf.get(HConstants.TEMPORARY_FS_DIRECTORY_KEY,
> HConstants.DEFAULT_TEMPORARY_HDFS_DIRECTORY);
> .
> .
> Path tmp = new Path(tmpDirName);
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20993) [Auth] IPC client fallback to simple auth allowed doesn't work
[
https://issues.apache.org/jira/browse/HBASE-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581818#comment-16581818
]
Jack Bearden commented on HBASE-20993:
--
Ah okay, I see what you did there. Do you want to take this [~reidchan]? I get
the feeling you may be pressed to get this out the door, and I may be just
slowing you down. If not, I'll gladly keep working. Up to you man
> [Auth] IPC client fallback to simple auth allowed doesn't work
> --
>
> Key: HBASE-20993
> URL: https://issues.apache.org/jira/browse/HBASE-20993
> Project: HBase
> Issue Type: Bug
> Components: Client, security
>Affects Versions: 1.2.6
>Reporter: Reid Chan
>Assignee: Jack Bearden
>Priority: Critical
> Attachments: HBASE-20993.001.patch, HBASE-20993.branch-1.002.patch,
> HBASE-20993.branch-1.2.001.patch, HBASE-20993.branch-1.wip.002.patch,
> HBASE-20993.branch-1.wip.patch
>
>
> It is easily reproducible.
> client's hbase-site.xml: hadoop.security.authentication:kerberos,
> hbase.security.authentication:kerberos,
> hbase.ipc.client.fallback-to-simple-auth-allowed:true, keytab and principal
> are right set
> A simple auth hbase cluster, a kerberized hbase client application.
> application trying to r/w/c/d table will have following exception:
> {code}
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:617)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:162)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:743)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:740)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:740)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:58383)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1592)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1530)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1552)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1581)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1738)
> at
> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4297)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4289)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsyncV2(HBaseAdmin.java:753)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:674)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:607)
> at
> org.playground.hbase.KerberizedClientFallback.main(KerberizedClientFallback.java:55)
> Caused by: GSSException: No valid credentials provided (Mechanism level:
> Failed to find any Kerberos tgt)
> at
> sun.security.jgss.krb5.Krb5InitCredential.getInstance(K
[jira] [Commented] (HBASE-20881) Introduce a region transition procedure to handle all the state transition for a region
[
https://issues.apache.org/jira/browse/HBASE-20881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581822#comment-16581822
]
Hadoop QA commented on HBASE-20881:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 26 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
9s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 4m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} The patch hbase-protocol-shaded passed checkstyle
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} hbase-client: The patch generated 0 new + 2
unchanged - 84 fixed = 2 total (was 86) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} hbase-procedure: The patch generated 0 new + 19
unchanged - 1 fixed = 19 total (was 20) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
14s{color} | {color:green} hbase-server: The patch generated 0 new + 246
unchanged - 73 fixed = 246 total (was 319) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch hbase-rsgroup passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
28s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 30s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
32s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
2s{color} | {color:green} hbase-client in the patch passed. {color}
[jira] [Comment Edited] (HBASE-21061) fix synchronization of org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
[
https://issues.apache.org/jira/browse/HBASE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581823#comment-16581823
]
Andrew Purtell edited comment on HBASE-21061 at 8/16/18 1:16 AM:
-
No it wasn't broken, it was judiciously set aside. Feel free to fix these,
though. Be careful, too much synchronization and tests will rather unexpectedly
hang, maybe not a thread you want to pull, but should if you have the
bandwidth...
Edit: Might just want to annotate these away for now, come back later.
was (Author: apurtell):
No it wasn't broken, it was judiciously set aside. Feel free to fix these,
though. Be careful, too much synchronization and tests will rather unexpectedly
hang, maybe not a thread you want to pull, but should if you have the
bandwidth...
> fix synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
> ---
>
> Key: HBASE-21061
> URL: https://issues.apache.org/jira/browse/HBASE-21061
> Project: HBase
> Issue Type: Sub-task
> Components: rpc
>Affects Versions: 1.5.0, 1.2.7, 1.3.3, 1.4.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
>
> fix findbugs highlighted issue
> {code}
> Inconsistent synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap; locked 75% of time
> Unsynchronized access at RpcServer.java:75% of time Unsynchronized access at
> RpcServer.java
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
