[jira] [Commented] (HBASE-20989) Minor, miscellaneous logging fixes
[
https://issues.apache.org/jira/browse/HBASE-20989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564844#comment-16564844
]
stack commented on HBASE-20989:
---
.002 addresses nice review by [~zyork] and [~liuml07].
> Minor, miscellaneous logging fixes
> --
>
> Key: HBASE-20989
> URL: https://issues.apache.org/jira/browse/HBASE-20989
> Project: HBase
> Issue Type: Task
> Components: logging
>Reporter: stack
>Assignee: stack
>Priority: Trivial
> Fix For: 2.0.2
>
> Attachments: HBASE-20989.branch-2.0.001.patch,
> HBASE-20989.branch-2.0.002.patch
>
>
> Minor logging fixes made this morning while staring at logs.
> In particular, change the AsyncRequestFutureImpl so it puts exception on end
> of the log line rather than in the middle because then we miss the important
> stuff like how long it has been trying...
> Below is new format.
> 2018-07-31 12:46:48,566 WARN [hconnection-0x9a19380-shared-pool12-t646]
> client.AsyncRequestFutureImpl(790): id=5, table=testRowMutation,
> attempt=1/16, on localhost,49798,1533066266628, tracking started Tue Jul 31
> 12:46:48 PDT 2018; not retrying, failed=1 - final failure, failureCount=1
> ops, last
> exception=org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException:
> org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column
> family bogus does not exist in region
> testRowMutation,,1533066407822.252dbbcb173e969f0eed4954e47dacdc. in table
> 'testRowMutation', {NAME => 'testFamily', VERSIONS => '1',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE => 'false',
> DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'NONE', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'}
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:7897)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkFamilies(HRegion.java:4288)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation.checkAndPreparePut(HRegion.java:3391)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.checkAndPrepareMutation(HRegion.java:3122)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.checkAndPrepareMutation(HRegion.java:3132)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation$1.visit(HRegion.java:3417)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.visitBatchOperations(HRegion.java:3015)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation.checkAndPrepare(HRegion.java:3397)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3834)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3768)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:1027)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doAtomicBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2648)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:42014)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> It currently is like this
> ve0528.halxg.cloudera.com_52178:2018-07-31 09:11:08,486 WARN
> [htable-pool3-t35] org.apache.hadoop.hbase.client.AsyncRequestFutureImpl:
> id=2, table=IntegrationTestBigLinkedList, attempt=17/16, failed=195ops, last
> exception=org.apache.hadoo
> p.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException:
> IntegrationTestBigLinkedList,\xFE9\x0C\xD4H\xE4[\xCBar!{U\x9C\x9B`,1533052059345.a47fce1dabbcffa6abef3c51b919abd2.
> is not online on ve0532.halxg.clouder
> a.com,16020,1533053378199
> .
> Also add logging of pid to drop table procedure... otherwise it runs silently
> and on big cluster it can be gone for a long time w/o logging as it does hdfs
> ops.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20989) Minor, miscellaneous logging fixes
[
https://issues.apache.org/jira/browse/HBASE-20989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20989:
--
Attachment: HBASE-20989.branch-2.0.002.patch
> Minor, miscellaneous logging fixes
> --
>
> Key: HBASE-20989
> URL: https://issues.apache.org/jira/browse/HBASE-20989
> Project: HBase
> Issue Type: Task
> Components: logging
>Reporter: stack
>Assignee: stack
>Priority: Trivial
> Fix For: 2.0.2
>
> Attachments: HBASE-20989.branch-2.0.001.patch,
> HBASE-20989.branch-2.0.002.patch
>
>
> Minor logging fixes made this morning while staring at logs.
> In particular, change the AsyncRequestFutureImpl so it puts exception on end
> of the log line rather than in the middle because then we miss the important
> stuff like how long it has been trying...
> Below is new format.
> 2018-07-31 12:46:48,566 WARN [hconnection-0x9a19380-shared-pool12-t646]
> client.AsyncRequestFutureImpl(790): id=5, table=testRowMutation,
> attempt=1/16, on localhost,49798,1533066266628, tracking started Tue Jul 31
> 12:46:48 PDT 2018; not retrying, failed=1 - final failure, failureCount=1
> ops, last
> exception=org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException:
> org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column
> family bogus does not exist in region
> testRowMutation,,1533066407822.252dbbcb173e969f0eed4954e47dacdc. in table
> 'testRowMutation', {NAME => 'testFamily', VERSIONS => '1',
> EVICT_BLOCKS_ON_CLOSE => 'false', NEW_VERSION_BEHAVIOR => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE => 'false',
> DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'NONE', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', BLOCKCACHE =>
> 'true', BLOCKSIZE => '65536'}
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:7897)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkFamilies(HRegion.java:4288)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation.checkAndPreparePut(HRegion.java:3391)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.checkAndPrepareMutation(HRegion.java:3122)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.checkAndPrepareMutation(HRegion.java:3132)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation$1.visit(HRegion.java:3417)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.visitBatchOperations(HRegion.java:3015)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation.checkAndPrepare(HRegion.java:3397)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3834)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3768)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:1027)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doAtomicBatchOp(RSRpcServices.java:952)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2648)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:42014)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> It currently is like this
> ve0528.halxg.cloudera.com_52178:2018-07-31 09:11:08,486 WARN
> [htable-pool3-t35] org.apache.hadoop.hbase.client.AsyncRequestFutureImpl:
> id=2, table=IntegrationTestBigLinkedList, attempt=17/16, failed=195ops, last
> exception=org.apache.hadoo
> p.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException:
> IntegrationTestBigLinkedList,\xFE9\x0C\xD4H\xE4[\xCBar!{U\x9C\x9B`,1533052059345.a47fce1dabbcffa6abef3c51b919abd2.
> is not online on ve0532.halxg.clouder
> a.com,16020,1533053378199
> .
> Also add logging of pid to drop table procedure... otherwise it runs silently
> and on big cluster it can be gone for a long time w/o logging as it does hdfs
> ops.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20975) Lock may not be taken while rolling back procedure
[
https://issues.apache.org/jira/browse/HBASE-20975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564839#comment-16564839
]
Hadoop QA commented on HBASE-20975:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2.0 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
40s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
3s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
24s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
12s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
43s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} branch-2.0 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} hbase-procedure: The patch generated 0 new + 19
unchanged - 1 fixed = 19 total (was 20) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s{color} | {color:green} The patch hbase-server passed checkstyle {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
16s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 33s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
40s{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}124m 15s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
44s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}170m 38s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.procedure.TestDeleteTableProcedure |
| | hadoop.hbase.master.procedure.TestProcedureAdmin |
| | hadoop.hbase.master.procedure.TestMasterObserverPostCalls |
| | hadoop.hbase.master.procedure.TestCloneSnapshotProcedure |
| | hadoop.hbase.TestNamespace |
[jira] [Commented] (HBASE-20896) Port HBASE-20866 to branch-1 and branch-1.4
[
https://issues.apache.org/jira/browse/HBASE-20896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564838#comment-16564838
]
Hadoop QA commented on HBASE-20896:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} branch-1.4 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
31s{color} | {color:green} branch-1.4 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} branch-1.4 passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} branch-1.4 passed with JDK v1.7.0_191 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} branch-1.4 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
36s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} branch-1.4 passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} branch-1.4 passed with JDK v1.7.0_191 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
14s{color} | {color:green} the patch passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed with JDK v1.7.0_191 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
35s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
6m 10s{color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.5 2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed with JDK v1.7.0_191 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
11s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:2cf636a |
| JIRA Issue | HBASE-20896 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12933865/HBASE-20896.branch-1.4.005.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 69d01b9a7f88 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
[jira] [Commented] (HBASE-20896) Port HBASE-20866 to branch-1 and branch-1.4
[ https://issues.apache.org/jira/browse/HBASE-20896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564825#comment-16564825 ] Vikas Vishwakarma commented on HBASE-20896: --- Thanks [~reidchan] , fixed the indent issue and updated the patch. Will add patch for master and branch-2 also in HBASE-20897 and ping you again for help with review :) (hopefully will not require much changes now since this part of the code was backported from the master branch) [~apurtell] kindly take a look at the final patch. > Port HBASE-20866 to branch-1 and branch-1.4 > > > Key: HBASE-20896 > URL: https://issues.apache.org/jira/browse/HBASE-20896 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Vikas Vishwakarma >Priority: Major > Fix For: 1.5.0, 1.4.7 > > Attachments: HBASE-20896.branch-1.4.001.patch, > HBASE-20896.branch-1.4.002.patch, HBASE-20896.branch-1.4.003.patch, > HBASE-20896.branch-1.4.004.patch, HBASE-20896.branch-1.4.005.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20192) RedirectServlet not getting registered in HMaster.putUpJettyServer() in local mode
[ https://issues.apache.org/jira/browse/HBASE-20192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-20192: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > RedirectServlet not getting registered in HMaster.putUpJettyServer() in local > mode > -- > > Key: HBASE-20192 > URL: https://issues.apache.org/jira/browse/HBASE-20192 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 2.0.0-beta-2 >Reporter: Samir Ahmic >Assignee: Samir Ahmic >Priority: Major > Attachments: HBASE-20192.branch-2.0.01.patch, Screen Shot 2018-03-14 > at 09.23.34.png, Screen Shot 2018-03-14 at 09.24.06.png > > > Jetty is returning 404 when trying to open master-status page from RS status > page in local cluster mode. > After some debugging it looks like request never hits jetty RedirectServlet > so i assume RedirectServlet is not properly registered in jetty. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18842) The hbase shell clone_snaphost command returns bad error message
[
https://issues.apache.org/jira/browse/HBASE-18842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-18842:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> The hbase shell clone_snaphost command returns bad error message
>
>
> Key: HBASE-18842
> URL: https://issues.apache.org/jira/browse/HBASE-18842
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Thoralf Gutierrez
>Assignee: Thoralf Gutierrez
>Priority: Minor
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 2.0.1, 1.4.5
>
> Attachments:
> 0001-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0002-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0003-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0004-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch,
> 0005-HBASE-18842-Fix-unknown-namespace-message-in-clone_s.patch
>
>
> When you call the hbase shell clone_snapshot command with a target namespace
> that doesn't exist, you get an error message, but the variable used to
> identify the inexistent namespace is wrong:
> {noformat}
> hbase(main):001:0> clone_snapshot 'someSnapshotName',
> 'someNamespaceName:someTableName'
> ERROR: Unknown namespace someSnapshotName!
> Create a new table by cloning the snapshot content.
> There're no copies of data involved.
> And writing on the newly created table will not influence the snapshot data.
> Examples:
> hbase> clone_snapshot 'snapshotName', 'tableName'
> hbase> clone_snapshot 'snapshotName', 'namespace:tableName'
> {noformat}
> It should rather say:
> {noformat}
> ERROR: Unknown namespace someNamespaceName!
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20769) getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl
[
https://issues.apache.org/jira/browse/HBASE-20769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-20769:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> getSplits() has a out of bounds problem in TableSnapshotInputFormatImpl
> ---
>
> Key: HBASE-20769
> URL: https://issues.apache.org/jira/browse/HBASE-20769
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0, 1.4.0, 2.0.0
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.3.3, 1.4.6, 2.0.2
>
> Attachments: HBASE-20769.branch-1.001.patch,
> HBASE-20769.branch-1.3.001.patch, HBASE-20769.master.001.patch,
> HBASE-20769.master.002.patch, HBASE-20769.master.003.patch,
> HBASE-20769.master.004.patch
>
>
> When numSplits > 1, getSplits may create split that has start row smaller
> than user specified scan's start row or stop row larger than user specified
> scan's stop row.
> {code}
> byte[][] sp = sa.split(hri.getStartKey(), hri.getEndKey(), numSplits,
> true);
> for (int i = 0; i < sp.length - 1; i++) {
> if (PrivateCellUtil.overlappingKeys(scan.getStartRow(),
> scan.getStopRow(), sp[i],
> sp[i + 1])) {
> List hosts =
> calculateLocationsForInputSplit(conf, htd, hri, tableDir,
> localityEnabled);
> Scan boundedScan = new Scan(scan);
> boundedScan.setStartRow(sp[i]);
> boundedScan.setStopRow(sp[i + 1]);
> splits.add(new InputSplit(htd, hri, hosts, boundedScan,
> restoreDir));
> }
> }
> {code}
> Since we split keys by the range of regions, when sp[i] < scan.getStartRow()
> or sp[i + 1] > scan.getStopRow(), the created bounded scan may contain range
> that over user defined scan.
> fix should be simple:
> {code}
> boundedScan.setStartRow(
> Bytes.compareTo(scan.getStartRow(), sp[i]) > 0 ? scan.getStartRow() : sp[i]);
> boundedScan.setStopRow(
> Bytes.compareTo(scan.getStopRow(), sp[i + 1]) < 0 ? scan.getStopRow() : sp[i
> + 1]);
> {code}
> I will also try to add UTs to help discover this problem
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18369) hbase thrift web-ui not available
[
https://issues.apache.org/jira/browse/HBASE-18369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-18369:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> hbase thrift web-ui not available
> -
>
> Key: HBASE-18369
> URL: https://issues.apache.org/jira/browse/HBASE-18369
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Affects Versions: 2.0.0
>Reporter: Peter Somogyi
>Assignee: Tamas Penzes
>Priority: Major
> Fix For: 2.0.0-alpha-3, 2.0.0
>
> Attachments: HBASE-18369.master.001.patch,
> HBASE-18369.master.002.patch
>
>
> In standalone mode hbase thrift web-ui is gives 404 error.
> {code}
> $ ./bin/start-hbase.sh
> running master, logging to
> /Users/peter.somogyi/hbase/bin/../logs/hbase-peter.somogyi-master-psomogyi-MBP.local.out
> $ ./bin/hbase thrift start
> ...
> Jul 12, 2017 12:32:03 PM org.apache.jasper.servlet.JspServlet serviceJspFile
> SEVERE: PWC6117: File
> "/Users/peter.somogyi/hbase/hbase-server/target/hbase-webapps/thrift/thrift.jsp"
> not found
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19240) Fix error-prone errors, part four?
[ https://issues.apache.org/jira/browse/HBASE-19240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-19240: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Fix error-prone errors, part four? > -- > > Key: HBASE-19240 > URL: https://issues.apache.org/jira/browse/HBASE-19240 > Project: HBase > Issue Type: Bug >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Fix For: 1.5.0, 2.0.0-beta-1, 2.0.0 > > Attachments: HBASE-19240.branch-1.patch, > HBASE-19240.branch-1.v2.patch, HBASE-19240.patch, HBASE-19240.v2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19584) hbase-thirdparty L&N refer to items not actually in the src release
[
https://issues.apache.org/jira/browse/HBASE-19584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-19584:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> hbase-thirdparty L&N refer to items not actually in the src release
> ---
>
> Key: HBASE-19584
> URL: https://issues.apache.org/jira/browse/HBASE-19584
> Project: HBase
> Issue Type: Bug
>Reporter: Mike Drob
>Assignee: Josh Elser
>Priority: Major
> Fix For: thirdparty-2.1.0
>
> Attachments: HBASE-19584.001.patch
>
>
> From [~elserj]'s vote on 2.0-RC0:
> {quote}
> * L&N not entirely accurate, IMO. They state that things are included in the
> src release which are not. I think it would be more appropriate to push the
> relevant information down into src/main/apppended-resources for each module
> (e.g. hbase-shaded-protobuf would have
> src/main/appended-resources/\{LICENSE,NOTICE}) which have the relevant L&N
> content for the products being bundled. Thus, we'd have nothing in the
> 3rdparty L&N which reflects the src release.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18447) MetricRegistryInfo#hashCode uses hashCode instead of toHashCode
[ https://issues.apache.org/jira/browse/HBASE-18447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-18447: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > MetricRegistryInfo#hashCode uses hashCode instead of toHashCode > --- > > Key: HBASE-18447 > URL: https://issues.apache.org/jira/browse/HBASE-18447 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 1.4.0, 2.0.0 >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Minor > Fix For: 1.4.0, 2.0.0-alpha-2, 2.0.0 > > Attachments: HBASE-18447.master.001.patch > > > With commons-lang 2.6 .hashCode and .toHashCode gives back the same result > but with version 2.4 the hashCode gives back the HashCodeBuilder's hash. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19780) Change execution phase of checkstyle plugin back to default 'verify'
[
https://issues.apache.org/jira/browse/HBASE-19780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-19780:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> Change execution phase of checkstyle plugin back to default 'verify'
>
>
> Key: HBASE-19780
> URL: https://issues.apache.org/jira/browse/HBASE-19780
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
>Priority: Major
> Fix For: 2.0.0-beta-2, 2.0.0
>
> Attachments: HBASE-19780.master.001.patch,
> HBASE-19780.master.002.patch, HBASE-19780.master.003.patch
>
>
> Not able to run following command successfully:
> {{mvn -DskipTests install site
> -Dmaven.repo.local=/Users/appy/Desktop/temp_repo}}
> Use a clean separate repo so that existing packages don't pollute the build.
> Error is following.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on project
> hbase: failed to get report for
> org.apache.maven.plugins:maven-javadoc-plugin: Failed to execute goal
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check (checkstyle) on
> project hbase-error-prone: Execution checkstyle of goal
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check failed: Plugin
> org.apache.maven.plugins:maven-checkstyle-plugin:2.17 or one of its
> dependencies could not be resolved: Failure to find
> org.apache.hbase:hbase-checkstyle:jar:2.0.0-beta-1 in
> http://repository.apache.org/snapshots/ was cached in the local repository,
> resolution will not be reattempted until the update interval of
> apache.snapshots has elapsed or updates are forced -> [Help 1]
> {noformat}
> Note that master build goes pass this point.
> Need to figure out what's the difference and fix the overall build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18335) clean up configuration guide
[ https://issues.apache.org/jira/browse/HBASE-18335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-18335: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > clean up configuration guide > > > Key: HBASE-18335 > URL: https://issues.apache.org/jira/browse/HBASE-18335 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 1.3.1, 1.2.6, 1.1.11, 2.0.0-alpha-1 >Reporter: Artem Ervits >Assignee: Artem Ervits >Priority: Major > Fix For: 2.0.0 > > Attachments: HBASE-18335-0.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19032) Set Content-Type header for patches uploaded by submit-patch.py
[ https://issues.apache.org/jira/browse/HBASE-19032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-19032: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Set Content-Type header for patches uploaded by submit-patch.py > --- > > Key: HBASE-19032 > URL: https://issues.apache.org/jira/browse/HBASE-19032 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy >Priority: Major > Fix For: 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-4, 2.0.0 > > Attachments: HBASE-19032.master.001.patch > > > The patches uploaded by submit-patch.py script get automatically downloaded > when clicked. > Quite annoying. Sometimes it's easier to see the patch in the browser itself. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20822) TestAsyncNonMetaRegionLocator is flakey
[ https://issues.apache.org/jira/browse/HBASE-20822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-20822: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > TestAsyncNonMetaRegionLocator is flakey > --- > > Key: HBASE-20822 > URL: https://issues.apache.org/jira/browse/HBASE-20822 > Project: HBase > Issue Type: Bug > Components: asyncclient >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.1.0, 2.0.2, 2.2.0 > > Attachments: HBASE-20822.patch, HBASE-20822.patch, > org.apache.hadoop.hbase.client.TestAsyncNonMetaRegionLocator-output.txt > > > This is a critical one, the error is result mismatch. Need to find out why. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-17879) Avoid NPE in snapshot.jsp when accessing without any request parameter
[ https://issues.apache.org/jira/browse/HBASE-17879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-17879: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Avoid NPE in snapshot.jsp when accessing without any request parameter > -- > > Key: HBASE-17879 > URL: https://issues.apache.org/jira/browse/HBASE-17879 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 2.0.0 >Reporter: Abhishek Kumar >Priority: Trivial > Fix For: 2.0.0 > > Attachments: HBASE-17879-V1.patch, HBASE-17879-master-v1.patch, > HBASE-17879-master.patch, HBASE-17879.patch, NPE_screenshot.png, > patched_screenshot.png > > > When accessing snapshot jsp with below url inadvertently NPE comes in UI: > Requested URL: > http://:/snapshot.jsp? > Response: > java.lang.NullPointerException > at > org.apache.hadoop.hbase.generated.master.snapshot_jsp._jspService(snapshot_jsp.java:66) > at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:98) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) > at > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20742) Always create WAL directory for region server
[ https://issues.apache.org/jira/browse/HBASE-20742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-20742: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Always create WAL directory for region server > - > > Key: HBASE-20742 > URL: https://issues.apache.org/jira/browse/HBASE-20742 > Project: HBase > Issue Type: Bug > Components: wal >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-20742.patch > > > After HBASE-20708, when master restart, we will scan the wal directory to > find out the live servers. In most cases this is OK, as when we create a > HRegion instance at RS side, we will create a WAL for it, and the directory > which contains the server name will be there, even if user always use > SKIP_WAL. > But there could still be problem as the directory is created in the > implementation of WAL, not in the initialization of region server, so if user > uses DisabledWALProvider then we will be in trouble. > So let's fix it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20230) Incorrrect log message in RSRpcService
[
https://issues.apache.org/jira/browse/HBASE-20230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-20230:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> Incorrrect log message in RSRpcService
> --
>
> Key: HBASE-20230
> URL: https://issues.apache.org/jira/browse/HBASE-20230
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1
>Reporter: Vishal Khandelwal
>Assignee: Xu Cang
>Priority: Minor
> Fix For: 1.5.0, 1.3.3, 1.4.6
>
> Attachments: HBASE-20230.branch-1.3.001.patch
>
>
> At RPCServices:3040, exception is thrown for version lesser 1.3 but check is
> for version 1.4
> VersionInfoUtil.hasMinimumVersion(context.getClientVersionInfo(), 1, 4)
> >> throw new UnknownScannerException("Throwing UnknownScannerException to
> >> reset the client"
> + " scanner state for clients older than 1.3.", e);
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18265) Correct the link unuseful in regionServer's region state UI
[ https://issues.apache.org/jira/browse/HBASE-18265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-18265: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Correct the link unuseful in regionServer's region state UI > --- > > Key: HBASE-18265 > URL: https://issues.apache.org/jira/browse/HBASE-18265 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 3.0.0 >Reporter: Shibin Zhang >Assignee: Shibin Zhang >Priority: Trivial > Fix For: 2.0.0-alpha-2, 2.0.0 > > Attachments: HBASE-18265.patch > > > when i watch for region state ,i found this link maybe update in > regionserver's region state UI > IP:60030/rs-status is in RegionListTmpl.jamon > See http://hbase.org";>HBase Home for further explication. > http://hbase.org is unuseful , should update to http://hbase.apache.org -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19503) Fix TestWALOpenAfterDNRollingStart for AsyncFSWAL
[ https://issues.apache.org/jira/browse/HBASE-19503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-19503: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Fix TestWALOpenAfterDNRollingStart for AsyncFSWAL > - > > Key: HBASE-19503 > URL: https://issues.apache.org/jira/browse/HBASE-19503 > Project: HBase > Issue Type: Bug > Components: Replication, wal >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0-beta-1, 2.0.0 > > Attachments: HBASE-19503.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18803) Mapreduce job get failed caused by NoClassDefFoundError: org/apache/commons/lang3/ArrayUtils
[ https://issues.apache.org/jira/browse/HBASE-18803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-18803: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Mapreduce job get failed caused by NoClassDefFoundError: > org/apache/commons/lang3/ArrayUtils > > > Key: HBASE-18803 > URL: https://issues.apache.org/jira/browse/HBASE-18803 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-alpha-2 >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Major > Fix For: 2.0.0-alpha-4, 2.0.0 > > Attachments: HBASE-18803-v1-master.patch > > > 2017-09-12 17:44:13,020 INFO [main] mapreduce.Job: Task Id : > attempt_1505170797673_0010_m_00_0, Status : FAILED > Error: java.io.IOException: java.lang.reflect.InvocationTargetException > at > org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:221) > at > org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:114) > at > org.apache.hadoop.hbase.PerformanceEvaluation$EvaluationMapTask.map(PerformanceEvaluation.java:311) > at > org.apache.hadoop.hbase.PerformanceEvaluation$EvaluationMapTask.map(PerformanceEvaluation.java:264) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:219) > ... 11 more > Caused by: java.lang.NoClassDefFoundError: org/apache/commons/lang3/ArrayUtils > at org.apache.hadoop.hbase.HConstants.(HConstants.java:1144) > at > org.apache.hadoop.hbase.client.ConnectionUtils.getPauseTime(ConnectionUtils.java:84) > at > org.apache.hadoop.hbase.client.AsyncProcess.(AsyncProcess.java:209) > at > org.apache.hadoop.hbase.client.ConnectionImplementation.(ConnectionImplementation.java:266) > ... 16 more -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20778) Make it so WALPE runs on DFS
[ https://issues.apache.org/jira/browse/HBASE-20778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-20778: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Make it so WALPE runs on DFS > > > Key: HBASE-20778 > URL: https://issues.apache.org/jira/browse/HBASE-20778 > Project: HBase > Issue Type: Bug > Components: test >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.1.0, 2.0.2 > > Attachments: HBASE-20778.branch-2.0.001.patch > > > WALPE is broke for running on DFS. The old issue is the cause HBASE-9908 > (making stuff work on windows) though it went in a long time ago. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20384) [AMv2] Logging format improvements; use encoded name rather than full region name marking transitions
[ https://issues.apache.org/jira/browse/HBASE-20384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-20384: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > [AMv2] Logging format improvements; use encoded name rather than full region > name marking transitions > -- > > Key: HBASE-20384 > URL: https://issues.apache.org/jira/browse/HBASE-20384 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-20384.branch-2.0.001.patch > > > We use encoded name near everywhere. Makes logging regular-looking at least > and eases tracing. In a few places we still do full region name. Let me fix > (ran into it trying to debug...) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18900) Backport HBASE-17290 "Potential loss of data for replication of bulk loaded hfiles" to 1.3
[ https://issues.apache.org/jira/browse/HBASE-18900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-18900: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Backport HBASE-17290 "Potential loss of data for replication of bulk loaded > hfiles" to 1.3 > -- > > Key: HBASE-18900 > URL: https://issues.apache.org/jira/browse/HBASE-18900 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Attachments: 17290-branch-1.3.patch, 18900.branch-1.3.patch > > > HBASE-17290 fixes data loss bug. > Bulk loaded hfile replication support is in branch-1.3 > This issue is to backport HBASE-17290 to branch-1.3 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18356) Enable TestFavoredStochasticBalancerPickers#testPickers that was disabled by Proc-V2 AM in HBASE-14614
[ https://issues.apache.org/jira/browse/HBASE-18356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-18356: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Enable TestFavoredStochasticBalancerPickers#testPickers that was disabled by > Proc-V2 AM in HBASE-14614 > -- > > Key: HBASE-18356 > URL: https://issues.apache.org/jira/browse/HBASE-18356 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-alpha-1 >Reporter: Stephen Yuan Jiang >Assignee: Thiruvel Thirumoolan >Priority: Major > Fix For: 2.0.0-beta-1, 2.0.0 > > Attachments: HBASE-18356.master.001.patch, > HBASE-18356.master.002.patch > > > The testPickers in TestFavoredStochasticBalancerPickers hangs after applying > the change in Core Proc-V2 AM in HBASE-14614. It was disabled. > This JIRA tracks the work to enable it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18352) Enable TestMasterOperationsForRegionReplicas#testCreateTableWithMultipleReplicas disabled by Proc-V2 AM in HBASE-14614
[ https://issues.apache.org/jira/browse/HBASE-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-18352: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Enable > TestMasterOperationsForRegionReplicas#testCreateTableWithMultipleReplicas > disabled by Proc-V2 AM in HBASE-14614 > -- > > Key: HBASE-18352 > URL: https://issues.apache.org/jira/browse/HBASE-18352 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-alpha-1 >Reporter: Stephen Yuan Jiang >Assignee: huaxiang sun >Priority: Major > Fix For: 2.0.0-beta-1, 2.0.0 > > Attachments: HBASE-18352.master.001.patch, > HBASE-18352.master.002.patch, HBASE-18946_1.patch > > > The following replica tests were disabled by Core Proc-V2 AM in HBASE-14614: > - Disabled parts of...testCreateTableWithMultipleReplicas in > TestMasterOperationsForRegionReplicas There is an issue w/ assigning more > replicas if number of replicas is changed on us. See '/* DISABLED! FOR > NOW'. > ** NOTE We moved fixing of the below two tests out to HBASE-19268 > - Disabled testRegionReplicasOnMidClusterHighReplication in > TestStochasticLoadBalancer2 > - Disabled testFlushAndCompactionsInPrimary in TestRegionReplicas > This JIRA tracks the work to enable them (or modify/remove if not applicable). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18983) Upgrade to latest error-prone
[ https://issues.apache.org/jira/browse/HBASE-18983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-18983: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Upgrade to latest error-prone > - > > Key: HBASE-18983 > URL: https://issues.apache.org/jira/browse/HBASE-18983 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Fix For: 2.0.0-beta-1, 2.0.0 > > Attachments: HBASE-18983.patch > > > There's some trouble with lambda expressions in our current version as > reported in https://github.com/google/error-prone/issues/768 > Also, error-prone 2.1.x has a bunch of nice upgrades that would be good to > get. https://groups.google.com/forum/#!topic/error-prone-announce/iXLlHsi_n5U -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18613) Race condition between master restart and test code when restoring distributed cluster after integration test
[
https://issues.apache.org/jira/browse/HBASE-18613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-18613:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> Race condition between master restart and test code when restoring
> distributed cluster after integration test
> -
>
> Key: HBASE-18613
> URL: https://issues.apache.org/jira/browse/HBASE-18613
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Minor
>
> Noticed the following in some internal testing (line numbers likely are
> skewed)
> {noformat}
> 2017-08-16 21:20:25,557| 2017-08-16 21:20:25,553 WARN [main]
> client.ConnectionManager$HConnectionImplementation: Checking master connection
> 2017-08-16 21:20:25,557| com.google.protobuf.ServiceException:
> org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Call to
> master1.domain.com/10.0.2.131:16000 failed on local exception:
> org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Connection to
> master1.domain.com/10.0.2.131:16000 is closing. Call id=581, waitTime=1
> 2017-08-16 21:20:25,557| at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:223)
> 2017-08-16 21:20:25,558| at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
> 2017-08-16 21:20:25,560| at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:62739)
> 2017-08-16 21:20:25,560| at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceState.isMasterRunning(ConnectionManager.java:1448)
> 2017-08-16 21:20:25,561| at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.isKeepAliveMasterConnectedAndRunning(ConnectionManag
> er.java:2124)
> 2017-08-16 21:20:25,561| at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1712)
> 2017-08-16 21:20:25,562| at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getMaster(ConnectionManager.java:1701)
> 2017-08-16 21:20:25,562| at
> org.apache.hadoop.hbase.DistributedHBaseCluster.getMasterAdminService(DistributedHBaseCluster.java:153)
> 2017-08-16 21:20:25,563| at
> org.apache.hadoop.hbase.DistributedHBaseCluster.waitForActiveAndReadyMaster(DistributedHBaseCluster.java:184)
> 2017-08-16 21:20:25,563| at
> org.apache.hadoop.hbase.HBaseCluster.waitForActiveAndReadyMaster(HBaseCluster.java:204)
> 2017-08-16 21:20:25,563| at
> org.apache.hadoop.hbase.DistributedHBaseCluster.restoreMasters(DistributedHBaseCluster.java:278)
> 2017-08-16 21:20:25,563| at
> org.apache.hadoop.hbase.DistributedHBaseCluster.restoreClusterStatus(DistributedHBaseCluster.java:239)
> 2017-08-16 21:20:25,563| at
> org.apache.hadoop.hbase.HBaseCluster.restoreInitialStatus(HBaseCluster.java:235)
> 2017-08-16 21:20:25,564| at
> org.apache.hadoop.hbase.IntegrationTestingUtility.restoreCluster(IntegrationTestingUtility.java:99)
> 2017-08-16 21:20:25,564| at
> org.apache.hadoop.hbase.IntegrationTestBase.cleanUpCluster(IntegrationTestBase.java:200)
> 2017-08-16 21:20:25,564| at
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover.cleanUpCluster(IntegrationTestDDLMasterFailover.java:146)
> 2017-08-16 21:20:25,564| at
> org.apache.hadoop.hbase.IntegrationTestBase.cleanUp(IntegrationTestBase.java:140)
> 2017-08-16 21:20:25,564| at
> org.apache.hadoop.hbase.IntegrationTestBase.doWork(IntegrationTestBase.java:125)
> 2017-08-16 21:20:25,565| at
> org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:112)
> 2017-08-16 21:20:25,565| at
> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> 2017-08-16 21:20:25,565| at
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover.main(IntegrationTestDDLMasterFailover.java:832)
> 2017-08-16 21:20:25,566| Caused by:
> org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Call to
> master1.domain.com/10.0.2.131:16000 failed on local exception:
> org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Connection to
> master1.domain.com/10.0.2.131:16000 is closing. Call id=581, waitTime=1
> 2017-08-16 21:20:25,566| at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1258)
> 2017-08-16 21:20:25,566| at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1229)
> 2017-08-16 21:20:25,566| at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
> 2017-08-16 21:20:25,566| ... 20 more
> 2017-08-16 21:
[jira] [Issue Comment Deleted] (HBASE-18887) After full backup passed on hdfs root and incremental failed, full backup cannot be cleaned
[
https://issues.apache.org/jira/browse/HBASE-18887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-18887:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> After full backup passed on hdfs root and incremental failed, full backup
> cannot be cleaned
> ---
>
> Key: HBASE-18887
> URL: https://issues.apache.org/jira/browse/HBASE-18887
> Project: HBase
> Issue Type: Bug
>Reporter: Vishal Khandelwal
>Assignee: Vladimir Rodionov
>Priority: Major
> Labels: backup
> Fix For: 2.0.0-alpha-4, 2.0.0
>
> Attachments: HBASE-18887-v1.patch
>
>
> >>
> ./bin/hbase backup create full hdfs://localhost:8020/ -t test1
> 2017-09-27 10:19:38,885 INFO [main] impl.BackupManifest: Manifest file
> stored to hdfs://localhost:8020/backup_1506487766386/.backup.manifest
> 2017-09-27 10:19:38,937 INFO [main] impl.TableBackupClient: Backup
> backup_1506487766386 completed.
> Backup session backup_1506487766386 finished. Status: SUCCESS
> >>
> 2017-09-27 10:20:48,211 INFO [main] mapreduce.JobSubmitter: Cleaning up the
> staging area
> /tmp/hadoop-yarn/staging/vkhandelwal/.staging/job_1506419443344_0045
> 2017-09-27 10:20:48,215 ERROR [main] impl.TableBackupClient: Unexpected
> exception in incremental-backup: incremental copy backup_1506487845361Can not
> convert from directory (check Hadoop, HBase and WALPlayer M/R job logs)
> java.io.IOException: Can not convert from directory (check Hadoop, HBase and
> WALPlayer M/R job logs)
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.walToHFiles(IncrementalTableBackupClient.java:363)
> at
> {code} ./bin/hbase backup create full hdfs://localhost:8020/ -t test1 {code}
> 2017-09-27 10:19:38,885 INFO [main] impl.BackupManifest: Manifest file
> stored to hdfs://localhost:8020/backup_1506487766386/.backup.manifest
> 2017-09-27 10:19:38,937 INFO [main] impl.TableBackupClient: Backup
> backup_1506487766386 completed.
> Backup session backup_1506487766386 finished. Status: SUCCESS
> {code} ./bin/hbase backup create incremental hdfs://localhost:8020/ -t test1
> {code}
> 2017-09-27 10:20:48,215 ERROR [main] impl.TableBackupClient: Unexpected
> exception in incremental-backup: incremental copy backup_1506487845361Can not
> convert from directory (check Hadoop, HBase and WALPlayer M/R job logs)
> java.io.IOException: Can not convert from directory (check Hadoop, HBase and
> WALPlayer M/R job logs)
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.walToHFiles(IncrementalTableBackupClient.java:363)
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.convertWALsToHFiles(IncrementalTableBackupClient.java:322)
> at
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.execute(IncrementalTableBackupClient.java:232)
> at
> org.apache.hadoop.hbase.backup.impl.BackupAdminImpl.backupTables(BackupAdminImpl.java:601)
> at
> org.apache.hadoop.hbase.backup.impl.BackupCommands$CreateCommand.execute(BackupCommands.java:336)
> at
> org.apache.hadoop.hbase.backup.BackupDriver.parseAndRun(BackupDriver.java:137)
> at
> org.apache.hadoop.hbase.backup.BackupDriver.doWork(BackupDriver.java:170)
> at
> org.apache.hadoop.hbase.backup.BackupDriver.run(BackupDriver.java:203)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.hbase.backup.BackupDriver.main(BackupDriver.java:178)
> Caused by: java.lang.IllegalArgumentException: Can not create a Path from an
> empty string
> at org.apache.hadoop.fs.Path.checkPathArg(Path.java:126)
> at org.apache.hadoop.fs.Path.(Path.java:134)
> at org.apache.hadoop.util.StringUtils.stringToPath(StringUtils.java:245)
> at
> org.apache.hadoop.hbase.mapreduce.WALInputFormat.getInputPaths(WALInputFormat.java:301)
> at
> org.apache.hadoop.hbase.mapreduce.WALInputFormat.getSplits(WALInputFormat.java:274)
> at
> org.apache.hadoop.hbase.mapreduce.WALInputFormat.getSplits(WALInputFormat.java:264)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:301)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:318)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:196)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
>
[jira] [Issue Comment Deleted] (HBASE-20139) NPE in RSRpcServices.get() when getRegion throws an exception
[
https://issues.apache.org/jira/browse/HBASE-20139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-20139:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> NPE in RSRpcServices.get() when getRegion throws an exception
> -
>
> Key: HBASE-20139
> URL: https://issues.apache.org/jira/browse/HBASE-20139
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1
>Reporter: Abhishek Singh Chouhan
>Assignee: Abhishek Singh Chouhan
>Priority: Minor
> Fix For: 1.3.2, 1.5.0, 1.4.3
>
> Attachments: HBASE-20139.branch-1.001.patch,
> HBASE-20139.branch-1.3.001.patch, HBASE-20139.branch-1.3.001.patch
>
>
> We can get a NPE in RsRpcServices at
> {code:java}
> } finally {
> if (regionServer.metricsRegionServer != null) {
> regionServer.metricsRegionServer.updateGet(
> -> region.getTableDesc().getTableName(), EnvironmentEdgeManager.currentTime()
> - before);
> }
> if (quota != null) {
> quota.close();
> }{code}
> when region itself is null which might happen when getRegion throws an
> exception, this is then sent back to the client which is not able to handle
> this/make sense of it.
> {code:java}
> 2018-03-06 08:31:25,100 DEBUG [0,queue=4,port=60020] ipc.RpcServer -
> RpcServer.FifoWFPBQ.default.handler=30,queue=4,port=60020: callId: 5605567
> service: ClientService methodName: Get size: 79 connection: xyz:58736
> deadline: 9223372036854775807
> java.io.IOException
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2431)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2246)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:35068)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2373)
> ... 3 more{code}
> This has been fixed by [~stack] over at HBASE-18946 for master, backporting
> the same to branch-1, 1.3 and 1.4
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19756) Master NPE during completed failed proc eviction
[
https://issues.apache.org/jira/browse/HBASE-19756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-19756:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> Master NPE during completed failed proc eviction
>
>
> Key: HBASE-19756
> URL: https://issues.apache.org/jira/browse/HBASE-19756
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.0, 1.3.1
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
>Priority: Major
> Fix For: 1.3.2, 1.4.1, 1.5.0, 2.0.0
>
> Attachments: HBASE-19756.branch-1.4.001.patch,
> HBASE-19756.branch-1.4.002.patch, HBASE-19756.branch-1.4.003.patch,
> HBASE-19756.master.001.patch
>
>
> When procedures like Create table fails due to say AccessDeniedException,
> then a rollback procedure is created. When the rollback is being cleaned up,
> it results in an NPE because those nonce procs aren't persisted
> Stack trace when this happens:
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.procedure2.store.ProcedureStoreTracker.delete(ProcedureStoreTracker.java:385)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.updateStoreTracker(WALProcedureStore.java:547)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.pushData(WALProcedureStore.java:504)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.delete(WALProcedureStore.java:453)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$CompletedProcedureCleaner.periodicExecute(ProcedureExecutor.java:184)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.timeoutLoop(ProcedureExecutor.java:995)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$500(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$3.run(ProcedureExecutor.java:507)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20102) AssignmentManager#shutdown doesn't shut down scheduled executor
[ https://issues.apache.org/jira/browse/HBASE-20102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-20102: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > AssignmentManager#shutdown doesn't shut down scheduled executor > --- > > Key: HBASE-20102 > URL: https://issues.apache.org/jira/browse/HBASE-20102 > Project: HBase > Issue Type: Bug > Components: master, Region Assignment >Affects Versions: 1.4.2 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 1.5.0, 1.4.3 > > Attachments: HBASE-20102-branch-1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20645) Fix security_available method in security.rb
[
https://issues.apache.org/jira/browse/HBASE-20645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-20645:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> Fix security_available method in security.rb
> -
>
> Key: HBASE-20645
> URL: https://issues.apache.org/jira/browse/HBASE-20645
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 2.0.1
>
> Attachments: HBASE-20645.patch
>
>
> "exists?" method expects parameter tableName to be String but ACL_TABLE_NAME
> is of org.apache.hadoop.hbase.TableName form.
> {code}
> raise(ArgumentError, 'DISABLED: Security features are not available') unless \
>
> exists?(org.apache.hadoop.hbase.security.access.AccessControlLists::ACL_TABLE_NAME.getNameAsString)
> {code}
> Impact of the bug:-
> So , if a user is running any security related
> command(revoke,user_permission) and there is an exception(MasterNotRunning)
> while checking security capabilities, then instead of seeing the underlying
> exception, user is seeing
> {code}
> ERROR: no method 'valueOf' for arguments (org.apache.hadoop.hbase.TableName)
> on Java::OrgApacheHadoopHbase::TableName
> available overloads:
> (java.lang.String)
> (byte[])
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20330) ProcedureExecutor.start() gets stuck in recover lease on store.
[
https://issues.apache.org/jira/browse/HBASE-20330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-20330:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> ProcedureExecutor.start() gets stuck in recover lease on store.
> ---
>
> Key: HBASE-20330
> URL: https://issues.apache.org/jira/browse/HBASE-20330
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Affects Versions: 2.0.0-beta-2
>Reporter: Umesh Agashe
>Assignee: Umesh Agashe
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: hbase-20330.master.001.patch,
> hbase-20330.master.002.patch, hbase-20330.master.003.patch,
> hbase-20330.master.004.patch, hbase-20330.master.005.patch
>
>
> We have instance in our internal testing where master log is getting filled
> with following messages:
> {code}
> 2018-04-02 17:11:17,566 INFO org.apache.hadoop.hbase.util.FSHDFSUtils:
> Recover lease on dfs file
> hdfs://ns1/hbase/MasterProcWALs/pv2-0018.log
> 2018-04-02 17:11:17,567 INFO org.apache.hadoop.hbase.util.FSHDFSUtils:
> Recovered lease, attempt=0 on
> file=hdfs://ns1/hbase/MasterProcWALs/pv2-0018.log after 1ms
> 2018-04-02 17:11:17,574 WARN
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore: Unable to
> read tracker for hdfs://ns1/hbase/MasterProcWALs/pv2-0018.log
> - Invalid Trailer version. got 111 expected 1
> 2018-04-02 17:11:17,576 ERROR
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore: Log file with
> id=19 already exists
> org.apache.hadoop.fs.FileAlreadyExistsException:
> /hbase/MasterProcWALs/pv2-0019.log for client 10.17.202.11
> already exists
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirWriteFileOp.startFile(FSDirWriteFileOp.java:381)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2442)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2339)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:764)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:451)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675)
> {code}
> Debugging it further with [~appy], [~avirmani] and [~xiaochen] we found that
> when WALProcedureStore#rollWriter() fails and returns false for some reason,
> it keeps looping continuously.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20028) NPE when comparing versions in AM after RS ZK expiration
[
https://issues.apache.org/jira/browse/HBASE-20028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-20028:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> NPE when comparing versions in AM after RS ZK expiration
>
>
> Key: HBASE-20028
> URL: https://issues.apache.org/jira/browse/HBASE-20028
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
>
> {noformat}
> 2018-02-20 16:36:41,794 ERROR [Thread-85] assignment.AssignmentManager:
> java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.util.VersionInfo.compareVersion(VersionInfo.java:122)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.lambda$getExcludedServersForSystemTable$5(AssignmentManager.java:1860)
> at java.util.Collections.max(Collections.java:712)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.getExcludedServersForSystemTable(AssignmentManager.java:1859)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.lambda$checkIfShouldMoveSystemRegionAsync$0(AssignmentManager.java:464){noformat}
> Looks like a race condition around an RS losing its ZK lock. If AM tries to
> see if it should move a Region to a server who we've seen that the lock was
> lost but the RS hasn't yet been processed as "dead", we can get into a
> situation where {{HMaster.getRegionServerVersion()}} returns null and causes
> this to fail.
> Looks like a simple filter on the servers to preclude null versions would fix
> the problem.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18947) HBase backups backup all tables once backed up irrespective of the table names passed to it.
[
https://issues.apache.org/jira/browse/HBASE-18947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-18947:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> HBase backups backup all tables once backed up irrespective of the table
> names passed to it.
> -
>
> Key: HBASE-18947
> URL: https://issues.apache.org/jira/browse/HBASE-18947
> Project: HBase
> Issue Type: Bug
>Reporter: Amit Kabra
>Assignee: Amit Kabra
>Priority: Major
> Attachments: HBASE-18947.patch
>
>
> Take backup of test1,test2,test3,test11,test12,test13
> and then take backup of only test2
> {code}./hbase backup -d create incremental hdfs://localhost:8020/test/ -t
> test2{code}
> It should only backup test2 but it backup all tables once backed up. This can
> be seen in hdfs as backed up tables and logs show the same :
> Logs show :
> 2017-09-25 19:29:39,170 DEBUG [main] impl.IncrementalTableBackupClient: For
> incremental backup, current table set is [test1,test2,test3,test11,
> test12,test13]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19523) TestLogRolling is flakey
[ https://issues.apache.org/jira/browse/HBASE-19523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-19523: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > TestLogRolling is flakey > > > Key: HBASE-19523 > URL: https://issues.apache.org/jira/browse/HBASE-19523 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Duo Zhang >Priority: Major > > https://builds.apache.org/job/PreCommit-HBASE-Build/10475/testReport/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-17936) Refine sum endpoint example in ref guide
[ https://issues.apache.org/jira/browse/HBASE-17936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-17936: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Refine sum endpoint example in ref guide > > > Key: HBASE-17936 > URL: https://issues.apache.org/jira/browse/HBASE-17936 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Xiang Li >Assignee: Xiang Li >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-17936.master.000.patch > > > Refine chapter 90.2, the example java code of sum endpoint of > http://hbase.apache.org/book.html#cp_example > - error > - typo > - genericity > - indent -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19986) If HBaseTestClassRule timesout a test, thread dump.
[ https://issues.apache.org/jira/browse/HBASE-19986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-19986: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > If HBaseTestClassRule timesout a test, thread dump. > --- > > Key: HBASE-19986 > URL: https://issues.apache.org/jira/browse/HBASE-19986 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0-beta-2, 2.0.0 > > Attachments: HBASE-19986.branch-2.001.patch, > HBASE-19986.branch-2.002.patch, HBASE-19986.branch-2.003.patch > > > We set look for stuck thread in our timeout rule but it is super conservative > in what it prints.. it looks for a RUNNABLE thread and prints first found > ONLY. Pretty useless for us. If a test timesout, often the printing has > stopped in the stderr/stdout. > I'm trying to debug TestAsyncRegionAdminApi. It says test timed out after 10 > minutes but we've stopped printing to the logs and here is what junit prints: > --- > Test set: org.apache.hadoop.hbase.client.TestAsyncRegionAdminApi > --- > Tests run: 25, Failures: 0, Errors: 2, Skipped: 2, Time elapsed: 572.508 s > <<< FAILURE! - in org.apache.hadoop.hbase.client.TestAsyncRegionAdminApi > org.apache.hadoop.hbase.client.TestAsyncRegionAdminApi Time elapsed: 14.642 > s <<< ERROR! > org.junit.runners.model.TestTimedOutException: test timed out after 600 > seconds > at > org.apache.hadoop.hbase.client.TestAsyncRegionAdminApi.testMergeRegions(TestAsyncRegionAdminApi.java:363) > org.apache.hadoop.hbase.client.TestAsyncRegionAdminApi Time elapsed: 14.642 > s <<< ERROR! > java.lang.Exception: Appears to be stuck in thread Socket Reader #1 for port > 35917 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18717) [pom cleanup] endpoints in hbase-server require explicit inclusion of hbase-protocol
[ https://issues.apache.org/jira/browse/HBASE-18717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-18717: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > [pom cleanup] endpoints in hbase-server require explicit inclusion of > hbase-protocol > > > Key: HBASE-18717 > URL: https://issues.apache.org/jira/browse/HBASE-18717 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > > hbase-server needs an explicit reference to hbase-protocol because it carries > a few endpoints. See below. Better would be to move the endpoints out to > hbase-endpoint. Some can't be moved because they are not pure CPEPs (look for > related issues) and actually should be native to hbase rather than > implemented as CPEPs. Meantime, we have to add reference to hbase-protocol to > hbase-server just because of these CPEPs. > Here are the errors seen by [~uagashe] over in HBASE-18674 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18592) [hbase-thirdparty] Doc on new hbase-thirdparty dependency for the refguide
[ https://issues.apache.org/jira/browse/HBASE-18592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-18592: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > [hbase-thirdparty] Doc on new hbase-thirdparty dependency for the refguide > -- > > Key: HBASE-18592 > URL: https://issues.apache.org/jira/browse/HBASE-18592 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0 > > > Add a bit to the refguide on the new hbase-thirdparty lib and why it exists. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19726) Failed to start HMaster due to infinite retrying on meta assign
[
https://issues.apache.org/jira/browse/HBASE-19726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-19726:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> Failed to start HMaster due to infinite retrying on meta assign
> ---
>
> Key: HBASE-19726
> URL: https://issues.apache.org/jira/browse/HBASE-19726
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2, 2.0.0
>
> Attachments: 19726.patch
>
>
> This is what I got at first, an exception when trying to write something to
> meta when meta has not been onlined yet.
> {noformat}
> 2018-01-07,21:03:14,389 INFO org.apache.hadoop.hbase.master.HMaster: Running
> RecoverMetaProcedure to ensure proper hbase:meta deploy.
> 2018-01-07,21:03:14,637 INFO
> org.apache.hadoop.hbase.master.procedure.RecoverMetaProcedure: Start pid=1,
> state=RUNNABLE:RECOVER_META_SPLIT_LOGS; RecoverMetaProcedure
> failedMetaServer=null, splitWal=true
> 2018-01-07,21:03:14,645 INFO org.apache.hadoop.hbase.master.MasterWalManager:
> Log folder
> hdfs://c402tst-community/hbase/c402tst-community/WALs/c4-hadoop-tst-st27.bj,38900,1515330173896
> belongs to an existing region server
> 2018-01-07,21:03:14,646 INFO org.apache.hadoop.hbase.master.MasterWalManager:
> Log folder
> hdfs://c402tst-community/hbase/c402tst-community/WALs/c4-hadoop-tst-st29.bj,38900,1515330177232
> belongs to an existing region server
> 2018-01-07,21:03:14,648 INFO
> org.apache.hadoop.hbase.master.procedure.RecoverMetaProcedure: pid=1,
> state=RUNNABLE:RECOVER_META_ASSIGN_REGIONS; RecoverMetaProcedure
> failedMetaServer=null, splitWal=true; Retaining meta assignment to server=null
> 2018-01-07,21:03:14,653 INFO
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: Initialized
> subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_TRANSITION_QUEUE;
> AssignProcedure table=hbase:meta, region=1588230740}]
> 2018-01-07,21:03:14,660 INFO
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler: pid=2,
> ppid=1, state=RUNNABLE:REGION_TRANSITION_QUEUE; AssignProcedure
> table=hbase:meta, region=1588230740 hbase:meta hbase:meta,,1.1588230740
> 2018-01-07,21:03:14,663 INFO
> org.apache.hadoop.hbase.master.assignment.AssignProcedure: Start pid=2,
> ppid=1, state=RUNNABLE:REGION_TRANSITION_QUEUE; AssignProcedure
> table=hbase:meta, region=1588230740; rit=OFFLINE, location=null;
> forceNewPlan=false, retain=false
> 2018-01-07,21:03:14,831 INFO
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator: Setting hbase:meta
> (replicaId=0) location in ZooKeeper as
> c4-hadoop-tst-st27.bj,38900,1515330173896
> 2018-01-07,21:03:14,841 INFO
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure: Dispatch
> pid=2, ppid=1, state=RUNNABLE:REGION_TRANSITION_DISPATCH; AssignProcedure
> table=hbase:meta, region=1588230740; rit=OPENING,
> location=c4-hadoop-tst-st27.bj,38900,1515330173896
> 2018-01-07,21:03:14,992 INFO
> org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher: Using
> procedure batch rpc execution for
> serverName=c4-hadoop-tst-st27.bj,38900,1515330173896 version=3145728
> 2018-01-07,21:03:15,593 ERROR
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl: Cannot get replica 0
> location for
> {"totalColumns":1,"row":"hbase:meta","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":1515330195514}]},"ts":1515330195514}
> 2018-01-07,21:03:15,594 WARN
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure:
> Retryable error trying to transition: pid=2, ppid=1,
> state=RUNNABLE:REGION_TRANSITION_FINISH; AssignProcedure table=hbase:meta,
> region=1588230740; rit=OPEN,
> location=c4-hadoop-tst-st27.bj,38900,1515330173896
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1
> action: IOException: 1 time, servers with issues: null
> at
> org.apache.hadoop.hbase.client.BatchErrors.makeException(BatchErrors.java:54)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.getErrors(AsyncRequestFutureImpl.java:1250)
> at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:457)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:570)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.put(MetaTableAccessor.java:1450)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.putToMetaTable(MetaTableAccessor.java:1439)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1785)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1151)
> at
> org.apache.hadoop.hbase.master.TableStateManager.udpateMetaState(TableStateMa
[jira] [Issue Comment Deleted] (HBASE-20363) TestNamespaceAuditor.testRegionMerge is flaky
[
https://issues.apache.org/jira/browse/HBASE-20363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-20363:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> TestNamespaceAuditor.testRegionMerge is flaky
> -
>
> Key: HBASE-20363
> URL: https://issues.apache.org/jira/browse/HBASE-20363
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-20363-addendum.patch, HBASE-20363.patch
>
>
> I think it is easy to find out the problem. We haven't done a compaction
> after merging and then try to split the region. The split will fail because
> of there are still reference files.
> https://builds.apache.org/job/HBASE-Flaky-Tests/28972/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.namespace.TestNamespaceAuditor-output.txt
> {noformat}
> 2018-04-08 05:29:49,742 INFO
> [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=43056]
> master.HMaster$2(1644): Client=jenkins//67.195.81.155 split
> TestNamespaceAuditor_regiontest:table2,,1523165387443.c6dd29ca77051607ab50a1edfa5f076f.
> 2018-04-08 05:29:49,745 DEBUG
> [RpcServer.priority.FPBQ.Fifo.handler=1,queue=0,port=38141]
> regionserver.HRegion(1360): Region
> TestNamespaceAuditor_regiontest:table2,,1523165387443.c6dd29ca77051607ab50a1edfa5f076f.
> is not mergeable because it has references
> 2018-04-08 05:29:49,746 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=43056]
> assignment.SplitTableRegionProcedure(174): Splittable=false rit=OPEN,
> location=asf911.gq1.ygridcore.net,38141,1523165245520
> 2018-04-08 05:29:49,747 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=43056]
> ipc.CallRunner(142): callId: 1092 service: MasterService methodName:
> SplitRegion size: 113 connection: 67.195.81.155:58584 deadline:
> 1523165449742, exception=org.apache.hadoop.hbase.DoNotRetryIOException:
> c6dd29ca77051607ab50a1edfa5f076f NOT splittable
> 2018-04-08 05:29:49,752 INFO [Time-limited test] client.HBaseAdmin$15(907):
> Started disable of TestNames
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18455) TestClientScannerRPCTimeout is flaky
[ https://issues.apache.org/jira/browse/HBASE-18455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-18455: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > TestClientScannerRPCTimeout is flaky > > > Key: HBASE-18455 > URL: https://issues.apache.org/jira/browse/HBASE-18455 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0, 1.5.0 >Reporter: Andrew Purtell >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20896) Port HBASE-20866 to branch-1 and branch-1.4
[ https://issues.apache.org/jira/browse/HBASE-20896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikas Vishwakarma updated HBASE-20896: -- Attachment: HBASE-20896.branch-1.4.005.patch > Port HBASE-20866 to branch-1 and branch-1.4 > > > Key: HBASE-20896 > URL: https://issues.apache.org/jira/browse/HBASE-20896 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Vikas Vishwakarma >Priority: Major > Fix For: 1.5.0, 1.4.7 > > Attachments: HBASE-20896.branch-1.4.001.patch, > HBASE-20896.branch-1.4.002.patch, HBASE-20896.branch-1.4.003.patch, > HBASE-20896.branch-1.4.004.patch, HBASE-20896.branch-1.4.005.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20707) Move MissingSwitchDefault check from checkstyle to error-prone
[ https://issues.apache.org/jira/browse/HBASE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-20707: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Move MissingSwitchDefault check from checkstyle to error-prone > -- > > Key: HBASE-20707 > URL: https://issues.apache.org/jira/browse/HBASE-20707 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Fix For: 3.0.0, 2.1.0 > > Attachments: HBASE-20707.patch > > > Both checkstyle and error-prone can alert when a switch statement is missing > a default. However, because checkstyle does it via static analysis and > error-prone does it during compilation, e-p can detect when all cases of an > enum have been covered, and will _not_ warn about the needed default case. > In fact, checkstyle explicitly mentions in their docs that even if you cover > all enum cases now, you should still have a default label because the enum > could change in the future. Which seems silly to me, because your analysis > tools should still be running in the future and would catch it then. > Se we should migrate the check from checkstyle to a slightly smarter > error-prone check. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20583) SplitLogWorker should handle FileNotFoundException when split a wal
[
https://issues.apache.org/jira/browse/HBASE-20583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-20583:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> SplitLogWorker should handle FileNotFoundException when split a wal
> ---
>
> Key: HBASE-20583
> URL: https://issues.apache.org/jira/browse/HBASE-20583
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 2.0.1
>
> Attachments: HBASE-20583.master.001.patch,
> HBASE-20583.master.001.patch
>
>
> When a split task is finished, master will delete the wal first, then remove
> the task's zk node. So if master crashed after delelte the wal, the zk task
> node may be leaved on zk. When master resubmit this task, the task will
> failed by FileNotFoundException.
> We also handle FileNotFoundException in WALSplitter. But not handle this in
> SplitLogWorker.
>
> {code:java}
> try {
> in = getReader(path, reporter);
> } catch (EOFException e) {
> if (length <= 0) {
> // TODO should we ignore an empty, not-last log file if skip.errors
> // is false? Either way, the caller should decide what to do. E.g.
> // ignore if this is the last log in sequence.
> // TODO is this scenario still possible if the log has been
> // recovered (i.e. closed)
> LOG.warn("Could not open {} for reading. File is empty", path, e);
> }
> // EOFException being ignored
> return null;
> }
> } catch (IOException e) {
> if (e instanceof FileNotFoundException) {
> // A wal file may not exist anymore. Nothing can be recovered so move on
> LOG.warn("File {} does not exist anymore", path, e);
> return null;
> }
> }{code}
> {code:java}
> // Here fs.getFileStatus may throw FileNotFoundException, too. We should
> handle this exception as the WALSplitter.getReader.
> try {
> if (!WALSplitter.splitLogFile(walDir, fs.getFileStatus(new Path(walDir,
> filename)),
> fs, conf, p, sequenceIdChecker,
> server.getCoordinatedStateManager().getSplitLogWorkerCoordination(),
> factory)) {
> return Status.PREEMPTED;
> }
> }
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19188) Build fails on branch-1 using maven-3.5.2
[
https://issues.apache.org/jira/browse/HBASE-19188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-19188:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> Build fails on branch-1 using maven-3.5.2
> -
>
> Key: HBASE-19188
> URL: https://issues.apache.org/jira/browse/HBASE-19188
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 1.4.0, 1.3.1, 1.2.6, 1.5.0
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Blocker
> Fix For: 1.4.0, 1.3.2, 1.2.7, 1.2.6.1
>
> Attachments: HBASE-19188.branch-1.2.001.patch,
> HBASE-19188.branch-1.2.002.patch
>
>
> With maven 3.5.2 the build fails on branch-1-2, branch-1.3, branch-1.4 and
> branch-1. On branch-1.1, branch-2 and master the build succeeds. With older
> maven versions the build finishes.
> {code:title=Maven version}
> $ mvn -v
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=1024m;
> support was removed in 8.0
> Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d;
> 2017-10-18T09:58:13+02:00)
> Maven home: /Users/peter.somogyi/bin/apache-maven-3.5.2
> Java version: 1.8.0_141, vendor: Oracle Corporation
> Java home:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_141.jdk/Contents/Home/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "mac os x", version: "10.12.6", arch: "x86_64", family: "mac"
> {code}
> {code}
> $ mvn clean install -DskipTests
> ...
> [INFO] --- jamon-maven-plugin:2.4.1:translate (default) @ hbase-server ---
> [INFO]
> [INFO] --- maven-antrun-plugin:1.6:run (generate) @ hbase-server ---
> [INFO] Executing tasks
> main:
> log4j:WARN No appenders could be found for logger (org.apache.jasper.JspC).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more
> info.
> [INFO] Logging to org.slf4j.impl.MavenSimpleLogger(org.mortbay.log) via
> org.mortbay.log.Slf4jLog
> java.util.MissingResourceException: Can't find bundle for base name
> org.apache.jasper.resources.LocalStrings, locale en_US
> at
> java.util.ResourceBundle.throwMissingResourceException(ResourceBundle.java:1564)
> at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1387)
> at java.util.ResourceBundle.getBundle(ResourceBundle.java:773)
> at org.apache.jasper.compiler.Localizer.(Localizer.java:36)
> at
> org.apache.jasper.compiler.JspRuntimeContext.(JspRuntimeContext.java:103)
> at org.apache.jasper.JspC.initServletContext(JspC.java:1242)
> at org.apache.jasper.JspC.execute(JspC.java:1103)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
> at org.apache.tools.ant.TaskAdapter.execute(TaskAdapter.java:154)
> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
> at sun.reflect.GeneratedMethodAccessor122.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
> at org.apache.tools.ant.Task.perform(Task.java:348)
> at org.apache.tools.ant.Target.execute(Target.java:390)
> at org.apache.tools.ant.Target.performTasks(Target.java:411)
> at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
> at org.apache.tools.ant.Project.executeTarget(Project.java:1366)
> at
> org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:270)
> at
> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.executeForkedExecutions(MojoExecutor.java:353)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:198)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
> at
> org.apache.maven.lifecycle.int
[jira] [Issue Comment Deleted] (HBASE-18129) truncate_preserve fails when the truncate method doesn't exists on the master
[
https://issues.apache.org/jira/browse/HBASE-18129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-18129:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> truncate_preserve fails when the truncate method doesn't exists on the master
> -
>
> Key: HBASE-18129
> URL: https://issues.apache.org/jira/browse/HBASE-18129
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 1.2.5, 2.0.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Fix For: 1.4.0, 2.0.0
>
> Attachments: HBASE-18129-branch-1-v1.patch.patch,
> HBASE-18129-branch-1-v2.patch, HBASE-18129-branch-1-v3.patch,
> HBASE-18129-branch-1.patch, HBASE-18129-master-v1.patch,
> HBASE-18129-master.patch
>
>
> Recently, I runs a rolling upgrade from HBase 0.98.x to HBase 1.2.5. During
> the master hasn't been upgraded yet, I truncate a table by the command
> truncate_preserve of 1.2.5, but failed.
> {code}
> hbase(main):001:0> truncate_preserve 'cf_logs'
> Truncating 'cf_logs' table (it may take a while):
> - Disabling table...
> - Truncating table...
> - Dropping table...
> - Creating table with region boundaries...
> ERROR: no method 'createTable' for arguments
> (org.apache.hadoop.hbase.HTableDescriptor,org.jruby.java.proxies.ArrayJavaProxy)
> on Java::OrgApacheHadoopHbaseClient::HBaseAdmin
> {code}
> After checking code and commit history, I found it's HBASE-12833 which causes
> this bug.so we should fix it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18358) Backport HBASE-18099 'FlushSnapshotSubprocedure should wait for concurrent Region#flush() to finish' to branch-1.3
[
https://issues.apache.org/jira/browse/HBASE-18358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-18358:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> Backport HBASE-18099 'FlushSnapshotSubprocedure should wait for concurrent
> Region#flush() to finish' to branch-1.3
> --
>
> Key: HBASE-18358
> URL: https://issues.apache.org/jira/browse/HBASE-18358
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2, 2.0.0
>
> Attachments: 18358.branch-1.3.patch, 18358.v2.txt, 18358.v3.txt
>
>
> HBASE-18099 was only integrated to branch-1 and above in consideration of
> backward compatibility.
> This issue is to backport the fix to branch-1.3 and branch-1.2.
> Quoting Gary's suggestion from the tail of HBASE-18099 :
> {quote}
> Sure, don't add the method to Region, just to HRegion, check for an instance
> of HRegion in FlushSnapshotSubprocedure and cast the instance before calling
> the method.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18606) Tests in hbase-spark module fail with UnsatisfiedLinkError
[
https://issues.apache.org/jira/browse/HBASE-18606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-18606:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> Tests in hbase-spark module fail with UnsatisfiedLinkError
> --
>
> Key: HBASE-18606
> URL: https://issues.apache.org/jira/browse/HBASE-18606
> Project: HBase
> Issue Type: Bug
> Components: spark, test
>Affects Versions: 3.0.0, 2.0.0-alpha-3, 2.0.0-alpha-4
>Reporter: Ted Yu
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 18606.v1.txt, 18606.v2.txt,
> HBASE-18606.branch-2.001.patch
>
>
> The following can be observed in recent trunk builds:
> {code}
> [31m java.io.IOException: Shutting down[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:232)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1065)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:936)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:930)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:41)[0m
> [31m at
> org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)[0m
> [31m at
> org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:30)[0m
> [31m ...[0m
> [31m Cause: java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMasterorg.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
> NativeStaticallyReferencedJniMethods.epollin()I[0m
> [31m at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:145)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:217)[0m
> [31m at
> org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:152)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:214)[0m
> [31m at
> org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)[0m
> [31m at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)[0m
> {code}
> This is due to scalatest-maven-plugin missing systemPropertyVariables for
> shaded netty.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18743) HFiles in use by a table which has the same name and namespace with a default table cloned from snapshot may be deleted when that snapshot and default tabl
[
https://issues.apache.org/jira/browse/HBASE-18743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-18743:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> HFiles in use by a table which has the same name and namespace with a default
> table cloned from snapshot may be deleted when that snapshot and default
> table are deleted
>
>
> Key: HBASE-18743
> URL: https://issues.apache.org/jira/browse/HBASE-18743
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.12
>Reporter: wenbang
>Assignee: wenbang
>Priority: Critical
> Fix For: 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-3, 2.0.0
>
> Attachments: HBASE-18743-branch-1.patch, HBASE_18743.patch,
> HBASE_18743_v1.patch, HBASE_18743_v2.patch
>
>
> We recently had a critical production issue in which HFiles that were still
> in use by a table were deleted.
> This appears to have been caused by conditions in which table have the same
> namespace and name with a default table cloned from snapshot.when snapshot
> and default table be deleted,HFiles that are still in use may be deleted.
> For example:
> Table with default namespace is: "t1"
> The namespace of the new table is the same as the name of the default table,
> and is generated by snapshot cloned : "t1: t1"
> When the snapshot and the default namespace table are deleted, the new table
> is also deleted in the used HFiles
> This is because the creation of the BackReferenceFile get the table Name is
> not normal, resulting in can not find the reference file, hfilecleaner to
> delete the HFiles in used, when the table has not been major compact
> {code:java}
> public static boolean create(final Configuration conf, final FileSystem fs,
> final Path dstFamilyPath, final TableName linkedTable, final String
> linkedRegion,
> final String hfileName, final boolean createBackRef) throws IOException
> {
> String familyName = dstFamilyPath.getName();
> String regionName = dstFamilyPath.getParent().getName();
> String tableName =
> FSUtils.getTableName(dstFamilyPath.getParent().getParent())
> .getNameAsString();
> {code}
> {code:java}
> public static TableName getTableName(Path tablePath) {
> return TableName.valueOf(tablePath.getParent().getName(),
> tablePath.getName());
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20554) "WALs outstanding" message from CleanerChore is noisy
[
https://issues.apache.org/jira/browse/HBASE-20554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-20554:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> "WALs outstanding" message from CleanerChore is noisy
> -
>
> Key: HBASE-20554
> URL: https://issues.apache.org/jira/browse/HBASE-20554
> Project: HBase
> Issue Type: Bug
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 3.0.0, 2.1.0, 1.5.0, 2.0.1, 1.4.5
>
> Attachments: HBASE-20554.patch
>
>
> WARN level "WALs outstanding" from CleanerChore should be DEBUG and are not
> always correct.
> I left a cluster configured for ITBLL (retaining all WALs for post hoc
> analysis) and in the morning found the master log full of "WALs outstanding"
> warnings from CleanerChore.
> Should this really be a warning?
> {quote}
> 2018-05-09 16:42:03,893 WARN
> [node-1.cluster,16000,1525851521469_ChoreService_2] cleaner.CleanerChore:
> WALs outstanding under hdfs://node-1.cluster/hbase/oldWALs
> {quote}
> If someone has configured really long WAL retention then having WALs in
> oldWALs will be normal.
> Also, it seems the warning is sometimes incorrect.
> {quote}
> 2018-05-09 16:42:24,751 WARN
> [node-1.cluster,16000,1525851521469_ChoreService_1] cleaner.CleanerChore:
> WALs outstanding under hdfs://node-1.cluster/hbase/archive
> {quote}
> There are no WALs under archive/.
> Even at DEBUG level, if it is not correct, then it can lead an operator to be
> concerned about nothing, so better to just remove it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18355) Enable export snapshot tests that were disabled by Proc-V2 AM in HBASE-14614
[ https://issues.apache.org/jira/browse/HBASE-18355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-18355: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Enable export snapshot tests that were disabled by Proc-V2 AM in HBASE-14614 > > > Key: HBASE-18355 > URL: https://issues.apache.org/jira/browse/HBASE-18355 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-alpha-1 >Reporter: Stephen Yuan Jiang >Assignee: huaxiang sun >Priority: Major > Fix For: 2.0.0-alpha-4, 2.0.0 > > Attachments: HBASE-18355-master_v001.patch, > HBASE-18355-master_v002.patch > > > The Proc-V2 AM in HBASE-14614 disabled the following tests: > - Disabled TestExportSnapshot Hangs. > - Disabled TestSecureExportSnapshot > - Disabled TestMobSecureExportSnapshot and TestMobExportSnapshot > This JIRA tracks the work to enable them. If MOB requires more work, we > could split to 2 tickets. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20795) Allow option in BBKVComparator.compare to do comparison without sequence id
[
https://issues.apache.org/jira/browse/HBASE-20795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-20795:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> Allow option in BBKVComparator.compare to do comparison without sequence id
> ---
>
> Key: HBASE-20795
> URL: https://issues.apache.org/jira/browse/HBASE-20795
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.1
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 2.0.2
>
> Attachments: HBASE-20795.patch
>
>
> CellComparatorImpl#compare(final Cell a, final Cell b, boolean
> ignoreSequenceid) needs to ignore sequence id in comparison if
> ignoreSequenceId parameter is set to true but BBKVComparator.compare used
> internally for the cell of type ByteBufferKeyValue doesn't consider this.
> {code}
> @Override
> public int compare(final Cell a, final Cell b, boolean ignoreSequenceid) {
> int diff = 0;
> // "Peel off" the most common path.
> if (a instanceof ByteBufferKeyValue && b instanceof ByteBufferKeyValue) {
> diff = BBKVComparator.compare((ByteBufferKeyValue)a,
> (ByteBufferKeyValue)b);
> if (diff != 0) {
> return diff;
> }
> } else {
> diff = compareRows(a, b);
> if (diff != 0) {
> return diff;
> }
> diff = compareWithoutRow(a, b);
> if (diff != 0) {
> return diff;
> }
> }
> // Negate following comparisons so later edits show up first mvccVersion:
> later sorts first
> return ignoreSequenceid? diff: Long.compare(b.getSequenceId(),
> a.getSequenceId());
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19954) Separate TestBlockReorder into individual tests to avoid ShutdownHook suppression error against hadoop3
[
https://issues.apache.org/jira/browse/HBASE-19954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-19954:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> Separate TestBlockReorder into individual tests to avoid ShutdownHook
> suppression error against hadoop3
>
>
> Key: HBASE-19954
> URL: https://issues.apache.org/jira/browse/HBASE-19954
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 2.0.0-beta-2, 2.0.0
>
> Attachments: 19954.v1.txt, 19954.v2.txt, 19954.v3.txt, 19954.v4.txt
>
>
> TestBlockReorder fails against hadoop 3 due to the following error:
> {code}
> 2018-02-08 17:22:23,389 ERROR [RS:0;cn012:38253]
> helpers.MarkerIgnoringBase(159): * ABORTING region server
> cn012.l42scl.hortonworks.com,38253,1518110542271: Unhandled: Failed
> suppression of fs shutdown hook:
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@6a0c707e *
> java.lang.RuntimeException: Failed suppression of fs shutdown hook:
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@6a0c707e
> at
> org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:207)
> at
> org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:85)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:927)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:187)
> {code}
> This is because when MiniDFSCluster is stopped in first subtest, hadoop 3
> clears ShutdownHook.
> When the 2nd test starts, suppression of ShutdownHook fails.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20467) Precommit personality should only run checkstyle once if we're going to run it at the root.
[
https://issues.apache.org/jira/browse/HBASE-20467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-20467:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> Precommit personality should only run checkstyle once if we're going to run
> it at the root.
> ---
>
> Key: HBASE-20467
> URL: https://issues.apache.org/jira/browse/HBASE-20467
> Project: HBase
> Issue Type: Bug
> Components: community, test
>Reporter: Sean Busbey
>Assignee: Nihal Jain
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0
>
> Attachments: HBASE-20467.master.001.patch
>
>
> shows up v egregiously on HBASE-20332, but I've seen in a few places where
> we'll do checkstyle at top level as well on individual modules.
> eg.:
> {code}
> +1checkstyle 0m 25s hbase-common: The patch generated 0 new + 88
> unchanged - 1 fixed = 88 total (was 89)
> +1checkstyle 0m 12s The patch hbase-hadoop2-compat passed checkstyle
> +1checkstyle 0m 30s The patch hbase-client passed checkstyle
> +1checkstyle 0m 11s The patch hbase-replication passed checkstyle
> +1checkstyle 1m 15s The patch hbase-server passed checkstyle
> +1checkstyle 0m 17s The patch hbase-mapreduce passed checkstyle
> +1checkstyle 0m 8s The patch hbase-testing-util passed checkstyle
> +1checkstyle 0m 28s The patch hbase-thrift passed checkstyle
> +1checkstyle 0m 11s The patch hbase-rsgroup passed checkstyle
> +1checkstyle 0m 10s The patch hbase-shell passed checkstyle
> +1checkstyle 0m 12s The patch hbase-endpoint passed checkstyle
> +1checkstyle 0m 14s The patch hbase-backup passed checkstyle
> +1checkstyle 0m 21s The patch hbase-it passed checkstyle
> +1checkstyle 0m 14s The patch hbase-examples passed checkstyle
> +1checkstyle 0m 16s The patch hbase-rest passed checkstyle
> +1checkstyle 0m 9s The patch hbase-external-blockcache passed
> checkstyle
> +1checkstyle 0m 10s The patch hbase-shaded passed checkstyle
> +1checkstyle 0m 8s The patch hbase-shaded-mapreduce passed
> checkstyle
> +1checkstyle 0m 8s The patch hbase-shaded-check-invariants passed
> checkstyle
> +1checkstyle 0m 9s The patch
> hbase-shaded-with-hadoop-check-invariants passed checkstyle
> +1checkstyle 2m 38s root: The patch generated 0 new + 429 unchanged
> - 1 fixed = 429 total (was 430)
> {code}
> we should use the same shortcut we do with other modules when root is present
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18645) Loads of tests timing out....
[ https://issues.apache.org/jira/browse/HBASE-18645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-18645: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Loads of tests timing out > - > > Key: HBASE-18645 > URL: https://issues.apache.org/jira/browse/HBASE-18645 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: Chia-Ping Tsai >Priority: Critical > Attachments: HBASE-18645.master.001.patch, > HBASE-18645.master.001.patch > > > Whats up? Why are tests mostly timing out? When did it start? I can't seem to > make it happen locally so tough doing a bisect. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19065) HRegion#bulkLoadHFiles() should wait for concurrent Region#flush() to finish
[
https://issues.apache.org/jira/browse/HBASE-19065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-19065:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> HRegion#bulkLoadHFiles() should wait for concurrent Region#flush() to finish
>
>
> Key: HBASE-19065
> URL: https://issues.apache.org/jira/browse/HBASE-19065
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 1.4.0, 1.3.2, 1.2.7, 2.0.0-beta-1, 2.0.0
>
> Attachments: 19065.v1.txt, 19065.v2.txt, 19065.v2.txt
>
>
> When I was debugging bulk load failure, I saw the following in region server
> log:
> {code}
> 2017-10-17 23:05:28,795 DEBUG
> [B.defaultRpcServer.handler=0,queue=0,port=16020] regionserver.HRegion: NOT
> flushing memstore for region mx_,
> f449669a8b0341e4edbd2ebdacc72094f449669a8b0341e4edbd2ebdacc7209420150711,1504909319142.52d496ba39036e0c2cc9522895ad438f.,
> flushing=true, writesEnabled=true
> 2017-10-17 23:05:28,796 ERROR
> [B.defaultRpcServer.handler=0,queue=0,port=16020]
> access.SecureBulkLoadEndpoint: Failed to complete bulk load
> java.io.IOException: Could not bulk load with an assigned sequential ID
> because the flush didn't run. Reason for not flushing: Not flushing since
> already flushing
> at
> org.apache.hadoop.hbase.regionserver.HRegion.bulkLoadHFiles(HRegion.java:5282)
> at
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint$1.run(SecureBulkLoadEndpoint.java:292)
> at
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint$1.run(SecureBulkLoadEndpoint.java:275)
> {code}
> There was concurrent flush which got misinterpreted by bulkLoadHFiles().
> HRegion#bulkLoadHFiles() should wait for the concurrent flush to complete.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19185) ClassNotFoundException: com.fasterxml.jackson.*
[ https://issues.apache.org/jira/browse/HBASE-19185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-19185: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > ClassNotFoundException: com.fasterxml.jackson.* > --- > > Key: HBASE-19185 > URL: https://issues.apache.org/jira/browse/HBASE-19185 > Project: HBase > Issue Type: Bug > Components: mapreduce >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Critical > Fix For: 2.0.0-beta-1, 2.0.0 > > Attachments: HBASE-19185.v0.patch > > > HBASE-16338 update jackson to 2.9.1 which is different from hadoop-2.7.1. We > should bring our bedding and towels and not be trying to use jackson > dependencies from hadoop. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20506) Add doc and test for unused RetryCounter, useful-looking utility
[ https://issues.apache.org/jira/browse/HBASE-20506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-20506: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Add doc and test for unused RetryCounter, useful-looking utility > > > Key: HBASE-20506 > URL: https://issues.apache.org/jira/browse/HBASE-20506 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Minor > Fix For: 3.0.0, 2.1.0 > > Attachments: 20506.txt, HBASE-20506.master.001.patch, > HBASE-20506.master.002.patch > > > I thought I could use RetryCounter, old facility added years ago, for doing > backoff calculations. In the end, it didn't work for me because it is lacking > pb serialization. While trying to use it, I added a bit of doc and a test. > Might help the next dev that trips along this way. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18556) change splitable to mergeable in MergeTableRegionsProcedure
[ https://issues.apache.org/jira/browse/HBASE-18556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-18556: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > change splitable to mergeable in MergeTableRegionsProcedure > --- > > Key: HBASE-18556 > URL: https://issues.apache.org/jira/browse/HBASE-18556 > Project: HBase > Issue Type: Bug >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18292) Fix flaky test hbase.master.locking.TestLockProcedure#testLocalMasterLockRecovery()
[ https://issues.apache.org/jira/browse/HBASE-18292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-18292: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Fix flaky test > hbase.master.locking.TestLockProcedure#testLocalMasterLockRecovery() > --- > > Key: HBASE-18292 > URL: https://issues.apache.org/jira/browse/HBASE-18292 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-alpha-1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Major > Fix For: 2.0.0-alpha-2, 2.0.0 > > Attachments: HBASE-18292.master.001.patch > > > Assert before waitProcedure() assumes that procedure executer is still > running but this assumption is incorrect. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20612) TestReplicationKillSlaveRSWithSeparateOldWALs sometimes fail because it uses an expired cluster conn
[
https://issues.apache.org/jira/browse/HBASE-20612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-20612:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> TestReplicationKillSlaveRSWithSeparateOldWALs sometimes fail because it uses
> an expired cluster conn
>
>
> Key: HBASE-20612
> URL: https://issues.apache.org/jira/browse/HBASE-20612
> Project: HBase
> Issue Type: Bug
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20612.v1.patch, HBASE-20612.v2.patch,
> org.apache.hadoop.hbase.replication.TestReplicationKillSlaveRSWithSeparateOldWALs-output.txt
>
>
> {code}
> 2018-05-22 06:40:00,614 INFO [Thread-961] regionserver.HRegionServer(2144):
> * STOPPING region server 'asf911.gq1.ygridcore.net,42867,1526971178277'
> *
> 2018-05-22 06:40:00,614 INFO [Thread-961] regionserver.HRegionServer(2158):
> STOPPED: Stopping as part of the test
>
> 2018-05-22 06:41:01,018 DEBUG [Time-limited test]
> client.ResultBoundedCompletionService(226): Replica 0 returns
> java.net.SocketTimeoutException: callTimeout=6, callDuration=60515: Call
> to asf911.gq1.ygridcore.net/67.195.81.155:42867 failed on local exception:
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=34, waitTime=59332,
> rpcTimeout=59322 row 'eee' on table 'test' at
> region=test,eee,1526971188643.5aab2dd2e1d02b4e40be6d00422acd21.,
> hostname=asf911.gq1.ygridcore.net,42867,1526971178277, seqNum=2
> java.net.SocketTimeoutException: callTimeout=6, callDuration=60515: Call
> to asf911.gq1.ygridcore.net/67.195.81.155:42867 failed on local exception:
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=34, waitTime=59332,
> rpcTimeout=59322 row 'eee' on table 'test' at
> region=test,eee,1526971188643.5aab2dd2e1d02b4e40be6d00422acd21.,
> hostname=asf911.gq1.ygridcore.net,42867,1526971178277, seqNum=2
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:159)
> at
> org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:80)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Call to
> asf911.gq1.ygridcore.net/67.195.81.155:42867 failed on local exception:
> org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=34, waitTime=59332,
> rpcTimeout=59322
> at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:180)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:390)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:95)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:410)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:406)
> at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:96)
> at
> org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:199)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:663)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:738)
> at
> org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:466)
> ... 1 more
> Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=34,
> waitTime=59332, rpcTimeout=59322
> at
> org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:200)
> ... 4 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20789) TestBucketCache#testCacheBlockNextBlockMetadataMissing is flaky
[ https://issues.apache.org/jira/browse/HBASE-20789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-20789: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > TestBucketCache#testCacheBlockNextBlockMetadataMissing is flaky > --- > > Key: HBASE-20789 > URL: https://issues.apache.org/jira/browse/HBASE-20789 > Project: HBase > Issue Type: Bug >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.4.6, 2.0.2 > > Attachments: > 0001-HBASE-20789-TestBucketCache-testCacheBlockNextBlockM.patch, > HBASE-20789.branch-1.v1.patch, HBASE-20789.v1.patch, HBASE-20789.v2.patch, > HBASE-20789.v3.patch, HBASE-20789.v4.patch, HBASE-20789.v5.patch, > bucket-33718.out > > > The UT failed frequently in our internal branch-2... Will dig into the UT. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20664) Variable shared across multiple threads
[ https://issues.apache.org/jira/browse/HBASE-20664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-20664: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Variable shared across multiple threads > --- > > Key: HBASE-20664 > URL: https://issues.apache.org/jira/browse/HBASE-20664 > Project: HBase > Issue Type: Bug >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.5, 1.2.6.1, > 1.3.2.1, 2.0.0.1 > > > Some static analysis found a variable which was used across multiple threads > without any synchronization that would allow race conditions. > The variable does not need to be a member of the class, instead just made a > local variable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20855) PeerConfigTracker only supporting one listener will cause problem when there is a recovered replication queue
[
https://issues.apache.org/jira/browse/HBASE-20855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-20855:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> PeerConfigTracker only supporting one listener will cause problem when there
> is a recovered replication queue
> -
>
> Key: HBASE-20855
> URL: https://issues.apache.org/jira/browse/HBASE-20855
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.0, 1.5.0
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
> Fix For: 1.5.0, 1.4.6
>
> Attachments: HBASE-20855.branch-1.001.patch,
> HBASE-20855.branch-1.002.patch, HBASE-20855.branch-1.003.patch,
> HBASE-20855.branch-1.004.patch, HBASE-20855.branch-1.005.patch,
> HBASE-20855.branch-1.006.patch, HBASE-20855.branch-1.007.patch
>
>
> {code}
> public void init(Context context) throws IOException {
> this.ctx = context;
> if (this.ctx != null){
> ReplicationPeer peer = this.ctx.getReplicationPeer();
> if (peer != null){
> peer.trackPeerConfigChanges(this);
> } else {
> LOG.warn("Not tracking replication peer config changes for Peer Id " +
> this.ctx.getPeerId() +
> " because there's no such peer");
> }
> }
> }
> {code}
> As we know, replication source will set itself to the PeerConfigTracker in
> ReplicationPeer. When there is one or more recovered queue, each queue will
> generate a new replication source, But they share the same ReplicationPeer.
> Then when it calls setListener, the new generated one will cover the older
> one. Thus there will only has one ReplicationPeer that receive the peer
> config change notify.
> {code}
> public synchronized void setListener(ReplicationPeerConfigListener listener){
> this.listener = listener;
> }
> {code}
>
> To solve this, PeerConfigTracker need to support multiple listener and
> listener should be removed when the replication endpoint terminated.
> I will upload a patch later with fix and UT.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20017) BufferedMutatorImpl submit the same mutation repeatedly
[ https://issues.apache.org/jira/browse/HBASE-20017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-20017: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > BufferedMutatorImpl submit the same mutation repeatedly > --- > > Key: HBASE-20017 > URL: https://issues.apache.org/jira/browse/HBASE-20017 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 1.4.0, 1.4.1, 1.4.2 >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Blocker > Fix For: 1.5.0, 1.4.2, 2.0.0 > > Attachments: HBASE-20017.branch-1.4.v0.patch.patch, > HBASE-20017.branch-1.v0.patch.patch, HBASE-20017.v0.patch.patch > > > BMI pass a iter of inner buffer to AsyncProcess to take the undealt > mutations, AsyncProcess call iter#next to get the mutation and then call > iter#remove to delete the mutation from inner buffer. Hence, There's a good > chance that the mutation is processed repeatedly in case there are a bunch > of threads which are running the flush. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19712) Fix TestSnapshotQuotaObserverChore#testSnapshotSize
[ https://issues.apache.org/jira/browse/HBASE-19712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-19712: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Fix TestSnapshotQuotaObserverChore#testSnapshotSize > --- > > Key: HBASE-19712 > URL: https://issues.apache.org/jira/browse/HBASE-19712 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Major > Fix For: 2.0.0-beta-1, 2.0.0 > > Attachments: HBASE-19712.v0.patch, HBASE-19712.v1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18005) read replica: handle the case that region server hosting both primary replica and meta region is down
[ https://issues.apache.org/jira/browse/HBASE-18005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-18005: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > read replica: handle the case that region server hosting both primary replica > and meta region is down > - > > Key: HBASE-18005 > URL: https://issues.apache.org/jira/browse/HBASE-18005 > Project: HBase > Issue Type: Bug >Reporter: huaxiang sun >Assignee: huaxiang sun >Priority: Major > Fix For: 1.4.0, 2.0.0 > > Attachments: HBASE-18005-branch-1-v001.patch, > HBASE-18005-master-001.patch, HBASE-18005-master-002.patch, > HBASE-18005-master-003.patch, HBASE-18005-master-004.patch, > HBASE-18005-master-005.patch, HBASE-18005-master-006.patch > > > Identified one corner case in testing that when the region server hosting > both primary replica and the meta region is down, the client tries to reload > the primary replica location from meta table, it is supposed to clean up only > the cached location for specific replicaId, but it clears caches for all > replicas. Please see > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java#L813 > Since it takes some time for regions to be reassigned (including meta > region), the following may throw exception > https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java#L173 > This exception needs to be caught and it needs to get cached location (in > this case, the primary replica's location is not available). If there are > cached locations for other replicas, it can still go ahead to get stale > values from secondary replicas. > With meta replica, it still helps to not clean up the caches for all replicas > as the info from primary meta replica is up-to-date. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18132) Low replication should be checked in period in case of datanode rolling upgrade
[ https://issues.apache.org/jira/browse/HBASE-18132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-18132: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Low replication should be checked in period in case of datanode rolling > upgrade > --- > > Key: HBASE-18132 > URL: https://issues.apache.org/jira/browse/HBASE-18132 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0, 1.1.10 >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Fix For: 1.4.0, 1.3.2, 1.2.7, 2.0.0 > > Attachments: HBASE-18132-branch-1.patch, > HBASE-18132-branch-1.v2.patch, HBASE-18132-branch-1.v3.patch, > HBASE-18132-branch-1.v4.patch, HBASE-18132.patch > > > For now, we just check low replication of WALs when there is a sync operation > (HBASE-2234), rolling the log if the replica of the WAL is less than > configured. But if the WAL has very little writes or no writes at all, low > replication will not be detected and thus no log will be rolled. > That is a problem when rolling updating datanode, all replica of the WAL with > no writes will be restarted and lead to the WAL file end up with a abnormal > state. Later operation of opening this file will be always failed. > I bring up a patch to check low replication of WALs at a configured period. > When rolling updating datanodes, we just make sure the restart interval time > between two nodes is bigger than the low replication check time, the WAL will > be closed and rolled normally. A UT in the patch will show everything. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18445) Upgrading Guava broke hadoop-3.0 profile
[
https://issues.apache.org/jira/browse/HBASE-18445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-18445:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> Upgrading Guava broke hadoop-3.0 profile
>
>
> Key: HBASE-18445
> URL: https://issues.apache.org/jira/browse/HBASE-18445
> Project: HBase
> Issue Type: Bug
> Components: build, dependencies, hadoop3
>Affects Versions: 2.0.0-alpha-2
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Blocker
> Fix For: 2.0.0-alpha-2, 2.0.0
>
> Attachments: HBASE-18445.patch
>
>
> After HBASE-17908 upgrade guava, we have failures with the hadoop-3.0 profile.
> {noformat}
> [INFO] --- maven-enforcer-plugin:1.4.1:enforce (banned-jsr305) @ hbase-client
> ---
> [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies failed
> with message:
> We don't allow the JSR305 jar from the Findbugs project, see HBASE-16321.
> Found Banned Dependency: com.google.code.findbugs:jsr305:jar:1.3.9
> Use 'mvn dependency:tree' to locate the source of the banned dependencies.
> {noformat}
> {noformat}
> [INFO] --- maven-dependency-plugin:2.10:tree (default-cli) @ hbase-client ---
> [INFO] org.apache.hbase:hbase-client:jar:2.0.0-alpha-2-SNAPSHOT
> [INFO] +- org.apache.hadoop:hadoop-auth:jar:3.0.0-alpha4:compile
> [INFO] | +- org.apache.httpcomponents:httpclient:jar:4.5.3:compile
> [INFO] | | \- org.apache.httpcomponents:httpcore:jar:4.4.6:compile
> [INFO] | +- com.nimbusds:nimbus-jose-jwt:jar:3.9:compile
> [INFO] | | +- net.jcip:jcip-annotations:jar:1.0:compile
> [INFO] | | \- net.minidev:json-smart:jar:1.1.1:compile
> [INFO] | +- org.apache.kerby:kerb-simplekdc:jar:1.0.0-RC2:compile
> [INFO] | | +- org.apache.kerby:kerby-config:jar:1.0.0-RC2:compile
> [INFO] | | +- org.apache.kerby:kerb-core:jar:1.0.0-RC2:compile
> [INFO] | | | +- org.apache.kerby:kerby-asn1:jar:1.0.0-RC2:compile
> [INFO] | | | \- org.apache.kerby:kerby-pkix:jar:1.0.0-RC2:compile
> [INFO] | | | \- org.apache.kerby:kerby-util:jar:1.0.0-RC2:compile
> [INFO] | | +- org.apache.kerby:kerb-client:jar:1.0.0-RC2:compile
> [INFO] | | | \- org.apache.kerby:kerb-common:jar:1.0.0-RC2:compile
> [INFO] | | +- org.apache.kerby:kerb-util:jar:1.0.0-RC2:compile
> [INFO] | | | \- org.apache.kerby:kerb-crypto:jar:1.0.0-RC2:compile
> [INFO] | | +- org.apache.kerby:kerb-server:jar:1.0.0-RC2:compile
> [INFO] | | | \- org.apache.kerby:kerb-identity:jar:1.0.0-RC2:compile
> [INFO] | | \- org.apache.kerby:kerb-admin:jar:1.0.0-RC2:compile
> [INFO] | \- com.google.guava:guava:jar:11.0.2:compile
> [INFO] | \- com.google.code.findbugs:jsr305:jar:1.3.9:compile
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18006) AsyncClientScanner does not retry openScan RPCs
[
https://issues.apache.org/jira/browse/HBASE-18006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-18006:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> AsyncClientScanner does not retry openScan RPCs
> ---
>
> Key: HBASE-18006
> URL: https://issues.apache.org/jira/browse/HBASE-18006
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: hbase-18006-test.patch
>
>
> I have been reading the code for the new async scan paths excessively, and
> noticed that there is a problem in the retrying layer for openScan RPCs.
> In AsyncClientScanner#callOpenScanner() we are doing a open scan RPC. The
> retrying logic comes from using the single rpc retrying caller in
> openScanner(). However, we have the logic for failing the scanner if any of
> the RPC calls here:
> {code}
> stub.scan(controller, request, resp -> {
> if (controller.failed()) {
> future.completeExceptionally(controller.getFailed());
> return;
> }
> future.complete(new OpenScannerResponse(loc, isRegionServerRemote,
> stub, controller, resp));
> });
> {code}
> So, if the open scan gets an UnknownScannerException or something, instead of
> retrying, it just fails the whole scan.
> [~Apache9] FYI.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18846) Accommodate the hbase-indexer/lily/SEP consumer deploy-type
[
https://issues.apache.org/jira/browse/HBASE-18846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-18846:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> Accommodate the hbase-indexer/lily/SEP consumer deploy-type
> ---
>
> Key: HBASE-18846
> URL: https://issues.apache.org/jira/browse/HBASE-18846
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-alpha-4, 2.0.0
>
> Attachments: HBASE-18846.master.001.patch,
> HBASE-18846.master.002.patch, HBASE-18846.master.003.patch,
> HBASE-18846.master.004.patch, HBASE-18846.master.005.patch,
> HBASE-18846.master.006.patch, HBASE-18846.master.007.patch,
> HBASE-18846.master.007.patch, IndexerConnection.java, hbase-site.xml,
> javadoc.txt
>
>
> This is a follow-on from HBASE-10504, Define a Replication Interface. There
> we defined a new, flexible replication endpoint for others to implement but
> it did little to help the case of the lily hbase-indexer. This issue takes up
> the case of the hbase-indexer.
> The hbase-indexer poses to hbase as a 'fake' peer cluster (For why
> hbase-indexer is implemented so, the advantage to having the indexing done in
> a separate process set that can be independently scaled, can participate in
> the same security realm, etc., see discussion in HBASE-10504). The
> hbase-indexer will start up a cut-down "RegionServer" processes that are just
> an instance of hbase RpcServer hosting an AdminProtos Service. They make
> themselves 'appear' to the Replication Source by hoisting up an ephemeral
> znode 'registering' as a RegionServer. The source cluster then streams
> WALEdits to the Admin Protos method:
> {code}
> public ReplicateWALEntryResponse replicateWALEntry(final RpcController
> controller,
> final ReplicateWALEntryRequest request) throws ServiceException {
> {code}
> The hbase-indexer relies on other hbase internals like Server so it can get a
> ZooKeeperWatcher instance and know the 'name' to use for this cut-down server.
> Thoughts on how to proceed include:
>
> * Better formalize its current digestion of hbase internals; make it so
> rpcserver is allowed to be used by others, etc. This would be hard to do
> given they use basics like Server, Protobuf serdes for WAL types, and
> AdminProtos Service. Any change in this wide API breaks (again)
> hbase-indexer. We have made a 'channel' for Coprocessor Endpoints so they
> continue to work though they use 'internal' types. They can use protos in
> hbase-protocol. hbase-protocol protos are in a limbo currently where they are
> sort-of 'public'; a TODO. Perhaps the hbase-indexer could do similar relying
> on the hbase-protocol (pb2.5) content and we could do something to reveal
> rpcserver and zk for hbase-indexer safe use.
> * Start an actual RegionServer only have it register the AdminProtos Service
> only -- not ClientProtos and the Service that does Master interaction, etc.
> [I checked, this is not as easy to do as I at first thought -- St.Ack] Then
> have the hbase-indexer implement an AdminCoprocessor to override the
> replicateWALEntry method (the Admin CP implementation may need work). This
> would narrow the hbase-indexer exposure to that of the Admin Coprocessor
> Interface
> * Over in HBASE-10504, [~enis] suggested "... if we want to provide
> isolation for the replication services in hbase, we can have a simple host as
> another daemon which hosts the ReplicationEndpoint implementation. RS's will
> use a built-in RE to send the edits to this layer, and the host will delegate
> it to the RE implementation. The flow would be something like: RS --> RE
> inside RS --> Host daemon for RE --> Actual RE implementation --> third party
> system..."
>
> Other crazy notions occur including the setup of an Admin Interface
> Coprocessor Endpoint. A new ReplicationEndpoint would feed the replication
> stream to the remote cluster via the CPEP registered channel.
> But time is short. Hopefully we can figure something that will work in 2.0
> timeframe w/o too much code movement.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18449) Fix client.locking.TestEntityLocks#testHeartbeatException
[
https://issues.apache.org/jira/browse/HBASE-18449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-18449:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> Fix client.locking.TestEntityLocks#testHeartbeatException
> -
>
> Key: HBASE-18449
> URL: https://issues.apache.org/jira/browse/HBASE-18449
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 2.0.0-alpha-2, 2.0.0
>
> Attachments: HBASE-18449.v0.patch
>
>
> {noformat}
> Wanted but not invoked:
> abortable.abort(
> ,
> isA(org.apache.hadoop.hbase.HBaseIOException)
> );
> -> at
> org.apache.hadoop.hbase.client.locking.TestEntityLocks.testHeartbeatException(TestEntityLocks.java:195)
> Actually, there were zero interactions with this mock.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19287) master hangs forever if RecoverMeta send assign meta region request to target server fail
[
https://issues.apache.org/jira/browse/HBASE-19287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-19287:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> master hangs forever if RecoverMeta send assign meta region request to target
> server fail
> -
>
> Key: HBASE-19287
> URL: https://issues.apache.org/jira/browse/HBASE-19287
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Affects Versions: 2.0.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>Priority: Major
> Fix For: 2.0.0-beta-1, 2.0.0
>
> Attachments: HBASE-19287-master-v3.patch,
> HBASE-19287-master-v3.patch, HBASE-19287-master-v4.patch,
> hbase-19287-master-v2.patch, master.patch
>
>
> 2017-11-10 19:26:56,019 INFO [ProcExecWrkr-1]
> procedure.RecoverMetaProcedure: pid=138,
> state=RUNNABLE:RECOVER_META_ASSIGN_REGIONS; RecoverMetaProcedure
> failedMetaServer=null, splitWal=true; Retaining meta assignment to
> server=hadoop-slave1.hadoop,16020,1510341981454
> 2017-11-10 19:26:56,029 INFO [ProcExecWrkr-1] procedure2.ProcedureExecutor:
> Initialized subprocedures=[{pid=139, ppid=138,
> state=RUNNABLE:REGION_TRANSITION_QUEUE; AssignProcedure table=hbase:meta,
> region=1588230740, target=hadoop-slave1.hadoop,16020,1510341981454}]
> 2017-11-10 19:26:56,067 INFO [ProcExecWrkr-2]
> procedure.MasterProcedureScheduler: pid=139, ppid=138,
> state=RUNNABLE:REGION_TRANSITION_QUEUE; AssignProcedure table=hbase:meta,
> region=1588230740, target=hadoop-slave1.hadoop,16020,1510341981454 hbase:meta
> hbase:meta,,1.1588230740
> 2017-11-10 19:26:56,071 INFO [ProcExecWrkr-2] assignment.AssignProcedure:
> Start pid=139, ppid=138, state=RUNNABLE:REGION_TRANSITION_QUEUE;
> AssignProcedure table=hbase:meta, region=1588230740,
> target=hadoop-slave1.hadoop,16020,1510341981454; rit=OFFLINE,
> location=hadoop-slave1.hadoop,16020,1510341981454; forceNewPlan=false,
> retain=false
> 2017-11-10 19:26:56,224 INFO [ProcExecWrkr-4] zookeeper.MetaTableLocator:
> Setting hbase:meta (replicaId=0) location in ZooKeeper as
> hadoop-slave2.hadoop,16020,1510341988652
> 2017-11-10 19:26:56,230 INFO [ProcExecWrkr-4]
> assignment.RegionTransitionProcedure: Dispatch pid=139, ppid=138,
> state=RUNNABLE:REGION_TRANSITION_DISPATCH; AssignProcedure table=hbase:meta,
> region=1588230740, target=hadoop-slave1.hadoop,16020,1510341981454;
> rit=OPENING, location=hadoop-slave2.hadoop,16020,1510341988652
> 2017-11-10 19:26:56,382 INFO [ProcedureDispatcherTimeoutThread]
> procedure.RSProcedureDispatcher: Using procedure batch rpc execution for
> serverName=hadoop-slave2.hadoop,16020,1510341988652 version=2097152
> 2017-11-10 19:26:57,542 INFO [main-EventThread]
> zookeeper.RegionServerTracker: RegionServer ephemeral node deleted,
> processing expiration [hadoop-slave2.hadoop,16020,1510341988652]
> 2017-11-10 19:26:57,543 INFO [main-EventThread] master.ServerManager: Master
> doesn't enable ServerShutdownHandler during initialization, delay expiring
> server hadoop-slave2.hadoop,16020,1510341988652
> 2017-11-10 19:26:58,875 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000]
> master.ServerManager: Registering
> server=hadoop-slave1.hadoop,16020,1510342016106
> 2017-11-10 19:27:05,832 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000]
> master.ServerManager: Registering
> server=hadoop-slave2.hadoop,16020,1510342023184
> 2017-11-10 19:27:05,832 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000]
> master.ServerManager: Triggering server recovery; existingServer
> hadoop-slave2.hadoop,16020,1510341988652 looks stale, new
> server:hadoop-slave2.hadoop,16020,1510342023184
> 2017-11-10 19:27:05,832 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000]
> master.ServerManager: Master doesn't enable ServerShutdownHandler during
> initialization, delay expiring server hadoop-slave2.hadoop,16020,1510341988652
> 2017-11-10 19:27:49,815 INFO
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16000]
> client.RpcRetryingCallerImpl: tarted=38594 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: hbase:meta,,1 is not
> online on hadoop-slave2.hadoop,16020,1510342023184
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3290)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1370)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2401)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41544)
> at org.apache.hado
[jira] [Issue Comment Deleted] (HBASE-18771) Incorrect StoreFileRefresh leading to split and compaction failures
[ https://issues.apache.org/jira/browse/HBASE-18771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-18771: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Incorrect StoreFileRefresh leading to split and compaction failures > --- > > Key: HBASE-18771 > URL: https://issues.apache.org/jira/browse/HBASE-18771 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.1 >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan >Priority: Blocker > Fix For: 1.4.0, 1.3.2, 2.0.0 > > Attachments: HBASE-18771.branch-1.3.001.patch, > HBASE-18771.branch-1.3.002.patch, HBASE-18771.branch-1.3.003.patch, > HBASE-18771.branch-1.3.004.patch, HBASE-18771.branch-1.3.005.patch, > HBASE-18771.master.001.patch, HBASE-18771.master.002.patch, > HBASE-18771.master.003.patch > > > We ran into issues of compaction and split failures with 1.3 similar to > HBASE-18186 and HBASE-17406. Here's what i believe is happening - > Lets say we have 4 store files that are compacted to form a new one. At this > point we now have 5 store files, however only 1(the newly formed) is open now > for the store and rest are waiting to get archived by HFileArchiver > Now before the files are archived we get a FNFE in a scanner. This results in > HRegion.RegionScannerImpl.handleFileNotFound(FileNotFoundException fnfe) > being called which results in region.refreshStoreFiles(true) -> > HStore.refreshStoreFiles() > HStore.refreshStoreFiles now checks the hdfs dir and adds the previously > compacted files back to the store, however these files are also present in > StoreFileManager's compactedFiles list. Now at this point HFileArchiver runs, > checks compactedFiles list and moves these files into the archive directory. > Now when compaction runs it gets: > 2017-09-04 12:30:13,899 ERROR [ctions-1504505399609] > regionserver.CompactSplitThread - Compaction selection failed regionName = > , storeName = 0, priority = 26, time = 1504528213899 > java.io.FileNotFoundException: File does not exist: hdfs:// > at > org.apache.hadoop.hdfs.DistributedFileSystem$23.doCall(DistributedFileSystem.java:1337) > at > org.apache.hadoop.hdfs.DistributedFileSystem$23.doCall(DistributedFileSystem.java:1329) > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > at > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1329) > at > org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:422) > at > org.apache.hadoop.hbase.regionserver.StoreFileInfo.getReferencedFileStatus(StoreFileInfo.java:342) > at > org.apache.hadoop.hbase.regionserver.StoreFileInfo.getFileStatus(StoreFileInfo.java:355) > at > org.apache.hadoop.hbase.regionserver.StoreFileInfo.getModificationTime(StoreFileInfo.java:360) > at > org.apache.hadoop.hbase.regionserver.StoreFile.getModificationTimeStamp(StoreFile.java:325) > at > org.apache.hadoop.hbase.regionserver.StoreUtils.getLowestTimestamp(StoreUtils.java:63) > at > org.apache.hadoop.hbase.regionserver.compactions.RatioBasedCompactionPolicy.shouldPerformMajorCompaction(RatioBasedCompactionPolicy.java:65) > at > org.apache.hadoop.hbase.regionserver.compactions.SortedCompactionPolicy.selectCompaction(SortedCompactionPolicy.java:82) > at > org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.select(DefaultStoreEngine.java:107) > at > org.apache.hadoop.hbase.regionserver.HStore.requestCompaction(HStore.java:1679) > Similarly if a split happens after archival we fail after PONR while opening > daughter regions due to FNFE. This results in parent offline and daughters > also in a limbo since they're unable to open. Since we get the error after > PONR we also end up aborting the RS. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19734) IntegrationTestReplication broken w/ separate clusters
[
https://issues.apache.org/jira/browse/HBASE-19734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-19734:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> IntegrationTestReplication broken w/ separate clusters
> --
>
> Key: HBASE-19734
> URL: https://issues.apache.org/jira/browse/HBASE-19734
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2, 2.0.0
>
> Attachments: HBASE-19734.001.branch-2.patch
>
>
> {noformat}
> Caused by:
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(java.io.IOException):
> java.io.IOException: Need clean namespaces or table-cfs config firstly when
> replicate_all flag is true
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:463)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.replication.ReplicationException: Need
> clean namespaces or table-cfs config firstly when replicate_all flag is true
> at
> org.apache.hadoop.hbase.master.replication.ReplicationManager.checkPeerConfig(ReplicationManager.java:134)
> at
> org.apache.hadoop.hbase.master.replication.ReplicationManager.addReplicationPeer(ReplicationManager.java:74)
> at
> org.apache.hadoop.hbase.master.HMaster.addReplicationPeer(HMaster.java:3325)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.addReplicationPeer(MasterRpcServices.java:1891)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
> ... 3 more
> {noformat}
> {code}
> // setup the replication on the source
> if (!source.equals(sink)) {
> ReplicationAdmin replicationAdmin = new
> ReplicationAdmin(source.getConfiguration());
> // remove any old replication peers
> for (String oldPeer : replicationAdmin.listPeerConfigs().keySet()) {
> replicationAdmin.removePeer(oldPeer);
> }
> // set the sink to be the target
> ReplicationPeerConfig peerConfig = new ReplicationPeerConfig();
> peerConfig.setClusterKey(sink.toString());
> // set the test table to be the table to replicate
> HashMap> toReplicate = new HashMap<>();
> toReplicate.put(tableName, new ArrayList<>(0));
> replicationAdmin.addPeer("TestPeer", peerConfig, toReplicate);
> {code}
> It seems like some change in semantics of ReplicationPeerConfig weren't
> correctly updated after HBASE-16868 (or a related change maybe).
> IntegrationTestReplication is trying to replicate a single table but we get
> into a state where we actually construct a Config object that says "replicate
> all tables" and "replicate just this one table".
> 1. We should catch invalid config objects when we construct it via the builder
> 2. Some Builder interface methods are missing Javadoc
> 3. We should update IntegrationTestReplication to use the new API.
> FYI [~zghaobac], [~stack]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19721) Unnecessary stubbings detected in test class: TestReversedScannerCallable
[
https://issues.apache.org/jira/browse/HBASE-19721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-19721:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> Unnecessary stubbings detected in test class: TestReversedScannerCallable
> -
>
> Key: HBASE-19721
> URL: https://issues.apache.org/jira/browse/HBASE-19721
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Jean-Marc Spaggiari
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.0.0-beta-1, 2.0.0
>
> Attachments: HBASE-19721.patch
>
>
> Found by JMS on the mailing list:
> {noformat}
> ---
> Test set: org.apache.hadoop.hbase.client.TestReversedScannerCallable
> ---
> Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.515 s <<<
> FAILURE! - in org.apache.hadoop.hbase.client.TestReversedScannerCallable
> unnecessary Mockito
> stubbings(org.apache.hadoop.hbase.client.TestReversedScannerCallable) Time
> elapsed: 0.014 s <<< ERROR!
> org.mockito.exceptions.misusing.UnnecessaryStubbingException:
> Unnecessary stubbings detected in test class: TestReversedScannerCallable
> Clean & maintainable test code requires zero unnecessary code.
> Following stubbings are unnecessary (click to navigate to relevant line of
> code):
> 1. -> at
> org.apache.hadoop.hbase.client.TestReversedScannerCallable.setUp(TestReversedScannerCallable.java:66)
> 2. -> at
> org.apache.hadoop.hbase.client.TestReversedScannerCallable.setUp(TestReversedScannerCallable.java:68)
> Please remove unnecessary stubbings. More info: javadoc for
> UnnecessaryStubbingException class.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19194) TestRSGroupsBase has some always false checks
[
https://issues.apache.org/jira/browse/HBASE-19194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-19194:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> TestRSGroupsBase has some always false checks
> -
>
> Key: HBASE-19194
> URL: https://issues.apache.org/jira/browse/HBASE-19194
> Project: HBase
> Issue Type: Bug
> Components: rsgroup, test
>Reporter: Mike Drob
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.4.0, 2.0.0
>
> Attachments: HBASE-19194-branch-1.patch, HBASE-19194.patch
>
>
> Discovered via error-prone analysis:
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.6.1:testCompile
> (default-testCompile) on project hbase-rsgroup: Compilation failure:
> Compilation failure:
> [ERROR]
> /Users/mdrob/IdeaProjects/hbase/hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsBase.java:[421,40]
> [CollectionIncompatibleType] Argument 'region' should not be passed to this
> method; its type RegionInfo is not compatible with its collection's type
> argument String
> [ERROR] (see http://errorprone.info/bugpattern/CollectionIncompatibleType)
> [ERROR]
> /Users/mdrob/IdeaProjects/hbase/hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsBase.java:[531,65]
> [CollectionIncompatibleType] Argument 'targetRegion' should not be passed to
> this method; its type String is not compatible with its collection's type
> argument RegionInfo
> [ERROR] (see http://errorprone.info/bugpattern/CollectionIncompatibleType)
> {noformat}
> Both of these are checks to {{List.contains}} with an element of the wrong
> type so they will always return false. The first is an optimization that we
> can probably live without (or short circuit on the Master, at least) the
> second is an assertion that will never fail, rendering the test ineffective.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-17985) Inline package manage updates with package installation in Yetus Dockerfile
[ https://issues.apache.org/jira/browse/HBASE-17985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-17985: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Inline package manage updates with package installation in Yetus Dockerfile > --- > > Key: HBASE-17985 > URL: https://issues.apache.org/jira/browse/HBASE-17985 > Project: HBase > Issue Type: Bug >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Blocker > Fix For: 1.4.0, 1.2.6, 1.3.2, 1.1.11, 2.0.0 > > Attachments: HBASE-17985.001.branch-1.patch, HBASE-17985.001.patch > > > Context: > https://lists.apache.org/thread.html/d34093557cc510bb8b1dc4b37f8a729b74577c7d4eaecdc3f1badea1@%3Cdev.hbase.apache.org%3E > The way Docker images are built for the Yetus-based PreCommit, we may > accidentally use a pre-built image that has a stale package-manager cache. If > the distribution updates their published packages (removing an older version, > adding a new one), our (stale) client will try to pull the older version > which is missing, failing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18625) Splitting of region with replica, doesn't update region list in serverHolding. A server crash leads to overlap.
[
https://issues.apache.org/jira/browse/HBASE-18625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-18625:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> Splitting of region with replica, doesn't update region list in
> serverHolding. A server crash leads to overlap.
> ---
>
> Key: HBASE-18625
> URL: https://issues.apache.org/jira/browse/HBASE-18625
> Project: HBase
> Issue Type: Bug
> Components: read replicas
>Affects Versions: 1.2.6
>Reporter: Igloo
>Assignee: huaxiang sun
>Priority: Critical
> Fix For: 1.3.2, 1.4.1, 1.5.0, 1.2.7
>
> Attachments: HBASE-18625-branch-1-v001.patch,
> HBASE-18625-branch-1-v002.patch
>
>
> The situation can appear in following steps in release hbase1.2.6
> 1. create 'testtable', 'info', {REGION_REPLICATION=>2}
> 2. write somerecords into 'testtable'
> 3. split the table 'testtable'
> 4. after the spliting, the serverHoldings in RegionStates still holds the
> regioninfo for the replica of parent region
> 5. restart the regionserver where the parent replica-region located
> 6. the offlined replica of parent region will be assigned in
> ServerCrashProcedure.
> hbase hbck 'testtable‘
> ERROR: Region { meta => null, hdfs => null, deployed =>
> qabb-qa-hdp-hbase1,16020,1503022958093;testtable,,1503022907686_0001.42d11cfe195b3cc4d08b2c078a687f6d
> ., replicaId => 1 } not in META, but deployed on
> qabb-qa-hdp-hbase1,16020,1503022958093
> 18 ERROR: No regioninfo in Meta or HDFS. { meta => null, hdfs => null,
> deployed =>
> qabb-qa-hdp-hbase1,16020,1503022958093;testtable,,1503022907686_0001.42d11cfe
>195b3cc4d08b2c078a687f6d., replicaId => 1 }
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20421) HBasecontext creates a connection but does not close it
[ https://issues.apache.org/jira/browse/HBASE-20421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-20421: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > HBasecontext creates a connection but does not close it > --- > > Key: HBASE-20421 > URL: https://issues.apache.org/jira/browse/HBASE-20421 > Project: HBase > Issue Type: Bug >Reporter: Yu Wang >Assignee: Yu Wang >Priority: Major > Labels: patch > Fix For: 3.0.0 > > Attachments: HBASE-20421.patch, HBASE-20421_master.patch, > HBASE-20421_master_1.patch, HBASE-20421_master_2.patch > > > HBasecontext creates a connection but does not turn it off -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19822) HFileCleaner threads stick around after shutdown stuck on queue#take
[ https://issues.apache.org/jira/browse/HBASE-19822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-19822: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > HFileCleaner threads stick around after shutdown stuck on queue#take > > > Key: HBASE-19822 > URL: https://issues.apache.org/jira/browse/HBASE-19822 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0-beta-2, 2.0.0 > > Attachments: HBASE-19822.master.001.patch, > HBASE-19822.master.addendum.patch > > > In thread dumps and in unit tests I see a lot of this: > Potentially hanging thread: > M:0;asf903:36485-HFileCleaner.large.0-1516277241553 > sun.misc.Unsafe.park(Native Method) > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:106) > > org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:232) > > org.apache.hadoop.hbase.master.cleaner.HFileCleaner$1.run(HFileCleaner.java:201) > Potentially hanging thread: > M:1;asf903:38770-HFileCleaner.large.0-1516277250526 > sun.misc.Unsafe.park(Native Method) > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:106) > > org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:232) > > org.apache.hadoop.hbase.master.cleaner.HFileCleaner$1.run(HFileCleaner.java:201) > Or long after a server has been killed in a unit test, I'll see this if a > thread dump: > Thread 1635 (M:0;asf903:36485-HFileCleaner.small.0-1516277241554): > State: WAITING > Blocked count: 0 > Waited count: 1 > Waiting on > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@626c1c7e > Stack: > sun.misc.Unsafe.park(Native Method) > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > > java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) > > org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:232) > > org.apache.hadoop.hbase.master.cleaner.HFileCleaner$2.run(HFileCleaner.java:216) > Thread 1634 (M:0;asf903:36485-HFileCleaner.large.0-1516277241553): > State: WAITING > Blocked count: 0 > Waited count: 2 > Waiting on > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2f961ff0 > Stack: > sun.misc.Unsafe.park(Native Method) > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) > org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:106) > > org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:232) > > org.apache.hadoop.hbase.master.cleaner.HFileCleaner$1.run(HFileCleaner.java:201) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19125) TestReplicator is flaky
[ https://issues.apache.org/jira/browse/HBASE-19125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-19125: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > TestReplicator is flaky > --- > > Key: HBASE-19125 > URL: https://issues.apache.org/jira/browse/HBASE-19125 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 1.4.1, 1.5.0, 2.0.0 > > > TestReplicator fails now and again. I had a look at the test. This is > something I contributed a while back but looking at it again it needs a > different approach. I'm going to disable it for now until this issue is > resolved. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18404) Small typo on ACID documentation page
[ https://issues.apache.org/jira/browse/HBASE-18404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-18404: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Small typo on ACID documentation page > - > > Key: HBASE-18404 > URL: https://issues.apache.org/jira/browse/HBASE-18404 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 1.3.1 >Reporter: Michael Crutcher >Assignee: Coral >Priority: Trivial > Fix For: 2.0.0 > > Attachments: HBASE-18404.patch > > > I noticed a couple of occurrences of the "word" wholely on the ACID semantics > doc page (https://hbase.apache.org/acid-semantics.html) > This should be "wholly". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-17909) Redundant exclusion of jruby-complete in pom of hbase-spark
[ https://issues.apache.org/jira/browse/HBASE-17909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-17909: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Redundant exclusion of jruby-complete in pom of hbase-spark > --- > > Key: HBASE-17909 > URL: https://issues.apache.org/jira/browse/HBASE-17909 > Project: HBase > Issue Type: Bug > Components: spark >Reporter: Xiang Li >Assignee: Xiang Li >Priority: Minor > Labels: easyfix, spark > Fix For: 3.0.0 > > Attachments: HBASE-17909.master.000.patch > > > In pom.xml of hbase-spark, jruby-complete is excluded from some of HBase > child modules, there are 2 issues: > 1. jruby-complete is excluded from the following child modules, > - hbase-client > - hbase-hadoop-compat > - hbase-hadoop2-compat > but those child modules does not introduce jruby-complete. jruby-complete is > only introduced by hbase-shell, and by transitive dependency, it is also > introduced by hbase-assembly, as hbase-assembly depends on hbase-shell. > But none of 3 modules listed above introduces jruby-complete. > 2. The exclusion of jruby-complete is stated twice in each of the 3 child > modules listed in 1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20134) support scripts use hard-coded /tmp
[
https://issues.apache.org/jira/browse/HBASE-20134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-20134:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> support scripts use hard-coded /tmp
> ---
>
> Key: HBASE-20134
> URL: https://issues.apache.org/jira/browse/HBASE-20134
> Project: HBase
> Issue Type: Bug
> Components: website
>Reporter: Mike Drob
>Assignee: Sean Busbey
>Priority: Minor
> Fix For: 1.3.2, 1.5.0, 1.2.7, 1.4.3, 2.0.0
>
> Attachments: HBASE-20134.0.patch
>
>
> {code}
> if [ -z "${working_dir}" ]; then
> echo "[DEBUG] defaulting to creating a directory in /tmp"
> working_dir=/tmp
> while [[ -e ${working_dir} ]]; do
> working_dir=/tmp/hbase-generate-website-${RANDOM}.${RANDOM}
> done
> mkdir "${working_dir}"
> else
> {code}
> This should likely use {{$TMPDIR}} or {{mktemp -d}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18390) Sleep too long when finding region location failed
[ https://issues.apache.org/jira/browse/HBASE-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-18390: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Sleep too long when finding region location failed > -- > > Key: HBASE-18390 > URL: https://issues.apache.org/jira/browse/HBASE-18390 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 1.3.1, 1.2.6, 1.1.11, 2.0.0-alpha-1 >Reporter: Phil Yang >Assignee: Phil Yang >Priority: Major > Fix For: 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2, 1.1.12, 2.0.0 > > Attachments: HBASE-18390.v01.patch, HBASE-18390.v02.patch, > HBASE-18390.v03.patch > > > If RegionServerCallable#prepare failed when getRegionLocation, the location > in this callable object is null. And before we retry we will sleep. However, > when location is null we will sleep at least 10 seconds. And the request will > be failed directly if operation timeout is less than 10 seconds. I think it > is no need to keep MIN_WAIT_DEAD_SERVER logic. Use backoff sleeping logic is > ok for most cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19884) BucketEntryGroup's equals, hashCode and compareTo methods are not consistent
[ https://issues.apache.org/jira/browse/HBASE-19884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-19884: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > BucketEntryGroup's equals, hashCode and compareTo methods are not consistent > > > Key: HBASE-19884 > URL: https://issues.apache.org/jira/browse/HBASE-19884 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-beta-1 >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Major > Fix For: 2.0.0-beta-2, 2.0.0 > > Attachments: HBASE-19884.master.001.patch, > HBASE-19884.master.001.patch, HBASE-19884.master.001.patch, > HBASE-19884.master.002.patch, HBASE-19884.master.003.patch > > > BucketEntryGroup currently uses different fields to calculate compareTo, > equals and hasCode. > In some cases !a.equals(b) but a.compareTo(b) == 0. Javadoc of Comparator > recommends that natural orderings be consistent with equals. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18225) Fix findbugs regression calling toString() on an array
[
https://issues.apache.org/jira/browse/HBASE-18225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-18225:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> Fix findbugs regression calling toString() on an array
> --
>
> Key: HBASE-18225
> URL: https://issues.apache.org/jira/browse/HBASE-18225
> Project: HBase
> Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Trivial
> Fix For: 2.0.0
>
> Attachments: HBASE-18225.001.patch
>
>
> Looks like we got a findbugs warning as a result of HBASE-18166
> {code}
> diff --git
> a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
>
> b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
> index 1d04944250..b7e0244aa2 100644
> ---
> a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
> +++
> b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
> @@ -2807,8 +2807,8 @@ public class RSRpcServices implements
> HBaseRPCErrorHandler,
> HRegionInfo hri = rsh.s.getRegionInfo();
> // Yes, should be the same instance
> if (regionServer.getOnlineRegion(hri.getRegionName()) != rsh.r) {
> - String msg = "Region was re-opened after the scanner" + scannerName +
> " was created: "
> - + hri.getRegionNameAsString();
> + String msg = "Region has changed on the scanner " + scannerName + ":
> regionName="
> + + hri.getRegionName() + ", scannerRegionName=" + rsh.r;
> {code}
> Looks like {{hri.getRegionNameAsString()}} was unintentionally changed to
> {{hri.getRegionName()}}, [~syuanjiang]/[~stack]?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18475) MasterProcedureScheduler incorrectly passes null Procedure to table locking
[
https://issues.apache.org/jira/browse/HBASE-18475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-18475:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> MasterProcedureScheduler incorrectly passes null Procedure to table locking
> ---
>
> Key: HBASE-18475
> URL: https://issues.apache.org/jira/browse/HBASE-18475
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.0.0-alpha-2, 2.0.0
>
> Attachments: HBASE-18475.0.patch
>
>
> Occasionally I get a series of these in tests during shutdown
> {code}
> 2017-07-27 16:24:26,774 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=37201]
> master.MasterRpcServices(1011): Checking to see if procedure is done
> procId=506
> 2017-07-27 16:24:26,777 INFO [main] client.HBaseAdmin$TableFuture(3591):
> Operation: CREATE, Table Name: default:foo failed with foo
> 2017-07-27 16:24:26,782 INFO
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=37201]
> master.HMaster$4(1755): Client=jenkins//172.26.21.67 create 'foo', {NAME =>
> 'family_1', VERSIONS => '1', EVICT_BLOCKS_ON_CLOSE => 'false',
> KEEP_DELETED_CELLS => 'FALSE', CACHE_DATA_ON_WRITE => 'false',
> DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', CACHE_INDEX_ON_WRITE =>
> 'false', IN_MEMORY => 'false', CACHE_BLOOMS_ON_WRITE => 'false',
> PREFETCH_BLOCKS_ON_OPEN => 'false', COMPRESSION => 'NONE', CACHE_DATA_IN_L1
> => 'false', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}
> 2017-07-27 16:24:26,884 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=37201]
> procedure2.ProcedureExecutor(792): Stored pid=507,
> state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=foo
> 2017-07-27 16:24:26,887 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=37201]
> master.MasterRpcServices(1011): Checking to see if procedure is done
> procId=507
> 2017-07-27 16:24:26,890 INFO [ProcExecWrkr-5]
> procedure2.ProcedureExecutor(1261): Rolled back pid=507, state=ROLLEDBACK,
> exception=org.apache.hadoop.hbase.TableExistsException via
> master-create-table:org.apache.hadoop.hbase.TableExistsException: foo;
> CreateTableProcedure table=foo exec-time=106msec
> 2017-07-27 16:24:26,890 WARN [ProcExecWrkr-5]
> procedure2.ProcedureExecutor$WorkerThread(1668): Worker terminating
> UNNATURALLY null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.requireTableExclusiveLock(MasterProcedureScheduler.java:590)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.access$300(MasterProcedureScheduler.java:106)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler$TableQueue.requireExclusiveLock(MasterProcedureScheduler.java:582)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.doPoll(MasterProcedureScheduler.java:215)
> at
> org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.dequeue(MasterProcedureScheduler.java:203)
> at
> org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:145)
> at
> org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:119)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1641)
> 2017-07-27 16:24:26,890 DEBUG [ProcExecWrkr-5]
> procedure2.ProcedureExecutor$WorkerThread(1670): Worker terminated.
> {code}
> Eventually all the workers are done.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20290) Typo in enable_table_replication error message
[
https://issues.apache.org/jira/browse/HBASE-20290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-20290:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> Typo in enable_table_replication error message
> --
>
> Key: HBASE-20290
> URL: https://issues.apache.org/jira/browse/HBASE-20290
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 1.2.6
>Reporter: Peter Somogyi
>Assignee: Gabor Bota
>Priority: Trivial
> Labels: beginner
> Fix For: 1.2.7
>
> Attachments: HBASE-20290.branch-1.2.001.patch
>
>
>
> Typo: comapred
> {noformat}
> hbase(main):020:0> enable_table_replication 'repl'
> ERROR: Table repl exists in peer cluster 1, but the table descriptors are not
> same when comapred with source cluster. Thus can not enable the table's
> replication switch.{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20521) TableOutputFormat.checkOutputSpecs conf checking sequence cause pig script run fail
[ https://issues.apache.org/jira/browse/HBASE-20521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-20521: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > TableOutputFormat.checkOutputSpecs conf checking sequence cause pig script > run fail > --- > > Key: HBASE-20521 > URL: https://issues.apache.org/jira/browse/HBASE-20521 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 2.0.0 > Environment: spark 2.2.1, hbase 2.0.0 >Reporter: Michael Jin >Assignee: Michael Jin >Priority: Major > Fix For: 3.0.0, 2.1.0, 2.0.1 > > Attachments: HBASE-20521.master.001.patch, > HBASE-20521.master.002.patch > > > HBASE-20295 fix null point exception of "conf" member variable, add > "context.getConfiguration()" in case when "conf" object was not been properly > initialized, and put it into the first priority checking sequence, this code > change affect user call "setConf" explicitly initialize "conf" object in > TableOutputFormat object, proposal to change checking sequence, use "conf" > object from "getConf" method first . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-18590) branch-1.4 needs a Jenkins commit build job
[ https://issues.apache.org/jira/browse/HBASE-18590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-18590: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > branch-1.4 needs a Jenkins commit build job > --- > > Key: HBASE-18590 > URL: https://issues.apache.org/jira/browse/HBASE-18590 > Project: HBase > Issue Type: Bug >Reporter: Jerry He >Assignee: Ted Yu >Priority: Critical > > The current HBase-1.4 job is actually branch-1. > https://builds.apache.org/job/HBase-1.4/ > Need a separate job for branch-1.4. And rename the current job to HBase-1.5. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-20675) The scan parameter include_stop_row=true does not work
[
https://issues.apache.org/jira/browse/HBASE-20675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gavin updated HBASE-20675:
--
Comment: was deleted
(was: A comment with security level 'jira-users' was removed.)
> The scan parameter include_stop_row=true does not work
> --
>
> Key: HBASE-20675
> URL: https://issues.apache.org/jira/browse/HBASE-20675
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.0.0
> Environment: HBase-Client Version: 2.0.0
> HBase Version: 1.2.0-cdh5.7.5
>Reporter: Jermy Li
>Assignee: Peter Somogyi
>Priority: Major
>
> we use
> [withStopRow|https://hbase.apache.org/2.0/apidocs/org/apache/hadoop/hbase/client/Scan.html#withStopRow-byte:A-boolean-]
> API to scan a rowkey range [startrow, stoprow], both inclusive, but the
> server can't return the last row including stoprow, however there is no
> exception.
> For example, the there are the following rows in hbase:
> ||rowkey||CF+qualifier+value||
> |1|{value1}|
> |2|{value2}
> |3|{value3}|
> {code:java}
> // Do scan like this
> Scan scan = new Scan();
> scan.withStartRow(bytes(1), true);
> scan.withStopRow(bytes(3), true);
> {code}
> The result returned only contains the first two rows: “1” and “2”, no “3”.
> Thanks.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19728) Add lock to filesCompacting in all place.
[ https://issues.apache.org/jira/browse/HBASE-19728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-19728: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Add lock to filesCompacting in all place. > - > > Key: HBASE-19728 > URL: https://issues.apache.org/jira/browse/HBASE-19728 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 1.5.0, 2.0.0 >Reporter: binlijin >Assignee: binlijin >Priority: Major > Fix For: 1.3.2, 1.5.0, 1.2.7, 1.4.2, 2.0.0 > > Attachments: HBASE-19728.branch-1.001.patch, > HBASE-19728.master.001.patch, HBASE-19728.master.002.patch, > HBASE-19728.master.002.patch > > > We find regionserver abort with the following exception: > 2017-05-09 17:40:06,369 FATAL > [regionserver/hadoop0349.et2.tbsite.net/11.251.152.199:16020-shortCompactions-1493026663275] > regionserver.HRegionServer: ABORTING region server > hadoop0349.et2.tbsite.net,16020,1493026637177: > Thread[regionserver/hadoop0349.et2.tbsite.net/11.251.152.199:16020-shortCompactions-1493026663275,5,main] > throw uncaught exception > java.lang.ArrayIndexOutOfBoundsException > at java.lang.System.arraycopy(Native Method) > at java.util.ArrayList.batchRemove(ArrayList.java:726) > at java.util.ArrayList.removeAll(ArrayList.java:690) > at > org.apache.hadoop.hbase.regionserver.HStore.finishCompactionRequest(HStore.java:1666) > at > org.apache.hadoop.hbase.regionserver.HStore.cancelRequestedCompaction(HStore.java:1656) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:504) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1147) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622) > at java.lang.Thread.run(Thread.java:834) > 2017-05-08 21:15:31,979 FATAL > [regionserver/hadoop1191.et2.tbsite.net/11.251.159.40:16020-longCompactions-1494249331978] > regionserver.HRegionServer: ABORTING region server > hadoop1191.et2.tbsite.net,16020,1493196567798: > Thread[regionserver/hadoop1191.et2.tbsite.net/11.251.159.40:16020-longCompactions-1494249331978,5,main] > throw uncaught exception > java.lang.IllegalArgumentException > at > com.google.common.base.Preconditions.checkArgument(Preconditions.java:76) > at > org.apache.hadoop.hbase.regionserver.compactions.RatioBasedCompactionPolicy.getCurrentEligibleFiles(RatioBasedCompactionPolicy.java:64) > at > org.apache.hadoop.hbase.regionserver.compactions.RatioBasedCompactionPolicy.preSelectCompactionForCoprocessor(RatioBasedCompactionPolicy.java:72) > at > org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.preSelect(DefaultStoreEngine.java:117) > at > org.apache.hadoop.hbase.regionserver.HStore.requestCompaction(HStore.java:1542) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread.selectCompaction(CompactSplitThread.java:362) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread.access$200(CompactSplitThread.java:58) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:491) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1147) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622) > at java.lang.Thread.run(Thread.java:834) > HStore#finishCompactionRequest do not require any HStore#lock's lock so > HStore.replaceStoreFiles need to synchronized on filesCompacting. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19656) Disable TestAssignmentManagerMetrics for beta-1
[ https://issues.apache.org/jira/browse/HBASE-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-19656: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > Disable TestAssignmentManagerMetrics for beta-1 > --- > > Key: HBASE-19656 > URL: https://issues.apache.org/jira/browse/HBASE-19656 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0-beta-1, 2.0.0 > > Attachments: HBASE-19656.branch-2.001.patch > > > TestAssignmentManagerMetrics fails reliably but up on apache and locally. > [~balazs.meszaros] is working on the issue over in HBASE-19598. For now I'm > going to disable it because it seems to be only test that always fails on > branch-2. Will follow this w/ an issue to reenable for beta-2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19167) RowCounter not working on 2.0.0-alpha3
[ https://issues.apache.org/jira/browse/HBASE-19167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated HBASE-19167: -- Comment: was deleted (was: A comment with security level 'jira-users' was removed.) > RowCounter not working on 2.0.0-alpha3 > -- > > Key: HBASE-19167 > URL: https://issues.apache.org/jira/browse/HBASE-19167 > Project: HBase > Issue Type: Bug > Components: hadoop2, mapreduce, shell >Affects Versions: 2.0.0-alpha-3 > Environment: Ubuntu 16.04, Distributed 3-Node Deployment, all running > 2.0.0-alpha3 >Reporter: Silvan Heller >Priority: Blocker > Fix For: 2.0.0-beta-1, 2.0.0 > > > Steps to reproduce: > * Create a table 'w' (or any other name) > * Execute in shell: ./hbase org.apache.hadoop.hbase.mapreduce.RowCounter w > (Where 'w' is my tablename) > Expected Behavior: > * I receive how many rows my table has > Actual Behavior: > * Jobs fail because org.apache.commons.lang3.ArrayUtils is not present on > the classpath. Detailed Stacktrace below. > == STACKTRACE > 2017-11-03 10:26:26,773 INFO [main] mapreduce.Job: Task Id : > attempt_1507127903800_0002_m_01_2, Status : FAILED > Error: java.lang.ClassNotFoundException: org.apache.commons.lang3.ArrayUtils > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at org.apache.hadoop.hbase.HConstants.(HConstants.java:1144) > at > org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toScan(ProtobufUtil.java:1091) > at > org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.convertStringToScan(TableMapReduceUtil.java:576) > at > org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:122) > at > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76) > at > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:749) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
