date:20180307


 [ 
https://issues.apache.org/jira/browse/HBASE-20157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yu Li resolved HBASE-20157.
---
Resolution: Duplicate

> WAL file might get broken
> -
>
> Key: HBASE-20157
> URL: https://issues.apache.org/jira/browse/HBASE-20157
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 1.1.0
>Reporter: Zephyr Guo
>Assignee: Zephyr Guo
>Priority: Major
> Fix For: 2.0.0
>
>
> WAL file can get corrupted by HBASE-16824. 
> When calling Writer.close() and Writer.sync() in the same time, a HDFS 
> bug(HDFS-13243) will be triggered. And, if this did happen, the last block in 
> WAL will get broken(NN mark it as CorruptBlock).
> My purpose of reporting this scenario here is to help those who come across 
> the same problem like me. (HBASE-16824 has been fixed, though) 
> {panel:title=RS log}
> 2018-02-05 07:58:54,212 INFO 
> [regionserver/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com/10.0.0.218:16020.logRoller]
>  hdfs.DFSClient: Could not complete 
> /hbase/WALs/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com,16020,1517453470107/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com%2C16020%2C1517453470107.default.1517788719683
>  retrying...
> 2018-02-05 07:59:00,612 INFO 
> [regionserver/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com/10.0.0.218:16020.logRoller]
>  hdfs.DFSClient: Could not complete 
> /hbase/WALs/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com,16020,1517453470107/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com%2C16020%2C1517453470107.default.1517788719683
>  retrying...
> {panel}
> {panel:title=NN log}
> 2018-02-05 07:58:48,011 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* 
> fsync: 
> /hbase/WALs/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com,16020,1517453470107/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com%2C16020%2C1517453470107.default.1517788719683
>  for DFSClient_NONMAPREDUCE_1109936977_1
> 2018-02-05 07:58:48,011 INFO 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: BLOCK* 
> blk_1080650145_6909339\{UCState=COMMITTED, truncateBlock=null, 
> primaryNodeIndex=-1, 
> replicas=[ReplicaUC[[DISK]DS-a4e579e7-4721-4c22-9b61-f1d00b33c45f:NORMAL:10.0.0.218:50010|RBW],
>  
> ReplicaUC[[DISK]DS-5d3d7878-876d-4a5a-97bc-5535c4cf8d59:NORMAL:10.0.0.220:50010|RBW],
>  
> ReplicaUC[[DISK]DS-ccc314b2-e2ad-4c1f-99a5-a39e3677a83b:NORMAL:10.0.0.221:50010|RBW]]}
>  is not COMPLETE (ucState = COMMITTED, replication# = 0 < minimum = 2) in 
> file 
> /hbase/WALs/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com,16020,1517453470107/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com%2C16020%2C1517453470107.default.1517788719683
> 2018-02-05 07:58:48,111 INFO BlockStateChange: BLOCK 
> NameSystem.addToCorruptReplicasMap: blk_1080650145 added as corrupt on 
> 10.0.0.221:50010 by 
> hb-j5e517al6xib80rkb-005.hbase.rds.aliyuncs.com/10.0.0.221 because block is 
> COMMITTED and reported length 1957330 does not match length in block map 80594
> 2018-02-05 07:58:48,224 INFO BlockStateChange: BLOCK 
> NameSystem.addToCorruptReplicasMap: blk_1080650145 added as corrupt on 
> 10.0.0.218:50010 by 
> hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com/10.0.0.218 because block is 
> COMMITTED and reported length 1957330 does not match length in block map 80594
> 2018-02-05 07:58:48,224 INFO BlockStateChange: BLOCK 
> NameSystem.addToCorruptReplicasMap: blk_1080650145 added as corrupt on 
> 10.0.0.220:50010 by 
> hb-j5e517al6xib80rkb-003.hbase.rds.aliyuncs.com/10.0.0.220 because block is 
> COMMITTED and reported length 1957330 does not match length in block map 80594
> 2018-02-05 07:58:48,511 INFO 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: BLOCK* 
> blk_1080650145_6909339\{UCState=COMMITTED, truncateBlock=null, 
> primaryNodeIndex=-1, 
> replicas=[ReplicaUC[[DISK]DS-a4e579e7-4721-4c22-9b61-f1d00b33c45f:NORMAL:10.0.0.218:50010|RBW],
>  
> ReplicaUC[[DISK]DS-5d3d7878-876d-4a5a-97bc-5535c4cf8d59:NORMAL:10.0.0.220:50010|RBW],
>  
> ReplicaUC[[DISK]DS-ccc314b2-e2ad-4c1f-99a5-a39e3677a83b:NORMAL:10.0.0.221:50010|RBW]]}
>  is not COMPLETE (ucState = COMMITTED, replication# = 3 >= minimum = 2) in 
> file 
> /hbase/WALs/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com,16020,1517453470107/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com%2C16020%2C1517453470107.default.1517788719683
> {panel}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Reopened] (HBASE-20157) WAL file might get broken


 [ 
https://issues.apache.org/jira/browse/HBASE-20157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yu Li reopened HBASE-20157:
---

Please allow me to confirm, that you mean the problem reported here is already 
fixed through HBASE-16824, or the problem is caused by HBASE-16824?

If the former, then no need to create this JIRA (although we sincerely 
appreciate your generous sharing, I'd say it's duplicated by HBASE-16824). And 
if the later, it's a bug and we need to find the root cause and fix it instead 
of just close it (and we recommend to raise a thread in the @user mailing list 
first and open the JIRA after someone confirmed it's a bug and located the root 
cause).

Thanks.

> WAL file might get broken
> -
>
> Key: HBASE-20157
> URL: https://issues.apache.org/jira/browse/HBASE-20157
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 1.1.0
>Reporter: Zephyr Guo
>Assignee: Zephyr Guo
>Priority: Major
> Fix For: 2.0.0
>
>
> WAL file can get corrupted by HBASE-16824. 
> When calling Writer.close() and Writer.sync() in the same time, a HDFS 
> bug(HDFS-13243) will be triggered. And, if this did happen, the last block in 
> WAL will get broken(NN mark it as CorruptBlock).
> My purpose of reporting this scenario here is to help those who come across 
> the same problem like me. (HBASE-16824 has been fixed, though) 
> {panel:title=RS log}
> 2018-02-05 07:58:54,212 INFO 
> [regionserver/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com/10.0.0.218:16020.logRoller]
>  hdfs.DFSClient: Could not complete 
> /hbase/WALs/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com,16020,1517453470107/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com%2C16020%2C1517453470107.default.1517788719683
>  retrying...
> 2018-02-05 07:59:00,612 INFO 
> [regionserver/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com/10.0.0.218:16020.logRoller]
>  hdfs.DFSClient: Could not complete 
> /hbase/WALs/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com,16020,1517453470107/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com%2C16020%2C1517453470107.default.1517788719683
>  retrying...
> {panel}
> {panel:title=NN log}
> 2018-02-05 07:58:48,011 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* 
> fsync: 
> /hbase/WALs/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com,16020,1517453470107/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com%2C16020%2C1517453470107.default.1517788719683
>  for DFSClient_NONMAPREDUCE_1109936977_1
> 2018-02-05 07:58:48,011 INFO 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: BLOCK* 
> blk_1080650145_6909339\{UCState=COMMITTED, truncateBlock=null, 
> primaryNodeIndex=-1, 
> replicas=[ReplicaUC[[DISK]DS-a4e579e7-4721-4c22-9b61-f1d00b33c45f:NORMAL:10.0.0.218:50010|RBW],
>  
> ReplicaUC[[DISK]DS-5d3d7878-876d-4a5a-97bc-5535c4cf8d59:NORMAL:10.0.0.220:50010|RBW],
>  
> ReplicaUC[[DISK]DS-ccc314b2-e2ad-4c1f-99a5-a39e3677a83b:NORMAL:10.0.0.221:50010|RBW]]}
>  is not COMPLETE (ucState = COMMITTED, replication# = 0 < minimum = 2) in 
> file 
> /hbase/WALs/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com,16020,1517453470107/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com%2C16020%2C1517453470107.default.1517788719683
> 2018-02-05 07:58:48,111 INFO BlockStateChange: BLOCK 
> NameSystem.addToCorruptReplicasMap: blk_1080650145 added as corrupt on 
> 10.0.0.221:50010 by 
> hb-j5e517al6xib80rkb-005.hbase.rds.aliyuncs.com/10.0.0.221 because block is 
> COMMITTED and reported length 1957330 does not match length in block map 80594
> 2018-02-05 07:58:48,224 INFO BlockStateChange: BLOCK 
> NameSystem.addToCorruptReplicasMap: blk_1080650145 added as corrupt on 
> 10.0.0.218:50010 by 
> hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com/10.0.0.218 because block is 
> COMMITTED and reported length 1957330 does not match length in block map 80594
> 2018-02-05 07:58:48,224 INFO BlockStateChange: BLOCK 
> NameSystem.addToCorruptReplicasMap: blk_1080650145 added as corrupt on 
> 10.0.0.220:50010 by 
> hb-j5e517al6xib80rkb-003.hbase.rds.aliyuncs.com/10.0.0.220 because block is 
> COMMITTED and reported length 1957330 does not match length in block map 80594
> 2018-02-05 07:58:48,511 INFO 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: BLOCK* 
> blk_1080650145_6909339\{UCState=COMMITTED, truncateBlock=null, 
> primaryNodeIndex=-1, 
> replicas=[ReplicaUC[[DISK]DS-a4e579e7-4721-4c22-9b61-f1d00b33c45f:NORMAL:10.0.0.218:50010|RBW],
>  
> ReplicaUC[[DISK]DS-5d3d7878-876d-4a5a-97bc-5535c4cf8d59:NORMAL:10.0.0.220:50010|RBW],
>  
> ReplicaUC[[DISK]DS-ccc314b2-e2ad-4c1f-99a5-a39e3677a83b:NORMAL:10.0.0.221:50010|RBW]]}
>  is not COMPLETE (ucState = COMMITTED, replication# = 3 >= minimum = 2) in 
> file 
>

[jira] [Created] (HBASE-20159) Support using separate ZK quorums for client

Yu Li created HBASE-20159:
-

 Summary: Support using separate ZK quorums for client
 Key: HBASE-20159
 URL: https://issues.apache.org/jira/browse/HBASE-20159
 Project: HBase
  Issue Type: New Feature
Reporter: Yu Li
Assignee: Yu Li


Currently we are using the same zookeeper quorums for client and server, which 
makes us under risk that if some client connection boost exhausted zookeeper, 
RegionServer might abort due to zookeeper session loss. Actually we have 
suffered from this many times in production.

Here we propose to allow client to use different ZK quorums, through below 
settings:
{noformat}
hbase.client.zookeeper.quorum
hbase.client.zookeeper.property.clientPort
hbase.client.zookeeper.observer.mode
{noformat}

The first two are for specifying client zookeeper properties, and the third one 
indicating whether the client ZK nodes are in observer mode. If the client ZK 
are not observer nodes, HMaster will take responsibility to synchronize 
necessary meta information (such as meta location and master address, etc.) 
from server to client ZK



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HBASE-20158) Enhance regionserver self health check to avoid stale server

Yu Li created HBASE-20158:
-

 Summary: Enhance regionserver self health check to avoid stale 
server
 Key: HBASE-20158
 URL: https://issues.apache.org/jira/browse/HBASE-20158
 Project: HBase
  Issue Type: New Feature
Reporter: Yu Li
Assignee: Yu Li


Currently we have many good metrics to monitor our cluster status, such as 
totalCallTime/processCallTime/queueCallTime etc. But these metrics won't work 
if server got stale and the client call timed out, for example during RS fullgc 
or there're some bad disk on HDFS and the read IO got stuck.

We also have a periodic health check chore introduced by HBASE-7351 which allow 
us to launch some external script periodically to perform some self detection. 
However this won't work if the server's system resource has ran out, for 
example no new native thread could be created, no new network connection could 
be setup, etc. Notice that although no new thread could not be launched, 
running thread won't be affected so zookeeper session is still alive and RS 
still regarded as alive, but clients cannot access since no new connection 
could be setup.

Here we propose a new HealthChecker called DirectHealthChecker. In this new 
checker we won't launch any outer script, but picking some regions on the RS 
and send some rpc request to itself, regarding the server as unhealthy if the 
call failure ratio exceeds some limit, and send the metrics out to our 
monitoring system. More details please refer to the coming patch



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (HBASE-20157) WAL file might get broken

2018-03-07 Thread Zephyr Guo (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-20157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zephyr Guo resolved HBASE-20157.

Resolution: Fixed

> WAL file might get broken
> -
>
> Key: HBASE-20157
> URL: https://issues.apache.org/jira/browse/HBASE-20157
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 1.1.0
>Reporter: Zephyr Guo
>Assignee: Zephyr Guo
>Priority: Major
> Fix For: 2.0.0
>
>
> WAL file can get corrupted by HBASE-16824. 
> When calling Writer.close() and Writer.sync() in the same time, a HDFS 
> bug(HDFS-13243) will be triggered. And, if this did happen, the last block in 
> WAL will get broken(NN mark it as CorruptBlock).
> My purpose of reporting this scenario here is to help those who come across 
> the same problem like me. (HBASE-16824 has been fixed, though) 
> {panel:title=RS log}
> 2018-02-05 07:58:54,212 INFO 
> [regionserver/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com/10.0.0.218:16020.logRoller]
>  hdfs.DFSClient: Could not complete 
> /hbase/WALs/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com,16020,1517453470107/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com%2C16020%2C1517453470107.default.1517788719683
>  retrying...
> 2018-02-05 07:59:00,612 INFO 
> [regionserver/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com/10.0.0.218:16020.logRoller]
>  hdfs.DFSClient: Could not complete 
> /hbase/WALs/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com,16020,1517453470107/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com%2C16020%2C1517453470107.default.1517788719683
>  retrying...
> {panel}
> {panel:title=NN log}
> 2018-02-05 07:58:48,011 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* 
> fsync: 
> /hbase/WALs/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com,16020,1517453470107/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com%2C16020%2C1517453470107.default.1517788719683
>  for DFSClient_NONMAPREDUCE_1109936977_1
> 2018-02-05 07:58:48,011 INFO 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: BLOCK* 
> blk_1080650145_6909339\{UCState=COMMITTED, truncateBlock=null, 
> primaryNodeIndex=-1, 
> replicas=[ReplicaUC[[DISK]DS-a4e579e7-4721-4c22-9b61-f1d00b33c45f:NORMAL:10.0.0.218:50010|RBW],
>  
> ReplicaUC[[DISK]DS-5d3d7878-876d-4a5a-97bc-5535c4cf8d59:NORMAL:10.0.0.220:50010|RBW],
>  
> ReplicaUC[[DISK]DS-ccc314b2-e2ad-4c1f-99a5-a39e3677a83b:NORMAL:10.0.0.221:50010|RBW]]}
>  is not COMPLETE (ucState = COMMITTED, replication# = 0 < minimum = 2) in 
> file 
> /hbase/WALs/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com,16020,1517453470107/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com%2C16020%2C1517453470107.default.1517788719683
> 2018-02-05 07:58:48,111 INFO BlockStateChange: BLOCK 
> NameSystem.addToCorruptReplicasMap: blk_1080650145 added as corrupt on 
> 10.0.0.221:50010 by 
> hb-j5e517al6xib80rkb-005.hbase.rds.aliyuncs.com/10.0.0.221 because block is 
> COMMITTED and reported length 1957330 does not match length in block map 80594
> 2018-02-05 07:58:48,224 INFO BlockStateChange: BLOCK 
> NameSystem.addToCorruptReplicasMap: blk_1080650145 added as corrupt on 
> 10.0.0.218:50010 by 
> hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com/10.0.0.218 because block is 
> COMMITTED and reported length 1957330 does not match length in block map 80594
> 2018-02-05 07:58:48,224 INFO BlockStateChange: BLOCK 
> NameSystem.addToCorruptReplicasMap: blk_1080650145 added as corrupt on 
> 10.0.0.220:50010 by 
> hb-j5e517al6xib80rkb-003.hbase.rds.aliyuncs.com/10.0.0.220 because block is 
> COMMITTED and reported length 1957330 does not match length in block map 80594
> 2018-02-05 07:58:48,511 INFO 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: BLOCK* 
> blk_1080650145_6909339\{UCState=COMMITTED, truncateBlock=null, 
> primaryNodeIndex=-1, 
> replicas=[ReplicaUC[[DISK]DS-a4e579e7-4721-4c22-9b61-f1d00b33c45f:NORMAL:10.0.0.218:50010|RBW],
>  
> ReplicaUC[[DISK]DS-5d3d7878-876d-4a5a-97bc-5535c4cf8d59:NORMAL:10.0.0.220:50010|RBW],
>  
> ReplicaUC[[DISK]DS-ccc314b2-e2ad-4c1f-99a5-a39e3677a83b:NORMAL:10.0.0.221:50010|RBW]]}
>  is not COMPLETE (ucState = COMMITTED, replication# = 3 >= minimum = 2) in 
> file 
> /hbase/WALs/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com,16020,1517453470107/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com%2C16020%2C1517453470107.default.1517788719683
> {panel}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HBASE-20157) WAL file might get broken

2018-03-07 Thread Zephyr Guo (JIRA)

Zephyr Guo created HBASE-20157:
--

 Summary: WAL file might get broken
 Key: HBASE-20157
 URL: https://issues.apache.org/jira/browse/HBASE-20157
 Project: HBase
  Issue Type: Bug
  Components: wal
Affects Versions: 1.1.0
Reporter: Zephyr Guo
Assignee: Zephyr Guo
 Fix For: 2.0.0


WAL file can get corrupted by HBASE-16824. 
When calling Writer.close() and Writer.sync() in the same time, a HDFS 
bug(HDFS-13243) will be triggered. And, if this did happen, the last block in 
WAL will get broken(NN mark it as CorruptBlock).

My purpose of reporting this scenario here is to help those who come across the 
same problem like me. (HBASE-16824 has been fixed, though) 


{panel:title=RS log}


2018-02-05 07:58:54,212 INFO 
[regionserver/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com/10.0.0.218:16020.logRoller]
 hdfs.DFSClient: Could not complete 
/hbase/WALs/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com,16020,1517453470107/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com%2C16020%2C1517453470107.default.1517788719683
 retrying...
2018-02-05 07:59:00,612 INFO 
[regionserver/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com/10.0.0.218:16020.logRoller]
 hdfs.DFSClient: Could not complete 
/hbase/WALs/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com,16020,1517453470107/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com%2C16020%2C1517453470107.default.1517788719683
 retrying...
{panel}
{panel:title=NN log}


2018-02-05 07:58:48,011 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* fsync: 
/hbase/WALs/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com,16020,1517453470107/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com%2C16020%2C1517453470107.default.1517788719683
 for DFSClient_NONMAPREDUCE_1109936977_1
2018-02-05 07:58:48,011 INFO 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: BLOCK* 
blk_1080650145_6909339\{UCState=COMMITTED, truncateBlock=null, 
primaryNodeIndex=-1, 
replicas=[ReplicaUC[[DISK]DS-a4e579e7-4721-4c22-9b61-f1d00b33c45f:NORMAL:10.0.0.218:50010|RBW],
 
ReplicaUC[[DISK]DS-5d3d7878-876d-4a5a-97bc-5535c4cf8d59:NORMAL:10.0.0.220:50010|RBW],
 
ReplicaUC[[DISK]DS-ccc314b2-e2ad-4c1f-99a5-a39e3677a83b:NORMAL:10.0.0.221:50010|RBW]]}
 is not COMPLETE (ucState = COMMITTED, replication# = 0 < minimum = 2) in file 
/hbase/WALs/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com,16020,1517453470107/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com%2C16020%2C1517453470107.default.1517788719683
2018-02-05 07:58:48,111 INFO BlockStateChange: BLOCK 
NameSystem.addToCorruptReplicasMap: blk_1080650145 added as corrupt on 
10.0.0.221:50010 by hb-j5e517al6xib80rkb-005.hbase.rds.aliyuncs.com/10.0.0.221 
because block is COMMITTED and reported length 1957330 does not match length in 
block map 80594
2018-02-05 07:58:48,224 INFO BlockStateChange: BLOCK 
NameSystem.addToCorruptReplicasMap: blk_1080650145 added as corrupt on 
10.0.0.218:50010 by hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com/10.0.0.218 
because block is COMMITTED and reported length 1957330 does not match length in 
block map 80594
2018-02-05 07:58:48,224 INFO BlockStateChange: BLOCK 
NameSystem.addToCorruptReplicasMap: blk_1080650145 added as corrupt on 
10.0.0.220:50010 by hb-j5e517al6xib80rkb-003.hbase.rds.aliyuncs.com/10.0.0.220 
because block is COMMITTED and reported length 1957330 does not match length in 
block map 80594
2018-02-05 07:58:48,511 INFO 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: BLOCK* 
blk_1080650145_6909339\{UCState=COMMITTED, truncateBlock=null, 
primaryNodeIndex=-1, 
replicas=[ReplicaUC[[DISK]DS-a4e579e7-4721-4c22-9b61-f1d00b33c45f:NORMAL:10.0.0.218:50010|RBW],
 
ReplicaUC[[DISK]DS-5d3d7878-876d-4a5a-97bc-5535c4cf8d59:NORMAL:10.0.0.220:50010|RBW],
 
ReplicaUC[[DISK]DS-ccc314b2-e2ad-4c1f-99a5-a39e3677a83b:NORMAL:10.0.0.221:50010|RBW]]}
 is not COMPLETE (ucState = COMMITTED, replication# = 3 >= minimum = 2) in file 
/hbase/WALs/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com,16020,1517453470107/hb-j5e517al6xib80rkb-004.hbase.rds.aliyuncs.com%2C16020%2C1517453470107.default.1517788719683
{panel}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HBASE-20156) Allow regionserver to live during HDFS failure

Yu Li created HBASE-20156:
-

 Summary: Allow regionserver to live during HDFS failure
 Key: HBASE-20156
 URL: https://issues.apache.org/jira/browse/HBASE-20156
 Project: HBase
  Issue Type: New Feature
Reporter: Yu Li


Currently if something is wrong with HDFS, for example NN fencing or get into 
safe mode, RS will abort itself immediately after detecting it (such as log 
roll or flush fail). And if we have a large scale cluster with dense writing 
workload, there will be a huge amount of WAL to split and replay when HDFS is 
back, and the recovery time might be tens of minutes or even hours (actually we 
experienced this more than once in production, there're always some surprise 
like unstable power supply for NN that we never expected...).

Here we propose to add an option to allow RS not aborting during HDFS failure, 
instead we will throw exceptions to clients indicating we're out of service, 
while we could get recovered right after HDFS is back.

This will also make it possible to restart HDFS in some extreme case, and allow 
us to survive if anything wrong happened during HDFS upgrading.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Apache Hbase | Requesting for Invite |

2018-03-07 Thread Shiva Kumar SS

Hi Team,

It it immense pleasure to interact with Apache Hbase PMC Group, My name is
Shiva Kumar.

I have around 10+ Years of work experience in startups, MNC and have been
an entrepreneur once. here is my linkedin profile
https://www.linkedin.com/in/shivakumarss/

currently i am associated with Big data team where we extensively use
Hadoop, Hbase and spark for our use cases.

I have voluntarily went through the code and raised a minor enhancement
request by looking at the code, later realized it is already part of 1.3.2+.


https://issues.apache.org/jira/browse/HBASE-20056

Have submitted my Individual Contributor License Agreement (ICLA) to apache
and my name is in the http://people.apache.org/unlistedclas.html

I would love to be part of Apache Hbase PMC team as Individual contributor,
I would appreciate if that opportunity is given to me.

Awaiting for your response.


Regards,
Shiva Kumar SS

Re: [ANNOUNCE] New HBase committer Zach York

2018-03-07 Thread Yu Li

Congratulations, Zach!

Best Regards,
Yu

On 8 March 2018 at 06:13, Mike Drob  wrote:

> Congratulations, Zach!
>
> On Wed, Mar 7, 2018 at 4:03 PM, Andrew Purtell 
> wrote:
>
> > Congratulations and welcome Zach!
> >
> >
> > On Wed, Mar 7, 2018 at 8:27 AM, Sean Busbey  wrote:
> >
> > > On behalf of the Apache HBase PMC, I am pleased to announce that Zach
> > > York has accepted the PMC's invitation to become a committer on the
> > > project.
> > >
> > > We appreciate all of Zach's great work thus far and look forward to
> > > continued involvement.
> > >
> > > Please join me in congratulating Zach!
> > >
> >
> >
> >
> > --
> > Best regards,
> > Andrew
> >
> > Words like orphans lost among the crosstalk, meaning torn from truth's
> > decrepit hands
> >- A23, Crosstalk
> >
>

[jira] [Created] (HBASE-20155) update branch-2 version to 2.1.0-SNAPSHOT

2018-03-07 Thread Sean Busbey (JIRA)

Sean Busbey created HBASE-20155:
---

 Summary: update branch-2 version to 2.1.0-SNAPSHOT
 Key: HBASE-20155
 URL: https://issues.apache.org/jira/browse/HBASE-20155
 Project: HBase
  Issue Type: Sub-task
  Components: build, community
Reporter: Sean Busbey
Assignee: Sean Busbey
 Fix For: 2.1.0


now that we have a branch-2.0, branch-2 should be set to the next minor release 
version number



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [VOTE] First release candidate for HBase 1.3.2 (RC0) is available

2018-03-07 Thread Chia-Ping Tsai

-1 as TestEndToEndSplitTransaction always fails. branch-1.4+ don't encounter 
this error.

On 2018/03/07 21:58:10, Francis Liu  wrote: 
> Hi,
> 
> I'm happy to announce the first release candidate for Apache HBase 1.3.2 is
> available to download and testing.
> 
> Artifacts are available here:
> 
> *https://dist.apache.org/repos/dist/dev/hbase/hbase-1.3.2RC0/
> *
> 
> Maven artifacts are available in the staging repository:
> 
> *https://repository.apache.org/content/repositories/orgapachehbase-1202
> *
> 
> All artifacts are signed with my code signing key D646D9F3, which is also
> in the project KEYS file at
> 
> http://www.apache.org/dist/hbase/KEYS
> 
> these artifacts correspond to commit hash
> 
> 4fc36c6e4ee84091d02a7e26fc45bae3fcb0e30f tagged as 1.3.2RC0.
> 
> HBase 1.3.2 is the second maintenance release in the HBase 1.3.z release
> line, continuing on the theme of bringing a stable, reliable database to the
> Hadoop and NoSQL communities. This release includes 241 resolved issues
> since 1.3.1 released 11 months ago.
> 
> The full list of issues addressed is available at
> 
> *https://s.apache.org/aMSp *
> 
> and also in the CHANGES.txt file in the root directory of the source
> tarball.
> 
> Please take a few minutes to verify the release and vote on
> releasing it:
> 
> [ ] +1 Release this package as Apache HBase 1.3.2
> [ ] +0 no opinion
> [ ] -1 Do not release this package because...
> 
> This Vote will run for one week and close Wed Mar 14, 2018 22:00 PST.
> 
> Thanks
> 
> Also extra thanks to Andy for helping me with this release.
> 
> Francis
>

[jira] [Created] (HBASE-20154) Make sure all tests have a small/medium/large category

Duo Zhang created HBASE-20154:
-

 Summary: Make sure all tests have a small/medium/large category
 Key: HBASE-20154
 URL: https://issues.apache.org/jira/browse/HBASE-20154
 Project: HBase
  Issue Type: Task
  Components: build, test
Reporter: Duo Zhang


For now if a test does not have such category, it will be ignored by pre commit 
and nightly. We need to find a way to confirm that all test have a category.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HBASE-20153) enable error-prone analysis in precommit

2018-03-07 Thread Mike Drob (JIRA)

Mike Drob created HBASE-20153:
-

 Summary: enable error-prone analysis in precommit
 Key: HBASE-20153
 URL: https://issues.apache.org/jira/browse/HBASE-20153
 Project: HBase
  Issue Type: Bug
  Components: community
Reporter: Mike Drob


We've done a lot of work to get rid of the error-prone errors, we should make 
sure they stay out. Let's enable errorProne profile and analysis in precommit.

[~busbey] - I tried figuring out how to pass flags ({{-PerrorProne}} to the mvn 
compile precommit check but was unable to unravel that thread. Any help is 
appreciated.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: DISCUSSION: "branch RM" roles for non-released branches branch-1, branch-2, and master

>

For eg. Andrew will caretake branch-1.4, Stack will caretake branch-2.0

Not as I originally proposed. In this example, Andrew will caretake
branch-1 and Stack will caretake branch-2.  Andrew and Stack will be making
more minor release branches more often and patch releases will become less
frequent. For example, I'm going to be done with branch-1.4 in six months
and on to branch-1.5. (Hypothetically.) Anyone is welcome to RM those
branch-x.y. As long as someone is actively RMing branch-x.y it stays alive.
That's how we'd come to a consensus on what is long term stable.


On Wed, Mar 7, 2018 at 5:55 PM, Apekshit Sharma  wrote:

> I like the thought, continuing to brainstorm
> In this method, the following holds true, right?
> - Care taking "branch-X.Y" will require least effort and will by default
> fall onto the shoulders of RM for X.Y version.
> 
> For eg. Andrew will caretake
> branch-1.4, Stack will caretake branch-2.0, and so on.
> - Care taking "branch-X" will require a bit more effort and it's uncertain
> how to assign one to such branches.
>   One way is, whoever will do the next minor release can own it. But the
> caveat in that is,
>   a) It's hard to find RMs as such, this may make it more difficult since
> one will have to own up the task much ahead in advance. Or, the effect
> could be opposite, since this way gives more heads up for carrying out RM
> responsibilities. Would be certainly interesting to see how it works out.
>   b) For branch-1 which might be close to finish, there's uncertainty -
> assign care taker but no future release (wasted effort?), OR, no care taker
> until we decide that there will be future release (more like status
> quo). We can make the decision when we are at the fork. For now, seems like
> we have one who's willing to take care of branch-1 to shepherd it towards
> 1.5.0smile.
>
>   Other way is, once can just caretake a branch without singing up to do
> the next release.
>   I don't think we have to choose one way or another (in fact, we might not
> have the luxury) and instead, can go with the flow (i.e. as contributions
> show up).
>
> - Care taking for "master": The most onerous task since almost everything
> goes in it - all patches, new features, bug fixes, test breaking changes,
> etc.  Defining 'care taking' of this branch will be a bit more difficult.
> Might be too big for single person, maybe round robin the task among
> committers/contributors (generates another opportunity for contributors to
> sparkle and become committers)?
> Unless this one is solved, we are not rectifying the problem mentioned by
> Stack previously - "...avoiding the hell that was 2.0.0 where its taken
> near on a year to stabilize (first alpha was last year) because it has over
> 5k JIRAs in it".
>
> To end on positive note, i volunteer to care take branch-2 once 2.0 GA is
> out.
>
> -- Appy
>
> On Thu, Mar 8, 2018 at 6:10 AM, Mike Drob  wrote:
>
> > If somebody volunteers to be the caretaker for 1.5.0, is there an
> implicit
> > expectation that they would take on the responsibilities for branch-1 as
> > well?
> >
> > On Wed, Mar 7, 2018 at 5:12 PM, Stack  wrote:
> >
> > > (Moving discussion to DISCUSSION thread from "NOTICE: made branch-2.0
> > > from..." -- my fault for starting it in wrong place)
> > >
> > > A while back, Andrew made a PROPOSAL for 'branch RM's [1]. I like this
> > > suggestion. I see it as a means of avoiding the hell that was 2.0.0
> where
> > > its taken near on a year to stabilize (first alpha was last year)
> because
> > > it has over 5k JIRAs in it; RMs can help keep their branch tidy and
> free
> > of
> > > flotsam.
> > >
> > > Andrew added more to his notion of 'branch RM' over in the NOTICE
> > thread. I
> > > repeat it below:
> > >
> > > "For a while leading up to 1.4.0 I was effectively the branch-1 RM in
> > > practice. Slacked off a bit since, but I'd like to continue with that
> if
> > > you're agreeable. I think that branch RM role involves informally:
> > > - Keeping an eye on commits to branch. Periodic review of recent
> commits.
> > > Not acting as a gate on commits and not needing to be pinged or in the
> > loop
> > > for every commit, except perhaps for short periods of time around
> > branching
> > > for new minors.
> > > - Keeping an eye on test stability. Undertaking get well projects like
> > > bisecting and reverting offending commits to resolve test suite decay.
> > > - Periodically kicking off new minor releases. Depends on branch and
> need
> > > for what's on deck."
> > >
> > > Interested in what folks think.
> > > St.Ack
> > >
> > > 1. *https://lists.apache.org/thread.html/
> 247697bcfb97adeeec2d14241856ca
> > > 7a77a167c9efe0dbc83b0a746f@%3Cdev.hbase.apache.org%3E
> > >  > > 7a77a167c9efe0dbc83b0a746f@%3Cdev.hbase.apache.org%3E>*
> > >
> >
>
>
>
> --
>
> -- Appy
>



-- 
Best regards,

Re: DISCUSSION: "branch RM" roles for non-released branches branch-1, branch-2, and master

> If somebody volunteers to be the caretaker for 1.5.0, is there an
implicit expectation that they would take on the responsibilities for
branch-1 as well?

Not as I originally proposed.

We will get away from RMs per branch for e.g. branch-1.y and focus on RMs
for each branch-x (and master). We are wasting precious RM effort on
branches that only produce patch releases. We should be focusing this
limited attention on branches that produce minor releases, and increase the
cadence of minor releases. This does not preclude maintenance of "long term
stable" patch release branches, like branch-1.2.


On Wed, Mar 7, 2018 at 4:40 PM, Mike Drob  wrote:

> If somebody volunteers to be the caretaker for 1.5.0, is there an implicit
> expectation that they would take on the responsibilities for branch-1 as
> well?
>
> On Wed, Mar 7, 2018 at 5:12 PM, Stack  wrote:
>
> > (Moving discussion to DISCUSSION thread from "NOTICE: made branch-2.0
> > from..." -- my fault for starting it in wrong place)
> >
> > A while back, Andrew made a PROPOSAL for 'branch RM's [1]. I like this
> > suggestion. I see it as a means of avoiding the hell that was 2.0.0 where
> > its taken near on a year to stabilize (first alpha was last year) because
> > it has over 5k JIRAs in it; RMs can help keep their branch tidy and free
> of
> > flotsam.
> >
> > Andrew added more to his notion of 'branch RM' over in the NOTICE
> thread. I
> > repeat it below:
> >
> > "For a while leading up to 1.4.0 I was effectively the branch-1 RM in
> > practice. Slacked off a bit since, but I'd like to continue with that if
> > you're agreeable. I think that branch RM role involves informally:
> > - Keeping an eye on commits to branch. Periodic review of recent commits.
> > Not acting as a gate on commits and not needing to be pinged or in the
> loop
> > for every commit, except perhaps for short periods of time around
> branching
> > for new minors.
> > - Keeping an eye on test stability. Undertaking get well projects like
> > bisecting and reverting offending commits to resolve test suite decay.
> > - Periodically kicking off new minor releases. Depends on branch and need
> > for what's on deck."
> >
> > Interested in what folks think.
> > St.Ack
> >
> > 1. *https://lists.apache.org/thread.html/247697bcfb97adeeec2d14241856ca
> > 7a77a167c9efe0dbc83b0a746f@%3Cdev.hbase.apache.org%3E
> >  > 7a77a167c9efe0dbc83b0a746f@%3Cdev.hbase.apache.org%3E>*
> >
>



-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

Re: DISCUSSION: "branch RM" roles for non-released branches branch-1, branch-2, and master

2018-03-07 Thread Apekshit Sharma

I like the thought, continuing to brainstorm
In this method, the following holds true, right?
- Care taking "branch-X.Y" will require least effort and will by default
fall onto the shoulders of RM for X.Y version. For eg. Andrew will caretake
branch-1.4, Stack will caretake branch-2.0, and so on.
- Care taking "branch-X" will require a bit more effort and it's uncertain
how to assign one to such branches.
  One way is, whoever will do the next minor release can own it. But the
caveat in that is,
  a) It's hard to find RMs as such, this may make it more difficult since
one will have to own up the task much ahead in advance. Or, the effect
could be opposite, since this way gives more heads up for carrying out RM
responsibilities. Would be certainly interesting to see how it works out.
  b) For branch-1 which might be close to finish, there's uncertainty -
assign care taker but no future release (wasted effort?), OR, no care taker
until we decide that there will be future release (more like status
quo). We can make the decision when we are at the fork. For now, seems like
we have one who's willing to take care of branch-1 to shepherd it towards
1.5.0smile.

  Other way is, once can just caretake a branch without singing up to do
the next release.
  I don't think we have to choose one way or another (in fact, we might not
have the luxury) and instead, can go with the flow (i.e. as contributions
show up).

- Care taking for "master": The most onerous task since almost everything
goes in it - all patches, new features, bug fixes, test breaking changes,
etc.  Defining 'care taking' of this branch will be a bit more difficult.
Might be too big for single person, maybe round robin the task among
committers/contributors (generates another opportunity for contributors to
sparkle and become committers)?
Unless this one is solved, we are not rectifying the problem mentioned by
Stack previously - "...avoiding the hell that was 2.0.0 where its taken
near on a year to stabilize (first alpha was last year) because it has over
5k JIRAs in it".

To end on positive note, i volunteer to care take branch-2 once 2.0 GA is
out.

-- Appy

On Thu, Mar 8, 2018 at 6:10 AM, Mike Drob  wrote:

> If somebody volunteers to be the caretaker for 1.5.0, is there an implicit
> expectation that they would take on the responsibilities for branch-1 as
> well?
>
> On Wed, Mar 7, 2018 at 5:12 PM, Stack  wrote:
>
> > (Moving discussion to DISCUSSION thread from "NOTICE: made branch-2.0
> > from..." -- my fault for starting it in wrong place)
> >
> > A while back, Andrew made a PROPOSAL for 'branch RM's [1]. I like this
> > suggestion. I see it as a means of avoiding the hell that was 2.0.0 where
> > its taken near on a year to stabilize (first alpha was last year) because
> > it has over 5k JIRAs in it; RMs can help keep their branch tidy and free
> of
> > flotsam.
> >
> > Andrew added more to his notion of 'branch RM' over in the NOTICE
> thread. I
> > repeat it below:
> >
> > "For a while leading up to 1.4.0 I was effectively the branch-1 RM in
> > practice. Slacked off a bit since, but I'd like to continue with that if
> > you're agreeable. I think that branch RM role involves informally:
> > - Keeping an eye on commits to branch. Periodic review of recent commits.
> > Not acting as a gate on commits and not needing to be pinged or in the
> loop
> > for every commit, except perhaps for short periods of time around
> branching
> > for new minors.
> > - Keeping an eye on test stability. Undertaking get well projects like
> > bisecting and reverting offending commits to resolve test suite decay.
> > - Periodically kicking off new minor releases. Depends on branch and need
> > for what's on deck."
> >
> > Interested in what folks think.
> > St.Ack
> >
> > 1. *https://lists.apache.org/thread.html/247697bcfb97adeeec2d14241856ca
> > 7a77a167c9efe0dbc83b0a746f@%3Cdev.hbase.apache.org%3E
> >  > 7a77a167c9efe0dbc83b0a746f@%3Cdev.hbase.apache.org%3E>*
> >
>



-- 

-- Appy

Re: DISCUSSION: "branch RM" roles for non-released branches branch-1, branch-2, and master

If somebody volunteers to be the caretaker for 1.5.0, is there an implicit
expectation that they would take on the responsibilities for branch-1 as
well?

On Wed, Mar 7, 2018 at 5:12 PM, Stack  wrote:

> (Moving discussion to DISCUSSION thread from "NOTICE: made branch-2.0
> from..." -- my fault for starting it in wrong place)
>
> A while back, Andrew made a PROPOSAL for 'branch RM's [1]. I like this
> suggestion. I see it as a means of avoiding the hell that was 2.0.0 where
> its taken near on a year to stabilize (first alpha was last year) because
> it has over 5k JIRAs in it; RMs can help keep their branch tidy and free of
> flotsam.
>
> Andrew added more to his notion of 'branch RM' over in the NOTICE thread. I
> repeat it below:
>
> "For a while leading up to 1.4.0 I was effectively the branch-1 RM in
> practice. Slacked off a bit since, but I'd like to continue with that if
> you're agreeable. I think that branch RM role involves informally:
> - Keeping an eye on commits to branch. Periodic review of recent commits.
> Not acting as a gate on commits and not needing to be pinged or in the loop
> for every commit, except perhaps for short periods of time around branching
> for new minors.
> - Keeping an eye on test stability. Undertaking get well projects like
> bisecting and reverting offending commits to resolve test suite decay.
> - Periodically kicking off new minor releases. Depends on branch and need
> for what's on deck."
>
> Interested in what folks think.
> St.Ack
>
> 1. *https://lists.apache.org/thread.html/247697bcfb97adeeec2d14241856ca
> 7a77a167c9efe0dbc83b0a746f@%3Cdev.hbase.apache.org%3E
>  7a77a167c9efe0dbc83b0a746f@%3Cdev.hbase.apache.org%3E>*
>

DISCUSSION: "branch RM" roles for non-released branches branch-1, branch-2, and master

(Moving discussion to DISCUSSION thread from "NOTICE: made branch-2.0
from..." -- my fault for starting it in wrong place)

A while back, Andrew made a PROPOSAL for 'branch RM's [1]. I like this
suggestion. I see it as a means of avoiding the hell that was 2.0.0 where
its taken near on a year to stabilize (first alpha was last year) because
it has over 5k JIRAs in it; RMs can help keep their branch tidy and free of
flotsam.

Andrew added more to his notion of 'branch RM' over in the NOTICE thread. I
repeat it below:

"For a while leading up to 1.4.0 I was effectively the branch-1 RM in
practice. Slacked off a bit since, but I'd like to continue with that if
you're agreeable. I think that branch RM role involves informally:
- Keeping an eye on commits to branch. Periodic review of recent commits.
Not acting as a gate on commits and not needing to be pinged or in the loop
for every commit, except perhaps for short periods of time around branching
for new minors.
- Keeping an eye on test stability. Undertaking get well projects like
bisecting and reverting offending commits to resolve test suite decay.
- Periodically kicking off new minor releases. Depends on branch and need
for what's on deck."

Interested in what folks think.
St.Ack

1. 
*https://lists.apache.org/thread.html/247697bcfb97adeeec2d14241856ca7a77a167c9efe0dbc83b0a746f@%3Cdev.hbase.apache.org%3E
*

[jira] [Created] (HBASE-20152) [AMv2] DisableTableProcedure versus ServerCrashProcedure

stack created HBASE-20152:
-

Summary: [AMv2] DisableTableProcedure versus ServerCrashProcedure
Key: HBASE-20152
URL: https://issues.apache.org/jira/browse/HBASE-20152
Project: HBase
Issue Type: Bug
Components: amv2
Reporter: stack
Assignee: stack

Seeing a small spate of issues where disabled tables/regions are being
assigned. Usually they happen when a DisableTableProcedure is running
concurrent with a ServerCrashProcedure. See below. See associated HBASE-20131.
This is umbrella issue for fixing.

.h2 Deadlock
>From HBASE-20137, 'TestRSGroups is Flakey',
>https://issues.apache.org/jira/browse/HBASE-20137?focusedCommentId=16390325=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16390325

{code}
* SCP is running because a server was aborted in test.
* SCP starts AssignProcedure of region X from crashed server.
* DisableTable Procedure runs because test has finished and we're doing table
delete. Queues
* UnassignProcedure for region X.
* Disable Unassign gets Lock on region X first.
* SCP AssignProcedure tries to get lock, waits on lock.
* DisableTable Procedure UnassignProcedure RPC fails because server is down
(Thats why the SCP).
* Tries to expire the server it failed the RPC against. Fails (currently being
SCP'd).
* DisableTable Procedure Unassign is suspended. It is a suspend with lock on
region X held
* SCP can't run because lock on X is held
* Test timesout.
{code}

.h2 Delete of online Regions
Saw this in nightly failure #452 for branch-2 in
TestSplitTransactionOnCluster.org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster

{code}
* DisableTableProcedure is queued before SCP.
* DisableTableProcedure Unassign fails because can't RPC to crashed server and
can't expire.
* Unassign is Stuck in suspend.
* SCP runs and cleans up suspended Disable Unassign.
* SCP completes which includes assign of Disable Unassign region.
* Disable Unassign completes
* Disable completes.
* A scheduled Drop Table Procedure runs (its end of test).
* Succeeds deleting regions that are actually assigned (see above where SCP
assigned region).
{code}

--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: NOTICE: Made branch-2.0 from tip of branch-2; hbase-2.0.0 will be spawned from branch-2.0

> per an old Andrew Purtell suggestion that we do a left-shift on version
numbers when it comes to RM oversight
> I can't find his email. Please add to this thread if any of you know what
I'm on about (smile).

Here: https://lists.apache.org/thread.html/247697bcfb97adeeec2d14241856ca
7a77a167c9efe0dbc83b0a746f@%3Cdev.hbase.apache.org%3E

For a while leading up to 1.4.0 I was effectively the branch-1 RM in
practice. Slacked off a bit since, but I'd like to continue with that if
you're agreeable. I think that branch RM role involves informally:
- Keeping an eye on commits to branch. Periodic review of recent commits.
Not acting as a gate on commits and not needing to be pinged or in the loop
for every commit, except perhaps for short periods of time around branching
for new minors.
- Keeping an eye on test stability. Undertaking get well projects like
bisecting and reverting offending commits to resolve test suite decay.
- Periodically kicking off new minor releases. Depends on branch and need
for what's on deck.


On Wed, Mar 7, 2018 at 10:41 AM, Stack  wrote:

> Please get my approval committing to branch-2.0. I'm trying to keep the
> branch stable as we head toward our first 2.0.0 release candidate.
>
> Related, please do not treat branch-2 as a dumping ground going forward.
> I'll try and keep an eye on it too (per an old Andrew Purtell suggestion
> that we do a left-shift on version numbers when it comes to RM
> oversight[2]). Only bug-fixes and pre-discussed features should go on
> branch-2[1] to ship in minor 2.x releases. Master and the our future hbase3
> is where the new stuff should be targeted.
>
> Thanks,
> St.Ack
>
> 1. The only branch-2 'feature' I know of currently is the serial
> replication work Duo and crew are currently working on.
> 2. I can't find his email. Please add to this thread if any of you know
> what I'm on about (smile).
>



-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

Re: [ANNOUNCE] New HBase committer Peter Somogyi

Welcome, Peter!

On Fri, Feb 23, 2018 at 7:51 AM, Anoop John  wrote:

> Congrats Peter..
>
> Anoop
>
>
> On Friday, February 23, 2018, ramkrishna vasudevan <
> ramkrishna.s.vasude...@gmail.com> wrote:
>
> > Congratulations Peter !!!
> >
> > On Fri, Feb 23, 2018 at 3:40 PM, Peter Somogyi 
> > wrote:
> >
> > > Thank you very much everyone!
> > >
> > > On Thu, Feb 22, 2018 at 8:08 PM, Sean Busbey 
> wrote:
> > >
> > > > On behalf of the Apache HBase PMC, I am pleased to announce that
> Peter
> > > > Somogyi has accepted the PMC's invitation to become a committer on
> the
> > > > project.
> > > >
> > > > We appreciate all of Peter's great work thus far and look forward to
> > > > continued involvement.
> > > >
> > > > Please join me in congratulating Peter!
> > > >
> > >
> >
>

Re: Re: [ANNOUNCE] Please welcome Mike Drob to the HBase PMC

Thanks, all! I will strive to live up to the trust and responsibility
placed in me!

On Wed, Feb 28, 2018 at 11:44 PM, ramkrishna vasudevan <
ramkrishna.s.vasude...@gmail.com> wrote:

> Congratulations Mike!!!
>
> On Thu, Mar 1, 2018 at 9:52 AM, Yu Li  wrote:
>
> > Congratulations, Mike!
> >
> > Best Regards,
> > Yu
> >
> > On 1 March 2018 at 11:56, Jerry He  wrote:
> >
> > > Congrats,  Mike!
> > >
> > > On Wed, Feb 28, 2018 at 6:19 PM Allan Yang 
> wrote:
> > >
> > > >  Congratulations Mike!
> > > >
> > > > Best Regards
> > > > Allan Yang
> > > >
> > > > 2018-03-01 10:12 GMT+08:00 OpenInx :
> > > >
> > > > > Congratulations !
> > > > >
> > > > > On Thu, Mar 1, 2018 at 9:54 AM, Chunhui Shen 
> wrote:
> > > > >
> > > > > > Congrats！
> > > > > > At 2018-03-01 07:56:11, "Andrew Purtell" 
> > > wrote:
> > > > > > >Congrats Mike!
> > > > > > >
> > > > > > >On Wed, Feb 28, 2018 at 7:54 AM, Sean Busbey  >
> > > > wrote:
> > > > > > >
> > > > > > >> On behalf of the Apache HBase PMC I am pleased to announce
> that
> > > Mike
> > > > > > >> Drob has accepted our invitation to become a PMC member on the
> > > > Apache
> > > > > > >> HBase project. We appreciate Mike stepping up to take more
> > > > > > >> responsibility in the HBase project.
> > > > > > >>
> > > > > > >>
> > > > > > >> As a reminder, if anyone would like to nominate another person
> > as
> > > a
> > > > > > >> committer or PMC member, even if you are not currently a
> > committer
> > > > or
> > > > > > >> PMC member, you can always drop a note to
> > > priv...@hbase.apache.org
> > > > to
> > > > > > >> let us know!
> > > > > > >>
> > > > > > >> - busbey
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >--
> > > > > > >Best regards,
> > > > > > >Andrew
> > > > > > >
> > > > > > >Words like orphans lost among the crosstalk, meaning torn from
> > > truth's
> > > > > > >decrepit hands
> > > > > > >   - A23, Crosstalk
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > ==
> > > > > Openinx  blog : http://openinx.github.io
> > > > >
> > > > > TO BE A GREAT HACKER !
> > > > > ==
> > > > >
> > > >
> > >
> >
>

Re: 回复：Re: [ANNOUNCE] Please welcome Ashish Singhi to the HBase PMC

Well deserved, Ashish!

On Wed, Feb 28, 2018 at 11:45 PM, ramkrishna vasudevan <
ramkrishna.s.vasude...@gmail.com> wrote:

> Congratulations, Ashish!!
>
> On Thu, Mar 1, 2018 at 9:52 AM, Yu Li  wrote:
>
> > Congratulations, Ashish!
> >
> > Best Regards,
> > Yu
> >
> > On 1 March 2018 at 11:57, Jerry He  wrote:
> >
> > > Congrats, Ashish!
> > >
> > > On Wed, Feb 28, 2018 at 6:18 PM Allan Yang 
> wrote:
> > >
> > > >  Congratulations  Ashish!
> > > >
> > > > Best Regards
> > > > Allan Yang
> > > >
> > > > 2018-03-01 10:11 GMT+08:00 OpenInx :
> > > >
> > > > > Congratulations !
> > > > >
> > > > > On Thu, Mar 1, 2018 at 9:55 AM, Chunhui Shen 
> wrote:
> > > > >
> > > > > > Congrats！
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > At 2018-03-01 09:24:47, "岭秀"  wrote:
> > > > > > >Congratulations, Ashish!!  well-deserved
> > > > > > >> > On behalf of the Apache HBase PMC I am pleased to announce
> > that
> > > > > Ashish
> > > > > > >> > Singhi has accepted our invitation to become a PMC member on
> > the
> > > > > > >> > Apache HBase project. We appreciate Ashish stepping up to
> take
> > > > more
> > > > > > >> > responsibility in the HBase project.
> > > > > > >> >
> > > > > > >> > As a reminder, if anyone would like to nominate another
> person
> > > as
> > > > a
> > > > > > >> > committer or PMC member, even if you are not currently a
> > > committer
> > > > > or
> > > > > > >> > PMC member, you can always drop a note to
> > > > priv...@hbase.apache.org
> > > > > to
> > > > > > >> > let us know!
> > > > > > >> >
> > > > > > >> > - busbey
> > > > > > >> >
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> --
> > > > > > >> Best regards,
> > > > > > >> Andrew
> > > > > > >>
> > > > > > >> Words like orphans lost among the crosstalk, meaning torn from
> > > > truth's
> > > > > > >> decrepit hands
> > > > > > >>- A23, Crosstalk
> > > > > > >>
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > ==
> > > > > Openinx  blog : http://openinx.github.io
> > > > >
> > > > > TO BE A GREAT HACKER !
> > > > > ==
> > > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] New HBase committer Zach York

Congratulations, Zach!

On Wed, Mar 7, 2018 at 4:03 PM, Andrew Purtell  wrote:

> Congratulations and welcome Zach!
>
>
> On Wed, Mar 7, 2018 at 8:27 AM, Sean Busbey  wrote:
>
> > On behalf of the Apache HBase PMC, I am pleased to announce that Zach
> > York has accepted the PMC's invitation to become a committer on the
> > project.
> >
> > We appreciate all of Zach's great work thus far and look forward to
> > continued involvement.
> >
> > Please join me in congratulating Zach!
> >
>
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
>- A23, Crosstalk
>

Re: Re: [ANNOUNCE] Please welcome Guanghao Zhang to the HBase PMC

Well deserved, Guanghao!

On Thu, Mar 1, 2018 at 1:20 AM, Phil Yang  wrote:

> Congratulations!
>
> Thanks,
> Phil
>
>
> 2018-03-01 13:44 GMT+08:00 ramkrishna vasudevan <
> ramkrishna.s.vasude...@gmail.com>:
>
> > Congratulations, Guanghao!!
> >
> > On Thu, Mar 1, 2018 at 9:52 AM, Yu Li  wrote:
> >
> > > Congratulations, Guanghao!
> > >
> > > Best Regards,
> > > Yu
> > >
> > > On 1 March 2018 at 11:59, Jerry He  wrote:
> > >
> > > >   Congrats,  Guanghao!
> > > >
> > > > On Wed, Feb 28, 2018 at 6:17 PM Allan Yang 
> > wrote:
> > > >
> > > > >  Congratulations Guanghao!
> > > > >
> > > > > Best Regards
> > > > > Allan Yang
> > > > >
> > > > > 2018-03-01 10:11 GMT+08:00 OpenInx :
> > > > >
> > > > > > Congratulations !
> > > > > >
> > > > > > On Thu, Mar 1, 2018 at 9:54 AM, Chunhui Shen 
> > wrote:
> > > > > >
> > > > > > > Congrats！
> > > > > > >
> > > > > > >
> > > > > > > At 2018-03-01 08:29:22, "Esteban Gutierrez" <
> > este...@cloudera.com>
> > > > > > wrote:
> > > > > > > >Congrats, Guanghao!
> > > > > > > >
> > > > > > > >--
> > > > > > > >Cloudera, Inc.
> > > > > > > >
> > > > > > > >
> > > > > > > >On Wed, Feb 28, 2018 at 3:56 PM, Andrew Purtell <
> > > > apurt...@apache.org>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > >> Congratulations!
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Wed, Feb 28, 2018 at 7:55 AM, Sean Busbey <
> > bus...@apache.org
> > > >
> > > > > > wrote:
> > > > > > > >>
> > > > > > > >> > On behalf of the Apache HBase PMC I am pleased to announce
> > > that
> > > > > > > >> > Guanghao Zhang has accepted our invitation to become a PMC
> > > > member
> > > > > on
> > > > > > > >> > the Apache HBase project. We appreciate Guanghao stepping
> up
> > > to
> > > > > take
> > > > > > > >> > more responsibility in the HBase project.
> > > > > > > >> >
> > > > > > > >> > As a reminder, if anyone would like to nominate another
> > person
> > > > as
> > > > > a
> > > > > > > >> > committer or PMC member, even if you are not currently a
> > > > committer
> > > > > > or
> > > > > > > >> > PMC member, you can always drop a note to
> > > > > priv...@hbase.apache.org
> > > > > > to
> > > > > > > >> > let us know!
> > > > > > > >> >
> > > > > > > >> > - busbey
> > > > > > > >> >
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> --
> > > > > > > >> Best regards,
> > > > > > > >> Andrew
> > > > > > > >>
> > > > > > > >> Words like orphans lost among the crosstalk, meaning torn
> from
> > > > > truth's
> > > > > > > >> decrepit hands
> > > > > > > >>- A23, Crosstalk
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > ==
> > > > > > Openinx  blog : http://openinx.github.io
> > > > > >
> > > > > > TO BE A GREAT HACKER !
> > > > > > ==
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] New HBase committer Zach York

Congratulations and welcome Zach!


On Wed, Mar 7, 2018 at 8:27 AM, Sean Busbey  wrote:

> On behalf of the Apache HBase PMC, I am pleased to announce that Zach
> York has accepted the PMC's invitation to become a committer on the
> project.
>
> We appreciate all of Zach's great work thus far and look forward to
> continued involvement.
>
> Please join me in congratulating Zach!
>



-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

[VOTE] First release candidate for HBase 1.3.2 (RC0) is available

2018-03-07 Thread Francis Liu

Hi,

I'm happy to announce the first release candidate for Apache HBase 1.3.2 is
available to download and testing.

Artifacts are available here:

*https://dist.apache.org/repos/dist/dev/hbase/hbase-1.3.2RC0/
*

Maven artifacts are available in the staging repository:

*https://repository.apache.org/content/repositories/orgapachehbase-1202
*

All artifacts are signed with my code signing key D646D9F3, which is also
in the project KEYS file at

http://www.apache.org/dist/hbase/KEYS

these artifacts correspond to commit hash

4fc36c6e4ee84091d02a7e26fc45bae3fcb0e30f tagged as 1.3.2RC0.

HBase 1.3.2 is the second maintenance release in the HBase 1.3.z release
line, continuing on the theme of bringing a stable, reliable database to the
Hadoop and NoSQL communities. This release includes 241 resolved issues
since 1.3.1 released 11 months ago.

The full list of issues addressed is available at

*https://s.apache.org/aMSp *

and also in the CHANGES.txt file in the root directory of the source
tarball.

Please take a few minutes to verify the release and vote on
releasing it:

[ ] +1 Release this package as Apache HBase 1.3.2
[ ] +0 no opinion
[ ] -1 Do not release this package because...

This Vote will run for one week and close Wed Mar 14, 2018 22:00 PST.

Thanks

Also extra thanks to Andy for helping me with this release.

Francis

[jira] [Created] (HBASE-20151) Bug with SingleColumnValueFilter and FamilyFilter

2018-03-07 Thread Steven Sadowski (JIRA)

Steven Sadowski created HBASE-20151:
---

 Summary: Bug with SingleColumnValueFilter and FamilyFilter
 Key: HBASE-20151
 URL: https://issues.apache.org/jira/browse/HBASE-20151
 Project: HBase
  Issue Type: Bug
 Environment: MacOS 10.13.3

HBase 1.3.1
Reporter: Steven Sadowski


When running the following queries, the result is sometimes return correctly 
and other times incorrectly based on the qualifier queried:

Setup:
{code:java}
create 'test', 'a', 'b'
test = get_table 'test'

test.put '1', 'a:1', nil
test.put '1', 'a:10', nil
test.put '1', 'b:2', nil
{code}
 
 This query works fine when the SCVF's qualifier has length 1 (i.e. '1') :
{code:java}
>   test.scan({ FILTER => "( 
> SingleColumnValueFilter('a','1',=,'binary:',true,true) AND 
> FamilyFilter(=,'binary:b') )"})
ROW   COLUMN+CELL
 1column=b:2, timestamp=1520455888059, 
value=
1 row(s) in 0.0060 seconds
{code}
 

The query should return the same result when passed a qualifier of length 2 
(i.e. '10') :
{code:java}
> test.scan({ FILTER => "( 
> SingleColumnValueFilter('a','10',=,'binary:',true,true) AND 
> FamilyFilter(=,'binary:b') )"})
ROW   COLUMN+CELL
0 row(s) in 0.0110 seconds
{code}
However, in this case, it does not return any row (expected result would be to 
return the same result as the first query).

 

Removing the family filter while the qualifier is '10' yields expected results:
{code:java}
> test.scan({ FILTER => "( 
> SingleColumnValueFilter('a','10',=,'binary:',true,true) )"})
ROW   COLUMN+CELL
 1column=a:1, timestamp=1520455887954, 
value=
 1column=a:10, timestamp=1520455888024, 
value=
 1column=b:2, timestamp=1520455888059, 
value=
1 row(s) in 0.0140 seconds
{code}
 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [ANNOUNCE] New HBase committer Zach York

2018-03-07 Thread Esteban Gutierrez

Congrats, Zach!

--
Cloudera, Inc.


On Wed, Mar 7, 2018 at 12:37 PM, Peter Somogyi  wrote:

> Congratulations!
>
> On Wed, Mar 7, 2018 at 9:43 AM, Huaxiang Sun  wrote:
>
> > Congratulations, Zach!
> >
> >
> > > On Mar 7, 2018, at 9:26 AM, Umesh Agashe  wrote:
> > >
> > > Congratulations, Zach!
> > >
> > > On Wed, Mar 7, 2018 at 8:40 AM, Ted Yu  wrote:
> > >
> > >> Congratulations, Zach !
> > >>
> > >> On Wed, Mar 7, 2018 at 8:27 AM, Sean Busbey 
> wrote:
> > >>
> > >>> On behalf of the Apache HBase PMC, I am pleased to announce that Zach
> > >>> York has accepted the PMC's invitation to become a committer on the
> > >>> project.
> > >>>
> > >>> We appreciate all of Zach's great work thus far and look forward to
> > >>> continued involvement.
> > >>>
> > >>> Please join me in congratulating Zach!
> > >>>
> > >>
> >
> >
>

NOTICE: Made branch-2.0 from tip of branch-2; hbase-2.0.0 will be spawned from branch-2.0

Please get my approval committing to branch-2.0. I'm trying to keep the
branch stable as we head toward our first 2.0.0 release candidate.

Related, please do not treat branch-2 as a dumping ground going forward.
I'll try and keep an eye on it too (per an old Andrew Purtell suggestion
that we do a left-shift on version numbers when it comes to RM
oversight[2]). Only bug-fixes and pre-discussed features should go on
branch-2[1] to ship in minor 2.x releases. Master and the our future hbase3
is where the new stuff should be targeted.

Thanks,
St.Ack

1. The only branch-2 'feature' I know of currently is the serial
replication work Duo and crew are currently working on.
2. I can't find his email. Please add to this thread if any of you know
what I'm on about (smile).

Re: [ANNOUNCE] New HBase committer Zach York

2018-03-07 Thread Peter Somogyi

Congratulations!

On Wed, Mar 7, 2018 at 9:43 AM, Huaxiang Sun  wrote:

> Congratulations, Zach!
>
>
> > On Mar 7, 2018, at 9:26 AM, Umesh Agashe  wrote:
> >
> > Congratulations, Zach!
> >
> > On Wed, Mar 7, 2018 at 8:40 AM, Ted Yu  wrote:
> >
> >> Congratulations, Zach !
> >>
> >> On Wed, Mar 7, 2018 at 8:27 AM, Sean Busbey  wrote:
> >>
> >>> On behalf of the Apache HBase PMC, I am pleased to announce that Zach
> >>> York has accepted the PMC's invitation to become a committer on the
> >>> project.
> >>>
> >>> We appreciate all of Zach's great work thus far and look forward to
> >>> continued involvement.
> >>>
> >>> Please join me in congratulating Zach!
> >>>
> >>
>
>

[jira] [Resolved] (HBASE-20150) Make branch-2.0 from branch-2 for 2.0.0 release


 [ 
https://issues.apache.org/jira/browse/HBASE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-20150.
---
Resolution: Fixed
  Assignee: stack

Pushed branch-2.0 at 96a42b7359674e787bd4d2e48e4622e9f5861645

> Make branch-2.0 from branch-2 for 2.0.0 release
> ---
>
> Key: HBASE-20150
> URL: https://issues.apache.org/jira/browse/HBASE-20150
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
>Priority: Major
>
> Create branch from which we will cut 2.0.0.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HBASE-20150) Make branch-2.0 from branch-2 for 2.0.0 release

stack created HBASE-20150:
-

 Summary: Make branch-2.0 from branch-2 for 2.0.0 release
 Key: HBASE-20150
 URL: https://issues.apache.org/jira/browse/HBASE-20150
 Project: HBase
  Issue Type: Sub-task
Reporter: stack


Create branch from which we will cut 2.0.0.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (HBASE-19814) Release hbase-2.0.0-beta-2; "rolling upgrade" release


 [ 
https://issues.apache.org/jira/browse/HBASE-19814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-19814.
---
Resolution: Fixed

Resolving as done.

> Release hbase-2.0.0-beta-2; "rolling upgrade" release
> -
>
> Key: HBASE-19814
> URL: https://issues.apache.org/jira/browse/HBASE-19814
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[ANNOUNCE] Apache HBase 2.0.0-beta-2 is now available for download

2018-03-07 Thread stack

The HBase team is happy to announce the immediate availability of Apache
HBase 2.0.0-beta-2!

Apache HBase is an open-source, distributed, versioned, non-relational
database. Apache HBase gives you low latency random access to billions of
rows with millions of columns atop non-specialized hardware. To learn more
about HBase, see https://hbase.apache.org/.

Download through an ASF mirror:

https://www.apache.org/dyn/closer.lua/hbase/

(It may take a while to show up on a particular mirror site)

hbase-2.0.0-beta-2 is our second beta release. It includes all that was in
previous alphas
and the previous beta (new assignment manager, offheap read/write path,
in-memory
compactions, etc.).

hbase-2.0.0-beta-2 is a not-for-production preview of hbase-2.0.0. It is
meant for devs and downstreamers to test drive and flag us if we messed up
on anything ahead of our rolling
2.0.0 release candidates ("GAs").

The list of features addressed in 2.0.0 so far can be found here [3]. There
are thousands.
The list of ~2k+ fixes in 2.0.0 exclusively can be found here [4].

I've updated our overview doc. on the state of 2.0.0 [6]. Next up are
actual 2.0.0 release
candidates. Look for them in a week or so after we do up a bit more doc and
test. Here is
the list of what is outstanding against 2.0.0 [5]. Only blockers will hold
up release.

One known issue is that the User API has not been properly filtered so it
shows more than
just InterfaceAudience Public content (HBASE-19663, to be fixed).

For instructions on verifying ASF release downloads, please see

https://www.apache.org/dyn/closer.cgi#verify

Project member signature keys can be found at

https://www.apache.org/dist/hbase/KEYS

This release was tagged with rel/2.0.0-beta-2.

Thanks to all the contributors who made this release possible!

Yours,
The HBase Dev Team

3. https://goo.gl/scYjJr
4. https://goo.gl/dFFT8b
5. *https://issues.apache.org/jira/projects/HBASE/versions/12327188
*
6. https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4
z9iEu_ktczrlKHK8N4SZzs/

Re: [ANNOUNCE] New HBase committer Zach York

2018-03-07 Thread Huaxiang Sun

Congratulations, Zach!


> On Mar 7, 2018, at 9:26 AM, Umesh Agashe  wrote:
> 
> Congratulations, Zach!
> 
> On Wed, Mar 7, 2018 at 8:40 AM, Ted Yu  wrote:
> 
>> Congratulations, Zach !
>> 
>> On Wed, Mar 7, 2018 at 8:27 AM, Sean Busbey  wrote:
>> 
>>> On behalf of the Apache HBase PMC, I am pleased to announce that Zach
>>> York has accepted the PMC's invitation to become a committer on the
>>> project.
>>> 
>>> We appreciate all of Zach's great work thus far and look forward to
>>> continued involvement.
>>> 
>>> Please join me in congratulating Zach!
>>> 
>>

Re: [ANNOUNCE] New HBase committer Zach York

2018-03-07 Thread Umesh Agashe

Congratulations, Zach!

On Wed, Mar 7, 2018 at 8:40 AM, Ted Yu  wrote:

> Congratulations, Zach !
>
> On Wed, Mar 7, 2018 at 8:27 AM, Sean Busbey  wrote:
>
> > On behalf of the Apache HBase PMC, I am pleased to announce that Zach
> > York has accepted the PMC's invitation to become a committer on the
> > project.
> >
> > We appreciate all of Zach's great work thus far and look forward to
> > continued involvement.
> >
> > Please join me in congratulating Zach!
> >
>

[RESULT][VOTE] The first hbase-2.0.0-beta-2 Release Candidate is available for download

Vote passes with 4 binding +1s, 3 non-binding, and a +0.

Let me push out this release.

Thanks to all for the robust vote.

S


On Wed, Mar 7, 2018 at 9:12 AM, Stack  wrote:

> Thanks Mike. I'll remove the .md5 when I post the release.
>
> I'm +1.
>
> Ran 1B ITBLL and it passed (10B did not. Work to do).
>
> St.Ack
>
>
> On Tue, Mar 6, 2018 at 8:50 PM, Mike Drob  wrote:
>
>> +0, I didn't have enough time on this RC to run a more complete suite of
>> tests like I was hoping for.
>>
>> sign/sums ok
>> SHOULD NOT supply a MD5 checksum file (because MD5 is too broken). [1]
>>
>> compile src 8u151 ok
>> compile src 8u151 w/ error-prone NOT OK - HBASE-19987
>>
>> [1]: http://www.apache.org/dev/release-distribution#sigs-and-sums
>>
>>
>>
>> On Fri, Mar 2, 2018 at 5:40 PM, Stack  wrote:
>>
>> > The first release candidate for HBase 2.0.0-beta-2 is up at
>> >
>> >  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.0-beta-2.RC0/
>> >
>> > Maven artifacts are available from a staging directory here:
>> >
>> >   https://repository.apache.org/content/repositories/orgap
>> achehbase-1199
>> >
>> > All was signed with my key at 8ACC93D2 [1]
>> >
>> > I tagged the RC as 2.0.0-beta-2RC0.2 at
>> > 9e9b347d667e1fc6165c9f8ae5ae7052147e8895
>> >
>> > hbase-2.0.0-beta-2 is a not-for-production preview of hbase-2.0.0. It is
>> > meant for devs and downstreamers to test drive and flag us if we messed
>> up
>> > on anything ahead of our rolling
>> > actual 2.0.0 release candidates ("GAs").
>> >
>> > hbase-2.0.0-beta-2 is our second beta release. More than 200 fixes have
>> > gone in since
>> > beta-1. Unit tests generallly pass when run against hadoop2 and
>> hadoop3[5].
>> > It includes
>> > all that was in previous alphas and beta (new assignment manager,
>> offheap
>> > read/write
>> > path, in-memory compactions, etc).The list of features addressed in
>> 2.0.0
>> > so far can be
>> > found here [3]. There are thousands. The list of ~3k+ fixes in 2.0.0
>> > exclusively can be
>> > found here [4]. Our overview doc. on the state of 2.0.0 is at [6].
>> >
>> > This beta was supposed to have as its focus rolling upgrade from
>> hbase-1.x
>> > versions but
>> > this is work not complete (At this late stage, it is looking like it
>> will
>> > be a post-2.0.0 project).
>> >
>> > This is our last hbase-2.0.0 beta release. Next up, we'll be rolling an
>> > actual 2.0.0 release
>> > candidate. Look for this in a week or two after beta-2 goes out, after
>> > we've done more
>> > testing and documentation (and we fix issues raised by you all against
>> this
>> > beta).
>> >
>> > One known issue, still unaddressed, is that the User API has not been
>> > properly filtered
>> > so it shows more than just InterfaceAudience Public content
>> (HBASE-19663,
>> > to be fixed
>> > by release).
>> >
>> > Please take this beta for a spin. Please vote on whether it ok to put
>> out
>> > this RC as our second
>> > beta (Note CHANGES has not yet been updated). Let the VOTE be open for
>> at
>> > least 72 hours
>> > (Lets say Wednesday morning, March 7th).
>> >
>> > Thanks,
>> > Your 2.0.0 Release Manager
>> >
>> > 1. http://pgp.mit.edu/pks/lookup?op=get=0x9816C7FC8ACC93D2
>> > 3. https://goo.gl/scYjJr
>> > 4. https://goo.gl/dFFT8b
>> > 5. https://builds.apache.org/job/HBase%20Nightly/job/branch-2/
>> > 
>> > 6. https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4
>> > z9iEu_ktczrlKHK8N4SZzs/
>> >
>>
>
>

Re: [VOTE] The first hbase-2.0.0-beta-2 Release Candidate is available for download

Thanks Mike. I'll remove the .md5 when I post the release.

I'm +1.

Ran 1B ITBLL and it passed (10B did not. Work to do).

St.Ack


On Tue, Mar 6, 2018 at 8:50 PM, Mike Drob  wrote:

> +0, I didn't have enough time on this RC to run a more complete suite of
> tests like I was hoping for.
>
> sign/sums ok
> SHOULD NOT supply a MD5 checksum file (because MD5 is too broken). [1]
>
> compile src 8u151 ok
> compile src 8u151 w/ error-prone NOT OK - HBASE-19987
>
> [1]: http://www.apache.org/dev/release-distribution#sigs-and-sums
>
>
>
> On Fri, Mar 2, 2018 at 5:40 PM, Stack  wrote:
>
> > The first release candidate for HBase 2.0.0-beta-2 is up at
> >
> >  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.0-beta-2.RC0/
> >
> > Maven artifacts are available from a staging directory here:
> >
> >   https://repository.apache.org/content/repositories/orgapachehbase-1199
> >
> > All was signed with my key at 8ACC93D2 [1]
> >
> > I tagged the RC as 2.0.0-beta-2RC0.2 at
> > 9e9b347d667e1fc6165c9f8ae5ae7052147e8895
> >
> > hbase-2.0.0-beta-2 is a not-for-production preview of hbase-2.0.0. It is
> > meant for devs and downstreamers to test drive and flag us if we messed
> up
> > on anything ahead of our rolling
> > actual 2.0.0 release candidates ("GAs").
> >
> > hbase-2.0.0-beta-2 is our second beta release. More than 200 fixes have
> > gone in since
> > beta-1. Unit tests generallly pass when run against hadoop2 and
> hadoop3[5].
> > It includes
> > all that was in previous alphas and beta (new assignment manager, offheap
> > read/write
> > path, in-memory compactions, etc).The list of features addressed in 2.0.0
> > so far can be
> > found here [3]. There are thousands. The list of ~3k+ fixes in 2.0.0
> > exclusively can be
> > found here [4]. Our overview doc. on the state of 2.0.0 is at [6].
> >
> > This beta was supposed to have as its focus rolling upgrade from
> hbase-1.x
> > versions but
> > this is work not complete (At this late stage, it is looking like it will
> > be a post-2.0.0 project).
> >
> > This is our last hbase-2.0.0 beta release. Next up, we'll be rolling an
> > actual 2.0.0 release
> > candidate. Look for this in a week or two after beta-2 goes out, after
> > we've done more
> > testing and documentation (and we fix issues raised by you all against
> this
> > beta).
> >
> > One known issue, still unaddressed, is that the User API has not been
> > properly filtered
> > so it shows more than just InterfaceAudience Public content (HBASE-19663,
> > to be fixed
> > by release).
> >
> > Please take this beta for a spin. Please vote on whether it ok to put out
> > this RC as our second
> > beta (Note CHANGES has not yet been updated). Let the VOTE be open for at
> > least 72 hours
> > (Lets say Wednesday morning, March 7th).
> >
> > Thanks,
> > Your 2.0.0 Release Manager
> >
> > 1. http://pgp.mit.edu/pks/lookup?op=get=0x9816C7FC8ACC93D2
> > 3. https://goo.gl/scYjJr
> > 4. https://goo.gl/dFFT8b
> > 5. https://builds.apache.org/job/HBase%20Nightly/job/branch-2/
> > 
> > 6. https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4
> > z9iEu_ktczrlKHK8N4SZzs/
> >
>

[jira] [Created] (HBASE-20149) Purge dev javadoc from bin tarball (or make a separate tarball of javadoc)

stack created HBASE-20149:
-

 Summary: Purge dev javadoc from bin tarball (or make a separate 
tarball of javadoc)
 Key: HBASE-20149
 URL: https://issues.apache.org/jira/browse/HBASE-20149
 Project: HBase
  Issue Type: Sub-task
Reporter: stack
 Fix For: 2.0.0


The bin tarball is too fat (Chia-Ping and Josh noticed it on the beta-2 vote). 
A note to the dev list subsequently resulted in suggestion that we just purge 
dev javadoc (or even all javadoc) from bin tarball (Andrew). Sean was good w/ 
it and suggested perhaps we could do a javadoc only tgz. Let me look into this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [ANNOUNCE] New HBase committer Zach York

2018-03-07 Thread Ted Yu

Congratulations, Zach !

On Wed, Mar 7, 2018 at 8:27 AM, Sean Busbey  wrote:

> On behalf of the Apache HBase PMC, I am pleased to announce that Zach
> York has accepted the PMC's invitation to become a committer on the
> project.
>
> We appreciate all of Zach's great work thus far and look forward to
> continued involvement.
>
> Please join me in congratulating Zach!
>

[ANNOUNCE] New HBase committer Zach York

2018-03-07 Thread Sean Busbey

On behalf of the Apache HBase PMC, I am pleased to announce that Zach
York has accepted the PMC's invitation to become a committer on the
project.

We appreciate all of Zach's great work thus far and look forward to
continued involvement.

Please join me in congratulating Zach!

[jira] [Created] (HBASE-20148) Make serial replication as a option for a peer instead of a table

Duo Zhang created HBASE-20148:
-

 Summary: Make serial replication as a option for a peer instead of 
a table
 Key: HBASE-20148
 URL: https://issues.apache.org/jira/browse/HBASE-20148
 Project: HBase
  Issue Type: Sub-task
Reporter: Duo Zhang


Discussed with [~zghaobac] and [~openinx] offline, HBASE-20147 can only be 
solved by adding new stages when creating a new replication peer. It will also 
effect he processing even if you do not have any table with serial replication. 
And also, the new logic introduced in ReplicationSource related classes will 
also be executed even if you do not have serial replication tables.

So, we think that it maybe a better choice to move the option to 
ReplicationPeerConfig. Since if there is one table in the peer requires serial 
replication, then all the tables in this peer will be automatically 'serial' 
since the wal for these tables are mixed with the one which requires serial 
replication.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HBASE-20147) Serial replication will be stuck if we create a table with serial replication but add it a peer after there are region moves

Duo Zhang created HBASE-20147:
-

 Summary: Serial replication will be stuck if we create a table 
with serial replication but add it a peer after there are region moves
 Key: HBASE-20147
 URL: https://issues.apache.org/jira/browse/HBASE-20147
 Project: HBase
  Issue Type: Sub-task
Reporter: Duo Zhang


The start point for serial replication is that, if we are in the first range 
then we are safe to push. And we will record replication barrier when the 
replication scope is set to SERIAL even if the table is not contained in any 
peers. So it could happen that, we record several barriers in the meta already 
and then we add a peer which contains this table. So when replicating, we will 
find that we are not in the first range, and then the replication will be stuck.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HBASE-20146) Regions are stuck to get opened when WAL is disabled

2018-03-07 Thread Ashish Singhi (JIRA)

Ashish Singhi created HBASE-20146:
-

 Summary: Regions are stuck to get opened when WAL is disabled
 Key: HBASE-20146
 URL: https://issues.apache.org/jira/browse/HBASE-20146
 Project: HBase
  Issue Type: Bug
  Components: wal
Affects Versions: 1.3.1
Reporter: Ashish Singhi


On a running cluster we had set {{hbase.regionserver.hlog.enabled}} to false, 
to disable the WAL for complete cluster, after restarting HBase service, 
regions are not getting opened leading to HMaster abort as Namespace table 
regions are not getting assigned. 

jstack for region open:
{noformat}
"RS_OPEN_PRIORITY_REGION-BLR106595:16045-1" #159 prio=5 os_prio=0 
tid=0x7fdfa4341000 nid=0x419d waiting on condition [0x7fdfa0467000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x87554448> (a 
java.util.concurrent.CountDownLatch$Sync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
at org.apache.hadoop.hbase.wal.WALKey.getWriteEntry(WALKey.java:98)
at 
org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeMarker(WALUtil.java:131)
at 
org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeRegionEventMarker(WALUtil.java:88)
at 
org.apache.hadoop.hbase.regionserver.HRegion.writeRegionOpenMarker(HRegion.java:1026)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6849)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6803)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6774)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6730)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6681)
at 
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:363)
at 
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:129)
at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}
This used to work with HBase 1.0.2 version.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HBASE-20144) The shutdown of master will hang if there are no live region server