[jira] [Commented] (HBASE-16058) TestHRegion fails on 1.4 builds

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337578#comment-15337578
 ] 

Hudson commented on HBASE-16058:


FAILURE: Integrated in HBase-1.4 #224 (See 
[https://builds.apache.org/job/HBase-1.4/224/])
HBASE-16058 TestHRegion fails on 1.4 builds (enis: rev 
1a989a1964e8bc5af3bb1c48f41da9dd06bd1257)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java


> TestHRegion fails on 1.4 builds
> ---
>
> Key: HBASE-16058
> URL: https://issues.apache.org/jira/browse/HBASE-16058
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Affects Versions: 1.4.0
>Reporter: Mikhail Antonov
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.4.0
>
> Attachments: hbase-16058-v1.branch-1.patch
>
>
> Results :
> Failed tests: 
> org.apache.hadoop.hbase.regionserver.TestHRegion.testFlushSizeAccounting(org.apache.hadoop.hbase.regionserver.TestHRegion)
>   Run 1: TestHRegion.testFlushSizeAccounting:465 expected:<384> but was:<368>
>   Run 2: TestHRegion.testFlushSizeAccounting:465 expected:<384> but was:<368>
>   Run 3: TestHRegion.testFlushSizeAccounting:465 expected:<384> but was:<368>
> org.apache.hadoop.hbase.regionserver.TestHRegion.testMemstoreSizeAccountingWithFailedPostBatchMutate(org.apache.hadoop.hbase.regionserver.TestHRegion)
>   Run 1: TestHRegion.testMemstoreSizeAccountingWithFailedPostBatchMutate:434 
> memstoreSize should be incremented expected:<448> but was:<432>
>   Run 2: TestHRegion.testMemstoreSizeAccountingWithFailedPostBatchMutate:434 
> memstoreSize should be incremented expected:<448> but was:<432>
>   Run 3: TestHRegion.testMemstoreSizeAccountingWithFailedPostBatchMutate:434 
> memstoreSize should be incremented expected:<448> but was:<432>
> Tests run: 2567, Failures: 2, Errors: 0, Skipped: 56



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16061) Allow logging to a buffered console

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337579#comment-15337579
 ] 

Hudson commented on HBASE-16061:


FAILURE: Integrated in HBase-1.4 #224 (See 
[https://builds.apache.org/job/HBase-1.4/224/])
HBASE-16061 Allow logging to a buffered console (eclark: rev 
e721aa1a8ac2db2697d699ca553fad4b26b5e633)
* conf/log4j.properties
* hbase-common/src/main/java/org/apache/hadoop/hbase/AsyncConsoleAppender.java


> Allow logging to a buffered console
> ---
>
> Key: HBASE-16061
> URL: https://issues.apache.org/jira/browse/HBASE-16061
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.2.1
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16061.v1.patch, HBASE-16061.v2.patch
>
>
> We've seen some stalls when logging to the ConsoleLogger that's getting 
> redirected to a file. We should allow people to use a buffered console 
> appender to keep from having stalls when the logging disk is busy.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337580#comment-15337580
 ] 

Hudson commented on HBASE-16023:


FAILURE: Integrated in HBase-1.4 #224 (See 
[https://builds.apache.org/job/HBase-1.4/224/])
HBASE-16023 Fastpath for the FIFO rpcscheduler Adds an executor that (stack: 
rev 411e3cdb6d47e622bf3aaec1c19380ba42d22b21)
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestSimpleRpcScheduler.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/BalancedQueueRpcExecutor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/FifoWithFastPathBalancedQueueRpcExecutor.java


> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[Mikhail Antonov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337527#comment-15337527
 ] 

Mikhail Antonov commented on HBASE-16023:
-

ah! somehow I was pretty sure this was pushed a day ago.

> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14964) Backport HBASE-14901 to brach-1 - There is duplicated code to create/manage encryption keys

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337512#comment-15337512
 ] 

Hadoop QA commented on HBASE-14964:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 20s 
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 
59s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 22s 
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 59s 
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 
5s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
29s {color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 1s 
{color} | {color:red} hbase-server in branch-1 has 1 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 1s 
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 16s 
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s 
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 45s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 45s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 14s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 14s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
37s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
16m 17s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 13s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 42s 
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 95m 48s 
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
30s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 141m 37s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12778270/HBASE-14964.1.branch-1.patch
 |
| JIRA Issue | HBASE-14964 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux asf910.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337510#comment-15337510
 ] 

stack commented on HBASE-16023:
---

Pushed this patch to branch-1.3 sir.

> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14743) Add metrics around HeapMemoryManager

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337509#comment-15337509
 ] 

stack commented on HBASE-14743:
---

Turn on the tuner always or only show this bean when tuner is on (I think we 
should do the former for 2.0).

> Add metrics around HeapMemoryManager
> 
>
> Key: HBASE-14743
> URL: https://issues.apache.org/jira/browse/HBASE-14743
> Project: HBase
>  Issue Type: Improvement
>Reporter: Elliott Clark
>Assignee: Reid Chan
>Priority: Minor
> Attachments: HBASE-14743.001.patch, HBASE-14743.002.patch, 
> HBASE-14743.003.patch, HBASE-14743.004.patch, HBASE-14743.005.patch, 
> HBASE-14743.006.patch, HBASE-14743.007.patch, HBASE-14743.008.patch, 
> HBASE-14743.009.patch, HBASE-14743.009.rw3.patch, HBASE-14743.009.v2.patch, 
> Screen Shot 2016-06-16 at 5.39.13 PM.png
>
>
> it would be good to know how many invocations there have been.
> How many decided to expand memstore.
> How many decided to expand block cache.
> How many decided to do nothing.
> etc.
> When that's done use those metrics to clean up the tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16062) Improper error handling in WAL Reader/Writer creation

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337498#comment-15337498
 ] 

Ted Yu commented on HBASE-16062:


+1 on v2, pending QA.

> Improper error handling in WAL Reader/Writer creation
> -
>
> Key: HBASE-16062
> URL: https://issues.apache.org/jira/browse/HBASE-16062
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-16062-v1.patch, HBASE-16062-v2.patch
>
>
> If creation of WAL reader/ writer fails for some reason RS may leak hanging 
> socket in CLOSE_WAIT state. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16060) 1.x clients cannot access table state talking to 2.0 cluster

[Jerry He (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337492#comment-15337492
 ] 

Jerry He commented on HBASE-16060:
--

The Semantic Versioning, we say in major release (2.0), Client-Server wire 
Compatibility is N.
How the rolling upgrade going to required?

> 1.x clients cannot access table state talking to 2.0 cluster
> 
>
> Key: HBASE-16060
> URL: https://issues.apache.org/jira/browse/HBASE-16060
> Project: HBase
>  Issue Type: Bug
>Reporter: Enis Soztutar
> Fix For: 2.0.0
>
>
> Since table state is migrated to meta instead of zk in 2.0, 1.x clients 
> talking to 2.0 cluster cannot access the table state. This causes some weird 
> behavior since from a client perspective, {{Admin.isTableEnabled()}} and 
> {{Admin.isTableDisabled()}} both return false. 
> One option we can do is to add code in 1.x clients so that they can access 
> the table state in meta if needed. Otherwise, we can mirror the table state 
> in zk (while keeping meta as the source of truth) during 2.x lifecycle so 
> that any 1.x client can still work correctly. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Issue Comment Deleted] (HBASE-14743) Add metrics around HeapMemoryManager

[Reid Chan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Reid Chan updated HBASE-14743:
--
Comment: was deleted

(was: send files to me? I attach them)

> Add metrics around HeapMemoryManager
> 
>
> Key: HBASE-14743
> URL: https://issues.apache.org/jira/browse/HBASE-14743
> Project: HBase
>  Issue Type: Improvement
>Reporter: Elliott Clark
>Assignee: Reid Chan
>Priority: Minor
> Attachments: HBASE-14743.001.patch, HBASE-14743.002.patch, 
> HBASE-14743.003.patch, HBASE-14743.004.patch, HBASE-14743.005.patch, 
> HBASE-14743.006.patch, HBASE-14743.007.patch, HBASE-14743.008.patch, 
> HBASE-14743.009.patch, HBASE-14743.009.rw3.patch, HBASE-14743.009.v2.patch, 
> Screen Shot 2016-06-16 at 5.39.13 PM.png
>
>
> it would be good to know how many invocations there have been.
> How many decided to expand memstore.
> How many decided to expand block cache.
> How many decided to do nothing.
> etc.
> When that's done use those metrics to clean up the tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16064) delete backup command shows HDFS permission error when deleting the intended backup

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-16064:
---
Summary: delete backup command shows HDFS permission error when deleting 
the intended backup  (was: HBase shell delete backup command shows HDFS 
permission error, after successfully deleting the intended backup)

> delete backup command shows HDFS permission error when deleting the intended 
> backup
> ---
>
> Key: HBASE-16064
> URL: https://issues.apache.org/jira/browse/HBASE-16064
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Assignee: Ted Yu
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: 16064.v1.txt
>
>
> HBase delete backup command shows error, after successfully deleting the 
> intended backup
> {code}
> hbase@cluster-name:~$ hbase backup delete backup_1465950334243
> 2016-06-15 00:36:18,883 INFO  [main] util.BackupClientUtil: No data has been 
> found in 
> hdfs://cluster-name:8020/user/hbase/backup_1465950334243/default/table_ttx7w0jgw8.
> 2016-06-15 00:36:18,894 ERROR [main] util.BackupClientUtil: Cleaning up 
> backup data of backup_1465950334243 at hdfs://cluster-name:8020/user/hbase 
> failed due to Permission denied: user=hbase, access=WRITE, 
> inode="/user/hbase":hdfs:hdfs:drwxr-xr-x
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:216)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1827)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:92)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:3822)
>   at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1071)
>   at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:619)
>   at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
>   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
>   at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
> .
> {code}
> Backup has been successfully deleted but the backup root dir under 
> /user/hbase dir still persists
> {code}
> hbase@cluster-name:~$ hdfs dfs -ls /user/hbase
> Found 6 items
> drwx--   - hbase hbase  0 2016-06-15 00:26 /user/hbase/.staging
> drwxr-xr-x   - hbase hbase  0 2016-06-15 00:36 
> /user/hbase/backup_1465950334243
> drwxr-xr-x   - hbase hbase  0 2016-06-15 00:26 
> /user/hbase/hbase-staging
> {code}
> /user/hbase/backup_1465950334243 is now empty though



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16062) Improper error handling in WAL Reader/Writer creation

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-16062:
--
Status: Patch Available  (was: Open)

> Improper error handling in WAL Reader/Writer creation
> -
>
> Key: HBASE-16062
> URL: https://issues.apache.org/jira/browse/HBASE-16062
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-16062-v1.patch, HBASE-16062-v2.patch
>
>
> If creation of WAL reader/ writer fails for some reason RS may leak hanging 
> socket in CLOSE_WAIT state. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16062) Improper error handling in WAL Reader/Writer creation

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-16062:
--
Attachment: HBASE-16062-v2.patch

v2 patch.

> Improper error handling in WAL Reader/Writer creation
> -
>
> Key: HBASE-16062
> URL: https://issues.apache.org/jira/browse/HBASE-16062
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-16062-v1.patch, HBASE-16062-v2.patch
>
>
> If creation of WAL reader/ writer fails for some reason RS may leak hanging 
> socket in CLOSE_WAIT state. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16061) Allow logging to a buffered console

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337482#comment-15337482
 ] 

Hudson commented on HBASE-16061:


FAILURE: Integrated in HBase-Trunk_matrix #1067 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/1067/])
HBASE-16061 Allow logging to a buffered console (eclark: rev 
65a8d77433cf72ce67e8b5e2efb35f2c2603c349)
* conf/log4j.properties
* hbase-common/src/main/java/org/apache/hadoop/hbase/AsyncConsoleAppender.java


> Allow logging to a buffered console
> ---
>
> Key: HBASE-16061
> URL: https://issues.apache.org/jira/browse/HBASE-16061
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.2.1
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16061.v1.patch, HBASE-16061.v2.patch
>
>
> We've seen some stalls when logging to the ConsoleLogger that's getting 
> redirected to a file. We should allow people to use a buffered console 
> appender to keep from having stalls when the logging disk is busy.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15467) Remove 1.x/2.0 TableDescriptor incompatibility

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337481#comment-15337481
 ] 

Hudson commented on HBASE-15467:


FAILURE: Integrated in HBase-Trunk_matrix #1067 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/1067/])
HBASE-15467 Remove 1.x/2.0 TableDescriptor incompatibility (enis: rev 
bdb0cc8808af7c3d08af4a506f34b8341726b58e)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/SnapshotTestingUtils.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateTableProcedure.java
* 
hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CloneSnapshotProcedure.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureTestingUtility.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotManifest.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestTableDescriptorModificationFromClient.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDefaultMemStore.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
* hbase-protocol/src/main/protobuf/HBase.proto
* hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/TestTableDescriptor.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/util/HMerge.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/util/Merge.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransactionOnCluster.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/TestHColumnDescriptorDefaultVersions.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestMergeTool.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestFSTableDescriptors.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CreateTableProcedure.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java


> Remove 1.x/2.0 TableDescriptor incompatibility
> --
>
> Key: HBASE-15467
> URL: https://issues.apache.org/jira/browse/HBASE-15467
> Project: HBase
>  Issue Type: Bug
>  Components: master, regionserver
>Affects Versions: 2.0.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Fix For: 2.0.0
>
> Attachments: HBASE-15467-v0.patch, HBASE-15467-v1.patch, 
> hbase-15467_v2.patch
>
>
> I'm experimenting with Master on "2.0" and RSs on 1.x and the first problem 
> that I get is on createTable where the Master is trying to write the HTD as 
> TableDescriptor instead of TableSchema and the RS is not able to read it.
> Since TableDescriptor does nothing for now. I'd say we can remove it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16062) Improper error handling in WAL Reader/Writer creation

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-16062:
--
Status: Open  (was: Patch Available)

> Improper error handling in WAL Reader/Writer creation
> -
>
> Key: HBASE-16062
> URL: https://issues.apache.org/jira/browse/HBASE-16062
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-16062-v1.patch
>
>
> If creation of WAL reader/ writer fails for some reason RS may leak hanging 
> socket in CLOSE_WAIT state. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16062) Improper error handling in WAL Reader/Writer creation

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337479#comment-15337479
 ] 

Hadoop QA commented on HBASE-16062:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s 
{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 
36s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s 
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s 
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
15s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 7s 
{color} | {color:red} hbase-server in master has 1 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s 
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s 
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 47s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 47s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
15s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 1s 
{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git 
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
28m 32s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 89m 53s {color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
25s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 135m 5s {color} 
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.quotas.TestQuotaTableUtil |
| Timed out junit tests | 
org.apache.hadoop.hbase.replication.TestReplicationSyncUpToolWithBulkLoadedData 
|
|   | org.apache.hadoop.hbase.replication.TestReplicationKillMasterRSCompressed 
|
|   | org.apache.hadoop.hbase.wal.TestWALSplit |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12811491/HBASE-16062-v1.patch |
| JIRA Issue | HBASE-16062 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 

[jira] [Commented] (HBASE-14756) Break out ClientBackoffPolicy factors into configurable and stackable components

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337473#comment-15337473
 ] 

Hadoop QA commented on HBASE-14756:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 
4s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 26s 
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s 
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
25s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s 
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s 
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 26s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 26s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 17s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
26m 19s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 58s 
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
8s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 23s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12770606/HBASE-14756.patch |
| JIRA Issue | HBASE-14756 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 6717f0e |
| Default Java | 1.7.0_80 |
| Multi-JDK versions |  /home/jenkins/tools/java/jdk1.8.0:1.8.0 
/home/jenkins/jenkins-slave/tools/hudson.model.JDK/JDK_1.7_latest_:1.7.0_80 |
| findbugs | v3.0.0 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/2289/testReport/ |
| 

[jira] [Commented] (HBASE-16062) Improper error handling in WAL Reader/Writer creation

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337467#comment-15337467
 ] 

Vladimir Rodionov commented on HBASE-16062:
---

{quote}
What if stream.close() throws exception ?
{quote}

I think reader.close() will throw as well. If you want I can add try ... catch 
inside catch, [~te...@apache.org].

> Improper error handling in WAL Reader/Writer creation
> -
>
> Key: HBASE-16062
> URL: https://issues.apache.org/jira/browse/HBASE-16062
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-16062-v1.patch
>
>
> If creation of WAL reader/ writer fails for some reason RS may leak hanging 
> socket in CLOSE_WAIT state. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13354) Add documentation and tests for external block cache.

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-13354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-13354:

Fix Version/s: (was: 1.3.0)
   1.3.1

> Add documentation and tests for external block cache.
> -
>
> Key: HBASE-13354
> URL: https://issues.apache.org/jira/browse/HBASE-13354
> Project: HBase
>  Issue Type: Bug
>  Components: BlockCache, documentation, test
>Affects Versions: 2.0.0, 1.1.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.1
>
>
> The new memcached integration needs some documentation and some testing 
> showing how it works and what can go wrong.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15347) Update CHANGES.txt for 1.3

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337465#comment-15337465
 ] 

Sean Busbey commented on HBASE-15347:
-

doing the work to find corrections needed in jira is worth it, IMHO. over time 
we'll get better at keeping accurate track in JIRA, which I think is our only 
hope for keeping a good record.

> Update CHANGES.txt for 1.3
> --
>
> Key: HBASE-15347
> URL: https://issues.apache.org/jira/browse/HBASE-15347
> Project: HBase
>  Issue Type: Sub-task
>  Components: documentation
>Affects Versions: 1.3.0
>Reporter: Mikhail Antonov
>Assignee: Mikhail Antonov
> Fix For: 1.3.0
>
>
> Going to post the steps in preparing changes file for 1.3 here.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (HBASE-14025) Update CHANGES.txt for 1.2

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337460#comment-15337460
 ] 

Sean Busbey edited comment on HBASE-14025 at 6/18/16 2:55 AM:
--

{quote}
if we're considering the Jira to be a source of truth here, then "project = 
HBase AND fixVersion = 1.3.0 AND fixVersion not in (1.2.0, 1.2.1, 1.2.2) AND 
status in (Resolved, Closed) ORDER BY issuetype DESC, updated ASC" jira filter 
should basically give the list of jiras (if we assumed that fixVersion are set 
accurately), if the jira fixVersion was always accurate?
{quote}

I would just look at those jiras with fixVersion 1.3.0 and fixVersion not in 
1.2.0, 1.1.0, 1.0.0. After the .0 release, it's hard to know how maintenance 
releases in the prior minor release line marry up with the new minor release 
line.

{quote}
bq. "find the commit where your release line branched off from the previous 
release.
bq. $ git merge-base 1.1.0 branch-1.2
bq. 8166142b2e815fc6ab30c14a5a546cd242bf3b4c"
But branch-1.3 was cut off branch-1 with whatever was there in the moment, so I 
don't see how it applies?
{quote}

right, but the prior minor release lines were also cut off of it. you can find 
where the branch for the prior minor release left branch-1 using merge-base, 
which then lets you check the commits that continued on branch-1 afterwards to 
see which ones also made it into the prior minor release.

That is, after the branch that led to 1.2.0 came off of branch-1 some set of 
JIRAS still had commits on both branches. the ones that happened on both you 
need not include in the CHANGES file for 1.3.0 because they will already be 
listed under 1.2.0.

does that make sense? If not, could make a diagram maybe?



was (Author: busbey):
{quote}
if we're considering the Jira to be a source of truth here, then "project = 
HBase AND fixVersion = 1.3.0 AND fixVersion not in (1.2.0, 1.2.1, 1.2.2) AND 
status in (Resolved, Closed) ORDER BY issuetype DESC, updated ASC" jira filter 
should basically give the list of jiras (if we assumed that fixVersion are set 
accurately), if the jira fixVersion was always accurate?
{quote}

I would just look at those jiras with fixVersion 1.3.0 and fixVersion not in 
1.2.0, 1.1.0, 1.0.0. After the .0 release, it's hard to know how maintenance 
releases in the prior minor release line marry up with the new minor release 
line.

{quote}
bq. "find the commit where your release line branched off from the previous 
release.
$ git merge-base 1.1.0 branch-1.2
8166142b2e815fc6ab30c14a5a546cd242bf3b4c"
But branch-1.3 was cut off branch-1 with whatever was there in the moment, so I 
don't see how it applies?
{quote}

right, but the prior minor release lines were also cut off of it. you can find 
where the branch for the prior minor release left branch-1 using merge-base, 
which then lets you check the commits that continued on branch-1 afterwards to 
see which ones also made it into the prior minor release.

That is, after the branch that led to 1.2.0 came off of branch-1 some set of 
JIRAS still had commits on both branches. the ones that happened on both you 
need not include in the CHANGES file for 1.3.0 because they will already be 
listed under 1.2.0.

does that make sense? If not, could make a diagram maybe?


> Update CHANGES.txt for 1.2
> --
>
> Key: HBASE-14025
> URL: https://issues.apache.org/jira/browse/HBASE-14025
> Project: HBase
>  Issue Type: Sub-task
>  Components: documentation
>Affects Versions: 1.2.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Fix For: 1.2.0
>
>
> Since it's more effort than I expected, making a ticket to track actually 
> updating CHANGES.txt so that new RMs have an idea what to expect.
> Maybe will make doc changes if there's enough here.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14025) Update CHANGES.txt for 1.2

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337460#comment-15337460
 ] 

Sean Busbey commented on HBASE-14025:
-

{quote}
if we're considering the Jira to be a source of truth here, then "project = 
HBase AND fixVersion = 1.3.0 AND fixVersion not in (1.2.0, 1.2.1, 1.2.2) AND 
status in (Resolved, Closed) ORDER BY issuetype DESC, updated ASC" jira filter 
should basically give the list of jiras (if we assumed that fixVersion are set 
accurately), if the jira fixVersion was always accurate?
{quote}

I would just look at those jiras with fixVersion 1.3.0 and fixVersion not in 
1.2.0, 1.1.0, 1.0.0. After the .0 release, it's hard to know how maintenance 
releases in the prior minor release line marry up with the new minor release 
line.

{quote}
bq. "find the commit where your release line branched off from the previous 
release.
$ git merge-base 1.1.0 branch-1.2
8166142b2e815fc6ab30c14a5a546cd242bf3b4c"
But branch-1.3 was cut off branch-1 with whatever was there in the moment, so I 
don't see how it applies?
{quote}

right, but the prior minor release lines were also cut off of it. you can find 
where the branch for the prior minor release left branch-1 using merge-base, 
which then lets you check the commits that continued on branch-1 afterwards to 
see which ones also made it into the prior minor release.

That is, after the branch that led to 1.2.0 came off of branch-1 some set of 
JIRAS still had commits on both branches. the ones that happened on both you 
need not include in the CHANGES file for 1.3.0 because they will already be 
listed under 1.2.0.

does that make sense? If not, could make a diagram maybe?


> Update CHANGES.txt for 1.2
> --
>
> Key: HBASE-14025
> URL: https://issues.apache.org/jira/browse/HBASE-14025
> Project: HBase
>  Issue Type: Sub-task
>  Components: documentation
>Affects Versions: 1.2.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Fix For: 1.2.0
>
>
> Since it's more effort than I expected, making a ticket to track actually 
> updating CHANGES.txt so that new RMs have an idea what to expect.
> Maybe will make doc changes if there's enough here.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337453#comment-15337453
 ] 

Hudson commented on HBASE-16023:


SUCCESS: Integrated in HBase-1.3-IT #715 (See 
[https://builds.apache.org/job/HBase-1.3-IT/715/])
HBASE-16023 Fastpath for the FIFO rpcscheduler Adds an executor that (stack: 
rev 0f8debc4c4b4403c5fff61e1ae7338d80831d988)
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java


> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16061) Allow logging to a buffered console

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337452#comment-15337452
 ] 

Hudson commented on HBASE-16061:


SUCCESS: Integrated in HBase-1.3-IT #715 (See 
[https://builds.apache.org/job/HBase-1.3-IT/715/])
HBASE-16061 Allow logging to a buffered console (eclark: rev 
75fb789a797de016217b24f4e6e9e3ef0a868cb3)
* conf/log4j.properties
* hbase-common/src/main/java/org/apache/hadoop/hbase/AsyncConsoleAppender.java


> Allow logging to a buffered console
> ---
>
> Key: HBASE-16061
> URL: https://issues.apache.org/jira/browse/HBASE-16061
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.2.1
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16061.v1.patch, HBASE-16061.v2.patch
>
>
> We've seen some stalls when logging to the ConsoleLogger that's getting 
> redirected to a file. We should allow people to use a buffered console 
> appender to keep from having stalls when the logging disk is busy.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15224) Undo "hbase.increment.fast.but.narrow.consistency" option; it is not necessary since HBASE-15213

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337450#comment-15337450
 ] 

Hudson commented on HBASE-15224:


SUCCESS: Integrated in HBase-1.3 #745 (See 
[https://builds.apache.org/job/HBase-1.3/745/])
HBASE-15224 Undo "hbase.increment.fast.but.narrow.consistency" option; 
(antonov: rev c327d9e380a88b5150c07a348d0f72857d1c14a7)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionIncrement.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestIncrementsFromClientSide.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestIncrementFromClientSideWithCoprocessor.java


> Undo  "hbase.increment.fast.but.narrow.consistency" option; it is not 
> necessary since HBASE-15213 
> --
>
> Key: HBASE-15224
> URL: https://issues.apache.org/jira/browse/HBASE-15224
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance
>Reporter: stack
>Assignee: stack
> Fix For: 1.2.0, 1.3.0, 1.1.4, 1.0.4
>
> Attachments: 
> 0001-HBASE-15224-Undo-hbase.increment.fast.branch-1.2.patch, 
> 0001-HBASE-15224-branch-1.2.patch, 15224.branch-1.2.addendum.patch, 
> 15224.branch-1.2.patch, HBASE-15224.branch-1.2.patch, 
> HBASE-15224v2-branch-1.2.patch
>
>
> Remove an escape valve no longer needed after HBASE-15213. Remove so folks 
> don't ever have to worry their pretty-little heads about it.
> Let the bulk of HBASE-15031 remain, the issue that added   
> "hbase.increment.fast.but.narrow.consistency" because it mostly cleanup we 
> want to keep.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16061) Allow logging to a buffered console

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337449#comment-15337449
 ] 

Hudson commented on HBASE-16061:


SUCCESS: Integrated in HBase-1.3 #745 (See 
[https://builds.apache.org/job/HBase-1.3/745/])
HBASE-16061 Allow logging to a buffered console (eclark: rev 
75fb789a797de016217b24f4e6e9e3ef0a868cb3)
* hbase-common/src/main/java/org/apache/hadoop/hbase/AsyncConsoleAppender.java
* conf/log4j.properties


> Allow logging to a buffered console
> ---
>
> Key: HBASE-16061
> URL: https://issues.apache.org/jira/browse/HBASE-16061
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.2.1
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16061.v1.patch, HBASE-16061.v2.patch
>
>
> We've seen some stalls when logging to the ConsoleLogger that's getting 
> redirected to a file. We should allow people to use a buffered console 
> appender to keep from having stalls when the logging disk is busy.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337451#comment-15337451
 ] 

Hudson commented on HBASE-16023:


SUCCESS: Integrated in HBase-1.3 #745 (See 
[https://builds.apache.org/job/HBase-1.3/745/])
HBASE-16023 Fastpath for the FIFO rpcscheduler Adds an executor that (stack: 
rev 0f8debc4c4b4403c5fff61e1ae7338d80831d988)
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java


> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16064) HBase shell delete backup command shows HDFS permission error, after successfully deleting the intended backup

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337448#comment-15337448
 ] 

Vladimir Rodionov commented on HBASE-16064:
---

Looks good to me, [~tedyu].

> HBase shell delete backup command shows HDFS permission error, after 
> successfully deleting the intended backup
> --
>
> Key: HBASE-16064
> URL: https://issues.apache.org/jira/browse/HBASE-16064
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Assignee: Ted Yu
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: 16064.v1.txt
>
>
> HBase delete backup command shows error, after successfully deleting the 
> intended backup
> {code}
> hbase@cluster-name:~$ hbase backup delete backup_1465950334243
> 2016-06-15 00:36:18,883 INFO  [main] util.BackupClientUtil: No data has been 
> found in 
> hdfs://cluster-name:8020/user/hbase/backup_1465950334243/default/table_ttx7w0jgw8.
> 2016-06-15 00:36:18,894 ERROR [main] util.BackupClientUtil: Cleaning up 
> backup data of backup_1465950334243 at hdfs://cluster-name:8020/user/hbase 
> failed due to Permission denied: user=hbase, access=WRITE, 
> inode="/user/hbase":hdfs:hdfs:drwxr-xr-x
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:216)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1827)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:92)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:3822)
>   at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1071)
>   at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:619)
>   at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
>   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
>   at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
> .
> {code}
> Backup has been successfully deleted but the backup root dir under 
> /user/hbase dir still persists
> {code}
> hbase@cluster-name:~$ hdfs dfs -ls /user/hbase
> Found 6 items
> drwx--   - hbase hbase  0 2016-06-15 00:26 /user/hbase/.staging
> drwxr-xr-x   - hbase hbase  0 2016-06-15 00:36 
> /user/hbase/backup_1465950334243
> drwxr-xr-x   - hbase hbase  0 2016-06-15 00:26 
> /user/hbase/hbase-staging
> {code}
> /user/hbase/backup_1465950334243 is now empty though



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-7105) RS throws NPE on forcing compaction from HBase shell on a single bulk imported file.

[Mikhail Antonov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-7105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337447#comment-15337447
 ] 

Mikhail Antonov commented on HBASE-7105:


any movement here? kicking to 1.4

> RS throws NPE on forcing compaction from HBase shell on a single bulk 
> imported file.
> 
>
> Key: HBASE-7105
> URL: https://issues.apache.org/jira/browse/HBASE-7105
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Reporter: Karthik Ranganathan
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 
> 0001-HBASE-7105-RS-throws-NPE-on-forcing-compaction-from-.patch, 
> 0001-HBASE-7105-RS-throws-NPE-on-forcing-compaction-from-.patch
>
>
> In StoreFile, we have:
> private AtomicBoolean majorCompaction = null;
> In StoreFile.open(), we do:
> b = metadataMap.get(MAJOR_COMPACTION_KEY);
> if (b != null) {
>   // init majorCompaction variable
> }
> Because the file was bulk imported, this is never initialized. Any subsequent 
> call to isMajorCompaction() NPE's.
> Fix is to initialize it to false.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11850) RpcClient can get stuck when closing

[Mikhail Antonov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-11850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337446#comment-15337446
 ] 

Mikhail Antonov commented on HBASE-11850:
-

Is this still relevant?

> RpcClient can get stuck when closing
> 
>
> Key: HBASE-11850
> URL: https://issues.apache.org/jira/browse/HBASE-11850
> Project: HBase
>  Issue Type: Bug
>  Components: Client
>Affects Versions: 0.99.0, 2.0.0
>Reporter: Nicolas Liochon
> Fix For: 2.0.0, 1.4.0
>
>
> we don't stop until the map in 'connections' is empty.
> the new connection is put in this map at creation, but if this connection is 
> not used it will never be removed.
> No totally sure of the fix yet. Probability is low (but not zero. I saw it 
> happening).
> The code is different in 0.98. It's 0.99+ issue only.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11850) RpcClient can get stuck when closing

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-11850:

Fix Version/s: (was: 1.3.0)
   1.4.0

> RpcClient can get stuck when closing
> 
>
> Key: HBASE-11850
> URL: https://issues.apache.org/jira/browse/HBASE-11850
> Project: HBase
>  Issue Type: Bug
>  Components: Client
>Affects Versions: 0.99.0, 2.0.0
>Reporter: Nicolas Liochon
> Fix For: 2.0.0, 1.4.0
>
>
> we don't stop until the map in 'connections' is empty.
> the new connection is put in this map at creation, but if this connection is 
> not used it will never be removed.
> No totally sure of the fix yet. Probability is low (but not zero. I saw it 
> happening).
> The code is different in 0.98. It's 0.99+ issue only.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12943) Set sun.net.inetaddr.ttl in HBase

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-12943:

Fix Version/s: (was: 1.3.0)
   1.3.1

> Set sun.net.inetaddr.ttl in HBase
> -
>
> Key: HBASE-12943
> URL: https://issues.apache.org/jira/browse/HBASE-12943
> Project: HBase
>  Issue Type: Bug
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
> Fix For: 2.0.0, 1.3.1
>
> Attachments: 12943-1-master.txt
>
>
> The default value of config: sun.net.inetaddr.ttl is -1 and the java 
> processes will cache the mapping of hostname to ip address  forever, See: 
> http://docs.oracle.com/javase/7/docs/technotes/guides/net/properties.html
> But things go wrong when a regionserver with same hostname and different ip 
> address rejoins the hbase cluster. The HMaster will get wrong ip address of 
> the regionserver from this cache and every region assignment to this 
> regionserver will be blocked for a time because the HMaster can't communicate 
> with the regionserver.
> A tradeoff is to set the sun.net.inetaddr.ttl to 10m or 1h and make the wrong 
> cache expired.
> Suggestions are welcomed. Thanks~



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13452) HRegion warning about memstore size miscalculation is not actionable

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-13452:

Fix Version/s: (was: 1.3.0)
   1.4.0

> HRegion warning about memstore size miscalculation is not actionable
> 
>
> Key: HBASE-13452
> URL: https://issues.apache.org/jira/browse/HBASE-13452
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Dev Lakhani
> Fix For: 2.0.0, 1.4.0, 1.3.1
>
>
> During normal operation the HRegion class reports a message related to 
> memstore flushing in HRegion.class :
>   if (!canFlush) {
> addAndGetGlobalMemstoreSize(-memstoreSize.get());
>   } else if (memstoreSize.get() != 0) {
> LOG.error("Memstore size is " + memstoreSize.get());
>   }
> The log file is filled with lots of 
> Memstore size is 558744
> Memstore size is 4390632
> Memstore size is 558744 
> ...
> These message are uninformative, clog up the logs and offers no root cause 
> nor solution. Maybe the message needs to be more informative, changed to WARN 
> or some further information provided.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13452) HRegion warning about memstore size miscalculation is not actionable

[Mikhail Antonov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337445#comment-15337445
 ] 

Mikhail Antonov commented on HBASE-13452:
-

Dropped to major and kicked to 1.4

> HRegion warning about memstore size miscalculation is not actionable
> 
>
> Key: HBASE-13452
> URL: https://issues.apache.org/jira/browse/HBASE-13452
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Dev Lakhani
> Fix For: 2.0.0, 1.4.0, 1.3.1
>
>
> During normal operation the HRegion class reports a message related to 
> memstore flushing in HRegion.class :
>   if (!canFlush) {
> addAndGetGlobalMemstoreSize(-memstoreSize.get());
>   } else if (memstoreSize.get() != 0) {
> LOG.error("Memstore size is " + memstoreSize.get());
>   }
> The log file is filled with lots of 
> Memstore size is 558744
> Memstore size is 4390632
> Memstore size is 558744 
> ...
> These message are uninformative, clog up the logs and offers no root cause 
> nor solution. Maybe the message needs to be more informative, changed to WARN 
> or some further information provided.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13452) HRegion warning about memstore size miscalculation is not actionable

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-13452:

Fix Version/s: 1.3.1

> HRegion warning about memstore size miscalculation is not actionable
> 
>
> Key: HBASE-13452
> URL: https://issues.apache.org/jira/browse/HBASE-13452
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Dev Lakhani
> Fix For: 2.0.0, 1.4.0, 1.3.1
>
>
> During normal operation the HRegion class reports a message related to 
> memstore flushing in HRegion.class :
>   if (!canFlush) {
> addAndGetGlobalMemstoreSize(-memstoreSize.get());
>   } else if (memstoreSize.get() != 0) {
> LOG.error("Memstore size is " + memstoreSize.get());
>   }
> The log file is filled with lots of 
> Memstore size is 558744
> Memstore size is 4390632
> Memstore size is 558744 
> ...
> These message are uninformative, clog up the logs and offers no root cause 
> nor solution. Maybe the message needs to be more informative, changed to WARN 
> or some further information provided.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14610) IntegrationTestRpcClient from HBASE-14535 is failing with Async RPC client

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-14610:

Fix Version/s: (was: 1.3.0)
   1.3.1

> IntegrationTestRpcClient from HBASE-14535 is failing with Async RPC client
> --
>
> Key: HBASE-14610
> URL: https://issues.apache.org/jira/browse/HBASE-14610
> Project: HBase
>  Issue Type: Bug
>  Components: IPC/RPC
>Reporter: Enis Soztutar
> Fix For: 2.0.0, 1.0.4, 1.1.6, 1.3.1, 1.2.3
>
> Attachments: output
>
>
> HBASE-14535 introduces an IT to simulate a running cluster with RPC servers 
> and RPC clients doing requests against the servers. 
> It passes with the sync client, but fails with async client. Probably we need 
> to take a look. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13452) HRegion warning about memstore size miscalculation is not actionable

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-13452:

Priority: Major  (was: Critical)

> HRegion warning about memstore size miscalculation is not actionable
> 
>
> Key: HBASE-13452
> URL: https://issues.apache.org/jira/browse/HBASE-13452
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Dev Lakhani
> Fix For: 2.0.0, 1.3.0
>
>
> During normal operation the HRegion class reports a message related to 
> memstore flushing in HRegion.class :
>   if (!canFlush) {
> addAndGetGlobalMemstoreSize(-memstoreSize.get());
>   } else if (memstoreSize.get() != 0) {
> LOG.error("Memstore size is " + memstoreSize.get());
>   }
> The log file is filled with lots of 
> Memstore size is 558744
> Memstore size is 4390632
> Memstore size is 558744 
> ...
> These message are uninformative, clog up the logs and offers no root cause 
> nor solution. Maybe the message needs to be more informative, changed to WARN 
> or some further information provided.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14329) Report region in transition only ever operates on one region

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-14329:

Fix Version/s: (was: 1.3.0)
   1.3.1

> Report region in transition only ever operates on one region
> 
>
> Key: HBASE-14329
> URL: https://issues.apache.org/jira/browse/HBASE-14329
> Project: HBase
>  Issue Type: Bug
>  Components: Region Assignment
>Reporter: Elliott Clark
> Fix For: 1.3.1
>
>
> Report region in transition takes a list of regions but it only ever operates 
> on one region however more than one region can be reported.
> Seems like this could cause some serious weirdness on failure cases.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14803) Add some debug logs to StoreFileScanner

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-14803:

Fix Version/s: (was: 1.3.0)
   1.4.0

> Add some debug logs to StoreFileScanner
> ---
>
> Key: HBASE-14803
> URL: https://issues.apache.org/jira/browse/HBASE-14803
> Project: HBase
>  Issue Type: Bug
>Reporter: Jean-Marc Spaggiari
>Assignee: Jean-Marc Spaggiari
>Priority: Minor
>  Labels: beginner
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-14803.v0-trunk.patch, HBASE-14803.v1-trunk.patch, 
> HBASE-14803.v2-trunk.patch, HBASE-14803.v3-trunk.patch, 
> HBASE-14803.v4-trunk.patch, HBASE-14803.v4-trunk.patch, 
> HBASE-14803.v5-trunk.patch
>
>
> To validate some behaviors I had to add some logs into StoreFileScanner.
> I think it can be interesting for other people looking for debuging. So 
> sharing the modifications here.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16061) Allow logging to a buffered console

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337443#comment-15337443
 ] 

Hadoop QA commented on HBASE-16061:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s 
{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s 
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 
1s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 57s 
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 49s 
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m 
11s {color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s 
{color} | {color:blue} Skipped branch modules with no Java source: . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 3s 
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 52s 
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s 
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 34s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m 34s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 51s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 51s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m 
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
27m 23s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s 
{color} | {color:blue} Skipped patch modules with no Java source: . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 
56s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 5s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 40s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 46s 
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 100m 1s 
{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
34s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 166m 2s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || 

[jira] [Updated] (HBASE-15635) Mean age of Blocks in cache (seconds) on webUI should be greater than zero

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-15635:

Fix Version/s: (was: 1.3.0)
   1.3.1

> Mean age of Blocks in cache (seconds) on webUI should be greater than zero
> --
>
> Key: HBASE-15635
> URL: https://issues.apache.org/jira/browse/HBASE-15635
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.98.17
>Reporter: Heng Chen
>Assignee: Heng Chen
> Fix For: 2.0.0, 1.4.0, 1.0.5, 1.1.6, 1.3.1, 0.98.21, 1.2.3
>
> Attachments: 7BFFAF68-0807-400C-853F-706B498449E1.png, 
> HBASE-15635.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14756) Break out ClientBackoffPolicy factors into configurable and stackable components

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-14756:

Fix Version/s: (was: 1.3.0)
   1.4.0

> Break out ClientBackoffPolicy factors into configurable and stackable 
> components
> 
>
> Key: HBASE-14756
> URL: https://issues.apache.org/jira/browse/HBASE-14756
> Project: HBase
>  Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Heng Chen
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-14756.patch
>
>
> Currently ExponentialClientBackoffPolicy evaluates three load parameters sent 
> back in results from the server. The policy is fixed in implementation. 
> Instead it should be possible to define the collection of considered load 
> factors via configuration, and for each selected term parameterize how the 
> load factor should contribute to the backoff calculation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16056) Procedure v2 - fix master crash for FileNotFound

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337441#comment-15337441
 ] 

Hudson commented on HBASE-16056:


SUCCESS: Integrated in HBase-1.2 #653 (See 
[https://builds.apache.org/job/HBase-1.2/653/])
HBASE-16056 Procedure v2 - fix master crash for FileNotFound (matteo.bertozzi: 
rev 8ea80b4556bb82b2c1ae8e31fa8a104264a5c400)
* 
hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/wal/TestWALProcedureStore.java
* 
hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java


> Procedure v2 - fix master crash for FileNotFound
> 
>
> Key: HBASE-16056
> URL: https://issues.apache.org/jira/browse/HBASE-16056
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Affects Versions: 2.0.0, 1.3.0, 1.2.1, 1.1.5
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Minor
> Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-16056-v0.patch, HBASE-16056-v1.patch, 
> HBASE-16056-v2.patch
>
>
> [~syuanjiang] and [~tedyu] reported a backup master not able to start with 
> FileNotFound during proc-v2 lease recovery. (another restart should have 
> solved the problem)
> {noformat}
> FileNotFoundException: File does not exist: 
> /hbase/MasterProcWALs/state-01.log
> namenode.INodeFile.valueOf(INodeFile.java:61) at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLease(FSNamesystem.java:2877)
>  at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.recoverLease(NameNodeRpcServer.java:753)
>  at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.recoverLease(ClientNamenodeProtocolServerSideTranslatorPB.java:671)
>  
> {noformat}
> this may happen when the other master is still active (e.g. GC) and tries to 
> remove files while the other master tries to become active. This operation is 
> retryable so the code should able to handle that.   



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-7218) Rename Snapshot

[Mikhail Antonov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-7218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337440#comment-15337440
 ] 

Mikhail Antonov commented on HBASE-7218:


(err, pushed - meant moved to)

> Rename Snapshot
> ---
>
> Key: HBASE-7218
> URL: https://issues.apache.org/jira/browse/HBASE-7218
> Project: HBase
>  Issue Type: New Feature
>  Components: snapshots
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Minor
> Fix For: 1.4.0
>
> Attachments: HBASE-7218-v0.patch, HBASE-7218-v1.patch
>
>
> Add the ability to rename a snapshot.
> HBaseAdmin.renameSnapshot(oldName, newName)
> shell: snapshot_rename 'oldName', 'newName'



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14743) Add metrics around HeapMemoryManager

[Reid Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337438#comment-15337438
 ] 

Reid Chan commented on HBASE-14743:
---

send files to me? I attach them

> Add metrics around HeapMemoryManager
> 
>
> Key: HBASE-14743
> URL: https://issues.apache.org/jira/browse/HBASE-14743
> Project: HBase
>  Issue Type: Improvement
>Reporter: Elliott Clark
>Assignee: Reid Chan
>Priority: Minor
> Attachments: HBASE-14743.001.patch, HBASE-14743.002.patch, 
> HBASE-14743.003.patch, HBASE-14743.004.patch, HBASE-14743.005.patch, 
> HBASE-14743.006.patch, HBASE-14743.007.patch, HBASE-14743.008.patch, 
> HBASE-14743.009.patch, HBASE-14743.009.rw3.patch, HBASE-14743.009.v2.patch, 
> Screen Shot 2016-06-16 at 5.39.13 PM.png
>
>
> it would be good to know how many invocations there have been.
> How many decided to expand memstore.
> How many decided to expand block cache.
> How many decided to do nothing.
> etc.
> When that's done use those metrics to clean up the tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16058) TestHRegion fails on 1.4 builds

[Enis Soztutar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Enis Soztutar updated HBASE-16058:
--
   Resolution: Fixed
 Assignee: Enis Soztutar
Fix Version/s: 2.0.0
   Status: Resolved  (was: Patch Available)

Committed.


> TestHRegion fails on 1.4 builds
> ---
>
> Key: HBASE-16058
> URL: https://issues.apache.org/jira/browse/HBASE-16058
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Affects Versions: 1.4.0
>Reporter: Mikhail Antonov
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.4.0
>
> Attachments: hbase-16058-v1.branch-1.patch
>
>
> Results :
> Failed tests: 
> org.apache.hadoop.hbase.regionserver.TestHRegion.testFlushSizeAccounting(org.apache.hadoop.hbase.regionserver.TestHRegion)
>   Run 1: TestHRegion.testFlushSizeAccounting:465 expected:<384> but was:<368>
>   Run 2: TestHRegion.testFlushSizeAccounting:465 expected:<384> but was:<368>
>   Run 3: TestHRegion.testFlushSizeAccounting:465 expected:<384> but was:<368>
> org.apache.hadoop.hbase.regionserver.TestHRegion.testMemstoreSizeAccountingWithFailedPostBatchMutate(org.apache.hadoop.hbase.regionserver.TestHRegion)
>   Run 1: TestHRegion.testMemstoreSizeAccountingWithFailedPostBatchMutate:434 
> memstoreSize should be incremented expected:<448> but was:<432>
>   Run 2: TestHRegion.testMemstoreSizeAccountingWithFailedPostBatchMutate:434 
> memstoreSize should be incremented expected:<448> but was:<432>
>   Run 3: TestHRegion.testMemstoreSizeAccountingWithFailedPostBatchMutate:434 
> memstoreSize should be incremented expected:<448> but was:<432>
> Tests run: 2567, Failures: 2, Errors: 0, Skipped: 56



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14964) Backport HBASE-14901 to brach-1 - There is duplicated code to create/manage encryption keys

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-14964:

Fix Version/s: (was: 1.3.0)
   1.4.0

> Backport HBASE-14901 to brach-1 - There is duplicated code to create/manage 
> encryption keys
> ---
>
> Key: HBASE-14964
> URL: https://issues.apache.org/jira/browse/HBASE-14964
> Project: HBase
>  Issue Type: Improvement
>  Components: encryption
>Reporter: Nate Edel
>Assignee: Nate Edel
>Priority: Minor
> Fix For: 1.4.0
>
> Attachments: HBASE-14964-branch-1.1.patch, 
> HBASE-14964.1.branch-1.patch, HBASE-14964.patch
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> There is duplicated code from MobUtils.createEncryptionContext in HStore, and 
> there is a subset of that code in HFileReaderImpl.
> Refactored key selection 
> Moved both to EncryptionUtil.java
> Can't figure out how to write a unit test for this, but there's no new code 
> just refactoring.
> A lot of the Mob stuff hasn't been backported, so this is a very small patch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14743) Add metrics around HeapMemoryManager

[Appy (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337432#comment-15337432
 ] 

Appy commented on HBASE-14743:
--

+1 (Will upload snapshots which convinced me that this works when i get perms)

> Add metrics around HeapMemoryManager
> 
>
> Key: HBASE-14743
> URL: https://issues.apache.org/jira/browse/HBASE-14743
> Project: HBase
>  Issue Type: Improvement
>Reporter: Elliott Clark
>Assignee: Reid Chan
>Priority: Minor
> Attachments: HBASE-14743.001.patch, HBASE-14743.002.patch, 
> HBASE-14743.003.patch, HBASE-14743.004.patch, HBASE-14743.005.patch, 
> HBASE-14743.006.patch, HBASE-14743.007.patch, HBASE-14743.008.patch, 
> HBASE-14743.009.patch, HBASE-14743.009.rw3.patch, HBASE-14743.009.v2.patch, 
> Screen Shot 2016-06-16 at 5.39.13 PM.png
>
>
> it would be good to know how many invocations there have been.
> How many decided to expand memstore.
> How many decided to expand block cache.
> How many decided to do nothing.
> etc.
> When that's done use those metrics to clean up the tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14743) Add metrics around HeapMemoryManager

[Appy (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337431#comment-15337431
 ] 

Appy commented on HBASE-14743:
--

Skimming through histograms code, it seems that they handle value 0 a bit 
differently. I think that's the reason for the weird thing (num_ops >  sum of 
bins count) i am seeing.

> Add metrics around HeapMemoryManager
> 
>
> Key: HBASE-14743
> URL: https://issues.apache.org/jira/browse/HBASE-14743
> Project: HBase
>  Issue Type: Improvement
>Reporter: Elliott Clark
>Assignee: Reid Chan
>Priority: Minor
> Attachments: HBASE-14743.001.patch, HBASE-14743.002.patch, 
> HBASE-14743.003.patch, HBASE-14743.004.patch, HBASE-14743.005.patch, 
> HBASE-14743.006.patch, HBASE-14743.007.patch, HBASE-14743.008.patch, 
> HBASE-14743.009.patch, HBASE-14743.009.rw3.patch, HBASE-14743.009.v2.patch, 
> Screen Shot 2016-06-16 at 5.39.13 PM.png
>
>
> it would be good to know how many invocations there have been.
> How many decided to expand memstore.
> How many decided to expand block cache.
> How many decided to do nothing.
> etc.
> When that's done use those metrics to clean up the tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14743) Add metrics around HeapMemoryManager

[Appy (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337427#comment-15337427
 ] 

Appy commented on HBASE-14743:
--

not able to attach files to this jira :-/
waiting for perms.

> Add metrics around HeapMemoryManager
> 
>
> Key: HBASE-14743
> URL: https://issues.apache.org/jira/browse/HBASE-14743
> Project: HBase
>  Issue Type: Improvement
>Reporter: Elliott Clark
>Assignee: Reid Chan
>Priority: Minor
> Attachments: HBASE-14743.001.patch, HBASE-14743.002.patch, 
> HBASE-14743.003.patch, HBASE-14743.004.patch, HBASE-14743.005.patch, 
> HBASE-14743.006.patch, HBASE-14743.007.patch, HBASE-14743.008.patch, 
> HBASE-14743.009.patch, HBASE-14743.009.rw3.patch, HBASE-14743.009.v2.patch, 
> Screen Shot 2016-06-16 at 5.39.13 PM.png
>
>
> it would be good to know how many invocations there have been.
> How many decided to expand memstore.
> How many decided to expand block cache.
> How many decided to do nothing.
> etc.
> When that's done use those metrics to clean up the tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13271) Table#puts(List) operation is indeterminate; needs fixing

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-13271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-13271:

Fix Version/s: (was: 1.3.0)
   1.4.0

> Table#puts(List) operation is indeterminate; needs fixing
> --
>
> Key: HBASE-13271
> URL: https://issues.apache.org/jira/browse/HBASE-13271
> Project: HBase
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 1.0.0
>Reporter: stack
>Priority: Critical
> Fix For: 2.0.0, 1.4.0
>
>
> Another API issue found by [~larsgeorge]:
> "Table.put(List {code}
> [Mar-17 9:21 AM] Lars George: Table.put(List) is weird since you cannot 
> flush partial lists
> [Mar-17 9:21 AM] Lars George: Say out of 5 the third is broken, then the 
> put() call returns with a local exception (say empty Put) and then you have 2 
> that are in the buffer
> [Mar-17 9:21 AM] Lars George: but how to you force commit them?
> [Mar-17 9:22 AM] Lars George: In the past you would call flushCache(), but 
> that is "gone" now
> [Mar-17 9:22 AM] Lars George: and flush() is not available on a Table
> [Mar-17 9:22 AM] Lars George: And you cannot access the underlying 
> BufferedMutation neither
> [Mar-17 9:23 AM] Lars George: You can *only* add more Puts if you can, or 
> call close()
> [Mar-17 9:23 AM] Lars George: that is just weird to explain
> {code}
> So, Table needs to get flush back or we deprecate this method or it flushes 
> immediately and does not return until complete in the implementation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14743) Add metrics around HeapMemoryManager

[Appy (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337420#comment-15337420
 ] 

Appy commented on HBASE-14743:
--

Testing the metrics:
Set these configs to turn on tuner and set memstore size to ~ 250bytes (4G * 
0.0005)
{noformat}
  
hbase.regionserver.global.memstore.size.max.range
0.5
  
  
hbase.regionserver.global.memstore.size.min.range
0.0
  
  
hfile.block.cache.size.min.range
0.0
  
  
hfile.block.cache.size.max.range
0.5
  
  
hbase.regionserver.global.memstore.size
0.0005
  
{noformat}

So i see the metrics being updated. But one weird thing is, one moment 
blockedflush histogram has all these values, and they are all gone in next 
refresh. You can compare num_ops values to get sense of relative timing of 
snapshots.

> Add metrics around HeapMemoryManager
> 
>
> Key: HBASE-14743
> URL: https://issues.apache.org/jira/browse/HBASE-14743
> Project: HBase
>  Issue Type: Improvement
>Reporter: Elliott Clark
>Assignee: Reid Chan
>Priority: Minor
> Attachments: HBASE-14743.001.patch, HBASE-14743.002.patch, 
> HBASE-14743.003.patch, HBASE-14743.004.patch, HBASE-14743.005.patch, 
> HBASE-14743.006.patch, HBASE-14743.007.patch, HBASE-14743.008.patch, 
> HBASE-14743.009.patch, HBASE-14743.009.rw3.patch, HBASE-14743.009.v2.patch, 
> Screen Shot 2016-06-16 at 5.39.13 PM.png
>
>
> it would be good to know how many invocations there have been.
> How many decided to expand memstore.
> How many decided to expand block cache.
> How many decided to do nothing.
> etc.
> When that's done use those metrics to clean up the tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14187) Add Thrift 1 RPC to batch gets in a single call

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-14187:

Fix Version/s: (was: 1.3.0)
   1.4.0

> Add Thrift 1 RPC to batch gets in a single call
> ---
>
> Key: HBASE-14187
> URL: https://issues.apache.org/jira/browse/HBASE-14187
> Project: HBase
>  Issue Type: Improvement
>  Components: Thrift
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Fix For: 2.0.0, 1.4.0
>
>
> add a method to pull a set of columns from a set of non-contiguous rows in a 
> single RPC call.
> e.g.
> {code}
>/**
>  * Parallel get. For a given table and column, return for
>  * the given rows.
>  *
>  * @param tableName table to get from
>  * @param column column to get
>  * @param rows a list of rows to get
>  * @result list of TRowResult for each item
>  */
> list parallelGet(1:Text tableName,
>  2:Text column,
>  3:list rows)
>  throws (1:IOError io)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14007) Writing to table through MR should fail upfront if table does not exist/disabled

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-14007:

Fix Version/s: (was: 1.3.0)
   1.4.0

> Writing to table through MR should fail upfront if table does not 
> exist/disabled
> 
>
> Key: HBASE-14007
> URL: https://issues.apache.org/jira/browse/HBASE-14007
> Project: HBase
>  Issue Type: Improvement
>  Components: mapreduce
>Affects Versions: 1.1.1
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
>Priority: Minor
>  Labels: mapreduce
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-14007.patch
>
>
> TableOutputFormat.checkOutputSpecs() needs to be fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-6618) Implement FuzzyRowFilter with ranges support

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-6618:
---
Fix Version/s: (was: 1.3.0)
   1.4.0

> Implement FuzzyRowFilter with ranges support
> 
>
> Key: HBASE-6618
> URL: https://issues.apache.org/jira/browse/HBASE-6618
> Project: HBase
>  Issue Type: New Feature
>  Components: Filters
>Reporter: Alex Baranau
>Assignee: Alex Baranau
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-6618-algo-desc-bits.png, HBASE-6618-algo.patch, 
> HBASE-6618.patch, HBASE-6618_2.path, HBASE-6618_3.path, HBASE-6618_4.patch, 
> HBASE-6618_5.patch
>
>
> Apart from current ability to specify fuzzy row filter e.g. for 
>  format as _0004 (where 0004 - actionId) it would be 
> great to also have ability to specify the "fuzzy range" , e.g. _0004, 
> ..., _0099.
> See initial discussion here: http://search-hadoop.com/m/WVLJdX0Z65
> Note: currently it is possible to provide multiple fuzzy row rules to 
> existing FuzzyRowFilter, but in case when the range is big (contains 
> thousands of values) it is not efficient.
> Filter should perform efficient fast-forwarding during the scan (this is what 
> distinguishes it from regex row filter).
> While such functionality may seem like a proper fit for custom filter (i.e. 
> not including into standard filter set) it looks like the filter may be very 
> re-useable. We may judge based on the implementation that will hopefully be 
> added.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-7115) [shell] Provide a way to register custom filters with the Filter Language Parser

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-7115:
---
Fix Version/s: (was: 1.3.0)
   1.4.0

> [shell] Provide a way to register custom filters with the Filter Language 
> Parser
> 
>
> Key: HBASE-7115
> URL: https://issues.apache.org/jira/browse/HBASE-7115
> Project: HBase
>  Issue Type: Improvement
>  Components: Filters, shell
>Affects Versions: 0.95.2
>Reporter: Aditya Kishore
>Assignee: Aditya Kishore
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-7115_trunk.patch, HBASE-7115_trunk.patch, 
> HBASE-7115_trunk_v2.patch
>
>
> HBASE-5428 added this capability to thrift interface but the configuration 
> parameter name is "thrift" specific.
> This patch introduces a more generic parameter "hbase.user.filters" using 
> which the user defined custom filters can be specified in the configuration 
> and loaded in any client that needs to use the filter language parser.
> The patch then uses this new parameter to register any user specified filters 
> while invoking the HBase shell.
> Example usage: Let's say I have written a couple of custom filters with class 
> names *{{org.apache.hadoop.hbase.filter.custom.SuperDuperFilter}}* and 
> *{{org.apache.hadoop.hbase.filter.custom.SilverBulletFilter}}* and I want to 
> use them from HBase shell using the filter language.
> To do that, I would add the following configuration to {{hbase-site.xml}}
> {panel}{{}}
> {{  hbase.user.filters}}
> {{  
> }}*{{SuperDuperFilter}}*{{:org.apache.hadoop.hbase.filter.custom.SuperDuperFilter,}}*{{SilverBulletFilter}}*{{:org.apache.hadoop.hbase.filter.custom.SilverBulletFilter}}
> {{}}{panel}
> Once this is configured, I can launch HBase shell and use these filters in my 
> {{get}} or {{scan}} just the way I would use a built-in filter.
> {code}
> hbase(main):001:0> scan 't', {FILTER => "SuperDuperFilter(true) AND 
> SilverBulletFilter(42)"}
> ROW  COLUMN+CELL
>  status  column=cf:a, 
> timestamp=30438552, value=world_peace
> 1 row(s) in 0. seconds
> {code}
> To use this feature in any client, the client needs to make the following 
> function call as part of its initialization.
> {code}
> ParseFilter.registerUserFilters(configuration);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-7115) [shell] Provide a way to register custom filters with the Filter Language Parser

[Mikhail Antonov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337417#comment-15337417
 ] 

Mikhail Antonov commented on HBASE-7115:


bumping to 1.4. any progress here?

> [shell] Provide a way to register custom filters with the Filter Language 
> Parser
> 
>
> Key: HBASE-7115
> URL: https://issues.apache.org/jira/browse/HBASE-7115
> Project: HBase
>  Issue Type: Improvement
>  Components: Filters, shell
>Affects Versions: 0.95.2
>Reporter: Aditya Kishore
>Assignee: Aditya Kishore
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-7115_trunk.patch, HBASE-7115_trunk.patch, 
> HBASE-7115_trunk_v2.patch
>
>
> HBASE-5428 added this capability to thrift interface but the configuration 
> parameter name is "thrift" specific.
> This patch introduces a more generic parameter "hbase.user.filters" using 
> which the user defined custom filters can be specified in the configuration 
> and loaded in any client that needs to use the filter language parser.
> The patch then uses this new parameter to register any user specified filters 
> while invoking the HBase shell.
> Example usage: Let's say I have written a couple of custom filters with class 
> names *{{org.apache.hadoop.hbase.filter.custom.SuperDuperFilter}}* and 
> *{{org.apache.hadoop.hbase.filter.custom.SilverBulletFilter}}* and I want to 
> use them from HBase shell using the filter language.
> To do that, I would add the following configuration to {{hbase-site.xml}}
> {panel}{{}}
> {{  hbase.user.filters}}
> {{  
> }}*{{SuperDuperFilter}}*{{:org.apache.hadoop.hbase.filter.custom.SuperDuperFilter,}}*{{SilverBulletFilter}}*{{:org.apache.hadoop.hbase.filter.custom.SilverBulletFilter}}
> {{}}{panel}
> Once this is configured, I can launch HBase shell and use these filters in my 
> {{get}} or {{scan}} just the way I would use a built-in filter.
> {code}
> hbase(main):001:0> scan 't', {FILTER => "SuperDuperFilter(true) AND 
> SilverBulletFilter(42)"}
> ROW  COLUMN+CELL
>  status  column=cf:a, 
> timestamp=30438552, value=world_peace
> 1 row(s) in 0. seconds
> {code}
> To use this feature in any client, the client needs to make the following 
> function call as part of its initialization.
> {code}
> ParseFilter.registerUserFilters(configuration);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-10974) Improve DBEs read performance by avoiding byte array deep copies for key[] and value[]

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-10974:

Fix Version/s: (was: 1.3.0)
   1.4.0

> Improve DBEs read performance by avoiding byte array deep copies for key[] 
> and value[]
> --
>
> Key: HBASE-10974
> URL: https://issues.apache.org/jira/browse/HBASE-10974
> Project: HBase
>  Issue Type: Improvement
>  Components: Scanners
>Affects Versions: 0.99.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-10974_1.patch
>
>
> As part of HBASE-10801, we  tried to reduce the copy of the value [] in 
> forming the KV from the DBEs. 
> The keys required copying and this was restricting us in using Cells and 
> always wanted to copy to be done.
> The idea here is to replace the key byte[] as ByteBuffer and create a 
> consecutive stream of the keys (currently the same byte[] is used and hence 
> the copy).  Use offset and length to track this key bytebuffer.
> The copy of the encoded format to normal Key format is definitely needed and 
> can't be avoided but we could always avoid the deep copy of the bytes to form 
> a KV and thus use cells effectively. Working on a patch, will post it soon.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12895) Introduce BufferedTable

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-12895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-12895:

Fix Version/s: (was: 1.3.0)
   1.4.0

> Introduce BufferedTable
> ---
>
> Key: HBASE-12895
> URL: https://issues.apache.org/jira/browse/HBASE-12895
> Project: HBase
>  Issue Type: Improvement
>  Components: Client, Usability
>Reporter: Nick Dimiduk
> Fix For: 2.0.0, 1.4.0
>
>
> Over on HBASE-12728 we extract the feature previously known as "disabled 
> autoFlush" into a separate interface called {{BufferedMutator}}. This 
> interface is like {{Table}} only exposes mutation operations. It would be 
> useful to provide an adapter that wraps up a {{BufferedMutator}} instance, 
> providing the rest of the {{Table}} interface API. That way, it's easier for 
> existing code to "drop-in" the new API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13602) Add an option to fail wal recovery when lease recovery fails

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-13602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-13602:

Fix Version/s: (was: 1.3.0)
   1.4.0

> Add an option to fail wal recovery when lease recovery fails
> 
>
> Key: HBASE-13602
> URL: https://issues.apache.org/jira/browse/HBASE-13602
> Project: HBase
>  Issue Type: Improvement
>  Components: wal
>Reporter: Sean Busbey
>  Labels: operability
> Fix For: 2.0.0, 1.4.0
>
>
> Currently, if lease recovery doesn't succeed over an extended timeout 
> (default 15 minutes), then we issue a log message about possible data loss 
> and continue with recovering the edits in that file.
> In some deployments this potential for dataloss might be unacceptable. In 
> those situations it would be good to have a configurable setting that marks 
> the recovery failed instead. Should default to off (at least in branch-1)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16056) Procedure v2 - fix master crash for FileNotFound

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337409#comment-15337409
 ] 

Hudson commented on HBASE-16056:


FAILURE: Integrated in HBase-1.4 #223 (See 
[https://builds.apache.org/job/HBase-1.4/223/])
HBASE-16056 Procedure v2 - fix master crash for FileNotFound (matteo.bertozzi: 
rev fb9a8a09f7e9493f166b3402a5948647c3f28acf)
* 
hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/wal/TestWALProcedureStore.java
* 
hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java


> Procedure v2 - fix master crash for FileNotFound
> 
>
> Key: HBASE-16056
> URL: https://issues.apache.org/jira/browse/HBASE-16056
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Affects Versions: 2.0.0, 1.3.0, 1.2.1, 1.1.5
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Minor
> Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-16056-v0.patch, HBASE-16056-v1.patch, 
> HBASE-16056-v2.patch
>
>
> [~syuanjiang] and [~tedyu] reported a backup master not able to start with 
> FileNotFound during proc-v2 lease recovery. (another restart should have 
> solved the problem)
> {noformat}
> FileNotFoundException: File does not exist: 
> /hbase/MasterProcWALs/state-01.log
> namenode.INodeFile.valueOf(INodeFile.java:61) at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLease(FSNamesystem.java:2877)
>  at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.recoverLease(NameNodeRpcServer.java:753)
>  at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.recoverLease(ClientNamenodeProtocolServerSideTranslatorPB.java:671)
>  
> {noformat}
> this may happen when the other master is still active (e.g. GC) and tries to 
> remove files while the other master tries to become active. This operation is 
> retryable so the code should able to handle that.   



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15224) Undo "hbase.increment.fast.but.narrow.consistency" option; it is not necessary since HBASE-15213

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337410#comment-15337410
 ] 

Hudson commented on HBASE-15224:


FAILURE: Integrated in HBase-1.4 #223 (See 
[https://builds.apache.org/job/HBase-1.4/223/])
HBASE-15224 Undo "hbase.increment.fast.but.narrow.consistency" option; 
(antonov: rev 76cf0d799fe3ad596b9872988c262da0895d59c6)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestIncrementFromClientSideWithCoprocessor.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestIncrementsFromClientSide.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionIncrement.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java


> Undo  "hbase.increment.fast.but.narrow.consistency" option; it is not 
> necessary since HBASE-15213 
> --
>
> Key: HBASE-15224
> URL: https://issues.apache.org/jira/browse/HBASE-15224
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance
>Reporter: stack
>Assignee: stack
> Fix For: 1.2.0, 1.3.0, 1.1.4, 1.0.4
>
> Attachments: 
> 0001-HBASE-15224-Undo-hbase.increment.fast.branch-1.2.patch, 
> 0001-HBASE-15224-branch-1.2.patch, 15224.branch-1.2.addendum.patch, 
> 15224.branch-1.2.patch, HBASE-15224.branch-1.2.patch, 
> HBASE-15224v2-branch-1.2.patch
>
>
> Remove an escape valve no longer needed after HBASE-15213. Remove so folks 
> don't ever have to worry their pretty-little heads about it.
> Let the bulk of HBASE-15031 remain, the issue that added   
> "hbase.increment.fast.but.narrow.consistency" because it mostly cleanup we 
> want to keep.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16063) Race condition in new FIFO fastpath from HBASE-16023

[Mikhail Antonov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337408#comment-15337408
 ] 

Mikhail Antonov commented on HBASE-16063:
-

Yeah, thanks for the heads up [~stack], will keep an eye on that

> Race condition in new FIFO fastpath from HBASE-16023
> 
>
> Key: HBASE-16063
> URL: https://issues.apache.org/jira/browse/HBASE-16063
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 1.3.0
>Reporter: stack
>
> From [~ikeda] over in HBASE-16023 at 
> https://issues.apache.org/jira/browse/HBASE-16023?focusedCommentId=15331172=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15331172
> {quote}
> An concrete example of the race condition:
> 1. Worker checks no task.
> 2. Reader checks no ready handler.
> 3. Worker pushes itself as a ready handler and waits on the semaphore.
> 4. Reader queues a task to the queue, without directly passing it to the 
> ready handler nor releasing the semaphore.
> (1,3) and (2,4) should be exclusively executed. That depends on luck, and it 
> might be not severe
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13953) Expand IT jenkins builds to include IntegrationTestIngest

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-13953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-13953:

Fix Version/s: (was: 1.3.0)
   1.4.0

> Expand IT jenkins builds to include IntegrationTestIngest
> -
>
> Key: HBASE-13953
> URL: https://issues.apache.org/jira/browse/HBASE-13953
> Project: HBase
>  Issue Type: Improvement
>  Components: integration tests
>Reporter: Sean Busbey
> Fix For: 2.0.0, 1.4.0
>
>
> update the various IT builds in jenkins to add IntegrationTestIngest, do 
> whatever is needed to make them pass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13952) Expand IT jenkins builds to include IntegrationTestBulkLoad

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-13952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-13952:

Fix Version/s: (was: 1.3.0)
   1.4.0

> Expand IT jenkins builds to include IntegrationTestBulkLoad
> ---
>
> Key: HBASE-13952
> URL: https://issues.apache.org/jira/browse/HBASE-13952
> Project: HBase
>  Issue Type: Improvement
>  Components: integration tests
>Reporter: Sean Busbey
> Fix For: 2.0.0, 1.4.0
>
>
> update the various HBase-*version*-IT matrix jobs to include 
> IntegrationTestBulkLoad, solve whatever's needed to make them pass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HBASE-16065) hbase backup set describe command does not inform if the set does not exist

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov reassigned HBASE-16065:
-

Assignee: Vladimir Rodionov

> hbase backup set describe command does not inform if the set does not exist
> ---
>
> Key: HBASE-16065
> URL: https://issues.apache.org/jira/browse/HBASE-16065
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Assignee: Vladimir Rodionov
>Priority: Minor
>  Labels: backup
> Fix For: 2.0.0
>
>
> hbase backup set describe command does not inform if the set does not exist
> from hbase shell
> {code}
> hbase@hbase-test-rc-7:~> hbase backup set list
> test_set={t1,t2}
> hbase@cluster-name:~> hbase backup set describe test_set1
> test_set1={}
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-16065) hbase backup set describe command does not inform if the set does not exist

[Romil Choksi (JIRA)]

Romil Choksi created HBASE-16065:


 Summary: hbase backup set describe command does not inform if the 
set does not exist
 Key: HBASE-16065
 URL: https://issues.apache.org/jira/browse/HBASE-16065
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Romil Choksi
Priority: Minor
 Fix For: 2.0.0


hbase backup set describe command does not inform if the set does not exist

from hbase shell
{code}
hbase@hbase-test-rc-7:~> hbase backup set list
test_set={t1,t2}

hbase@cluster-name:~> hbase backup set describe test_set1
test_set1={}
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16064) HBase shell delete backup command shows HDFS permission error, after successfully deleting the intended backup

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337393#comment-15337393
 ] 

Hadoop QA commented on HBASE-16064:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 2s 
{color} | {color:blue} The patch file was not named according to hbase's naming 
conventions. Please see 
https://yetus.apache.org/documentation/0.2.1/precommit-patchnames for 
instructions. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color} 
| {color:red} HBASE-16064 does not apply to master. Rebase required? Wrong 
Branch? See https://yetus.apache.org/documentation/0.2.1/precommit-patchnames 
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12811500/16064.v1.txt |
| JIRA Issue | HBASE-16064 |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/2287/console |
| Powered by | Apache Yetus 0.2.1   http://yetus.apache.org |


This message was automatically generated.



> HBase shell delete backup command shows HDFS permission error, after 
> successfully deleting the intended backup
> --
>
> Key: HBASE-16064
> URL: https://issues.apache.org/jira/browse/HBASE-16064
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Assignee: Ted Yu
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: 16064.v1.txt
>
>
> HBase delete backup command shows error, after successfully deleting the 
> intended backup
> {code}
> hbase@cluster-name:~$ hbase backup delete backup_1465950334243
> 2016-06-15 00:36:18,883 INFO  [main] util.BackupClientUtil: No data has been 
> found in 
> hdfs://cluster-name:8020/user/hbase/backup_1465950334243/default/table_ttx7w0jgw8.
> 2016-06-15 00:36:18,894 ERROR [main] util.BackupClientUtil: Cleaning up 
> backup data of backup_1465950334243 at hdfs://cluster-name:8020/user/hbase 
> failed due to Permission denied: user=hbase, access=WRITE, 
> inode="/user/hbase":hdfs:hdfs:drwxr-xr-x
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:216)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1827)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:92)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:3822)
>   at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1071)
>   at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:619)
>   at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
>   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
>   at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
> .
> {code}
> Backup has been successfully deleted but the backup root dir under 
> /user/hbase dir still persists
> {code}
> hbase@cluster-name:~$ hdfs dfs -ls /user/hbase
> Found 6 items
> drwx--   - hbase hbase  0 2016-06-15 00:26 /user/hbase/.staging
> drwxr-xr-x   - hbase hbase  0 2016-06-15 00:36 
> /user/hbase/backup_1465950334243
> drwxr-xr-x   - hbase hbase  0 2016-06-15 00:26 
> /user/hbase/hbase-staging
> {code}
> /user/hbase/backup_1465950334243 is now empty though



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16064) HBase shell delete backup command shows HDFS permission error, after successfully deleting the intended backup

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337392#comment-15337392
 ] 

Ted Yu commented on HBASE-16064:


The cause was that /user/hbase was mistakenly deleted, leading to the 
Permission denied error.

Thanks to [~cnauroth] who helped analyze the error.

> HBase shell delete backup command shows HDFS permission error, after 
> successfully deleting the intended backup
> --
>
> Key: HBASE-16064
> URL: https://issues.apache.org/jira/browse/HBASE-16064
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Assignee: Ted Yu
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: 16064.v1.txt
>
>
> HBase delete backup command shows error, after successfully deleting the 
> intended backup
> {code}
> hbase@cluster-name:~$ hbase backup delete backup_1465950334243
> 2016-06-15 00:36:18,883 INFO  [main] util.BackupClientUtil: No data has been 
> found in 
> hdfs://cluster-name:8020/user/hbase/backup_1465950334243/default/table_ttx7w0jgw8.
> 2016-06-15 00:36:18,894 ERROR [main] util.BackupClientUtil: Cleaning up 
> backup data of backup_1465950334243 at hdfs://cluster-name:8020/user/hbase 
> failed due to Permission denied: user=hbase, access=WRITE, 
> inode="/user/hbase":hdfs:hdfs:drwxr-xr-x
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:216)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1827)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:92)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:3822)
>   at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1071)
>   at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:619)
>   at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
>   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
>   at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
> .
> {code}
> Backup has been successfully deleted but the backup root dir under 
> /user/hbase dir still persists
> {code}
> hbase@cluster-name:~$ hdfs dfs -ls /user/hbase
> Found 6 items
> drwx--   - hbase hbase  0 2016-06-15 00:26 /user/hbase/.staging
> drwxr-xr-x   - hbase hbase  0 2016-06-15 00:36 
> /user/hbase/backup_1465950334243
> drwxr-xr-x   - hbase hbase  0 2016-06-15 00:26 
> /user/hbase/hbase-staging
> {code}
> /user/hbase/backup_1465950334243 is now empty though



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HBASE-16064) HBase shell delete backup command shows HDFS permission error, after successfully deleting the intended backup

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu reassigned HBASE-16064:
--

Assignee: Ted Yu

> HBase shell delete backup command shows HDFS permission error, after 
> successfully deleting the intended backup
> --
>
> Key: HBASE-16064
> URL: https://issues.apache.org/jira/browse/HBASE-16064
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Assignee: Ted Yu
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: 16064.v1.txt
>
>
> HBase delete backup command shows error, after successfully deleting the 
> intended backup
> {code}
> hbase@cluster-name:~$ hbase backup delete backup_1465950334243
> 2016-06-15 00:36:18,883 INFO  [main] util.BackupClientUtil: No data has been 
> found in 
> hdfs://cluster-name:8020/user/hbase/backup_1465950334243/default/table_ttx7w0jgw8.
> 2016-06-15 00:36:18,894 ERROR [main] util.BackupClientUtil: Cleaning up 
> backup data of backup_1465950334243 at hdfs://cluster-name:8020/user/hbase 
> failed due to Permission denied: user=hbase, access=WRITE, 
> inode="/user/hbase":hdfs:hdfs:drwxr-xr-x
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:216)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1827)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:92)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:3822)
>   at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1071)
>   at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:619)
>   at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
>   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
>   at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
> .
> {code}
> Backup has been successfully deleted but the backup root dir under 
> /user/hbase dir still persists
> {code}
> hbase@cluster-name:~$ hdfs dfs -ls /user/hbase
> Found 6 items
> drwx--   - hbase hbase  0 2016-06-15 00:26 /user/hbase/.staging
> drwxr-xr-x   - hbase hbase  0 2016-06-15 00:36 
> /user/hbase/backup_1465950334243
> drwxr-xr-x   - hbase hbase  0 2016-06-15 00:26 
> /user/hbase/hbase-staging
> {code}
> /user/hbase/backup_1465950334243 is now empty though



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16064) HBase shell delete backup command shows HDFS permission error, after successfully deleting the intended backup

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-16064:
---
Attachment: 16064.v1.txt

> HBase shell delete backup command shows HDFS permission error, after 
> successfully deleting the intended backup
> --
>
> Key: HBASE-16064
> URL: https://issues.apache.org/jira/browse/HBASE-16064
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Assignee: Ted Yu
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: 16064.v1.txt
>
>
> HBase delete backup command shows error, after successfully deleting the 
> intended backup
> {code}
> hbase@cluster-name:~$ hbase backup delete backup_1465950334243
> 2016-06-15 00:36:18,883 INFO  [main] util.BackupClientUtil: No data has been 
> found in 
> hdfs://cluster-name:8020/user/hbase/backup_1465950334243/default/table_ttx7w0jgw8.
> 2016-06-15 00:36:18,894 ERROR [main] util.BackupClientUtil: Cleaning up 
> backup data of backup_1465950334243 at hdfs://cluster-name:8020/user/hbase 
> failed due to Permission denied: user=hbase, access=WRITE, 
> inode="/user/hbase":hdfs:hdfs:drwxr-xr-x
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:216)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1827)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:92)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:3822)
>   at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1071)
>   at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:619)
>   at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
>   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
>   at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
> .
> {code}
> Backup has been successfully deleted but the backup root dir under 
> /user/hbase dir still persists
> {code}
> hbase@cluster-name:~$ hdfs dfs -ls /user/hbase
> Found 6 items
> drwx--   - hbase hbase  0 2016-06-15 00:26 /user/hbase/.staging
> drwxr-xr-x   - hbase hbase  0 2016-06-15 00:36 
> /user/hbase/backup_1465950334243
> drwxr-xr-x   - hbase hbase  0 2016-06-15 00:26 
> /user/hbase/hbase-staging
> {code}
> /user/hbase/backup_1465950334243 is now empty though



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16064) HBase shell delete backup command shows HDFS permission error, after successfully deleting the intended backup

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-16064:
---
Status: Patch Available  (was: Open)

> HBase shell delete backup command shows HDFS permission error, after 
> successfully deleting the intended backup
> --
>
> Key: HBASE-16064
> URL: https://issues.apache.org/jira/browse/HBASE-16064
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Assignee: Ted Yu
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: 16064.v1.txt
>
>
> HBase delete backup command shows error, after successfully deleting the 
> intended backup
> {code}
> hbase@cluster-name:~$ hbase backup delete backup_1465950334243
> 2016-06-15 00:36:18,883 INFO  [main] util.BackupClientUtil: No data has been 
> found in 
> hdfs://cluster-name:8020/user/hbase/backup_1465950334243/default/table_ttx7w0jgw8.
> 2016-06-15 00:36:18,894 ERROR [main] util.BackupClientUtil: Cleaning up 
> backup data of backup_1465950334243 at hdfs://cluster-name:8020/user/hbase 
> failed due to Permission denied: user=hbase, access=WRITE, 
> inode="/user/hbase":hdfs:hdfs:drwxr-xr-x
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:216)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1827)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:92)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:3822)
>   at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1071)
>   at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:619)
>   at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
>   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
>   at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
> .
> {code}
> Backup has been successfully deleted but the backup root dir under 
> /user/hbase dir still persists
> {code}
> hbase@cluster-name:~$ hdfs dfs -ls /user/hbase
> Found 6 items
> drwx--   - hbase hbase  0 2016-06-15 00:26 /user/hbase/.staging
> drwxr-xr-x   - hbase hbase  0 2016-06-15 00:36 
> /user/hbase/backup_1465950334243
> drwxr-xr-x   - hbase hbase  0 2016-06-15 00:26 
> /user/hbase/hbase-staging
> {code}
> /user/hbase/backup_1465950334243 is now empty though



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-16064) HBase shell delete backup command shows HDFS permission error, after successfully deleting the intended backup

[Romil Choksi (JIRA)]

Romil Choksi created HBASE-16064:


 Summary: HBase shell delete backup command shows HDFS permission 
error, after successfully deleting the intended backup
 Key: HBASE-16064
 URL: https://issues.apache.org/jira/browse/HBASE-16064
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Romil Choksi
 Fix For: 2.0.0


HBase delete backup command shows error, after successfully deleting the 
intended backup

{code}
hbase@cluster-name:~$ hbase backup delete backup_1465950334243

2016-06-15 00:36:18,883 INFO  [main] util.BackupClientUtil: No data has been 
found in 
hdfs://cluster-name:8020/user/hbase/backup_1465950334243/default/table_ttx7w0jgw8.
2016-06-15 00:36:18,894 ERROR [main] util.BackupClientUtil: Cleaning up backup 
data of backup_1465950334243 at hdfs://cluster-name:8020/user/hbase failed due 
to Permission denied: user=hbase, access=WRITE, 
inode="/user/hbase":hdfs:hdfs:drwxr-xr-x
at 
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
at 
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
at 
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:216)
at 
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
at 
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1827)
at 
org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:92)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:3822)
at 
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1071)
at 
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:619)
at 
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
.
{code}

Backup has been successfully deleted but the backup root dir under /user/hbase 
dir still persists
{code}
hbase@cluster-name:~$ hdfs dfs -ls /user/hbase
Found 6 items
drwx--   - hbase hbase  0 2016-06-15 00:26 /user/hbase/.staging
drwxr-xr-x   - hbase hbase  0 2016-06-15 00:36 
/user/hbase/backup_1465950334243
drwxr-xr-x   - hbase hbase  0 2016-06-15 00:26 /user/hbase/hbase-staging
{code}

/user/hbase/backup_1465950334243 is now empty though



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16010) Put draining function through Admin API

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337387#comment-15337387
 ] 

Hadoop QA commented on HBASE-16010:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 3 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 3m 20s 
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 57s 
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 23s 
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 7m 
34s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m 
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m 
55s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 
52s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 24s 
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s 
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s 
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 35s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 35s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 35s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 24s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 24s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 24s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 7m 
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
46s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s 
{color} | {color:red} The patch has 30 line(s) that end in whitespace. Use git 
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
33m 44s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m 
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 10s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 59s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 20s 
{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 1s 
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 99m 19s 
{color} | {color:green} hbase-server in the patch passed. {color} |
| 

[jira] [Updated] (HBASE-14129) If any regionserver gets shutdown uncleanly during full cluster restart, locality looks to be lost

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-14129:

Fix Version/s: (was: 1.3.0)
   1.4.0

> If any regionserver gets shutdown uncleanly during full cluster restart, 
> locality looks to be lost
> --
>
> Key: HBASE-14129
> URL: https://issues.apache.org/jira/browse/HBASE-14129
> Project: HBase
>  Issue Type: Bug
>Reporter: churro morales
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-14129.patch
>
>
> We were doing a cluster restart the other day.  Some regionservers did not 
> shut down cleanly.  Upon restart our locality went from 99% to 5%.  Upon 
> looking at the AssignmentManager.joinCluster() code it calls 
> AssignmentManager.processDeadServersAndRegionsInTransition().
> If the failover flag gets set for any reason it seems we don't call 
> assignAllUserRegions().  Then it looks like the balancer does the work in 
> assigning those regions, we don't use a locality aware balancer and we lost 
> our region locality.
> I don't have a solid grasp on the reasoning for these checks but there could 
> be some potential workarounds here.
> 1. After shutting down your cluster, move your WALs aside (replay later).  
> 2. Clean up your zNodes 
> That seems to work, but requires a lot of manual labor.  Another solution 
> which I prefer would be to have a flag for ./start-hbase.sh --clean 
> If we start master with that flag then we do a check in 
> AssignmentManager.processDeadServersAndRegionsInTransition()  thus if this 
> flag is set we call: assignAllUserRegions() regardless of the failover state.
> I have a patch for the later solution, that is if I am understanding the 
> logic correctly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14644) Region in transition metric is broken

[huaxiang sun (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

huaxiang sun updated HBASE-14644:
-
Attachment: HBASE-14644-v002-addendum.patch

Addendum to start the chore when it is active. [~mbertozzi], sorry I missed 
that in v2, could you take a look the new diff? Thanks.

> Region in transition metric is broken
> -
>
> Key: HBASE-14644
> URL: https://issues.apache.org/jira/browse/HBASE-14644
> Project: HBase
>  Issue Type: Bug
>Reporter: Elliott Clark
>Assignee: huaxiang sun
> Fix For: 2.0.0, 1.3.0, 1.2.2
>
> Attachments: HBASE-14644-v001.patch, HBASE-14644-v002-addendum.patch, 
> HBASE-14644-v002.patch, HBASE-14644-v002.patch, branch-1.diff
>
>
> ritCount stays 0 no matter what



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16058) TestHRegion fails on 1.4 builds

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337369#comment-15337369
 ] 

Hadoop QA commented on HBASE-16058:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m 
14s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s 
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s 
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 
17s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
22s {color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 13s 
{color} | {color:red} hbase-server in branch-1 has 1 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 38s 
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s 
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 48s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 36s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 
6s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
15m 50s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 91m 50s 
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
19s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 126m 26s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12811482/hbase-16058-v1.branch-1.patch
 |
| JIRA Issue | HBASE-16058 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux asf911.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | branch-1 / 76cf0d7 |
| Default Java | 1.7.0_80 |
| Multi-JDK versions |  /home/jenkins/tools/java/jdk1.8.0:1.8.0 
/home/jenkins/jenkins-slave/tools/hudson.model.JDK/JDK_1.7_latest_:1.7.0_80 |
| findbugs | v3.0.0 |
| findbugs | 

[jira] [Updated] (HBASE-15388) Add ACL for some master methods

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-15388:

Fix Version/s: (was: 1.3.0)
   1.4.0

> Add ACL for some master methods
> ---
>
> Key: HBASE-15388
> URL: https://issues.apache.org/jira/browse/HBASE-15388
> Project: HBase
>  Issue Type: Bug
>Reporter: Enis Soztutar
> Fix For: 2.0.0, 1.4.0
>
>
> Some new methods and some old ones do not have ACLs. 
> A basic look at the master rpc endpoints results in 
>  - Catalog janitor methods 
>  - set balancer switch
>  - Normalizer methods 
>  - split merge switch 
>  - mob methods 
>  - others? 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14644) Region in transition metric is broken

[huaxiang sun (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337368#comment-15337368
 ] 

huaxiang sun commented on HBASE-14644:
--

When working on backporting, I found an issue. The PeriodicDoMetrics chore 
should only be scheduled when it is an active master. However, it is done in 
HMaster constructor currently. I will upload an addendum shortly.

> Region in transition metric is broken
> -
>
> Key: HBASE-14644
> URL: https://issues.apache.org/jira/browse/HBASE-14644
> Project: HBase
>  Issue Type: Bug
>Reporter: Elliott Clark
>Assignee: huaxiang sun
> Fix For: 2.0.0, 1.3.0, 1.2.2
>
> Attachments: HBASE-14644-v001.patch, HBASE-14644-v002.patch, 
> HBASE-14644-v002.patch, branch-1.diff
>
>
> ritCount stays 0 no matter what



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15783) AccessControlConstants#OP_ATTRIBUTE_ACL_STRATEGY_CELL_FIRST not used any more.

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-15783:

Fix Version/s: (was: 1.3.0)

> AccessControlConstants#OP_ATTRIBUTE_ACL_STRATEGY_CELL_FIRST not used any more.
> --
>
> Key: HBASE-15783
> URL: https://issues.apache.org/jira/browse/HBASE-15783
> Project: HBase
>  Issue Type: Bug
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0, 1.0.4, 1.4.0
>
>
> This is based on a mail in the user list. 
> OP_ATTRIBUTE_ACL_STRATEGY_CELL_FIRST in AccessControlConstants is not used 
> any more in the code and AccessControlconstants is Public. We need to either 
> bring in this feature or remove the constant from the Public APi which may be 
> misleading. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16032) Possible memory leak in StoreScanner

[Enis Soztutar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337365#comment-15337365
 ] 

Enis Soztutar commented on HBASE-16032:
---

With the following: 
{code}
 +// add observer at last to avoid memory leak when exception occurs during 
initialization
{code}

what happens if flush/compaction comes in between the heap reset and this. We 
will miss the new files or no? 

> Possible memory leak in StoreScanner
> 
>
> Key: HBASE-16032
> URL: https://issues.apache.org/jira/browse/HBASE-16032
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.2.1, 1.1.5, 0.98.20
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.1.6, 1.3.1, 0.98.21, 1.2.3
>
> Attachments: HBASE-16032.patch, HBASE-16032_v2.patch, 
> HBASE-16032_v3.patch
>
>
> We observed frequent fullGC of RS in our production environment, and after 
> analyzing the heapdump, we found large memory occupancy by 
> HStore#changedReaderObservers, the map is surprisingly containing 7500w 
> objects...
> After some debugging, I located some possible memory leak in StoreScanner 
> constructor:
> {code}
>   public StoreScanner(Store store, ScanInfo scanInfo, Scan scan, final 
> NavigableSet columns,
>   long readPt)
>   throws IOException {
> this(store, scan, scanInfo, columns, readPt, scan.getCacheBlocks());
> if (columns != null && scan.isRaw()) {
>   throw new DoNotRetryIOException("Cannot specify any column for a raw 
> scan");
> }
> matcher = new ScanQueryMatcher(scan, scanInfo, columns,
> ScanType.USER_SCAN, Long.MAX_VALUE, HConstants.LATEST_TIMESTAMP,
> oldestUnexpiredTS, now, store.getCoprocessorHost());
> this.store.addChangedReaderObserver(this);
> // Pass columns to try to filter out unnecessary StoreFiles.
> List scanners = getScannersNoCompaction();
> ...
> seekScanners(scanners, matcher.getStartKey(), explicitColumnQuery
> && lazySeekEnabledGlobally, parallelSeekEnabled);
> ...
> resetKVHeap(scanners, store.getComparator());
>   }
> {code}
> If there's any Exception thrown after 
> {{this.store.addChangedReaderObserver(this)}}, the returned scanner might be 
> null and there's no chance to remove the scanner from changedReaderObservers, 
> like in {{HRegion#get}}
> {code}
> RegionScanner scanner = null;
> try {
>   scanner = getScanner(scan);
>   scanner.next(results);
> } finally {
>   if (scanner != null)
> scanner.close();
> }
> {code}
> What's more, all exception thrown in the {{HRegion#getScanner}} path will 
> cause scanner==null then memory leak, so we also need to handle this part.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14753) TestShell is not invoked anymore

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-14753:

Fix Version/s: (was: 1.3.0)
   1.3.1

> TestShell is not invoked anymore
> 
>
> Key: HBASE-14753
> URL: https://issues.apache.org/jira/browse/HBASE-14753
> Project: HBase
>  Issue Type: Bug
>Reporter: Enis Soztutar
> Fix For: 2.0.0, 1.3.1, 1.2.3
>
>
> Not sure whether it is due to HBASE-14679 or not. 
> {code}
> mvn clean  test -Dtest=TestShell
> {code}
> does not run any test at all. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14506) Reenable TestDistributedLogSplitting#testWorkerAbort

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-14506:

Fix Version/s: (was: 1.3.0)
   1.3.1

> Reenable TestDistributedLogSplitting#testWorkerAbort
> 
>
> Key: HBASE-14506
> URL: https://issues.apache.org/jira/browse/HBASE-14506
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: stack
>Priority: Critical
> Fix For: 2.0.0, 1.3.1
>
> Attachments: 
> org.apache.hadoop.hbase.master.TestDistributedLogSplitting-output.txt
>
>
> See attached log. Flakey up on jenkins and down locally.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12148) Remove TimeRangeTracker as point of contention when many threads writing a Store

[Mikhail Antonov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Antonov updated HBASE-12148:

Fix Version/s: (was: 1.3.0)
   1.3.1

> Remove TimeRangeTracker as point of contention when many threads writing a 
> Store
> 
>
> Key: HBASE-12148
> URL: https://issues.apache.org/jira/browse/HBASE-12148
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance
>Affects Versions: 2.0.0, 0.99.1
>Reporter: stack
>Assignee: Walter Koetke
> Fix For: 2.0.0, 1.3.1, 0.98.21
>
> Attachments: 
> 0001-In-AtomicUtils-change-updateMin-and-updateMax-to-ret.patch, 
> 12148.addendum.txt, 12148.min_and_max_run_independent.patch, 12148.txt, 
> 12148.txt, 12148v2.txt, 12148v2.txt, 12148v4.patch, HBASE-12148-V3.patch, 
> HBASE-12148-V3.patch, HBASE-12148.branch-1.v5.patch, 
> HBASE-12148.branch-1.v5.patch, HBASE-12148.txt, HBASE-12148V2.txt, Screen 
> Shot 2014-10-01 at 3.39.46 PM.png, Screen Shot 2014-10-01 at 3.41.07 PM.png, 
> Screen Shot 2016-04-13 at 1.49.30 PM.png, Screen Shot 2016-04-13 at 2.02.22 
> PM.png, Screen Shot 2016-05-18 at 10.21.53 PM.png, TimeRangeTracker.tiff
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16062) Improper error handling in WAL Reader/Writer creation

[Andrew Purtell (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337350#comment-15337350
 ] 

Andrew Purtell commented on HBASE-16062:


+1

> Improper error handling in WAL Reader/Writer creation
> -
>
> Key: HBASE-16062
> URL: https://issues.apache.org/jira/browse/HBASE-16062
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-16062-v1.patch
>
>
> If creation of WAL reader/ writer fails for some reason RS may leak hanging 
> socket in CLOSE_WAIT state. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16062) Improper error handling in WAL Reader/Writer creation

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337349#comment-15337349
 ] 

Ted Yu commented on HBASE-16062:


Nice finding, Vlad.
{code}
317 if (stream != null) {
318   stream.close();   318   stream.close();
319 }   319 }
320 if (reader != null) {
321   reader.close();
322 }
{code}
What if stream.close() throws exception ?

> Improper error handling in WAL Reader/Writer creation
> -
>
> Key: HBASE-16062
> URL: https://issues.apache.org/jira/browse/HBASE-16062
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-16062-v1.patch
>
>
> If creation of WAL reader/ writer fails for some reason RS may leak hanging 
> socket in CLOSE_WAIT state. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16062) Improper error handling in WAL Reader/Writer creation

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-16062:
---
Status: Patch Available  (was: Open)

> Improper error handling in WAL Reader/Writer creation
> -
>
> Key: HBASE-16062
> URL: https://issues.apache.org/jira/browse/HBASE-16062
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-16062-v1.patch
>
>
> If creation of WAL reader/ writer fails for some reason RS may leak hanging 
> socket in CLOSE_WAIT state. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16063) Race condition in new FIFO fastpath from HBASE-16023

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337342#comment-15337342
 ] 

stack commented on HBASE-16063:
---

[~mantonov] FYI. HBASE-16023 seems fine given its been running for a bunch of 
days on my little cluster but you should know about above. I can't see how we'd 
lock up here. Seems highly unlikely but FYI.

> Race condition in new FIFO fastpath from HBASE-16023
> 
>
> Key: HBASE-16063
> URL: https://issues.apache.org/jira/browse/HBASE-16063
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 1.3.0
>Reporter: stack
>
> From [~ikeda] over in HBASE-16023 at 
> https://issues.apache.org/jira/browse/HBASE-16023?focusedCommentId=15331172=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15331172
> {quote}
> An concrete example of the race condition:
> 1. Worker checks no task.
> 2. Reader checks no ready handler.
> 3. Worker pushes itself as a ready handler and waits on the semaphore.
> 4. Reader queues a task to the queue, without directly passing it to the 
> ready handler nor releasing the semaphore.
> (1,3) and (2,4) should be exclusively executed. That depends on luck, and it 
> might be not severe
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15467) Remove 1.x/2.0 TableDescriptor incompatibility

[Enis Soztutar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Enis Soztutar updated HBASE-15467:
--
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Pushed to master. 

> Remove 1.x/2.0 TableDescriptor incompatibility
> --
>
> Key: HBASE-15467
> URL: https://issues.apache.org/jira/browse/HBASE-15467
> Project: HBase
>  Issue Type: Bug
>  Components: master, regionserver
>Affects Versions: 2.0.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Fix For: 2.0.0
>
> Attachments: HBASE-15467-v0.patch, HBASE-15467-v1.patch, 
> hbase-15467_v2.patch
>
>
> I'm experimenting with Master on "2.0" and RSs on 1.x and the first problem 
> that I get is on createTable where the Master is trying to write the HTD as 
> TableDescriptor instead of TableSchema and the RS is not able to read it.
> Since TableDescriptor does nothing for now. I'd say we can remove it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[Mikhail Antonov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337332#comment-15337332
 ] 

Mikhail Antonov commented on HBASE-16023:
-

[~stack] pushed what?

> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16063) Race condition in new FIFO fastpath from HBASE-16023

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337330#comment-15337330
 ] 

stack commented on HBASE-16063:
---

And I've been running this intensely over the last week or so.

> Race condition in new FIFO fastpath from HBASE-16023
> 
>
> Key: HBASE-16063
> URL: https://issues.apache.org/jira/browse/HBASE-16063
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 1.3.0
>Reporter: stack
>
> From [~ikeda] over in HBASE-16023 at 
> https://issues.apache.org/jira/browse/HBASE-16023?focusedCommentId=15331172=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15331172
> {quote}
> An concrete example of the race condition:
> 1. Worker checks no task.
> 2. Reader checks no ready handler.
> 3. Worker pushes itself as a ready handler and waits on the semaphore.
> 4. Reader queues a task to the queue, without directly passing it to the 
> ready handler nor releasing the semaphore.
> (1,3) and (2,4) should be exclusively executed. That depends on luck, and it 
> might be not severe
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16059) Region normalizer fails to trigger merge action where one of the regions is empty

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337325#comment-15337325
 ] 

Ted Yu commented on HBASE-16059:


{code}
Failed tests: 
  
TestMultithreadedTableMapper.testMultithreadedTableMapper:132->runTestOnTable:155
 null
Flaked tests: 
org.apache.hadoop.hbase.TestPartialResultsFromClientSide.testExpectedNumberOfCellsPerPartialResult(org.apache.hadoop.hbase.TestPartialResultsFromClientSide)
  Run 1: 
TestPartialResultsFromClientSide.testExpectedNumberOfCellsPerPartialResult » 
TableNotFound
  Run 2: PASS

org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient.testRestoreSchemaChange(org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient)
  Run 1: 
TestRestoreSnapshotFromClient.testRestoreSchemaChange:207->countRows:321 » 
Runtime
  Run 2: PASS

org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testRegionCrossingHFileSplitRowBloom(org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles)
  Run 1: 
TestLoadIncrementalHFiles.testRegionCrossingHFileSplitRowBloom:192->testRegionCrossingHFileSplit:227->runTest:259->runTest:269->runTest:295
 » ScannerTimeout
  Run 2: PASS

org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan2.testScanYYYToEmpty(org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan2)
  Run 1: 
TestTableInputFormatScan2.testScanYYYToEmpty:97->TestTableInputFormatScanBase.testScan:240
 » TableNotFound
  Run 2: PASS

org.apache.hadoop.hbase.regionserver.TestScannerRetriableFailure.testFaultyScanner(org.apache.hadoop.hbase.regionserver.TestScannerRetriableFailure)
  Run 1: TestScannerRetriableFailure.testFaultyScanner:113->checkTableRows:150 
» ScannerTimeout
  Run 2: PASS

org.apache.hadoop.hbase.regionserver.wal.TestAsyncLogRollPeriod.testWithEdits(org.apache.hadoop.hbase.regionserver.wal.TestAsyncLogRollPeriod)
  Run 1: 
TestAsyncLogRollPeriod>AbstractTestLogRollPeriod.testWithEdits:124->Object.wait:-2
 » TestTimedOut
  Run 2: PASS

org.apache.hadoop.hbase.regionserver.wal.TestAsyncLogRolling.testLogRollOnDatanodeDeath(org.apache.hadoop.hbase.regionserver.wal.TestAsyncLogRolling)
  Run 1: TestAsyncLogRolling.testLogRollOnDatanodeDeath:63 expected:<1> but 
was:<0>
  Run 2: PASS


Tests run: 1900, Failures: 1, Errors: 0, Skipped: 44, Flakes: 7
{code}
Failed test was not related to the patch.

> Region normalizer fails to trigger merge action where one of the regions is 
> empty
> -
>
> Key: HBASE-16059
> URL: https://issues.apache.org/jira/browse/HBASE-16059
> Project: HBase
>  Issue Type: Bug
>  Components: master
>Reporter: Romil Choksi
>Assignee: Ted Yu
>  Labels: normalization
> Fix For: 2.0.0
>
> Attachments: 16059.v1.txt
>
>
> Region normalizer failed to trigger merge action where expected
> Steps to reproduce:
> - Pre-split the test table into 5 regions with keys 1,3,7,8
> - Insert some data for each of the split. 27K rows for regions starting with 
> key 1, and 100K rows for each of the regions with start key 3,7 and 8
> - Scan the test table, and verify that these regions exists -  1) STARTKEY => 
> ‘' ENDKEY => ’1’  2)  STARTKEY => ’1’ ENDKEY => ’3’
> - Turn on normalization, verify normalization switch is enabled and that 
> normalization is true for test table
> - Run normalizer a few times
> - Scan test table again, verify that regions don’t exist anymore  1) STARTKEY 
> => ‘' ENDKEY => ’1’  2)  STARTKEY => ’1’ ENDKEY => ’3’, but instead a new 
> region is created with  STARTKEY => ’’ ENDKEY => ’3’
> The test now fails, with the last step failing at assertion. 
> Looking into the Master log, I see that normalization plan was computed for 
> the test table but it decides that no normalization is needed for the test 
> table, and that the regions look good.
> {code:title = Master.log}
> 2016-06-17 00:41:46,895 DEBUG 
> [B.defaultRpcServer.handler=4,queue=1,port=2] 
> normalizer.SimpleRegionNormalizer: Computing normalization plan for table: 
> table_zrof6ea383, number of regions: 5
> 2016-06-17 00:41:46,895 DEBUG 
> [B.defaultRpcServer.handler=4,queue=1,port=2] 
> normalizer.SimpleRegionNormalizer: Table table_zrof6ea383, total aggregated 
> regions size: 13
> 2016-06-17 00:41:46,896 DEBUG 
> [B.defaultRpcServer.handler=4,queue=1,port=2] 
> normalizer.SimpleRegionNormalizer: Table table_zrof6ea383, average region 
> size: 2.6
> 2016-06-17 00:41:46,896 DEBUG 
> [B.defaultRpcServer.handler=4,queue=1,port=2] 
> normalizer.SimpleRegionNormalizer: No normalization needed, regions look good 
> for table: table_zrof6ea383
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16063) Race condition in new FIFO fastpath from HBASE-16023

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337324#comment-15337324
 ] 

stack commented on HBASE-16063:
---

I do not see how we could lock up. If the unlikely event of all handlers doing 
step #3 in the above before #4 happened, the next request would free us up 
again 

> Race condition in new FIFO fastpath from HBASE-16023
> 
>
> Key: HBASE-16063
> URL: https://issues.apache.org/jira/browse/HBASE-16063
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 1.3.0
>Reporter: stack
>
> From [~ikeda] over in HBASE-16023 at 
> https://issues.apache.org/jira/browse/HBASE-16023?focusedCommentId=15331172=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15331172
> {quote}
> An concrete example of the race condition:
> 1. Worker checks no task.
> 2. Reader checks no ready handler.
> 3. Worker pushes itself as a ready handler and waits on the semaphore.
> 4. Reader queues a task to the queue, without directly passing it to the 
> ready handler nor releasing the semaphore.
> (1,3) and (2,4) should be exclusively executed. That depends on luck, and it 
> might be not severe
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16056) Procedure v2 - fix master crash for FileNotFound

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337319#comment-15337319
 ] 

Hudson commented on HBASE-16056:


FAILURE: Integrated in HBase-Trunk_matrix #1066 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/1066/])
HBASE-16056 Procedure v2 - fix master crash for FileNotFound (matteo.bertozzi: 
rev 568e37d3831c65dfe4dc90c72fede2671b89cf7a)
* 
hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java
* 
hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/wal/TestWALProcedureStore.java


> Procedure v2 - fix master crash for FileNotFound
> 
>
> Key: HBASE-16056
> URL: https://issues.apache.org/jira/browse/HBASE-16056
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Affects Versions: 2.0.0, 1.3.0, 1.2.1, 1.1.5
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Minor
> Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-16056-v0.patch, HBASE-16056-v1.patch, 
> HBASE-16056-v2.patch
>
>
> [~syuanjiang] and [~tedyu] reported a backup master not able to start with 
> FileNotFound during proc-v2 lease recovery. (another restart should have 
> solved the problem)
> {noformat}
> FileNotFoundException: File does not exist: 
> /hbase/MasterProcWALs/state-01.log
> namenode.INodeFile.valueOf(INodeFile.java:61) at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLease(FSNamesystem.java:2877)
>  at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.recoverLease(NameNodeRpcServer.java:753)
>  at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.recoverLease(ClientNamenodeProtocolServerSideTranslatorPB.java:671)
>  
> {noformat}
> this may happen when the other master is still active (e.g. GC) and tries to 
> remove files while the other master tries to become active. This operation is 
> retryable so the code should able to handle that.   



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16018) Better documentation of ReplicationPeers

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337321#comment-15337321
 ] 

Hudson commented on HBASE-16018:


FAILURE: Integrated in HBase-Trunk_matrix #1066 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/1066/])
HBASE-16018 Refactored the ReplicationPeers interface to clear up what (eclark: 
rev 61ff6ced5bea3586129fb0844bbf64b122775b42)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationTrackerZKImpl.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateHBaseImpl.java
* 
hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeersZKImpl.java
* 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java
* 
hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeers.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java


> Better documentation of ReplicationPeers
> 
>
> Key: HBASE-16018
> URL: https://issues.apache.org/jira/browse/HBASE-16018
> Project: HBase
>  Issue Type: Improvement
>Reporter: Joseph
>Assignee: Joseph
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16018.patch
>
>
> Some of the ReplicationPeers interface's methods are not documented and are 
> tied to a ZooKeeper implementation of ReplicationPeers. Also some method 
> names are a little confusing.
> Review board at: https://reviews.apache.org/r/48696/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16053) Master code is not setting the table in ENABLING state in create table

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337320#comment-15337320
 ] 

Hudson commented on HBASE-16053:


FAILURE: Integrated in HBase-Trunk_matrix #1066 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/1066/])
HBASE-16053 Master code is not setting the table in ENABLING state in (enis: 
rev 81a9c1ac31333f11ad4a5defb9c625daf492513b)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CreateTableProcedure.java


> Master code is not setting the table in ENABLING state in create table
> --
>
> Key: HBASE-16053
> URL: https://issues.apache.org/jira/browse/HBASE-16053
> Project: HBase
>  Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0
>
> Attachments: hbase-16053_v1.patch
>
>
> Unit test logs are filled with the following, because in master, unlike 
> branch-1, we are missing the code which sets the table in ENABLING mode 
> before assignment in CreateTableProcedure. 
> {code}
> 2016-06-10 17:48:15,832 ERROR 
> [B.defaultRpcServer.handler=0,queue=0,port=60448] 
> master.TableStateManager(134): Unable to get table testRegionCache state
> org.apache.hadoop.hbase.TableNotFoundException: testRegionCache
>   at 
> org.apache.hadoop.hbase.master.TableStateManager.getTableState(TableStateManager.java:174)
>   at 
> org.apache.hadoop.hbase.master.TableStateManager.isTableState(TableStateManager.java:131)
>   at 
> org.apache.hadoop.hbase.master.AssignmentManager.onRegionOpen(AssignmentManager.java:2320)
>   at 
> org.apache.hadoop.hbase.master.AssignmentManager.onRegionTransition(AssignmentManager.java:2900)
>   at 
> org.apache.hadoop.hbase.master.MasterRpcServices.reportRegionStateTransition(MasterRpcServices.java:1334)
>   at 
> org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:8623)
>   at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2273)
>   at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:116)
>   at 
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:138)
>   at org.apache.hadoop.hbase.ipc.RpcExecutor$2.run(RpcExecutor.java:113)
>   at java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16023) Fastpath for the FIFO rpcscheduler

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337316#comment-15337316
 ] 

stack commented on HBASE-16023:
---

I made HBASE-16063. 

> Fastpath for the FIFO rpcscheduler
> --
>
> Key: HBASE-16023
> URL: https://issues.apache.org/jira/browse/HBASE-16023
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance, rpc
>Affects Versions: 2.0.0, 1.3.0
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-16023.addenum.patch, 
> HBASE-16023.branch-1.001.patch, hits.nofifo.fifoplusfp.fifownofp.hacks.png
>
>
> This is an idea copied from kudu where we skip queuing a request if there is 
> a handler ready to go; we just do a direct handoff from reader to handler.
> Makes for close to a %20 improvement in random read workloadc testing moving 
> the bottleneck to HBASE-15716 and to returning the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16056) Procedure v2 - fix master crash for FileNotFound

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337310#comment-15337310
 ] 

Hudson commented on HBASE-16056:


SUCCESS: Integrated in HBase-1.1-JDK7 #1733 (See 
[https://builds.apache.org/job/HBase-1.1-JDK7/1733/])
HBASE-16056 Procedure v2 - fix master crash for FileNotFound (matteo.bertozzi: 
rev ececf19dbaae38773f4b58454439a0914c4f8375)
* 
hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java
* 
hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/wal/TestWALProcedureStore.java


> Procedure v2 - fix master crash for FileNotFound
> 
>
> Key: HBASE-16056
> URL: https://issues.apache.org/jira/browse/HBASE-16056
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Affects Versions: 2.0.0, 1.3.0, 1.2.1, 1.1.5
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Minor
> Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-16056-v0.patch, HBASE-16056-v1.patch, 
> HBASE-16056-v2.patch
>
>
> [~syuanjiang] and [~tedyu] reported a backup master not able to start with 
> FileNotFound during proc-v2 lease recovery. (another restart should have 
> solved the problem)
> {noformat}
> FileNotFoundException: File does not exist: 
> /hbase/MasterProcWALs/state-01.log
> namenode.INodeFile.valueOf(INodeFile.java:61) at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLease(FSNamesystem.java:2877)
>  at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.recoverLease(NameNodeRpcServer.java:753)
>  at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.recoverLease(ClientNamenodeProtocolServerSideTranslatorPB.java:671)
>  
> {noformat}
> this may happen when the other master is still active (e.g. GC) and tries to 
> remove files while the other master tries to become active. This operation is 
> retryable so the code should able to handle that.   



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-16063) Race condition in new FIFO fastpath from HBASE-16023

[stack (JIRA)]

stack created HBASE-16063:
-

 Summary: Race condition in new FIFO fastpath from HBASE-16023
 Key: HBASE-16063
 URL: https://issues.apache.org/jira/browse/HBASE-16063
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 1.3.0
Reporter: stack


>From [~ikeda] over in HBASE-16023 at 
>https://issues.apache.org/jira/browse/HBASE-16023?focusedCommentId=15331172=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15331172

{quote}
An concrete example of the race condition:
1. Worker checks no task.
2. Reader checks no ready handler.
3. Worker pushes itself as a ready handler and waits on the semaphore.
4. Reader queues a task to the queue, without directly passing it to the ready 
handler nor releasing the semaphore.
(1,3) and (2,4) should be exclusively executed. That depends on luck, and it 
might be not severe
{quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)