[jira] [Commented] (HBASE-17251) Add a timeout parameter when locating region

2016-12-06 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15728001#comment-15728001
 ] 

Hadoop QA commented on HBASE-17251:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 3 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s 
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 
24s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 42s 
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s 
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
56s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 51s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
26m 22s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 7s 
{color} | {color:red} hbase-client generated 1 new + 0 unchanged - 0 fixed = 1 
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 46s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 0s 
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 97m 38s 
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
27s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 141m 12s {color} 
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-client |
|  |  Dead store to callTimeoutNs in 
org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(HRegionLocation)
  At 
AsyncSingleRequestRpcRetryingCaller.java:org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(HRegionLocation)
  At AsyncSingleRequestRpcRetryingCaller.java:[line 171] |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12842092/HBASE-17251.patch |
| JIRA Issue | HBASE-17251 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux 92592276b169 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 1eb24e4 |
| Default 

[jira] [Updated] (HBASE-17272) Doc how to run Standalone HBase over an HDFS instance; all daemons in one JVM but persisting to an HDFS instance

2016-12-06 Thread stack (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-17272:
--
Release Note: Adds section at 
http://hbase.apache.org/book.html#standalone.over.hdfs on how to make 
standalone persist to an hdfs instance (where standalone is all daemons in the 
one jvm).

> Doc how to run Standalone HBase over an HDFS instance; all daemons in one JVM 
> but persisting to an HDFS instance
> 
>
> Key: HBASE-17272
> URL: https://issues.apache.org/jira/browse/HBASE-17272
> Project: HBase
>  Issue Type: Task
>  Components: documentation
>Reporter: stack
> Attachments: 17272.txt
>
>
> It is possible to run hbase in standalone mode -- all daemons in the one JVM 
> (Master+RegionServers+ZooKeeper) -- but persisting to HDFS rather than the 
> usual local filesystem. That this works was news to me. This configuration is 
> wanted for the default deploy of the timeline server v2. The standalone mode 
> hbase shoudl be good enough for light loading. The persistence to HDFS makes 
> it so data can live across the coming and goings of nodes.
> This issue is about doc'ing how to setup this config.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727936#comment-15727936
 ] 

Hadoop QA commented on HBASE-17182:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
11s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 
52s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 15s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 15s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
23m 12s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 0s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 23s 
{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
7s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 59s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12842103/HBASE-17182-V8.patch |
| JIRA Issue | HBASE-17182 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux e3624c940dc0 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 1eb24e4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4826/testReport/ |
| modules | C: hbase-thrift U: hbase-thrift |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4826/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>Assignee: Jian Yi
> Attachments: 17182-V5.patch, 17182-V6.patch, HBASE-17182-V1.patch, 
> HBASE-17182-V2.patch, HBASE-17182-V3.patch, 

[jira] [Commented] (HBASE-17249) Get/Scan's setTimeRange/setColumnFamilyTimeRange can take the TimeRange reference as the parameter instead of creating a new setColumnFamilyTimeRange instance

2016-12-06 Thread Anoop Sam John (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727925#comment-15727925
 ] 

Anoop Sam John commented on HBASE-17249:


Pushed to master.  Thanks for the patch.

> Get/Scan's setTimeRange/setColumnFamilyTimeRange can take the TimeRange 
> reference as the parameter instead of creating a new setColumnFamilyTimeRange 
> instance
> --
>
> Key: HBASE-17249
> URL: https://issues.apache.org/jira/browse/HBASE-17249
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-17249-master-001.patch, 
> HBASE-17249-master-002.patch, HBASE-17249-master-003.patch
>
>
> Going through the code, found For Get/Scan's 
> setTimeRange/setColumnFamilyTimeRange, it can use  TimeRange as reference 
> instead of creating a new one.
> Reference:
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java#L500
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java#L506
> We can implement this in a similar way as filter:
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java#L510
> I checked it is same with branch-1.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17249) Get/Scan's setTimeRange/setColumnFamilyTimeRange can take the TimeRange reference as the parameter instead of creating a new setColumnFamilyTimeRange instance

2016-12-06 Thread Anoop Sam John (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anoop Sam John updated HBASE-17249:
---
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 2.0.0
   Status: Resolved  (was: Patch Available)

> Get/Scan's setTimeRange/setColumnFamilyTimeRange can take the TimeRange 
> reference as the parameter instead of creating a new setColumnFamilyTimeRange 
> instance
> --
>
> Key: HBASE-17249
> URL: https://issues.apache.org/jira/browse/HBASE-17249
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-17249-master-001.patch, 
> HBASE-17249-master-002.patch, HBASE-17249-master-003.patch
>
>
> Going through the code, found For Get/Scan's 
> setTimeRange/setColumnFamilyTimeRange, it can use  TimeRange as reference 
> instead of creating a new one.
> Reference:
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java#L500
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java#L506
> We can implement this in a similar way as filter:
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java#L510
> I checked it is same with branch-1.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17273) Create an hbase-coprocessor module as repository for generally useful coprocessors

2016-12-06 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727914#comment-15727914
 ] 

stack commented on HBASE-17273:
---

.001 was an overcommit. Use .002.

> Create an hbase-coprocessor module as repository for generally useful 
> coprocessors
> --
>
> Key: HBASE-17273
> URL: https://issues.apache.org/jira/browse/HBASE-17273
> Project: HBase
>  Issue Type: Task
>  Components: Coprocessors
>Reporter: stack
> Attachments: HBASE-17273.master.001.patch, 
> HBASE-17273.master.002.patch
>
>
> Idea here is a module where we can carry coprocessors that are of general 
> utility. In particular, I am thinking of the coprocessor used by yarn 
> timeline server v2 which does aggregating and then replacing increments with 
> a sum cell. This seems generally useful. Other candidates might include our 
> cross-row transactions coprocessor is another that could go here pulled out 
> of hbase-server and so on.
> Currently only coprocessors bundled in hbase are examples and endpoints.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17273) Create an hbase-coprocessor module as repository for generally useful coprocessors

2016-12-06 Thread stack (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-17273:
--
Attachment: HBASE-17273.master.002.patch

> Create an hbase-coprocessor module as repository for generally useful 
> coprocessors
> --
>
> Key: HBASE-17273
> URL: https://issues.apache.org/jira/browse/HBASE-17273
> Project: HBase
>  Issue Type: Task
>  Components: Coprocessors
>Reporter: stack
> Attachments: HBASE-17273.master.001.patch, 
> HBASE-17273.master.002.patch
>
>
> Idea here is a module where we can carry coprocessors that are of general 
> utility. In particular, I am thinking of the coprocessor used by yarn 
> timeline server v2 which does aggregating and then replacing increments with 
> a sum cell. This seems generally useful. Other candidates might include our 
> cross-row transactions coprocessor is another that could go here pulled out 
> of hbase-server and so on.
> Currently only coprocessors bundled in hbase are examples and endpoints.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17234) Allow alternate Readers/Writers; currently hardcoded

2016-12-06 Thread stack (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-17234:
--
Attachment: (was: HBASE-17234.master.002.patch)

> Allow alternate Readers/Writers; currently hardcoded
> 
>
> Key: HBASE-17234
> URL: https://issues.apache.org/jira/browse/HBASE-17234
> Project: HBase
>  Issue Type: Task
>  Components: io
>Reporter: stack
> Attachments: HBASE-17234.master.001.patch
>
>
> Allow alternate HFile Reader and Writers. For Writers, we have WriterFactory 
> so you'd think it possible to supply a different Writer but in actuality, 
> WriterFactory is hardcoded.
> Read side does something else altogether complicated by fact that Reader 
> presumes trailer and that it has to take a Stream.
> Yeah, expecting someone would provide their own Reader/Writer is a little 
> unexpected but



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17249) Get/Scan's setTimeRange/setColumnFamilyTimeRange can take the TimeRange reference as the parameter instead of creating a new setColumnFamilyTimeRange instance

2016-12-06 Thread Anoop Sam John (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727907#comment-15727907
 ] 

Anoop Sam John commented on HBASE-17249:


+1

> Get/Scan's setTimeRange/setColumnFamilyTimeRange can take the TimeRange 
> reference as the parameter instead of creating a new setColumnFamilyTimeRange 
> instance
> --
>
> Key: HBASE-17249
> URL: https://issues.apache.org/jira/browse/HBASE-17249
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-17249-master-001.patch, 
> HBASE-17249-master-002.patch, HBASE-17249-master-003.patch
>
>
> Going through the code, found For Get/Scan's 
> setTimeRange/setColumnFamilyTimeRange, it can use  TimeRange as reference 
> instead of creating a new one.
> Reference:
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java#L500
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java#L506
> We can implement this in a similar way as filter:
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java#L510
> I checked it is same with branch-1.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17234) Allow alternate Readers/Writers; currently hardcoded

2016-12-06 Thread stack (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-17234:
--
Attachment: HBASE-17234.master.002.patch

> Allow alternate Readers/Writers; currently hardcoded
> 
>
> Key: HBASE-17234
> URL: https://issues.apache.org/jira/browse/HBASE-17234
> Project: HBase
>  Issue Type: Task
>  Components: io
>Reporter: stack
> Attachments: HBASE-17234.master.001.patch, 
> HBASE-17234.master.002.patch
>
>
> Allow alternate HFile Reader and Writers. For Writers, we have WriterFactory 
> so you'd think it possible to supply a different Writer but in actuality, 
> WriterFactory is hardcoded.
> Read side does something else altogether complicated by fact that Reader 
> presumes trailer and that it has to take a Stream.
> Yeah, expecting someone would provide their own Reader/Writer is a little 
> unexpected but



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17273) Create an hbase-coprocessor module as repository for generally useful coprocessors

2016-12-06 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727878#comment-15727878
 ] 

stack commented on HBASE-17273:
---

Posted patch is the shell of a hbase-coprocessor, a frame into which we could 
plug our first general utility coprocessors.

> Create an hbase-coprocessor module as repository for generally useful 
> coprocessors
> --
>
> Key: HBASE-17273
> URL: https://issues.apache.org/jira/browse/HBASE-17273
> Project: HBase
>  Issue Type: Task
>  Components: Coprocessors
>Reporter: stack
> Attachments: HBASE-17273.master.001.patch
>
>
> Idea here is a module where we can carry coprocessors that are of general 
> utility. In particular, I am thinking of the coprocessor used by yarn 
> timeline server v2 which does aggregating and then replacing increments with 
> a sum cell. This seems generally useful. Other candidates might include our 
> cross-row transactions coprocessor is another that could go here pulled out 
> of hbase-server and so on.
> Currently only coprocessors bundled in hbase are examples and endpoints.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17273) Create an hbase-coprocessor module as repository for generally useful coprocessors

2016-12-06 Thread stack (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-17273:
--
Attachment: HBASE-17273.master.001.patch

> Create an hbase-coprocessor module as repository for generally useful 
> coprocessors
> --
>
> Key: HBASE-17273
> URL: https://issues.apache.org/jira/browse/HBASE-17273
> Project: HBase
>  Issue Type: Task
>  Components: Coprocessors
>Reporter: stack
> Attachments: HBASE-17273.master.001.patch
>
>
> Idea here is a module where we can carry coprocessors that are of general 
> utility. In particular, I am thinking of the coprocessor used by yarn 
> timeline server v2 which does aggregating and then replacing increments with 
> a sum cell. This seems generally useful. Other candidates might include our 
> cross-row transactions coprocessor is another that could go here pulled out 
> of hbase-server and so on.
> Currently only coprocessors bundled in hbase are examples and endpoints.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Jian Yi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jian Yi updated HBASE-17182:

Attachment: HBASE-17182-V8.patch

Given a larger default value for SCAN_PERIOD

> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>Assignee: Jian Yi
> Attachments: 17182-V5.patch, 17182-V6.patch, HBASE-17182-V1.patch, 
> HBASE-17182-V2.patch, HBASE-17182-V3.patch, HBASE-17182-V4.patch, 
> HBASE-17182-V7.patch, HBASE-17182-V8.patch
>
>
> Client call openScanner, but client (coredump or others) not closeScanner, 
> the scanner will not be removed from scannerMap.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (HBASE-17273) Create an hbase-coprocessor module as repository for generally useful coprocessors

2016-12-06 Thread stack (JIRA)
stack created HBASE-17273:
-

 Summary: Create an hbase-coprocessor module as repository for 
generally useful coprocessors
 Key: HBASE-17273
 URL: https://issues.apache.org/jira/browse/HBASE-17273
 Project: HBase
  Issue Type: Task
  Components: Coprocessors
Reporter: stack


Idea here is a module where we can carry coprocessors that are of general 
utility. In particular, I am thinking of the coprocessor used by yarn timeline 
server v2 which does aggregating and then replacing increments with a sum cell. 
This seems generally useful. Other candidates might include our cross-row 
transactions coprocessor is another that could go here pulled out of 
hbase-server and so on.

Currently only coprocessors bundled in hbase are examples and endpoints.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-16014) Get and Put constructor argument lists are divergent

2016-12-06 Thread brandboat (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-16014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727851#comment-15727851
 ] 

brandboat commented on HBASE-16014:
---

Hello,I've already solved this issue. If there is any question, please let me 
know. Thanks a lot.

> Get and Put constructor argument lists are divergent
> 
>
> Key: HBASE-16014
> URL: https://issues.apache.org/jira/browse/HBASE-16014
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Nick Dimiduk
>Assignee: brandboat
> Fix For: 2.0.0
>
> Attachments: HBASE-16014_v0.patch, HBASE-16014_v1.patch
>
>
> API for construing Get and Put objects for a specific rowkey is quite 
> different. 
> [Put|http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Put.html#constructor_summary]
>  supports many more variations for specifying the target rowkey and timestamp 
> compared to 
> [Get|http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Get.html#constructor_summary].
>  Notably lacking are {{Get(byte[], int, int)}} and {{Get(ByteBuffer)}} 
> variations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727850#comment-15727850
 ] 

Hadoop QA commented on HBASE-17182:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 24s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
27s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 6s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 21s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 21s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 21s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
29s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
15s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s 
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git 
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
29m 3s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 29s 
{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
8s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 16s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12842094/HBASE-17182-V7.patch |
| JIRA Issue | HBASE-17182 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux 2b648b918031 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
 |
| git revision | master / 1eb24e4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4824/artifact/patchprocess/whitespace-eol.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4824/testReport/ |
| modules | C: hbase-thrift U: hbase-thrift |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4824/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>

[jira] [Commented] (HBASE-17267) Add OffheapMemoryTuner

2016-12-06 Thread Anoop Sam John (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727828#comment-15727828
 ] 

Anoop Sam John commented on HBASE-17267:


Agree.. This can better be a main jira. with linking to HBASE-15179.

> Add OffheapMemoryTuner
> --
>
> Key: HBASE-17267
> URL: https://issues.apache.org/jira/browse/HBASE-17267
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
>
> This JIRA is aimed at tuning the offheap memory. It is not straight forward 
> as we should not cross the available Direct_memory configured.
> Should include BC and offheap memstore configs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-15787) Change the flush related heuristics to work with offheap size configured

2016-12-06 Thread ramkrishna.s.vasudevan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-15787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-15787:
---
Attachment: HBASE-15787_6.patch

Correct findbugs and the testcase. 

> Change the flush related heuristics to work with offheap size configured
> 
>
> Key: HBASE-15787
> URL: https://issues.apache.org/jira/browse/HBASE-15787
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> Attachments: HBASE-15787.patch, HBASE-15787_1.patch, 
> HBASE-15787_4.patch, HBASE-15787_5.patch, HBASE-15787_6.patch
>
>
> With offheap MSLAB in place we may have to change the flush related 
> heuristics to work with offheap size configured rather than the java heap 
> size.
> Since we now have clear seperation of the memstore data size and memstore 
> heap size, for offheap memstore
> -> Decide if the global.offheap.memstore.size is breached for blocking 
> updates and force flushes. 
> -> If the onheap global.memstore.size is breached (due to heap overhead) even 
> then block updates and force flushes.
> -> The global.memstore.size.lower.limit is now by default 95% of the 
> global.memstore.size. So now we apply this 95% on the 
> global.offheap.memstore.size and also on global.memstore.size (as it was done 
> for onheap case).
> -> We will have new FlushTypes introduced
> {code}
>   ABOVE_ONHEAP_LOWER_MARK, /* happens due to lower mark breach of onheap 
> memstore settings
>   An offheap memstore can even breach the 
> onheap_lower_mark*/
>   ABOVE_ONHEAP_HIGHER_MARK,/* happens due to higher mark breach of onheap 
> memstore settings
>   An offheap memstore can even breach the 
> onheap_higher_mark*/
>   ABOVE_OFFHEAP_LOWER_MARK,/* happens due to lower mark breach of offheap 
> memstore settings*/
>   ABOVE_OFFHEAP_HIGHER_MARK;
> {code}
> -> regionServerAccounting does all the accounting.
> -> HeapMemoryTuner is what is litte tricky here. First thing to note is that 
> at no point it will try to increase or decrease the 
> global.offheap.memstore.size. If there is a heap pressure then it will try to 
> increase the memstore heap limit. 
> In case of offheap memstore there is always a chance that the heap pressure 
> does not increase. In that case we could ideally decrease the heap limit for 
> memstore. The current logic of heapmemory tuner is such that things will 
> naturally settle down. But on discussion what we thought is let us include 
> the flush count that happens due to offheap pressure but give that a lesser 
> weightage and thus ensure that the initial decrease on memstore heap limit 
> does not happen. Currently that fraction is set as 0.5. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-15787) Change the flush related heuristics to work with offheap size configured

2016-12-06 Thread ramkrishna.s.vasudevan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-15787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-15787:
---
Hadoop Flags: Reviewed
  Status: Patch Available  (was: Open)

> Change the flush related heuristics to work with offheap size configured
> 
>
> Key: HBASE-15787
> URL: https://issues.apache.org/jira/browse/HBASE-15787
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> Attachments: HBASE-15787.patch, HBASE-15787_1.patch, 
> HBASE-15787_4.patch, HBASE-15787_5.patch, HBASE-15787_6.patch
>
>
> With offheap MSLAB in place we may have to change the flush related 
> heuristics to work with offheap size configured rather than the java heap 
> size.
> Since we now have clear seperation of the memstore data size and memstore 
> heap size, for offheap memstore
> -> Decide if the global.offheap.memstore.size is breached for blocking 
> updates and force flushes. 
> -> If the onheap global.memstore.size is breached (due to heap overhead) even 
> then block updates and force flushes.
> -> The global.memstore.size.lower.limit is now by default 95% of the 
> global.memstore.size. So now we apply this 95% on the 
> global.offheap.memstore.size and also on global.memstore.size (as it was done 
> for onheap case).
> -> We will have new FlushTypes introduced
> {code}
>   ABOVE_ONHEAP_LOWER_MARK, /* happens due to lower mark breach of onheap 
> memstore settings
>   An offheap memstore can even breach the 
> onheap_lower_mark*/
>   ABOVE_ONHEAP_HIGHER_MARK,/* happens due to higher mark breach of onheap 
> memstore settings
>   An offheap memstore can even breach the 
> onheap_higher_mark*/
>   ABOVE_OFFHEAP_LOWER_MARK,/* happens due to lower mark breach of offheap 
> memstore settings*/
>   ABOVE_OFFHEAP_HIGHER_MARK;
> {code}
> -> regionServerAccounting does all the accounting.
> -> HeapMemoryTuner is what is litte tricky here. First thing to note is that 
> at no point it will try to increase or decrease the 
> global.offheap.memstore.size. If there is a heap pressure then it will try to 
> increase the memstore heap limit. 
> In case of offheap memstore there is always a chance that the heap pressure 
> does not increase. In that case we could ideally decrease the heap limit for 
> memstore. The current logic of heapmemory tuner is such that things will 
> naturally settle down. But on discussion what we thought is let us include 
> the flush count that happens due to offheap pressure but give that a lesser 
> weightage and thus ensure that the initial decrease on memstore heap limit 
> does not happen. Currently that fraction is set as 0.5. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Jian Yi (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727788#comment-15727788
 ] 

Jian Yi commented on HBASE-17182:
-

Given a larger value:
int scanPeriod = conf.getInt(SCAN_PERIOD, 10 * 60 * 1000);

> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>Assignee: Jian Yi
> Attachments: 17182-V5.patch, 17182-V6.patch, HBASE-17182-V1.patch, 
> HBASE-17182-V2.patch, HBASE-17182-V3.patch, HBASE-17182-V4.patch, 
> HBASE-17182-V7.patch
>
>
> Client call openScanner, but client (coredump or others) not closeScanner, 
> the scanner will not be removed from scannerMap.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17272) Doc how to run Standalone HBase over an HDFS instance; all daemons in one JVM but persisting to an HDFS instance

2016-12-06 Thread stack (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-17272:
--
Attachment: 17272.txt

Here is what I'll commit. Adds a section on standalone.over.hdfs. Removes scary 
bit about local fs not supporting flush/sync -- fixed a good while ago.

> Doc how to run Standalone HBase over an HDFS instance; all daemons in one JVM 
> but persisting to an HDFS instance
> 
>
> Key: HBASE-17272
> URL: https://issues.apache.org/jira/browse/HBASE-17272
> Project: HBase
>  Issue Type: Task
>  Components: documentation
>Reporter: stack
> Attachments: 17272.txt
>
>
> It is possible to run hbase in standalone mode -- all daemons in the one JVM 
> (Master+RegionServers+ZooKeeper) -- but persisting to HDFS rather than the 
> usual local filesystem. That this works was news to me. This configuration is 
> wanted for the default deploy of the timeline server v2. The standalone mode 
> hbase shoudl be good enough for light loading. The persistence to HDFS makes 
> it so data can live across the coming and goings of nodes.
> This issue is about doc'ing how to setup this config.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Jian Yi (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727764#comment-15727764
 ] 

Jian Yi commented on HBASE-17182:
-

  public void testLongLivedScan() throws Exception {
int numTrials = 6;
int trialPause = 1000;
int cleanUpInterval = 100;
int scan_period = 12;
Configuration conf = new Configuration(UTIL.getConfiguration());
// Set the ConnectionCache timeout to trigger halfway through the trials
conf.setInt(ThriftHBaseServiceHandler.MAX_IDLETIME, (numTrials / 2) * 
trialPause);
conf.setInt(ThriftHBaseServiceHandler.CLEANUP_INTERVAL, cleanUpInterval);
conf.setInt(ThriftHBaseServiceHandler.SCAN_PERIOD, scan_period);
ThriftHBaseServiceHandler handler = new ThriftHBaseServiceHandler(conf,
UserProvider.instantiate(conf));

> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>Assignee: Jian Yi
> Attachments: 17182-V5.patch, 17182-V6.patch, HBASE-17182-V1.patch, 
> HBASE-17182-V2.patch, HBASE-17182-V3.patch, HBASE-17182-V4.patch, 
> HBASE-17182-V7.patch
>
>
> Client call openScanner, but client (coredump or others) not closeScanner, 
> the scanner will not be removed from scannerMap.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (HBASE-17272) Doc how to run Standalone HBase over an HDFS instance; all daemons in one JVM but persisting to an HDFS instance

2016-12-06 Thread stack (JIRA)
stack created HBASE-17272:
-

 Summary: Doc how to run Standalone HBase over an HDFS instance; 
all daemons in one JVM but persisting to an HDFS instance
 Key: HBASE-17272
 URL: https://issues.apache.org/jira/browse/HBASE-17272
 Project: HBase
  Issue Type: Task
Reporter: stack


It is possible to run hbase in standalone mode -- all daemons in the one JVM 
(Master+RegionServers+ZooKeeper) -- but persisting to HDFS rather than the 
usual local filesystem. That this works was news to me. This configuration is 
wanted for the default deploy of the timeline server v2. The standalone mode 
hbase shoudl be good enough for light loading. The persistence to HDFS makes it 
so data can live across the coming and goings of nodes.

This issue is about doc'ing how to setup this config.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Jian Yi (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727753#comment-15727753
 ] 

Jian Yi commented on HBASE-17182:
-

If getScannerRows cost more than SCAN_PERIOD, the next call will fail.

> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>Assignee: Jian Yi
> Attachments: 17182-V5.patch, 17182-V6.patch, HBASE-17182-V1.patch, 
> HBASE-17182-V2.patch, HBASE-17182-V3.patch, HBASE-17182-V4.patch, 
> HBASE-17182-V7.patch
>
>
> Client call openScanner, but client (coredump or others) not closeScanner, 
> the scanner will not be removed from scannerMap.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17272) Doc how to run Standalone HBase over an HDFS instance; all daemons in one JVM but persisting to an HDFS instance

2016-12-06 Thread stack (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-17272:
--
Component/s: documentation

> Doc how to run Standalone HBase over an HDFS instance; all daemons in one JVM 
> but persisting to an HDFS instance
> 
>
> Key: HBASE-17272
> URL: https://issues.apache.org/jira/browse/HBASE-17272
> Project: HBase
>  Issue Type: Task
>  Components: documentation
>Reporter: stack
>
> It is possible to run hbase in standalone mode -- all daemons in the one JVM 
> (Master+RegionServers+ZooKeeper) -- but persisting to HDFS rather than the 
> usual local filesystem. That this works was news to me. This configuration is 
> wanted for the default deploy of the timeline server v2. The standalone mode 
> hbase shoudl be good enough for light loading. The persistence to HDFS makes 
> it so data can live across the coming and goings of nodes.
> This issue is about doc'ing how to setup this config.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Jian Yi (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727749#comment-15727749
 ] 

Jian Yi commented on HBASE-17182:
-

How long? Maybe timeout is not enough.

> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>Assignee: Jian Yi
> Attachments: 17182-V5.patch, 17182-V6.patch, HBASE-17182-V1.patch, 
> HBASE-17182-V2.patch, HBASE-17182-V3.patch, HBASE-17182-V4.patch, 
> HBASE-17182-V7.patch
>
>
> Client call openScanner, but client (coredump or others) not closeScanner, 
> the scanner will not be removed from scannerMap.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Jian Yi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jian Yi updated HBASE-17182:

Attachment: HBASE-17182-V7.patch

Restore interpret and set smaller timeout

> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>Assignee: Jian Yi
> Attachments: 17182-V5.patch, 17182-V6.patch, HBASE-17182-V1.patch, 
> HBASE-17182-V2.patch, HBASE-17182-V3.patch, HBASE-17182-V4.patch, 
> HBASE-17182-V7.patch
>
>
> Client call openScanner, but client (coredump or others) not closeScanner, 
> the scanner will not be removed from scannerMap.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Ted Yu (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727717#comment-15727717
 ] 

Ted Yu commented on HBASE-17182:


TestThriftHBaseServiceHandler#testLongLivedScan fails with patch.

Please investigate.

> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>Assignee: Jian Yi
> Attachments: 17182-V5.patch, 17182-V6.patch, HBASE-17182-V1.patch, 
> HBASE-17182-V2.patch, HBASE-17182-V3.patch, HBASE-17182-V4.patch
>
>
> Client call openScanner, but client (coredump or others) not closeScanner, 
> the scanner will not be removed from scannerMap.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17251) Add a timeout parameter when locating region

2016-12-06 Thread Duo Zhang (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-17251:
--
Attachment: HBASE-17251.patch

> Add a timeout parameter when locating region
> 
>
> Key: HBASE-17251
> URL: https://issues.apache.org/jira/browse/HBASE-17251
> Project: HBase
>  Issue Type: Sub-task
>  Components: asyncclient, Client
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
> Fix For: 2.0.0
>
> Attachments: HBASE-17251.patch
>
>
> Now we always use the default timeout configured for zk and meta.
> I think it is reasonable to always use the same timeout when accessing zk and 
> meta as the result will be shared by lots of threads. If we could do a 
> successful fetching then the result will be in cache so it does not make 
> sense to set the timeout to a very small value when fetching.
> But I think it is also important to let the user request finish in time even 
> if the user can only get a timeout exception. We should not block a user 
> request longer than operation timeout.
> So I think we could add a timeout parameter to the region locate method, and 
> if the location can not be fetched in time, we will just finished the 
> returned CompletableFuture with a timeout exception, but the actual fetching 
> operation can still go on and use its own timeout config.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17251) Add a timeout parameter when locating region

2016-12-06 Thread Duo Zhang (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-17251:
--
Assignee: Duo Zhang
  Status: Patch Available  (was: Open)

> Add a timeout parameter when locating region
> 
>
> Key: HBASE-17251
> URL: https://issues.apache.org/jira/browse/HBASE-17251
> Project: HBase
>  Issue Type: Sub-task
>  Components: asyncclient, Client
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0
>
> Attachments: HBASE-17251.patch
>
>
> Now we always use the default timeout configured for zk and meta.
> I think it is reasonable to always use the same timeout when accessing zk and 
> meta as the result will be shared by lots of threads. If we could do a 
> successful fetching then the result will be in cache so it does not make 
> sense to set the timeout to a very small value when fetching.
> But I think it is also important to let the user request finish in time even 
> if the user can only get a timeout exception. We should not block a user 
> request longer than operation timeout.
> So I think we could add a timeout parameter to the region locate method, and 
> if the location can not be fetched in time, we will just finished the 
> returned CompletableFuture with a timeout exception, but the actual fetching 
> operation can still go on and use its own timeout config.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (HBASE-17271) hbase-thrift QA tests only run one test

2016-12-06 Thread Ted Yu (JIRA)
Ted Yu created HBASE-17271:
--

 Summary: hbase-thrift QA tests only run one test
 Key: HBASE-17271
 URL: https://issues.apache.org/jira/browse/HBASE-17271
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu


>From 
>https://builds.apache.org/job/PreCommit-HBASE-Build/4822/artifact/patchprocess/patch-unit-hbase-thrift.txt
> , it is pretty clear that only one test was run - TestCallQueue

Even though the patch contains modified TestThriftHBaseServiceHandler.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-15787) Change the flush related heuristics to work with offheap size configured

2016-12-06 Thread ramkrishna.s.vasudevan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-15787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-15787:
---
Status: Open  (was: Patch Available)

> Change the flush related heuristics to work with offheap size configured
> 
>
> Key: HBASE-15787
> URL: https://issues.apache.org/jira/browse/HBASE-15787
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> Attachments: HBASE-15787.patch, HBASE-15787_1.patch, 
> HBASE-15787_4.patch, HBASE-15787_5.patch
>
>
> With offheap MSLAB in place we may have to change the flush related 
> heuristics to work with offheap size configured rather than the java heap 
> size.
> Since we now have clear seperation of the memstore data size and memstore 
> heap size, for offheap memstore
> -> Decide if the global.offheap.memstore.size is breached for blocking 
> updates and force flushes. 
> -> If the onheap global.memstore.size is breached (due to heap overhead) even 
> then block updates and force flushes.
> -> The global.memstore.size.lower.limit is now by default 95% of the 
> global.memstore.size. So now we apply this 95% on the 
> global.offheap.memstore.size and also on global.memstore.size (as it was done 
> for onheap case).
> -> We will have new FlushTypes introduced
> {code}
>   ABOVE_ONHEAP_LOWER_MARK, /* happens due to lower mark breach of onheap 
> memstore settings
>   An offheap memstore can even breach the 
> onheap_lower_mark*/
>   ABOVE_ONHEAP_HIGHER_MARK,/* happens due to higher mark breach of onheap 
> memstore settings
>   An offheap memstore can even breach the 
> onheap_higher_mark*/
>   ABOVE_OFFHEAP_LOWER_MARK,/* happens due to lower mark breach of offheap 
> memstore settings*/
>   ABOVE_OFFHEAP_HIGHER_MARK;
> {code}
> -> regionServerAccounting does all the accounting.
> -> HeapMemoryTuner is what is litte tricky here. First thing to note is that 
> at no point it will try to increase or decrease the 
> global.offheap.memstore.size. If there is a heap pressure then it will try to 
> increase the memstore heap limit. 
> In case of offheap memstore there is always a chance that the heap pressure 
> does not increase. In that case we could ideally decrease the heap limit for 
> memstore. The current logic of heapmemory tuner is such that things will 
> naturally settle down. But on discussion what we thought is let us include 
> the flush count that happens due to offheap pressure but give that a lesser 
> weightage and thus ensure that the initial decrease on memstore heap limit 
> does not happen. Currently that fraction is set as 0.5. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-14123) HBase Backup/Restore Phase 2

2016-12-06 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727689#comment-15727689
 ] 

Hadoop QA commented on HBASE-14123:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 20s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m 6s 
{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 47 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s 
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 40s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m 
21s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m 
20s {color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s 
{color} | {color:blue} Skipped patched modules with no Java source: . {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 49s 
{color} | {color:red} hbase-protocol-shaded in master has 24 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 17s 
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s 
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 
5s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 45s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 4m 45s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m 45s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m 
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m 
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m 
4s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s 
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
26m 42s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 4m 
17s {color} | {color:green} the patch passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s 
{color} | {color:blue} Skipped patched modules with no Java source: . {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 14s 
{color} | {color:red} hbase-server generated 5 new + 0 unchanged - 0 fixed = 5 
total (was 0) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 33s 
{color} | {color:red} hbase-server generated 1 new + 1 unchanged - 0 fixed = 2 
total (was 1) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 2m 20s 
{color} | {color:red} root generated 1 new + 19 unchanged - 0 fixed = 20 total 
(was 19) {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 42s 
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 27s 
{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | 

[jira] [Commented] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727686#comment-15727686
 ] 

Hadoop QA commented on HBASE-17182:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 
24s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
29s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 23s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 24s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 24s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
29s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
29m 38s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 23s 
{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
7s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 0s {color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12842086/17182-V6.patch |
| JIRA Issue | HBASE-17182 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux a326321e811c 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
 |
| git revision | master / 1eb24e4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4822/testReport/ |
| modules | C: hbase-thrift U: hbase-thrift |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4822/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>Assignee: Jian Yi
> Attachments: 17182-V5.patch, 17182-V6.patch, HBASE-17182-V1.patch, 
> HBASE-17182-V2.patch, HBASE-17182-V3.patch, HBASE-17182-V4.patch

[jira] [Commented] (HBASE-17267) Add OffheapMemoryTuner

2016-12-06 Thread ramkrishna.s.vasudevan (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727661#comment-15727661
 ] 

ramkrishna.s.vasudevan commented on HBASE-17267:


May be this need not be a subtask of write path offheaping? Move it to main 
JIRA?

> Add OffheapMemoryTuner
> --
>
> Key: HBASE-17267
> URL: https://issues.apache.org/jira/browse/HBASE-17267
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
>
> This JIRA is aimed at tuning the offheap memory. It is not straight forward 
> as we should not cross the available Direct_memory configured.
> Should include BC and offheap memstore configs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17268) Add OffheapMemoryTuner

2016-12-06 Thread ramkrishna.s.vasudevan (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727659#comment-15727659
 ] 

ramkrishna.s.vasudevan commented on HBASE-17268:


No idea why it got created twice. Thanks for taking care.

> Add OffheapMemoryTuner
> --
>
> Key: HBASE-17268
> URL: https://issues.apache.org/jira/browse/HBASE-17268
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
>
> This JIRA is aimed at tuning the offheap memory. It is not straight forward 
> as we should not cross the available Direct_memory configured.
> Should include BC and offheap memstore configs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Ted Yu (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-17182:
---
Attachment: 17182-V6.patch

Patch v6 shortens new test runtime.

> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>Assignee: Jian Yi
> Attachments: 17182-V5.patch, 17182-V6.patch, HBASE-17182-V1.patch, 
> HBASE-17182-V2.patch, HBASE-17182-V3.patch, HBASE-17182-V4.patch
>
>
> Client call openScanner, but client (coredump or others) not closeScanner, 
> the scanner will not be removed from scannerMap.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727598#comment-15727598
 ] 

Hadoop QA commented on HBASE-17182:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 
1s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
25s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 18s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 18s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
12s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s 
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git 
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
25m 35s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 3s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 23s 
{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
7s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 24s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12842081/17182-V5.patch |
| JIRA Issue | HBASE-17182 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux 61cf90c9ab09 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 1eb24e4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4821/artifact/patchprocess/whitespace-eol.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4821/testReport/ |
| modules | C: hbase-thrift U: hbase-thrift |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4821/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>Assignee: 

[jira] [Commented] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727595#comment-15727595
 ] 

Hadoop QA commented on HBASE-17182:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 22s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 23s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
31s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 25s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 25s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
17s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s 
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git 
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
31m 45s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 23s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 29s 
{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
9s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 43m 9s {color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12842080/HBASE-17182-V4.patch |
| JIRA Issue | HBASE-17182 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux 5bf7146da160 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
 |
| git revision | master / 1eb24e4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4820/artifact/patchprocess/whitespace-eol.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4820/testReport/ |
| modules | C: hbase-thrift U: hbase-thrift |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4820/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>

[jira] [Commented] (HBASE-17264) Process RIT with offline state will always fail to open in the first time

2016-12-06 Thread Allan Yang (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727535#comment-15727535
 ] 

Allan Yang commented on HBASE-17264:


hadoop.hbase.master.handler.TestEnableTableHandler passed locally.

> Process RIT with offline state will always fail to open in the first time
> -
>
> Key: HBASE-17264
> URL: https://issues.apache.org/jira/browse/HBASE-17264
> Project: HBase
>  Issue Type: Bug
>  Components: Region Assignment
>Affects Versions: 1.1.7
>Reporter: Allan Yang
>Assignee: Allan Yang
> Attachments: HBASE-17264-branch-1.1.patch
>
>
> In Assignment#processRegionsInTransition, when handling regions with 
> M_ZK_REGION_OFFLINE state, we used a handler to reassign this region. But, 
> when calling assign, we passed not to set the zk node 
> {code}
> case M_ZK_REGION_OFFLINE:
> // Insert in RIT and resend to the regionserver
> regionStates.updateRegionState(rt, State.PENDING_OPEN);
> final RegionState rsOffline = regionStates.getRegionState(regionInfo);
> this.executorService.submit(
>   new EventHandler(server, EventType.M_MASTER_RECOVERY) {
> @Override
> public void process() throws IOException {
>   ReentrantLock lock = 
> locker.acquireLock(regionInfo.getEncodedName());
>   try {
> RegionPlan plan = new RegionPlan(regionInfo, null, sn);
> addPlan(encodedName, plan);
> assign(rsOffline, false, false);  //we decide to not to 
> setOfflineInZK
>   } finally {
> lock.unlock();
>   }
> }
>   });
> break;
> {code}
> But, when setOfflineInZK is false, we passed a zk node vesion of -1 to the 
> regionserver, meaning the zk node does not exists. But actually the offline 
> zk node does exist with a different version. RegionServer will report fail to 
> open because of this.
> This situation is trully happened in our test environment. Though the master 
> will recevied the FAILED_OPEN zk event and retry later, but due to a another 
> bug(HBASE-17265). The Region will be remain in closed state forever.
> Master assign region in RIT
> {noformat}
> 2016-11-23 17:11:46,842 INFO  [example.org:30001.activeMasterManager] 
> master.AssignmentManager: Processing 57513956a7b671f4e8da1598c2e2970e in 
> state: M_ZK_REGION_OFFLINE
> 2016-11-23 17:11:46,842 INFO  [example.org:30001.activeMasterManager] 
> master.RegionStates: Transition {57513956a7b671f4e8da1598c2e2970e 
> state=OFFLINE, ts=1479892306738, server=example.org,30003,1475893095003} to 
> {57513956a7b671f4e8da1598c2e2970e state=PENDING_OPEN, ts=1479892306842, 
> server=example.org,30003,1479780976834}
> 2016-11-23 17:11:46,842 INFO  [example.org:30001.activeMasterManager] 
> master.AssignmentManager: Processed region 57513956a7b671f4e8da1598c2e2970e 
> in state M_ZK_REGION_OFFLINE, on server: example.org,30003,1479780976834
> 2016-11-23 17:11:46,843 INFO  [MASTER_SERVER_OPERATIONS-example.org:30001-0] 
> master.AssignmentManager: Assigning 
> test,QFO7M,1475986053104.57513956a7b671f4e8da1598c2e2970e. to 
> example.org,30003,1479780976834
> {noformat}
> RegionServer recevied the open region request, and new a RegionOpenHandler to 
> open the region, but only to find the RIT node's version is not as it 
> expected. RS transition the RIT ZK node to failed open in the end
> {noformat}
> 2016-11-23 17:11:46,860 WARN  [RS_OPEN_REGION-example.org:30003-1] 
> coordination.ZkOpenRegionCoordination: Failed transition from OFFLINE to 
> OPENING for region=57513956a7b671f4e8da1598c2e2970e
> 2016-11-23 17:11:46,861 WARN  [RS_OPEN_REGION-example.org:30003-1] 
> handler.OpenRegionHandler: Region was hijacked? Opening cancelled for 
> encodedName=57513956a7b671f4e8da1598c2e2970e
> 2016-11-23 17:11:46,860 WARN  [RS_OPEN_REGION-example.org:30003-1] 
> zookeeper.ZKAssign: regionserver:30003-0x15810b5f633015f, 
> quorum=hbase4dev04.et2sqa:2181,hbase4dev05.et2sqa:2181,hbase4dev06.et2sqa:2181,
>  baseZNode=/test-hbase11-func2 Attempt to transition the unassigned node for 
> 57513956a7b671f4e8da1598c2e2970e from M_ZK_REGION_OFFLINE to 
> RS_ZK_REGION_OPENING failed, the node existed but was version 3 not the 
> expected version -1
> {noformat}
> Master recevied this zk event and begin to handle RS_ZK_REGION_FAILED_OPEN
> {noformat}
> 2016-11-23 17:11:46,944 DEBUG [AM.ZK.Worker-pool2-t1] 
> master.AssignmentManager: Handling RS_ZK_REGION_FAILED_OPEN, 
> server=example.org,30003,1479780976834, 
> region=57513956a7b671f4e8da1598c2e2970e, 
> current_state={57513956a7b671f4e8da1598c2e2970e state=PENDING_OPEN, 
> ts=1479892306843, server=example.org,30003,1479780976834}
> {noformat}



--
This message was sent by Atlassian 

[jira] [Updated] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Ted Yu (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-17182:
---
Attachment: 17182-V5.patch

Restored interrupt status in v5.

> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>Assignee: Jian Yi
> Attachments: 17182-V5.patch, HBASE-17182-V1.patch, 
> HBASE-17182-V2.patch, HBASE-17182-V3.patch, HBASE-17182-V4.patch
>
>
> Client call openScanner, but client (coredump or others) not closeScanner, 
> the scanner will not be removed from scannerMap.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Ted Yu (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727524#comment-15727524
 ] 

Ted Yu commented on HBASE-17182:


{code}
Thread.sleep(6 + 5000);
{code}
Can you set the timeout shorter for the test so that it doesn't run 1 minute ?

Please base the change on patch v5.

> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>Assignee: Jian Yi
> Attachments: HBASE-17182-V1.patch, HBASE-17182-V2.patch, 
> HBASE-17182-V3.patch, HBASE-17182-V4.patch
>
>
> Client call openScanner, but client (coredump or others) not closeScanner, 
> the scanner will not be removed from scannerMap.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Jian Yi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jian Yi updated HBASE-17182:

Attachment: HBASE-17182-V4.patch

remove commented codes

> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>Assignee: Jian Yi
> Attachments: HBASE-17182-V1.patch, HBASE-17182-V2.patch, 
> HBASE-17182-V3.patch, HBASE-17182-V4.patch
>
>
> Client call openScanner, but client (coredump or others) not closeScanner, 
> the scanner will not be removed from scannerMap.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17257) Add column-aliasing capability to hbase-client

2016-12-06 Thread Daniel Vimont (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727501#comment-15727501
 ] 

Daniel Vimont commented on HBASE-17257:
---

While I haven't done a lot with this yet, I did quickly attempt a manual split 
of the NAMESPACE_TABLE, and the warning message listed above ("Cannot split 
meta region in HBase 0.20 and above") appeared in the HBase log, and the 
namespace table remained with just a single region. So, as far as I can see, 
meta tables are not able to be split, and this was apparently hardwired into 
HBase when it was integrated with Zookeeper in release 0.20.0.

> Add column-aliasing capability to hbase-client
> --
>
> Key: HBASE-17257
> URL: https://issues.apache.org/jira/browse/HBASE-17257
> Project: HBase
>  Issue Type: New Feature
>  Components: Client
>Affects Versions: 2.0.0
>Reporter: Daniel Vimont
>Assignee: Daniel Vimont
>  Labels: features
>
> Column aliasing will provide the option for a 1, 2, or 4 byte alias value to 
> be stored in each cell of an "alias enabled" column-family, in place of the 
> full-length column-qualifier. Aliasing is intended to operate completely 
> invisibly to the end-user developer, with absolutely no "awareness" of 
> aliasing required to be coded into a front-end application. No new public 
> hbase-client interfaces are to be introduced, and only a few new public 
> methods should need to be added to existing interfaces, primarily to allow an 
> administrator to designate that a new column-family is to be alias-enabled by 
> setting its aliasSize attribute to 1, 2, or 4.
> To facilitate such functionality, new subclasses of HTable, 
> BufferedMutatorImpl, and HTableMultiplexer are to be provided. The overriding 
> methods of these new subclasses will invoke methods of the new AliasManager 
> class to facilitate qualifier-to-alias conversions (for user-submitted Gets, 
> Scans, and Mutations) and alias-to-qualifier conversions (for Results 
> returned from HBase) for any Table that has one or more alias-enabled column 
> families. All conversion logic will be encapsulated in the new AliasManager 
> class, and all qualifier-to-alias mappings will be persisted in a new 
> aliasMappingTable in a new, reserved namespace.
> An informal polling of HBase users at HBaseCon East and at the 
> Strata/Hadoop-World conference in Sept. 2016 showed that Column Aliasing 
> could be a popular enhancement to standard HBase functionality, due to the 
> fact that full column-qualifiers are stored in each cell, and reducing this 
> qualifier storage requirement down to 1, 2, or 4 bytes per cell could prove 
> beneficial in terms of reduced storage and bandwidth needs. Aliasing is 
> intended chiefly for column-families which are of the "narrow and tall" 
> variety (i.e., that are designed to use relatively few distinct 
> column-qualifiers throughout a large number of rows, throughout the lifespan 
> of the column-family). A column-family that is set up with an alias-size of 1 
> byte can contain up to 255 unique column-qualifiers; a 2 byte alias-size 
> allows for up to 65,535 unique column-qualifiers; and a 4 byte alias-size 
> allows for up to 4,294,967,295 unique column-qualifiers.
> Fuller specifications will be entered into the comments section below. Note 
> that it may well not be viable to add aliasing support in the new "async" 
> classes that appear to be currently under development.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17249) Get/Scan's setTimeRange/setColumnFamilyTimeRange can take the TimeRange reference as the parameter instead of creating a new setColumnFamilyTimeRange instance

2016-12-06 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727469#comment-15727469
 ] 

Hadoop QA commented on HBASE-17249:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s 
{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 14s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 13s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 13s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 13s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
23m 50s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 0s 
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
8s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 42s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12842070/HBASE-17249-master-003.patch
 |
| JIRA Issue | HBASE-17249 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux cc7b81db5eb3 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
 |
| git revision | master / 1eb24e4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4819/testReport/ |
| modules | C: hbase-client U: hbase-client |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4819/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Get/Scan's setTimeRange/setColumnFamilyTimeRange can take the TimeRange 
> reference as the parameter instead of creating a new setColumnFamilyTimeRange 
> instance
> --
>
> Key: 

[jira] [Commented] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Ted Yu (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727460#comment-15727460
 ] 

Ted Yu commented on HBASE-17182:


{code}
+  public void testCleanScannerThread() throws Exception {
{code}
Rename the test method.
{code}
+} catch (InterruptedException e) {
+  // LOG.error(e);
{code}
Restore interrupt status.
{code}
+  // ResultScanner scanner = scannerInfo.getScanner();
+  // scanner.close(); // Close will throw UnknownScannerException
{code}
You can drop the above lines.


> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>Assignee: Jian Yi
> Attachments: HBASE-17182-V1.patch, HBASE-17182-V2.patch, 
> HBASE-17182-V3.patch
>
>
> Client call openScanner, but client (coredump or others) not closeScanner, 
> the scanner will not be removed from scannerMap.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727449#comment-15727449
 ] 

Hadoop QA commented on HBASE-17182:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
23s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 18s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 18s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
23m 56s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 2s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 22s 
{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
5s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 32m 8s {color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12842068/HBASE-17182-V3.patch |
| JIRA Issue | HBASE-17182 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux 01695132eb97 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 1eb24e4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4818/testReport/ |
| modules | C: hbase-thrift U: hbase-thrift |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4818/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>Assignee: Jian Yi
> Attachments: HBASE-17182-V1.patch, HBASE-17182-V2.patch, 
> HBASE-17182-V3.patch
>
>
> Client call openScanner, but client (coredump 

[jira] [Commented] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Jian Yi (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727442#comment-15727442
 ] 

Jian Yi commented on HBASE-17182:
-

I submit the new patch with your opinion.

> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>Assignee: Jian Yi
> Attachments: HBASE-17182-V1.patch, HBASE-17182-V2.patch, 
> HBASE-17182-V3.patch
>
>
> Client call openScanner, but client (coredump or others) not closeScanner, 
> the scanner will not be removed from scannerMap.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17249) Get/Scan's setTimeRange/setColumnFamilyTimeRange can take the TimeRange reference as the parameter instead of creating a new setColumnFamilyTimeRange instance

2016-12-06 Thread huaxiang sun (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

huaxiang sun updated HBASE-17249:
-
Attachment: HBASE-17249-master-003.patch

v3 patch to address Anoop's comments.

> Get/Scan's setTimeRange/setColumnFamilyTimeRange can take the TimeRange 
> reference as the parameter instead of creating a new setColumnFamilyTimeRange 
> instance
> --
>
> Key: HBASE-17249
> URL: https://issues.apache.org/jira/browse/HBASE-17249
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-17249-master-001.patch, 
> HBASE-17249-master-002.patch, HBASE-17249-master-003.patch
>
>
> Going through the code, found For Get/Scan's 
> setTimeRange/setColumnFamilyTimeRange, it can use  TimeRange as reference 
> instead of creating a new one.
> Reference:
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java#L500
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java#L506
> We can implement this in a similar way as filter:
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java#L510
> I checked it is same with branch-1.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Jian Yi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jian Yi updated HBASE-17182:

Attachment: HBASE-17182-V3.patch

rename CleanScannerThread to ScannerCleanerThread, and add SCAN_PERIOD to 
replace magic number.

> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>Assignee: Jian Yi
> Attachments: HBASE-17182-V1.patch, HBASE-17182-V2.patch, 
> HBASE-17182-V3.patch
>
>
> Client call openScanner, but client (coredump or others) not closeScanner, 
> the scanner will not be removed from scannerMap.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17270) Change Service to Task in BackupRestoreServerFactory

2016-12-06 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727352#comment-15727352
 ] 

Hadoop QA commented on HBASE-17270:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 21m 12s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 1s 
{color} | {color:green} HBASE-7912 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 1s 
{color} | {color:green} HBASE-7912 passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m 
2s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 1s 
{color} | {color:green} HBASE-7912 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 1s 
{color} | {color:green} HBASE-7912 passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 1s 
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 1s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 1s 
{color} | {color:green} the patch passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 1s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
26m 16s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m 
2s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 3s 
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 2s 
{color} | {color:green} the patch passed with JDK v1.7.0_121 {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 57s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.2 Server=1.12.2 Image:yetus/hbase:date2016-12-07 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12842061/HBASE-17270.HBASE-7912.v1.patch
 |
| JIRA Issue | HBASE-17270 |
| Optional Tests |  hadoopcheck  hbaseanti  javac  compile  javadoc  |
| uname | Linux 670c0b3a851d 3.13.0-100-generic #147-Ubuntu SMP Tue Oct 18 
16:48:51 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | nobuild |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | HBASE-7912 / 72dfdc8 |
| Default Java | 1.7.0_121 |
| Multi-JDK versions |  /usr/lib/jvm/java-8-oracle:1.8.0_111 
/usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121 |
| modules | C: . U: . |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4817/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Change Service to Task in BackupRestoreServerFactory
> 
>
> Key: HBASE-17270
> URL: https://issues.apache.org/jira/browse/HBASE-17270
> Project: HBase
>  Issue Type: Task
>Affects Versions: HBASE-7912
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Trivial
> Fix For: 2.0.0
>
> Attachments: HBASE-17270.HBASE-7912.v1.patch
>
>
> Small method name refactoring



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17152) Use new HFile API to open reader w/o block cache being created

2016-12-06 Thread Ted Yu (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-17152:
---
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

> Use new HFile API to open reader w/o block cache being created
> --
>
> Key: HBASE-17152
> URL: https://issues.apache.org/jira/browse/HBASE-17152
> Project: HBase
>  Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Trivial
>  Labels: backup
> Fix For: HBASE-7912
>
> Attachments: HBASE-17152.HBASE-7912.v1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (HBASE-17249) Get/Scan's setTimeRange/setColumnFamilyTimeRange can take the TimeRange reference as the parameter instead of creating a new setColumnFamilyTimeRange instance

2016-12-06 Thread huaxiang sun (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726284#comment-15726284
 ] 

huaxiang sun edited comment on HBASE-17249 at 12/7/16 1:20 AM:
---

Thanks [~anoop.hbase], will take care of the comments and upload a new patch.


was (Author: huaxiang):
Thanks [~anoopa], will take care of the comments and upload a new patch.

> Get/Scan's setTimeRange/setColumnFamilyTimeRange can take the TimeRange 
> reference as the parameter instead of creating a new setColumnFamilyTimeRange 
> instance
> --
>
> Key: HBASE-17249
> URL: https://issues.apache.org/jira/browse/HBASE-17249
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-17249-master-001.patch, 
> HBASE-17249-master-002.patch
>
>
> Going through the code, found For Get/Scan's 
> setTimeRange/setColumnFamilyTimeRange, it can use  TimeRange as reference 
> instead of creating a new one.
> Reference:
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java#L500
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java#L506
> We can implement this in a similar way as filter:
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java#L510
> I checked it is same with branch-1.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17270) Change Service to Task in BackupRestoreServerFactory

2016-12-06 Thread Ted Yu (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-17270:
---
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 2.0.0
   Status: Resolved  (was: Patch Available)

> Change Service to Task in BackupRestoreServerFactory
> 
>
> Key: HBASE-17270
> URL: https://issues.apache.org/jira/browse/HBASE-17270
> Project: HBase
>  Issue Type: Task
>Affects Versions: HBASE-7912
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Trivial
> Fix For: 2.0.0
>
> Attachments: HBASE-17270.HBASE-7912.v1.patch
>
>
> Small method name refactoring



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17152) Use new HFile API to open reader w/o block cache being created

2016-12-06 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727298#comment-15727298
 ] 

Hadoop QA commented on HBASE-17152:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 2s 
{color} | {color:green} HBASE-7912 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 2s 
{color} | {color:green} HBASE-7912 passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m 
2s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 1s 
{color} | {color:green} HBASE-7912 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 1s 
{color} | {color:green} HBASE-7912 passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 2s 
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 2s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 2s 
{color} | {color:green} the patch passed with JDK v1.7.0_121 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 2s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
23m 16s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m 
3s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 3s 
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 3s 
{color} | {color:green} the patch passed with JDK v1.7.0_121 {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 9s {color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:date2016-12-07 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12842060/HBASE-17152.HBASE-7912.v1.patch
 |
| JIRA Issue | HBASE-17152 |
| Optional Tests |  hadoopcheck  hbaseanti  javac  compile  javadoc  |
| uname | Linux 9217dcccfed0 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | nobuild |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | HBASE-7912 / 72dfdc8 |
| Default Java | 1.7.0_121 |
| Multi-JDK versions |  /usr/lib/jvm/java-8-oracle:1.8.0_111 
/usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121 |
| modules | C: . U: . |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4816/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Use new HFile API to open reader w/o block cache being created
> --
>
> Key: HBASE-17152
> URL: https://issues.apache.org/jira/browse/HBASE-17152
> Project: HBase
>  Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Trivial
>  Labels: backup
> Fix For: HBASE-7912
>
> Attachments: HBASE-17152.HBASE-7912.v1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Jian Yi (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727254#comment-15727254
 ] 

Jian Yi commented on HBASE-17182:
-

I test it, the server close the scanner before timeout. if call closeScanner 
will get UnknownScannerException.

org.apache.hadoop.hbase.UnknownScannerException: 
org.apache.hadoop.hbase.UnknownScannerException: Name: 49140346, already closed?
at 
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2374)
at 
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33648)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)
at 
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
at java.lang.Thread.run(Thread.java:745)

> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>Assignee: Jian Yi
> Attachments: HBASE-17182-V1.patch, HBASE-17182-V2.patch
>
>
> Client call openScanner, but client (coredump or others) not closeScanner, 
> the scanner will not be removed from scannerMap.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17152) Use new HFile API to open reader w/o block cache being created

2016-12-06 Thread Vladimir Rodionov (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-17152:
--
Labels: backup  (was: )

> Use new HFile API to open reader w/o block cache being created
> --
>
> Key: HBASE-17152
> URL: https://issues.apache.org/jira/browse/HBASE-17152
> Project: HBase
>  Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Trivial
>  Labels: backup
> Fix For: HBASE-7912
>
> Attachments: HBASE-17152.HBASE-7912.v1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17152) Use new HFile API to open reader w/o block cache being created

2016-12-06 Thread Vladimir Rodionov (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-17152:
--
Fix Version/s: HBASE-7912

> Use new HFile API to open reader w/o block cache being created
> --
>
> Key: HBASE-17152
> URL: https://issues.apache.org/jira/browse/HBASE-17152
> Project: HBase
>  Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Trivial
> Fix For: HBASE-7912
>
> Attachments: HBASE-17152.HBASE-7912.v1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17270) Change Service to Task in BackupRestoreServerFactory

2016-12-06 Thread Vladimir Rodionov (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-17270:
--
Status: Patch Available  (was: Open)

> Change Service to Task in BackupRestoreServerFactory
> 
>
> Key: HBASE-17270
> URL: https://issues.apache.org/jira/browse/HBASE-17270
> Project: HBase
>  Issue Type: Task
>Affects Versions: HBASE-7912
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-17270.HBASE-7912.v1.patch
>
>
> Small method name refactoring



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17270) Change Service to Task in BackupRestoreServerFactory

2016-12-06 Thread Vladimir Rodionov (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-17270:
--
Priority: Trivial  (was: Major)

> Change Service to Task in BackupRestoreServerFactory
> 
>
> Key: HBASE-17270
> URL: https://issues.apache.org/jira/browse/HBASE-17270
> Project: HBase
>  Issue Type: Task
>Affects Versions: HBASE-7912
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Trivial
> Attachments: HBASE-17270.HBASE-7912.v1.patch
>
>
> Small method name refactoring



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17270) Change Service to Task in BackupRestoreServerFactory

2016-12-06 Thread Vladimir Rodionov (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-17270:
--
Attachment: HBASE-17270.HBASE-7912.v1.patch

v1. cc :[~te...@apache.org]

> Change Service to Task in BackupRestoreServerFactory
> 
>
> Key: HBASE-17270
> URL: https://issues.apache.org/jira/browse/HBASE-17270
> Project: HBase
>  Issue Type: Task
>Affects Versions: HBASE-7912
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-17270.HBASE-7912.v1.patch
>
>
> Small method name refactoring



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17152) Use new HFile API to open reader w/o block cache being created

2016-12-06 Thread Vladimir Rodionov (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-17152:
--
Priority: Trivial  (was: Major)

> Use new HFile API to open reader w/o block cache being created
> --
>
> Key: HBASE-17152
> URL: https://issues.apache.org/jira/browse/HBASE-17152
> Project: HBase
>  Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Trivial
> Fix For: HBASE-7912
>
> Attachments: HBASE-17152.HBASE-7912.v1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17152) Use new HFile API to open reader w/o block cache being created

2016-12-06 Thread Vladimir Rodionov (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-17152:
--
Attachment: HBASE-17152.HBASE-7912.v1.patch

v1. cc: [~ted_yu]

> Use new HFile API to open reader w/o block cache being created
> --
>
> Key: HBASE-17152
> URL: https://issues.apache.org/jira/browse/HBASE-17152
> Project: HBase
>  Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-17152.HBASE-7912.v1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17152) Use new HFile API to open reader w/o block cache being created

2016-12-06 Thread Vladimir Rodionov (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-17152:
--
Status: Patch Available  (was: Open)

> Use new HFile API to open reader w/o block cache being created
> --
>
> Key: HBASE-17152
> URL: https://issues.apache.org/jira/browse/HBASE-17152
> Project: HBase
>  Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-17152.HBASE-7912.v1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17260) Procedure v2 - Add setOwner() overload taking a User instance

2016-12-06 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727094#comment-15727094
 ] 

Hudson commented on HBASE-17260:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #2085 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/2085/])
HBASE-17260 Procedure v2 - Add setOwner() overload taking a User 
(matteo.bertozzi: rev 1eb24e4e8e8d2df4ba5cd3aa0223dfd08e1a90aa)
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/AbstractStateMachineNamespaceProcedure.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/AbstractStateMachineTableProcedure.java
* (edit) 
hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java


> Procedure v2 - Add setOwner() overload taking a User instance
> -
>
> Key: HBASE-17260
> URL: https://issues.apache.org/jira/browse/HBASE-17260
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Trivial
> Fix For: 2.0.0
>
> Attachments: HBASE-17260-v0.patch
>
>
> since we should have a User instance in most of the cases, we should just be 
> able to pass it, rather than converting it to getShortName() every time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (HBASE-14123) HBase Backup/Restore Phase 2

2016-12-06 Thread Vladimir Rodionov (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727077#comment-15727077
 ] 

Vladimir Rodionov edited comment on HBASE-14123 at 12/6/16 11:32 PM:
-

v39 addresses the recent comments on RB:

https://reviews.apache.org/r/52748/

as well as the following JIRAs:

HBASE-17134
HBASE-17135
HBASE-17136
HBASE-17146
HBASE-17147
HBASE-17151
HBASE-17152
HBASE-17154
HBASE-17155
HBASE-17156

cc: [~saint@gmail.com], [~te...@apache.org], [~enis], [~mbertozzi] 

The patch was uploaded to RB.


was (Author: vrodionov):
v39 addresses the recent comments on RB:

https://reviews.apache.org/r/52748/

as well as the following JIRAs:

HBASE-17134
HBASE-17135
HBASE-17136
HBASE-17146
HBASE-17147
HBASE-17151
HBASE-17152
HBASE-17154
HBASE-17155
HBASE-17156

cc: [~saint@gmail.com], [~te...@apache.org], [~enis], [~mbertozzi] 

> HBase Backup/Restore Phase 2
> 
>
> Key: HBASE-14123
> URL: https://issues.apache.org/jira/browse/HBASE-14123
> Project: HBase
>  Issue Type: Umbrella
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: 14123-master.v14.txt, 14123-master.v15.txt, 
> 14123-master.v16.txt, 14123-master.v17.txt, 14123-master.v18.txt, 
> 14123-master.v19.txt, 14123-master.v2.txt, 14123-master.v20.txt, 
> 14123-master.v21.txt, 14123-master.v24.txt, 14123-master.v25.txt, 
> 14123-master.v27.txt, 14123-master.v28.txt, 14123-master.v29.full.txt, 
> 14123-master.v3.txt, 14123-master.v30.txt, 14123-master.v31.txt, 
> 14123-master.v32.txt, 14123-master.v33.txt, 14123-master.v34.txt, 
> 14123-master.v35.txt, 14123-master.v36.txt, 14123-master.v37.txt, 
> 14123-master.v38.txt, 14123-master.v5.txt, 14123-master.v6.txt, 
> 14123-master.v7.txt, 14123-master.v8.txt, 14123-master.v9.txt, 14123-v14.txt, 
> 14123.master.v39.patch, HBASE-14123-for-7912-v1.patch, 
> HBASE-14123-for-7912-v6.patch, HBASE-14123-v1.patch, HBASE-14123-v10.patch, 
> HBASE-14123-v11.patch, HBASE-14123-v12.patch, HBASE-14123-v13.patch, 
> HBASE-14123-v15.patch, HBASE-14123-v16.patch, HBASE-14123-v2.patch, 
> HBASE-14123-v3.patch, HBASE-14123-v4.patch, HBASE-14123-v5.patch, 
> HBASE-14123-v6.patch, HBASE-14123-v7.patch, HBASE-14123-v9.patch
>
>
> Phase 2 umbrella JIRA. See HBASE-7912 for design document and description. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-14123) HBase Backup/Restore Phase 2

2016-12-06 Thread Vladimir Rodionov (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-14123:
--
Attachment: 14123.master.v39.patch

v39 addresses the recent comments on RB:

https://reviews.apache.org/r/52748/

as well as the following JIRAs:

HBASE-17134
HBASE-17135
HBASE-17136
HBASE-17146
HBASE-17147
HBASE-17151
HBASE-17152
HBASE-17154
HBASE-17155
HBASE-17156

cc: [~saint@gmail.com], [~te...@apache.org], [~enis], [~mbertozzi] 

> HBase Backup/Restore Phase 2
> 
>
> Key: HBASE-14123
> URL: https://issues.apache.org/jira/browse/HBASE-14123
> Project: HBase
>  Issue Type: Umbrella
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: 14123-master.v14.txt, 14123-master.v15.txt, 
> 14123-master.v16.txt, 14123-master.v17.txt, 14123-master.v18.txt, 
> 14123-master.v19.txt, 14123-master.v2.txt, 14123-master.v20.txt, 
> 14123-master.v21.txt, 14123-master.v24.txt, 14123-master.v25.txt, 
> 14123-master.v27.txt, 14123-master.v28.txt, 14123-master.v29.full.txt, 
> 14123-master.v3.txt, 14123-master.v30.txt, 14123-master.v31.txt, 
> 14123-master.v32.txt, 14123-master.v33.txt, 14123-master.v34.txt, 
> 14123-master.v35.txt, 14123-master.v36.txt, 14123-master.v37.txt, 
> 14123-master.v38.txt, 14123-master.v5.txt, 14123-master.v6.txt, 
> 14123-master.v7.txt, 14123-master.v8.txt, 14123-master.v9.txt, 14123-v14.txt, 
> 14123.master.v39.patch, HBASE-14123-for-7912-v1.patch, 
> HBASE-14123-for-7912-v6.patch, HBASE-14123-v1.patch, HBASE-14123-v10.patch, 
> HBASE-14123-v11.patch, HBASE-14123-v12.patch, HBASE-14123-v13.patch, 
> HBASE-14123-v15.patch, HBASE-14123-v16.patch, HBASE-14123-v2.patch, 
> HBASE-14123-v3.patch, HBASE-14123-v4.patch, HBASE-14123-v5.patch, 
> HBASE-14123-v6.patch, HBASE-14123-v7.patch, HBASE-14123-v9.patch
>
>
> Phase 2 umbrella JIRA. See HBASE-7912 for design document and description. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-15437) Response size calculated in RPCServer for warning tooLarge responses does NOT count CellScanner payload

2016-12-06 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-15437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726962#comment-15726962
 ] 

Hadoop QA commented on HBASE-15437:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s 
{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
56s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 42s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
28m 47s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
58s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 94m 50s 
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
14s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 136m 17s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12842021/HBASE-15437-v5.patch |
| JIRA Issue | HBASE-15437 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux ecde998f3b4d 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 1eb24e4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4814/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4814/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Response size calculated in RPCServer for warning tooLarge responses does NOT 
> count CellScanner payload
> ---
>
> Key: HBASE-15437
> URL: https://issues.apache.org/jira/browse/HBASE-15437
> Project: HBase
>  

[jira] [Updated] (HBASE-17224) There are lots of spelling errors in the HBase logging and exception messages

2016-12-06 Thread Andrew Purtell (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-17224:
---
Fix Version/s: 0.98.24

> There are lots of spelling errors in the HBase logging and exception messages
> -
>
> Key: HBASE-17224
> URL: https://issues.apache.org/jira/browse/HBASE-17224
> Project: HBase
>  Issue Type: Bug
>  Components: Client, io, mapreduce, master, regionserver, security, 
> wal
>Reporter: Grant Sohn
>Assignee: Grant Sohn
>Priority: Trivial
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.24
>
> Attachments: HBASE-17224.1.patch, hbase-17224.branch-1.patch
>
>
> Found a bunch of spelling errors in log messages and exception messages such 
> as "Stoping" instead of "Stopping", "alligned" instead of "aligned".



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17211) Add more details in log when UnknownScannerException thrown in ScannerCallable

2016-12-06 Thread Andrew Purtell (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-17211:
---
Fix Version/s: 0.98.24

> Add more details in log when UnknownScannerException thrown in ScannerCallable
> --
>
> Key: HBASE-17211
> URL: https://issues.apache.org/jira/browse/HBASE-17211
> Project: HBase
>  Issue Type: Improvement
>Reporter: Yu Li
>Assignee: Yu Li
>Priority: Minor
> Fix For: 2.0.0, 1.4.0, 0.98.24
>
> Attachments: HBASE-17211.patch
>
>
> Currently if we try to close an already closed scanner, we may see below 
> warning log which is useless for debugging:
> {noformat}
> 2016-06-23 01:21:21,101 WARN [main] 
> org.apache.hadoop.hbase.client.ScannerCallable: Ignore, probably already 
> closed
> org.apache.hadoop.hbase.UnknownScannerException: 
> org.apache.hadoop.hbase.UnknownScannerException: Name: 18706910, already 
> closed?
> at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2304)
> at 
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2188)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:102)
> at 
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
> {noformat}
> and here we propose to add more details about the scan in log.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (HBASE-17270) Change Service to Task in BackupRestoreServerFactory

2016-12-06 Thread Vladimir Rodionov (JIRA)
Vladimir Rodionov created HBASE-17270:
-

 Summary: Change Service to Task in BackupRestoreServerFactory
 Key: HBASE-17270
 URL: https://issues.apache.org/jira/browse/HBASE-17270
 Project: HBase
  Issue Type: Task
Affects Versions: HBASE-7912
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov


Small method name refactoring



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17221) Abstract out an interface for RpcServer.Call

2016-12-06 Thread Jerry He (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jerry He updated HBASE-17221:
-
Hadoop Flags: Incompatible change,Reviewed
Release Note: 
Provide an interface RpcCall on the server side. 
RpcServer.Call now is marked as @InterfaceAudience.Private, and implements the 
interface RpcCall,

> Abstract out an interface for RpcServer.Call
> 
>
> Key: HBASE-17221
> URL: https://issues.apache.org/jira/browse/HBASE-17221
> Project: HBase
>  Issue Type: Improvement
>Reporter: Jerry He
>Assignee: Jerry He
> Fix For: 2.0.0
>
> Attachments: HBASE-17221-v2.patch, HBASE-17221-v3.patch, 
> HBASE-17221-v4.patch, HBASE-17221.patch
>
>
> RpcServer.Call is a concrete class, but it is marked as:
> {noformat}
> @InterfaceAudience.LimitedPrivate({HBaseInterfaceAudience.COPROC, 
> HBaseInterfaceAudience.PHOENIX})
> {noformat}
> Let's abstract out an interface out of it for potential consumers that want 
> to pass it around.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17221) Abstract out an interface for RpcServer.Call

2016-12-06 Thread Jerry He (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726619#comment-15726619
 ] 

Jerry He commented on HBASE-17221:
--

Updated, [~anoop.hbase]

> Abstract out an interface for RpcServer.Call
> 
>
> Key: HBASE-17221
> URL: https://issues.apache.org/jira/browse/HBASE-17221
> Project: HBase
>  Issue Type: Improvement
>Reporter: Jerry He
>Assignee: Jerry He
> Fix For: 2.0.0
>
> Attachments: HBASE-17221-v2.patch, HBASE-17221-v3.patch, 
> HBASE-17221-v4.patch, HBASE-17221.patch
>
>
> RpcServer.Call is a concrete class, but it is marked as:
> {noformat}
> @InterfaceAudience.LimitedPrivate({HBaseInterfaceAudience.COPROC, 
> HBaseInterfaceAudience.PHOENIX})
> {noformat}
> Let's abstract out an interface out of it for potential consumers that want 
> to pass it around.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-15437) Response size calculated in RPCServer for warning tooLarge responses does NOT count CellScanner payload

2016-12-06 Thread Jerry He (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-15437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726601#comment-15726601
 ] 

Jerry He commented on HBASE-15437:
--

HI, [~anoop.hbase]

Thanks for the review. All good points. 
v5 addressed your suggestions.

> Response size calculated in RPCServer for warning tooLarge responses does NOT 
> count CellScanner payload
> ---
>
> Key: HBASE-15437
> URL: https://issues.apache.org/jira/browse/HBASE-15437
> Project: HBase
>  Issue Type: Bug
>  Components: IPC/RPC
>Reporter: deepankar
>Assignee: Jerry He
> Attachments: HBASE-15437-v1.patch, HBASE-15437-v2.patch, 
> HBASE-15437-v3.patch, HBASE-15437-v4.patch, HBASE-15437-v5.patch, 
> HBASE-15437.patch
>
>
> After HBASE-13158 where we respond back to RPCs with cells in the payload , 
> the protobuf response will just have the count the cells to read from 
> payload, but there are set of features where we log warn in RPCServer 
> whenever the response is tooLarge, but this size now is not considering the 
> sizes of the cells in the PayloadCellScanner. Code form RPCServer
> {code}
>   long responseSize = result.getSerializedSize();
>   // log any RPC responses that are slower than the configured warn
>   // response time or larger than configured warning size
>   boolean tooSlow = (processingTime > warnResponseTime && 
> warnResponseTime > -1);
>   boolean tooLarge = (responseSize > warnResponseSize && warnResponseSize 
> > -1);
>   if (tooSlow || tooLarge) {
> // when tagging, we let TooLarge trump TooSmall to keep output simple
> // note that large responses will often also be slow.
> logResponse(new Object[]{param},
> md.getName(), md.getName() + "(" + param.getClass().getName() + 
> ")",
> (tooLarge ? "TooLarge" : "TooSlow"),
> status.getClient(), startTime, processingTime, qTime,
> responseSize);
>   }
> {code}
> Should this feature be not supported any more or should we add a method to 
> CellScanner or a new interface which returns the serialized size (but this 
> might not include the compression codecs which might be used during response 
> ?) Any other Idea this could be fixed ?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-15437) Response size calculated in RPCServer for warning tooLarge responses does NOT count CellScanner payload

2016-12-06 Thread Jerry He (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-15437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jerry He updated HBASE-15437:
-
Attachment: HBASE-15437-v5.patch

> Response size calculated in RPCServer for warning tooLarge responses does NOT 
> count CellScanner payload
> ---
>
> Key: HBASE-15437
> URL: https://issues.apache.org/jira/browse/HBASE-15437
> Project: HBase
>  Issue Type: Bug
>  Components: IPC/RPC
>Reporter: deepankar
>Assignee: Jerry He
> Attachments: HBASE-15437-v1.patch, HBASE-15437-v2.patch, 
> HBASE-15437-v3.patch, HBASE-15437-v4.patch, HBASE-15437-v5.patch, 
> HBASE-15437.patch
>
>
> After HBASE-13158 where we respond back to RPCs with cells in the payload , 
> the protobuf response will just have the count the cells to read from 
> payload, but there are set of features where we log warn in RPCServer 
> whenever the response is tooLarge, but this size now is not considering the 
> sizes of the cells in the PayloadCellScanner. Code form RPCServer
> {code}
>   long responseSize = result.getSerializedSize();
>   // log any RPC responses that are slower than the configured warn
>   // response time or larger than configured warning size
>   boolean tooSlow = (processingTime > warnResponseTime && 
> warnResponseTime > -1);
>   boolean tooLarge = (responseSize > warnResponseSize && warnResponseSize 
> > -1);
>   if (tooSlow || tooLarge) {
> // when tagging, we let TooLarge trump TooSmall to keep output simple
> // note that large responses will often also be slow.
> logResponse(new Object[]{param},
> md.getName(), md.getName() + "(" + param.getClass().getName() + 
> ")",
> (tooLarge ? "TooLarge" : "TooSlow"),
> status.getClient(), startTime, processingTime, qTime,
> responseSize);
>   }
> {code}
> Should this feature be not supported any more or should we add a method to 
> CellScanner or a new interface which returns the serialized size (but this 
> might not include the compression codecs which might be used during response 
> ?) Any other Idea this could be fixed ?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17241) Avoid compacting already compacted mob files with _del files

2016-12-06 Thread huaxiang sun (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726559#comment-15726559
 ] 

huaxiang sun commented on HBASE-17241:
--

Thanks Ted and Anoop for review!

> Avoid compacting already compacted  mob files with _del files
> -
>
> Key: HBASE-17241
> URL: https://issues.apache.org/jira/browse/HBASE-17241
> Project: HBase
>  Issue Type: Improvement
>  Components: mob
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
> Fix For: 2.0.0
>
> Attachments: HBASE-17241-master-002.patch, 
> HBASE-17241-master-003.patch, HBASE-17241.master.001.patch
>
>
> Today if there is only one file in the partition, and there is no _del files, 
> the file is skipped. With del file, the current logic is to compact the 
> already-compacted file with _del file. Let's say there is one mob file 
> regionA20161101***, which was compacted. On 12/1/2016, there is _del file 
> regionB20161201**_del, mob compaction kicks in, regionA20161101*** is less 
> than the threshold, and it is picked for compaction. Since there is a _del 
> file, regionA20161101 and regionB20161201***_del are compacted into 
> regionA20161101**_1 . After that, regionB20161201**_del cannot be deleted 
> since it is not a allFile compaction. The next mob compaction, 
> regionA20161101**_1 and regionB20161201**_del will be picked up again and be 
> compacted into regionA20161101***_2. So in this case, it will cause more 
> unnecessary IOs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17136) Fix logging message in LoadIncrementalHFiles

2016-12-06 Thread Ted Yu (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-17136:
---
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 2.0.0
   Status: Resolved  (was: Patch Available)

Integrated without the changes in import.

> Fix logging message in LoadIncrementalHFiles
> 
>
> Key: HBASE-17136
> URL: https://issues.apache.org/jira/browse/HBASE-17136
> Project: HBase
>  Issue Type: Bug
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-17136-v1.patch
>
>
> {quote}
> Want to fix this log message so it doesn't have java object ids in it? 
> 2016-11-17 14:11:14,639 DEBUG [LoadIncrementalHFiles-2] 
> mapreduce.LoadIncrementalHFiles: Going to connect to server 
> region=x_1_restore,999adfe23d3bda876af50397a462f7d8-18028,1479420656653.15bd3f57fb7364c5d738f8bf7be7a32c.,
>  hostname=ve0540.halxg.cloudera.com,16020,1479274141369, seqNum=2 for row 
> 999adfe23d3bda876af50397a462f7d8-18028 with hfile group [
> {[B@68c87fc3,hdfs://ve0524.halxg.cloudera.com:8020/user/stack/hbase-staging/restore/fa3bff72113aee83c50c812c722cd8f4/test_cf/33a9cac5e3944e338005dc87e70f85fb}
> ,
> {[B@68c87fc3,hdfs://ve0524.halxg.cloudera.com:8020/user/stack/hbase-staging/restore/fa3bff72113aee83c50c812c722cd8f4/test_cf/935f048447394f93856e917b8dc34a8f}
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17136) Fix logging message in LoadIncrementalHFiles

2016-12-06 Thread Ted Yu (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-17136:
---
Summary: Fix logging message in LoadIncrementalHFiles  (was: Fix logging 
message in backup restore)

> Fix logging message in LoadIncrementalHFiles
> 
>
> Key: HBASE-17136
> URL: https://issues.apache.org/jira/browse/HBASE-17136
> Project: HBase
>  Issue Type: Bug
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Minor
> Attachments: HBASE-17136-v1.patch
>
>
> {quote}
> Want to fix this log message so it doesn't have java object ids in it? 
> 2016-11-17 14:11:14,639 DEBUG [LoadIncrementalHFiles-2] 
> mapreduce.LoadIncrementalHFiles: Going to connect to server 
> region=x_1_restore,999adfe23d3bda876af50397a462f7d8-18028,1479420656653.15bd3f57fb7364c5d738f8bf7be7a32c.,
>  hostname=ve0540.halxg.cloudera.com,16020,1479274141369, seqNum=2 for row 
> 999adfe23d3bda876af50397a462f7d8-18028 with hfile group [
> {[B@68c87fc3,hdfs://ve0524.halxg.cloudera.com:8020/user/stack/hbase-staging/restore/fa3bff72113aee83c50c812c722cd8f4/test_cf/33a9cac5e3944e338005dc87e70f85fb}
> ,
> {[B@68c87fc3,hdfs://ve0524.halxg.cloudera.com:8020/user/stack/hbase-staging/restore/fa3bff72113aee83c50c812c722cd8f4/test_cf/935f048447394f93856e917b8dc34a8f}
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17136) Fix logging message in backup restore

2016-12-06 Thread Vladimir Rodionov (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726477#comment-15726477
 ] 

Vladimir Rodionov commented on HBASE-17136:
---

Ping [~tedyu]

> Fix logging message in backup restore
> -
>
> Key: HBASE-17136
> URL: https://issues.apache.org/jira/browse/HBASE-17136
> Project: HBase
>  Issue Type: Bug
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Minor
> Attachments: HBASE-17136-v1.patch
>
>
> {quote}
> Want to fix this log message so it doesn't have java object ids in it? 
> 2016-11-17 14:11:14,639 DEBUG [LoadIncrementalHFiles-2] 
> mapreduce.LoadIncrementalHFiles: Going to connect to server 
> region=x_1_restore,999adfe23d3bda876af50397a462f7d8-18028,1479420656653.15bd3f57fb7364c5d738f8bf7be7a32c.,
>  hostname=ve0540.halxg.cloudera.com,16020,1479274141369, seqNum=2 for row 
> 999adfe23d3bda876af50397a462f7d8-18028 with hfile group [
> {[B@68c87fc3,hdfs://ve0524.halxg.cloudera.com:8020/user/stack/hbase-staging/restore/fa3bff72113aee83c50c812c722cd8f4/test_cf/33a9cac5e3944e338005dc87e70f85fb}
> ,
> {[B@68c87fc3,hdfs://ve0524.halxg.cloudera.com:8020/user/stack/hbase-staging/restore/fa3bff72113aee83c50c812c722cd8f4/test_cf/935f048447394f93856e917b8dc34a8f}
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17241) Avoid compacting already compacted mob files with _del files

2016-12-06 Thread Ted Yu (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-17241:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Avoid compacting already compacted  mob files with _del files
> -
>
> Key: HBASE-17241
> URL: https://issues.apache.org/jira/browse/HBASE-17241
> Project: HBase
>  Issue Type: Improvement
>  Components: mob
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
> Fix For: 2.0.0
>
> Attachments: HBASE-17241-master-002.patch, 
> HBASE-17241-master-003.patch, HBASE-17241.master.001.patch
>
>
> Today if there is only one file in the partition, and there is no _del files, 
> the file is skipped. With del file, the current logic is to compact the 
> already-compacted file with _del file. Let's say there is one mob file 
> regionA20161101***, which was compacted. On 12/1/2016, there is _del file 
> regionB20161201**_del, mob compaction kicks in, regionA20161101*** is less 
> than the threshold, and it is picked for compaction. Since there is a _del 
> file, regionA20161101 and regionB20161201***_del are compacted into 
> regionA20161101**_1 . After that, regionB20161201**_del cannot be deleted 
> since it is not a allFile compaction. The next mob compaction, 
> regionA20161101**_1 and regionB20161201**_del will be picked up again and be 
> compacted into regionA20161101***_2. So in this case, it will cause more 
> unnecessary IOs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17269) Intermittent TestMasterProcedureSchedulerConcurrency failure

2016-12-06 Thread Matteo Bertozzi (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726386#comment-15726386
 ] 

Matteo Bertozzi commented on HBASE-17269:
-

this is known to be a bit flaky since it relies on timing, but it should be 
solved by HBASE-17067. but it will take a week or two to get that one in. we 
are trying to get in what that is depending on

> Intermittent TestMasterProcedureSchedulerConcurrency failure
> 
>
> Key: HBASE-17269
> URL: https://issues.apache.org/jira/browse/HBASE-17269
> Project: HBase
>  Issue Type: Test
>Reporter: Ted Yu
>Assignee: Matteo Bertozzi
>Priority: Minor
>
> TestMasterProcedureSchedulerConcurrency sometimes appeared as timed out test 
> in QA runs.
> In 
> https://builds.apache.org/job/HBase-TRUNK_matrix/2083/jdk=JDK%201.8%20(latest),label=Hadoop/testReport/org.apache.hadoop.hbase.master.procedure/TestMasterProcedureSchedulerConcurrency/testMasterProcedureSchedulerPerformanceEvaluation/
>  :
> I saw:
> {code}
> 2016-12-06 14:22:23,888 ERROR [Time-limited test] 
> util.AbstractHBaseTool(151): Error running command-line tool
> java.lang.InterruptedException
>   at java.lang.Object.wait(Native Method)
>   at java.lang.Thread.join(Thread.java:1249)
>   at java.lang.Thread.join(Thread.java:1323)
>   at 
> org.apache.hadoop.hbase.master.procedure.MasterProcedureSchedulerPerformanceEvaluation.runThreads(MasterProcedureSchedulerPerformanceEvaluation.java:239)
>   at 
> org.apache.hadoop.hbase.master.procedure.MasterProcedureSchedulerPerformanceEvaluation.doWork(MasterProcedureSchedulerPerformanceEvaluation.java:261)
>   at 
> org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:149)
>   at 
> org.apache.hadoop.hbase.master.procedure.MasterProcedureSchedulerPerformanceEvaluation.main(MasterProcedureSchedulerPerformanceEvaluation.java:294)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Assigned] (HBASE-17269) Intermittent TestMasterProcedureSchedulerConcurrency failure

2016-12-06 Thread Matteo Bertozzi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi reassigned HBASE-17269:
---

Assignee: Matteo Bertozzi

> Intermittent TestMasterProcedureSchedulerConcurrency failure
> 
>
> Key: HBASE-17269
> URL: https://issues.apache.org/jira/browse/HBASE-17269
> Project: HBase
>  Issue Type: Test
>Reporter: Ted Yu
>Assignee: Matteo Bertozzi
>Priority: Minor
>
> TestMasterProcedureSchedulerConcurrency sometimes appeared as timed out test 
> in QA runs.
> In 
> https://builds.apache.org/job/HBase-TRUNK_matrix/2083/jdk=JDK%201.8%20(latest),label=Hadoop/testReport/org.apache.hadoop.hbase.master.procedure/TestMasterProcedureSchedulerConcurrency/testMasterProcedureSchedulerPerformanceEvaluation/
>  :
> I saw:
> {code}
> 2016-12-06 14:22:23,888 ERROR [Time-limited test] 
> util.AbstractHBaseTool(151): Error running command-line tool
> java.lang.InterruptedException
>   at java.lang.Object.wait(Native Method)
>   at java.lang.Thread.join(Thread.java:1249)
>   at java.lang.Thread.join(Thread.java:1323)
>   at 
> org.apache.hadoop.hbase.master.procedure.MasterProcedureSchedulerPerformanceEvaluation.runThreads(MasterProcedureSchedulerPerformanceEvaluation.java:239)
>   at 
> org.apache.hadoop.hbase.master.procedure.MasterProcedureSchedulerPerformanceEvaluation.doWork(MasterProcedureSchedulerPerformanceEvaluation.java:261)
>   at 
> org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:149)
>   at 
> org.apache.hadoop.hbase.master.procedure.MasterProcedureSchedulerPerformanceEvaluation.main(MasterProcedureSchedulerPerformanceEvaluation.java:294)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (HBASE-17260) Procedure v2 - Add setOwner() overload taking a User instance

2016-12-06 Thread Matteo Bertozzi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-17260:

Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Procedure v2 - Add setOwner() overload taking a User instance
> -
>
> Key: HBASE-17260
> URL: https://issues.apache.org/jira/browse/HBASE-17260
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Trivial
> Fix For: 2.0.0
>
> Attachments: HBASE-17260-v0.patch
>
>
> since we should have a User instance in most of the cases, we should just be 
> able to pass it, rather than converting it to getShortName() every time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (HBASE-17269) Intermittent TestMasterProcedureSchedulerConcurrency failure

2016-12-06 Thread Ted Yu (JIRA)
Ted Yu created HBASE-17269:
--

 Summary: Intermittent TestMasterProcedureSchedulerConcurrency 
failure
 Key: HBASE-17269
 URL: https://issues.apache.org/jira/browse/HBASE-17269
 Project: HBase
  Issue Type: Test
Reporter: Ted Yu
Priority: Minor


TestMasterProcedureSchedulerConcurrency sometimes appeared as timed out test in 
QA runs.

In 
https://builds.apache.org/job/HBase-TRUNK_matrix/2083/jdk=JDK%201.8%20(latest),label=Hadoop/testReport/org.apache.hadoop.hbase.master.procedure/TestMasterProcedureSchedulerConcurrency/testMasterProcedureSchedulerPerformanceEvaluation/
 :

I saw:
{code}
2016-12-06 14:22:23,888 ERROR [Time-limited test] util.AbstractHBaseTool(151): 
Error running command-line tool
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1249)
at java.lang.Thread.join(Thread.java:1323)
at 
org.apache.hadoop.hbase.master.procedure.MasterProcedureSchedulerPerformanceEvaluation.runThreads(MasterProcedureSchedulerPerformanceEvaluation.java:239)
at 
org.apache.hadoop.hbase.master.procedure.MasterProcedureSchedulerPerformanceEvaluation.doWork(MasterProcedureSchedulerPerformanceEvaluation.java:261)
at 
org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:149)
at 
org.apache.hadoop.hbase.master.procedure.MasterProcedureSchedulerPerformanceEvaluation.main(MasterProcedureSchedulerPerformanceEvaluation.java:294)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (HBASE-16940) Address review of "Backup/Restore (HBASE-7912, HBASE-14030, HBASE-14123) mega patch" posted on RB

2016-12-06 Thread Ted Yu (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-16940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu resolved HBASE-16940.

Resolution: Fixed

Addressed the above comment on commit.

> Address review of "Backup/Restore (HBASE-7912, HBASE-14030, HBASE-14123) mega 
> patch" posted on RB 
> --
>
> Key: HBASE-16940
> URL: https://issues.apache.org/jira/browse/HBASE-16940
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-16940-v1.patch, HBASE-16940-v2.patch, 
> HBASE-16940.HBASE-7912.v2.patch, HBASE-16940.addendum, HBASE-16940.addendum.2
>
>
> Review 52748 remaining issues.
> https://reviews.apache.org/r/52748



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (HBASE-17268) Add OffheapMemoryTuner

2016-12-06 Thread Anoop Sam John (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-17268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anoop Sam John resolved HBASE-17268.

Resolution: Duplicate
  Assignee: (was: ramkrishna.s.vasudevan)

Dup of HBASE-17267

> Add OffheapMemoryTuner
> --
>
> Key: HBASE-17268
> URL: https://issues.apache.org/jira/browse/HBASE-17268
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver
>Affects Versions: 2.0.0
>Reporter: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
>
> This JIRA is aimed at tuning the offheap memory. It is not straight forward 
> as we should not cross the available Direct_memory configured.
> Should include BC and offheap memstore configs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Ted Yu (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726303#comment-15726303
 ] 

Ted Yu commented on HBASE-17182:


{code}
+  private final CleanScannerThread cleanScannerThread;
{code}
I think scannerCleanerThread would be better name (please change class name and 
variable name).
{code}
+cleanScannerThread = new CleanScannerThread();
{code}
Please set as background thread.
{code}
+  final long maxIdleTime = 6; // 1minutes
{code}
Please make the timeout same as hbase.client.scanner.timeout.period .
{code}
+} catch (InterruptedException e) {
+  // LOG.error(e);
{code}
Restore interrupt status.
{code}
+  String str = String.format("Remove timeout scanner(%d).", 
scannerId);
{code}
"timeout" -> "timedout"
{code}
+  // scanner.close(); // Close will throw UnknownScannerException
{code}
I think the close() call should be made, in case the scanner is still active 
server side.
We need to catch UnknownScannerException.

> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>Assignee: Jian Yi
> Attachments: HBASE-17182-V1.patch, HBASE-17182-V2.patch
>
>
> Client call openScanner, but client (coredump or others) not closeScanner, 
> the scanner will not be removed from scannerMap.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17249) Get/Scan's setTimeRange/setColumnFamilyTimeRange can take the TimeRange reference as the parameter instead of creating a new setColumnFamilyTimeRange instance

2016-12-06 Thread huaxiang sun (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726284#comment-15726284
 ] 

huaxiang sun commented on HBASE-17249:
--

Thanks [~anoopa], will take care of the comments and upload a new patch.

> Get/Scan's setTimeRange/setColumnFamilyTimeRange can take the TimeRange 
> reference as the parameter instead of creating a new setColumnFamilyTimeRange 
> instance
> --
>
> Key: HBASE-17249
> URL: https://issues.apache.org/jira/browse/HBASE-17249
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-17249-master-001.patch, 
> HBASE-17249-master-002.patch
>
>
> Going through the code, found For Get/Scan's 
> setTimeRange/setColumnFamilyTimeRange, it can use  TimeRange as reference 
> instead of creating a new one.
> Reference:
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java#L500
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java#L506
> We can implement this in a similar way as filter:
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java#L510
> I checked it is same with branch-1.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-16981) Expand Mob Compaction Partition policy from daily to weekly, monthly and beyond

2016-12-06 Thread huaxiang sun (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-16981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726279#comment-15726279
 ] 

huaxiang sun commented on HBASE-16981:
--

Thanks [~jingcheng.du]

> Expand Mob Compaction Partition policy from daily to weekly, monthly and 
> beyond
> ---
>
> Key: HBASE-16981
> URL: https://issues.apache.org/jira/browse/HBASE-16981
> Project: HBase
>  Issue Type: New Feature
>  Components: mob
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
> Attachments: HBASE-16981.master.001.patch, 
> HBASE-16981.master.002.patch, 
> Supportingweeklyandmonthlymobcompactionpartitionpolicyinhbase.pdf
>
>
> Today the mob region holds all mob files for all regions. With daily 
> partition mob compaction policy, after major mob compaction, there is still 
> one file per region daily. Given there is 365 days in one year, at least 365 
> files per region. Since HDFS has limitation for number of files under one 
> folder, this is not going to scale if there are lots of regions. To reduce 
> mob file number,  we want to introduce other partition policies such as 
> weekly, monthly to compact mob files within one week or month into one file. 
> This jira is create to track this effort.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (HBASE-16940) Address review of "Backup/Restore (HBASE-7912, HBASE-14030, HBASE-14123) mega patch" posted on RB

2016-12-06 Thread Ted Yu (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-16940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726260#comment-15726260
 ] 

Ted Yu edited comment on HBASE-16940 at 12/6/16 6:15 PM:
-

Please fix the reference in 
hbase-server/src/main/java/org/apache/hadoop/hbase/backup/BackupAdmin.java :
{code}
 * {@link HBaseBackupAdmin(Connection)} and call {@link #close()} afterwards.
{code}



was (Author: yuzhih...@gmail.com):
Please fix the reference in 
hbase-server/src/main/java/org/apache/hadoop/hbase/backup/BackupAdmin.java :
{code}
 * {@link HBaseBackupAdmin(Connection)} and call {@link #close()} afterwards.
{code}
{code}
+public class FullTableBackupClient extends TableBackupClient{
{code}
What about IncrementalTableBackupClient ?


> Address review of "Backup/Restore (HBASE-7912, HBASE-14030, HBASE-14123) mega 
> patch" posted on RB 
> --
>
> Key: HBASE-16940
> URL: https://issues.apache.org/jira/browse/HBASE-16940
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-16940-v1.patch, HBASE-16940-v2.patch, 
> HBASE-16940.HBASE-7912.v2.patch, HBASE-16940.addendum, HBASE-16940.addendum.2
>
>
> Review 52748 remaining issues.
> https://reviews.apache.org/r/52748



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-16940) Address review of "Backup/Restore (HBASE-7912, HBASE-14030, HBASE-14123) mega patch" posted on RB

2016-12-06 Thread Ted Yu (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-16940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726260#comment-15726260
 ] 

Ted Yu commented on HBASE-16940:


Please fix the reference in 
hbase-server/src/main/java/org/apache/hadoop/hbase/backup/BackupAdmin.java :
{code}
 * {@link HBaseBackupAdmin(Connection)} and call {@link #close()} afterwards.
{code}
{code}
+public class FullTableBackupClient extends TableBackupClient{
{code}
What about IncrementalTableBackupClient ?


> Address review of "Backup/Restore (HBASE-7912, HBASE-14030, HBASE-14123) mega 
> patch" posted on RB 
> --
>
> Key: HBASE-16940
> URL: https://issues.apache.org/jira/browse/HBASE-16940
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-16940-v1.patch, HBASE-16940-v2.patch, 
> HBASE-16940.HBASE-7912.v2.patch, HBASE-16940.addendum, HBASE-16940.addendum.2
>
>
> Review 52748 remaining issues.
> https://reviews.apache.org/r/52748



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-15787) Change the flush related heuristics to work with offheap size configured

2016-12-06 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-15787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726229#comment-15726229
 ] 

Hadoop QA commented on HBASE-15787:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 4 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
13s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s 
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git 
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
25m 35s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 57s 
{color} | {color:red} hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 
total (was 0) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 26s 
{color} | {color:red} hbase-server generated 1 new + 1 unchanged - 0 fixed = 2 
total (was 1) {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 14m 12s {color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
8s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 51m 40s {color} 
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
|  |  int value cast to float and then passed to Math.round in 
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.calculateMaxLogFiles(Configuration,
 long)  At AbstractFSWAL.java:and then passed to Math.round in 
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.calculateMaxLogFiles(Configuration,
 long)  At AbstractFSWAL.java:[line 289] |
| Failed junit tests | hadoop.hbase.regionserver.TestHeapMemoryManager |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12841993/HBASE-15787_5.patch |
| JIRA Issue | HBASE-15787 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux 9179a93d76fd 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / f112427 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4813/artifact/patchprocess/whitespace-eol.txt
 |
| findbugs | 

[jira] [Commented] (HBASE-16940) Address review of "Backup/Restore (HBASE-7912, HBASE-14030, HBASE-14123) mega patch" posted on RB

2016-12-06 Thread Vladimir Rodionov (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-16940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726217#comment-15726217
 ] 

Vladimir Rodionov commented on HBASE-16940:
---

Done. Most of the comments have been addressed, except four, which are relevant 
only for master branch.

All tests pass.

> Address review of "Backup/Restore (HBASE-7912, HBASE-14030, HBASE-14123) mega 
> patch" posted on RB 
> --
>
> Key: HBASE-16940
> URL: https://issues.apache.org/jira/browse/HBASE-16940
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-16940-v1.patch, HBASE-16940-v2.patch, 
> HBASE-16940.HBASE-7912.v2.patch, HBASE-16940.addendum, HBASE-16940.addendum.2
>
>
> Review 52748 remaining issues.
> https://reviews.apache.org/r/52748



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (HBASE-17268) Add OffheapMemoryTuner

2016-12-06 Thread ramkrishna.s.vasudevan (JIRA)
ramkrishna.s.vasudevan created HBASE-17268:
--

 Summary: Add OffheapMemoryTuner
 Key: HBASE-17268
 URL: https://issues.apache.org/jira/browse/HBASE-17268
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 2.0.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan


This JIRA is aimed at tuning the offheap memory. It is not straight forward as 
we should not cross the available Direct_memory configured.
Should include BC and offheap memstore configs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Assigned] (HBASE-16786) Procedure V2 - Move ZK-lock's uses to Procedure framework locks (LockProcedure)

2016-12-06 Thread Matteo Bertozzi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-16786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi reassigned HBASE-16786:
---

Assignee: Matteo Bertozzi  (was: Appy)

> Procedure V2 - Move ZK-lock's uses to Procedure framework locks 
> (LockProcedure)
> ---
>
> Key: HBASE-16786
> URL: https://issues.apache.org/jira/browse/HBASE-16786
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Appy
>Assignee: Matteo Bertozzi
> Attachments: HBASE-16786.master.001.patch, 
> HBASE-16786.master.002.patch, HBASE-16786.master.003.patch, 
> HBASE-16786.master.004.patch, HBASE-16786.master.005.patch, 
> HBASE-16786.master.006.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Assigned] (HBASE-16744) Procedure V2 - Lock procedures to allow clients to acquire locks on tables/namespaces/regions

2016-12-06 Thread Matteo Bertozzi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-16744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi reassigned HBASE-16744:
---

Assignee: Matteo Bertozzi  (was: Appy)

> Procedure V2 - Lock procedures to allow clients to acquire locks on 
> tables/namespaces/regions
> -
>
> Key: HBASE-16744
> URL: https://issues.apache.org/jira/browse/HBASE-16744
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Appy
>Assignee: Matteo Bertozzi
> Attachments: HBASE-16744.master.001.patch, 
> HBASE-16744.master.002.patch, HBASE-16744.master.003.patch, 
> HBASE-16744.master.004.patch, HBASE-16744.master.005.patch, 
> HBASE-16744.master.006.patch, HBASE-16744.master.007.patch, 
> HBASE-16744.master.008.patch, HBASE-16744.master.009.patch, 
> HBASE-16744.master.010.patch, HBASE-16744.master.011.patch, 
> HBASE-16744.master.012.patch, HBASE-16744.master.013.patch
>
>
> Will help us get rid of ZK locks.
> Will be useful for external tools like hbck, future backup manager, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (HBASE-17267) Add OffheapMemoryTuner

2016-12-06 Thread ramkrishna.s.vasudevan (JIRA)
ramkrishna.s.vasudevan created HBASE-17267:
--

 Summary: Add OffheapMemoryTuner
 Key: HBASE-17267
 URL: https://issues.apache.org/jira/browse/HBASE-17267
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 2.0.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan


This JIRA is aimed at tuning the offheap memory. It is not straight forward as 
we should not cross the available Direct_memory configured.
Should include BC and offheap memstore configs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-17138) Backport read-path offheap (HBASE-11425) to branch-1

2016-12-06 Thread ramkrishna.s.vasudevan (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726170#comment-15726170
 ] 

ramkrishna.s.vasudevan commented on HBASE-17138:


Checking this. Will revert back tomorrow.

> Backport read-path offheap (HBASE-11425) to branch-1
> 
>
> Key: HBASE-17138
> URL: https://issues.apache.org/jira/browse/HBASE-17138
> Project: HBase
>  Issue Type: Improvement
>Reporter: Yu Li
>Assignee: Yu Sun
>
> From the 
> [thread|http://mail-archives.apache.org/mod_mbox/hbase-user/201611.mbox/%3CCAM7-19%2Bn7cEiY4H9iLQ3N9V0NXppOPduZwk-hhgNLEaJfiV3kA%40mail.gmail.com%3E]
>  of sharing our experience and performance data of read-path offheap usage in 
> Alibaba search, we could see people are positive to have HBASE-11425 in 
> branch-1, so I'd like to create a JIRA and move the discussion and decision 
> making here.
> Echoing some comments from the mail thread:
> Bryan:
> Is the backported patch available anywhere? If it ends up not getting 
> officially backported to branch-1 due to 2.0 around the corner, some of us 
> who build our own deploy may want to integrate into our builds
> Andrew:
> Yes, please, the patches will be useful to the community even if we decide 
> not to backport into an official 1.x release.
> Enis:
> I don't see any reason why we cannot backport to branch-1.
> Ted:
> Opening a JIRA would be fine. This makes it easier for people to obtain the 
> patch(es)
> Nick:
> From the DISCUSS thread re: EOL of 1.1, it seems we'll continue to
> support 1.x releases for some time... I would guess these will be
> maintained until 2.2 at least. Therefore, offheap patches that have seen
> production exposure seem like a reasonable candidate for backport, perhaps in 
> a 1.4 or 1.5 release timeframe.
> Anoop:
> Because of some compatibility issues, we decide that this will be done in 2.0 
> only..  Ya as Andy said, it would be great to share the 1.x backported 
> patches.
> The following is all the jira ids we have back ported:
> HBASE-10930 Change Filters and GetClosestRowBeforeTracker to work with Cells 
> (Ram)
> HBASE-13373 Squash HFileReaderV3 together with HFileReaderV2 and 
> AbstractHFileReader; ditto for Scanners and BlockReader, etc.
> HBASE-13429 Remove deprecated seek/reseek methods from HFileScanner.
> HBASE-13450 - Purge RawBytescomparator from the writers and readers for 
> HBASE-10800 (Ram)
> HBASE-13501 - Deprecate/Remove getComparator() in HRegionInfo.
> HBASE-12048 Remove deprecated APIs from Filter.
> HBASE-10800 - Use CellComparator instead of KVComparator (Ram)
> HBASE-13679 Change ColumnTracker and SQM to deal with Cell instead of byte[], 
> int, int.
> HBASE-13642 Deprecate RegionObserver#postScannerFilterRow CP hook with 
> byte[],int,int args in favor of taking Cell arg.
> HBASE-13641 Deperecate Filter#filterRowKey(byte[] buffer, int offset, int 
> length) in favor of filterRowKey(Cell firstRowCell).
> HBASE-13827 Delayed scanner close in KeyValueHeap and StoreScanner.
> HBASE-13871 Change RegionScannerImpl to deal with Cell instead of byte[], 
> int, int.
> HBASE-11911 Break up tests into more fine grained categories (Alex Newman)
> HBASE-12059 Create hbase-annotations module
> HBASE-12106 Move test annotations to test artifact (Enis Soztutar)
> HBASE-13916 Create MultiByteBuffer an aggregation of ByteBuffers.
> HBASE-15679 Assertion on wrong variable in 
> TestReplicationThrottler#testThrottling
> HBASE-13931 Move Unsafe based operations to UnsafeAccess.
> HBASE-12345 Unsafe based ByteBuffer Comparator.
> HBASE-13998 Remove CellComparator#compareRows(byte[] left, int loffset, int 
> llength, byte[] right, int roffset, int rlength).
> HBASE-13998 Remove CellComparator#compareRows()- Addendum to fix javadoc warn
> HBASE-13579 Avoid isCellTTLExpired() for NO-TAG cases (partially backport 
> this patch)
> HBASE-13448 New Cell implementation with cached component offsets/lengths.
> HBASE-13387 Add ByteBufferedCell an extension to Cell.
> HBASE-13387 Add ByteBufferedCell an extension to Cell - addendum.
> HBASE-12650 Move ServerName to hbase-common module (partially backport this 
> patch)
> HBASE-12296 Filters should work with ByteBufferedCell.
> HBASE-14120 ByteBufferUtils#compareTo small optimization.
> HBASE-13510 - Purge ByteBloomFilter (Ram)
> HBASE-13451 - Make the HFileBlockIndex blockKeys to Cells so that it could be 
> easy to use in the CellComparators (Ram)
> HBASE-13614 - Avoid temp KeyOnlyKeyValue temp objects creations in read hot 
> path (Ram)
> HBASE-13939 - Make HFileReaderImpl.getFirstKeyInBlock() to return a Cell (Ram)
> HBASE-13307 Making methods under ScannerV2#next inlineable, faster
> HBASE-14020 Unsafe based optimized write in ByteBufferOutputStream.
> HBASE-13977 - Convert getKey and related APIs to Cell (Ram)
> HBASE-11927 

[jira] [Commented] (HBASE-17182) Memory leak from openScanner of HBase thrift2

2016-12-06 Thread Phil Yang (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-17182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726090#comment-15726090
 ] 

Phil Yang commented on HBASE-17182:
---

Oh, one concern is the close logic. I thought I had post it, sorry. If we close 
a expired scanner in the main thread when it is removed by the guava cache, the 
accessing may be blocked to wait the close RPC request. It is bad. But the 
expired scanner must also be expired at region server (and the TTL in thrift 
server should same as TTL at region server which is scanner timeout I think). 
So the close RPC will only get an UnknownScannerException. In that case I am 
not sure if we still need to close the scanner. Maybe let it be GCed is enough?
[~eyj...@gmail.com]'s patch also notice this issue. In the patch the close 
logic is commented out. 
[~yuzhih...@gmail.com] What do you think? Thanks.

> Memory leak from openScanner of HBase thrift2
> -
>
> Key: HBASE-17182
> URL: https://issues.apache.org/jira/browse/HBASE-17182
> Project: HBase
>  Issue Type: Bug
>  Components: Thrift
>Reporter: Jian Yi
>Assignee: Jian Yi
> Attachments: HBASE-17182-V1.patch, HBASE-17182-V2.patch
>
>
> Client call openScanner, but client (coredump or others) not closeScanner, 
> the scanner will not be removed from scannerMap.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HBASE-16940) Address review of "Backup/Restore (HBASE-7912, HBASE-14030, HBASE-14123) mega patch" posted on RB

2016-12-06 Thread Ted Yu (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-16940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726083#comment-15726083
 ] 

Ted Yu commented on HBASE-16940:


Alternatively you can mark the comments on https://reviews.apache.org/r/52748 
which are addressed by patch v2.

> Address review of "Backup/Restore (HBASE-7912, HBASE-14030, HBASE-14123) mega 
> patch" posted on RB 
> --
>
> Key: HBASE-16940
> URL: https://issues.apache.org/jira/browse/HBASE-16940
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
> Attachments: HBASE-16940-v1.patch, HBASE-16940-v2.patch, 
> HBASE-16940.HBASE-7912.v2.patch, HBASE-16940.addendum, HBASE-16940.addendum.2
>
>
> Review 52748 remaining issues.
> https://reviews.apache.org/r/52748



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


  1   2   >