[jira] [Commented] (HBASE-17288) Add warn log for huge Cell and huge row

[Guanghao Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750616#comment-15750616
 ] 

Guanghao Zhang commented on HBASE-17288:


bq. HConstants being Public, not to add these kind of constants here.
You mean the constant name or only add it to StoreScanner?
bq. Why KV ref where as we call it Cells these days 
They are old code since we use 0.94. I will change this. Thanks.
bq. We have MOB feature now where each of the Cell might be >1M in size.
If i am not wrong, the MOB feature only use MobStoreScanner. This log is for 
StoreScanner.

> Add warn log for huge Cell and huge row
> ---
>
> Key: HBASE-17288
> URL: https://issues.apache.org/jira/browse/HBASE-17288
> Project: HBase
>  Issue Type: Improvement
>  Components: scan
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
> Attachments: HBASE-17288-v1.patch, HBASE-17288.patch
>
>
> Some log examples from our production cluster.
> {code}
> 2016-12-10,17:08:11,478 WARN 
> org.apache.hadoop.hbase.regionserver.StoreScanner: adding a HUGE KV into 
> result list, kv size:1253360, 
> kv:10567114001-1-c/R:r1/1481360887152/Put/vlen=1253245/ts=923099, from 
> table X
> 2016-12-10,17:08:16,724 WARN 
> org.apache.hadoop.hbase.regionserver.StoreScanner: adding a HUGE KV into 
> result list, kv size:1048680, 
> kv:0220459/I:i_0/1481360889551/Put/vlen=1048576/ts=13642, from table XX
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17292) Add observer notification before bulk loaded hfile is moved to region directory

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750612#comment-15750612
 ] 

Hadoop QA commented on HBASE-17292:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 2s 
{color} | {color:blue} The patch file was not named according to hbase's naming 
conventions. Please see 
https://yetus.apache.org/documentation/0.3.0/precommit-patchnames for 
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s 
{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 
55s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
40s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
13s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s 
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git 
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
24m 55s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 89m 42s 
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
12s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 126m 14s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12843351/17292.v3.txt |
| JIRA Issue | HBASE-17292 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux 6fb704a4ddb4 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / a5a6036 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4925/artifact/patchprocess/whitespace-eol.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4925/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4925/console |
| Powered by | Apache Yetus 0.3.0   

[jira] [Updated] (HBASE-16972) Log more details for Scan#next request when responseTooSlow

[Yu Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yu Li updated HBASE-16972:
--
Affects Version/s: (was: 1.2.3)
   1.2.4
Fix Version/s: (was: 1.2.4)
   1.2.5

As per [~y0un5] point out in HBASE-16033, this one is not included in 1.2.4 
release. Update the affect and fix versions correspondingly.

> Log more details for Scan#next request when responseTooSlow
> ---
>
> Key: HBASE-16972
> URL: https://issues.apache.org/jira/browse/HBASE-16972
> Project: HBase
>  Issue Type: Improvement
>  Components: Operability
>Affects Versions: 1.1.7, 1.2.4
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.5, 1.1.8
>
> Attachments: HBASE-16972.patch, HBASE-16972.v2.patch, 
> HBASE-16972.v3.patch
>
>
> Currently for if responseTooSlow happens on the scan.next call, we will get 
> warn log like below:
> {noformat}
> 2016-10-31 11:43:23,430 WARN  
> [RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=60193] 
> ipc.RpcServer(2574):
> (responseTooSlow): 
> {"call":"Scan(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ScanRequest)",
> "starttimems":1477885403428,"responsesize":52,"method":"Scan","param":"scanner_id:
>  11 number_of_rows: 2147483647
> close_scanner: false next_call_seq: 0 client_handles_partials: true 
> client_handles_heartbeats: true
> track_scan_metrics: false renew: 
> false","processingtimems":2,"client":"127.0.0.1:60254","queuetimems":0,"class":"HMaster"}
> {noformat}
> From which we only have a {{scanner_id}} and impossible to know what exactly 
> this scan is about, like against which region of which table.
> After this JIRA, we will improve the message to something like below (notice 
> the last line):
> {noformat}
> 2016-10-31 11:43:23,430 WARN  
> [RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=60193] 
> ipc.RpcServer(2574):
> (responseTooSlow): 
> {"call":"Scan(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ScanRequest)",
> "starttimems":1477885403428,"responsesize":52,"method":"Scan","param":"scanner_id:
>  11 number_of_rows: 2147483647
> close_scanner: false next_call_seq: 0 client_handles_partials: true 
> client_handles_heartbeats: true
> track_scan_metrics: false renew: 
> false","processingtimems":2,"client":"127.0.0.1:60254","queuetimems":0,"class":"HMaster",
> "scandetails":"table: hbase:meta region: hbase:meta,,1.1588230740"}
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16033) Add more details in logging of responseTooSlow/TooLarge

[Yu Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yu Li updated HBASE-16033:
--
Affects Version/s: (was: 1.2.3)
   1.2.4
Fix Version/s: (was: 1.2.4)
   1.2.5

Checked the commit log and confirmed HBASE-16033 and HBASE-16972 is not 
included in 1.2.4 release. Update/Correct the affect and fix versions. Thanks 
[~y0un5] for pointing this out.

> Add more details in logging of responseTooSlow/TooLarge
> ---
>
> Key: HBASE-16033
> URL: https://issues.apache.org/jira/browse/HBASE-16033
> Project: HBase
>  Issue Type: Improvement
>  Components: Operability
>Affects Versions: 1.1.7, 1.2.4
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21, 1.2.5, 1.1.8
>
> Attachments: HBASE-16033.patch, HBASE-16033.patch, HBASE-16033.patch
>
>
> Currently the log message when responseTooSlow/TooLarge is like:
> {noformat}
> 2016-06-08 12:18:04,363 WARN  
> [B.defaultRpcServer.handler=127,queue=10,port=16020]
> ipc.RpcServer: (responseTooSlow): 
> {"processingtimems":13125,"call":"Multi(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$MultiRequest)",
> "client":"11.251.158.22:36331","starttimems":1465359471238,"queuetimems":1540116,
> "class":"HRegionServer","responsesize":17,"method":"Multi"}
> {noformat}
> which is kind of helpless for debugging since we don't know on which 
> table/region/row the request is against.
> What's more, we could see some if-else check in the {{RpcServer#logResponse}} 
> method which trying to do sth different when the {{param}} includes instance 
> of {{Operation}}, but there's only one place invoking {{logResponse}} and the 
> {{param}} is always an instance of {{Message}}. Checking the change history, 
> I believe this is a left-over cleanup in work of HBASE-8214 
> We will address the above issues, do some cleanup and improve the log just 
> like {{RpcServer$Call#toString}} does to include table/region/row information 
> of the request



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17288) Add warn log for huge keyvalue and huge row

[Anoop Sam John (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750574#comment-15750574
 ] 

Anoop Sam John commented on HBASE-17288:


HConstants being Public, not to add these kind of constants here.
bq.HUGE_KV_SIZE_IN_BYTE_WARN_NAME = "hbase.huge.keyvalue.size.in.byte";
Why KV ref where as we call it Cells these days 
We have MOB feature now where each of the Cell might be >1M in size. This is 
make too many warn logs with def value of 'ase.huge.keyvalue.size.in.byte'. 
(And also on row size)

> Add warn log for huge keyvalue and huge row
> ---
>
> Key: HBASE-17288
> URL: https://issues.apache.org/jira/browse/HBASE-17288
> Project: HBase
>  Issue Type: Improvement
>  Components: scan
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
> Attachments: HBASE-17288-v1.patch, HBASE-17288.patch
>
>
> Some log examples from our production cluster.
> {code}
> 2016-12-10,17:08:11,478 WARN 
> org.apache.hadoop.hbase.regionserver.StoreScanner: adding a HUGE KV into 
> result list, kv size:1253360, 
> kv:10567114001-1-c/R:r1/1481360887152/Put/vlen=1253245/ts=923099, from 
> table X
> 2016-12-10,17:08:16,724 WARN 
> org.apache.hadoop.hbase.regionserver.StoreScanner: adding a HUGE KV into 
> result list, kv size:1048680, 
> kv:0220459/I:i_0/1481360889551/Put/vlen=1048576/ts=13642, from table XX
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-17288) Add warn log for huge Cell and huge row

[Anoop Sam John (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anoop Sam John updated HBASE-17288:
---
Summary: Add warn log for huge Cell and huge row  (was: Add warn log for 
huge keyvalue and huge row)

> Add warn log for huge Cell and huge row
> ---
>
> Key: HBASE-17288
> URL: https://issues.apache.org/jira/browse/HBASE-17288
> Project: HBase
>  Issue Type: Improvement
>  Components: scan
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
> Attachments: HBASE-17288-v1.patch, HBASE-17288.patch
>
>
> Some log examples from our production cluster.
> {code}
> 2016-12-10,17:08:11,478 WARN 
> org.apache.hadoop.hbase.regionserver.StoreScanner: adding a HUGE KV into 
> result list, kv size:1253360, 
> kv:10567114001-1-c/R:r1/1481360887152/Put/vlen=1253245/ts=923099, from 
> table X
> 2016-12-10,17:08:16,724 WARN 
> org.apache.hadoop.hbase.regionserver.StoreScanner: adding a HUGE KV into 
> result list, kv size:1048680, 
> kv:0220459/I:i_0/1481360889551/Put/vlen=1048576/ts=13642, from table XX
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-17319) Truncate table with preserve after split may cause truncate fail

[Allan Yang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Allan Yang updated HBASE-17319:
---
Attachment: HBASE-17319-branch-1.patch

> Truncate table with preserve after split may cause truncate fail
> 
>
> Key: HBASE-17319
> URL: https://issues.apache.org/jira/browse/HBASE-17319
> Project: HBase
>  Issue Type: Bug
>  Components: Admin
>Affects Versions: 1.1.7, 1.2.4
>Reporter: Allan Yang
>Assignee: Allan Yang
> Attachments: HBASE-17319-branch-1.patch
>
>
> In truncateTableProcedue , when getting tables regions  from meta to recreate 
> new regions, split parents are not excluded, so the new regions can end up 
> with the same start key, and the same region dir:
> {noformat}
> 2016-12-14 20:15:22,231 WARN  [RegionOpenAndInitThread-writetest-1] 
> regionserver.HRegionFileSystem: Trying to create a region that already exists 
> on disk: 
> hdfs://hbasedev1/zhengyan-hbase11-func2/.tmp/data/default/writetest/9b2c8d1539cd92661703ceb8a4d518a1
> {noformat} 
> The truncateTableProcedue will retry forever and never get success.
> A attached unit test shows everything.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-17319) Truncate table with preserve after split may cause truncate fail

[Allan Yang (JIRA)]

Allan Yang created HBASE-17319:
--

 Summary: Truncate table with preserve after split may cause 
truncate fail
 Key: HBASE-17319
 URL: https://issues.apache.org/jira/browse/HBASE-17319
 Project: HBase
  Issue Type: Bug
  Components: Admin
Affects Versions: 1.2.4, 1.1.7
Reporter: Allan Yang
Assignee: Allan Yang


In truncateTableProcedue , when getting tables regions  from meta to recreate 
new regions, split parents are not excluded, so the new regions can end up with 
the same start key, and the same region dir:
{noformat}
2016-12-14 20:15:22,231 WARN  [RegionOpenAndInitThread-writetest-1] 
regionserver.HRegionFileSystem: Trying to create a region that already exists 
on disk: 
hdfs://hbasedev1/zhengyan-hbase11-func2/.tmp/data/default/writetest/9b2c8d1539cd92661703ceb8a4d518a1
{noformat} 
The truncateTableProcedue will retry forever and never get success.
A attached unit test shows everything.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-17319) Truncate table with preserve after split may cause truncate fail

[Allan Yang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Allan Yang updated HBASE-17319:
---
Status: Patch Available  (was: Open)

> Truncate table with preserve after split may cause truncate fail
> 
>
> Key: HBASE-17319
> URL: https://issues.apache.org/jira/browse/HBASE-17319
> Project: HBase
>  Issue Type: Bug
>  Components: Admin
>Affects Versions: 1.2.4, 1.1.7
>Reporter: Allan Yang
>Assignee: Allan Yang
>
> In truncateTableProcedue , when getting tables regions  from meta to recreate 
> new regions, split parents are not excluded, so the new regions can end up 
> with the same start key, and the same region dir:
> {noformat}
> 2016-12-14 20:15:22,231 WARN  [RegionOpenAndInitThread-writetest-1] 
> regionserver.HRegionFileSystem: Trying to create a region that already exists 
> on disk: 
> hdfs://hbasedev1/zhengyan-hbase11-func2/.tmp/data/default/writetest/9b2c8d1539cd92661703ceb8a4d518a1
> {noformat} 
> The truncateTableProcedue will retry forever and never get success.
> A attached unit test shows everything.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17081) Flush the entire CompactingMemStore content to disk

[ramkrishna.s.vasudevan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750516#comment-15750516
 ] 

ramkrishna.s.vasudevan commented on HBASE-17081:


[~anastas]
I tried committing the latest patch attached here. But there are some conflicts 
now - may be [~eshcar]'s new patch. Could  you rebase it? I will commit after 
that. Thanks.

> Flush the entire CompactingMemStore content to disk
> ---
>
> Key: HBASE-17081
> URL: https://issues.apache.org/jira/browse/HBASE-17081
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Anastasia Braginsky
>Assignee: Anastasia Braginsky
> Attachments: HBASE-15787_8.patch, HBASE-17081-V01.patch, 
> HBASE-17081-V02.patch, HBASE-17081-V03.patch, HBASE-17081-V04.patch, 
> HBASE-17081-V05.patch, HBASE-17081-V06.patch, HBASE-17081-V06.patch, 
> HBaseMeetupDecember2016-V02.pptx, Pipelinememstore_fortrunk_3.patch
>
>
> Part of CompactingMemStore's memory is held by an active segment, and another 
> part is divided between immutable segments in the compacting pipeline. Upon 
> flush-to-disk request we want to flush all of it to disk, in contrast to 
> flushing only tail of the compacting pipeline.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17081) Flush the entire CompactingMemStore content to disk

[Anoop Sam John (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750494#comment-15750494
 ] 

Anoop Sam John commented on HBASE-17081:


Sorry for being late. I was out of work for last 3 days.
And +1 for the latest patch :-)

> Flush the entire CompactingMemStore content to disk
> ---
>
> Key: HBASE-17081
> URL: https://issues.apache.org/jira/browse/HBASE-17081
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Anastasia Braginsky
>Assignee: Anastasia Braginsky
> Attachments: HBASE-15787_8.patch, HBASE-17081-V01.patch, 
> HBASE-17081-V02.patch, HBASE-17081-V03.patch, HBASE-17081-V04.patch, 
> HBASE-17081-V05.patch, HBASE-17081-V06.patch, HBASE-17081-V06.patch, 
> HBaseMeetupDecember2016-V02.pptx, Pipelinememstore_fortrunk_3.patch
>
>
> Part of CompactingMemStore's memory is held by an active segment, and another 
> part is divided between immutable segments in the compacting pipeline. Upon 
> flush-to-disk request we want to flush all of it to disk, in contrast to 
> flushing only tail of the compacting pipeline.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17318) Increment does not add new column if the increment amount is zero at first time writing

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750470#comment-15750470
 ] 

Hadoop QA commented on HBASE-17318:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 
24s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s 
{color} | {color:green} branch-1.2 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s 
{color} | {color:green} branch-1.2 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
54s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
16s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
53s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s 
{color} | {color:green} branch-1.2 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s 
{color} | {color:green} branch-1.2 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s 
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 32s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
54s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
12m 40s {color} | {color:green} The patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m 
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 7s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s 
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 76m 16s {color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
16s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 102m 52s {color} 
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestSplitTransactionOnCluster |
| Timed out junit tests | 
org.apache.hadoop.hbase.snapshot.TestSecureExportSnapshot |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:d6626eb |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12843349/HBASE-17318-branch-1.2-v2.patch
 |
| JIRA Issue | HBASE-17318 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux 4e7e61465011 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 

[jira] [Commented] (HBASE-14123) HBase Backup/Restore Phase 2

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750466#comment-15750466
 ] 

Vladimir Rodionov commented on HBASE-14123:
---

Patch has been posted to RB:
https://reviews.apache.org/r/52748/

> HBase Backup/Restore Phase 2
> 
>
> Key: HBASE-14123
> URL: https://issues.apache.org/jira/browse/HBASE-14123
> Project: HBase
>  Issue Type: Umbrella
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: 14123-master.v14.txt, 14123-master.v15.txt, 
> 14123-master.v16.txt, 14123-master.v17.txt, 14123-master.v18.txt, 
> 14123-master.v19.txt, 14123-master.v2.txt, 14123-master.v20.txt, 
> 14123-master.v21.txt, 14123-master.v24.txt, 14123-master.v25.txt, 
> 14123-master.v27.txt, 14123-master.v28.txt, 14123-master.v29.full.txt, 
> 14123-master.v3.txt, 14123-master.v30.txt, 14123-master.v31.txt, 
> 14123-master.v32.txt, 14123-master.v33.txt, 14123-master.v34.txt, 
> 14123-master.v35.txt, 14123-master.v36.txt, 14123-master.v37.txt, 
> 14123-master.v38.txt, 14123-master.v5.txt, 14123-master.v6.txt, 
> 14123-master.v7.txt, 14123-master.v8.txt, 14123-master.v9.txt, 14123-v14.txt, 
> 14123.master.v39.patch, 14123.master.v40.patch, 14123.master.v41.patch, 
> 14123.master.v42.patch, 14123.master.v44.patch, 
> HBASE-14123-for-7912-v1.patch, HBASE-14123-for-7912-v6.patch, 
> HBASE-14123-v1.patch, HBASE-14123-v10.patch, HBASE-14123-v11.patch, 
> HBASE-14123-v12.patch, HBASE-14123-v13.patch, HBASE-14123-v15.patch, 
> HBASE-14123-v16.patch, HBASE-14123-v2.patch, HBASE-14123-v3.patch, 
> HBASE-14123-v4.patch, HBASE-14123-v5.patch, HBASE-14123-v6.patch, 
> HBASE-14123-v7.patch, HBASE-14123-v9.patch
>
>
> Phase 2 umbrella JIRA. See HBASE-7912 for design document and description. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14123) HBase Backup/Restore Phase 2

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-14123:
--
Attachment: 14123.master.v44.patch

patch v44. Full refactoring. Backup has been moved into a separate module: 
hbase-backup. 

[~saint@gmail.com], let us finish the backup merge saga. 

> HBase Backup/Restore Phase 2
> 
>
> Key: HBASE-14123
> URL: https://issues.apache.org/jira/browse/HBASE-14123
> Project: HBase
>  Issue Type: Umbrella
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: 14123-master.v14.txt, 14123-master.v15.txt, 
> 14123-master.v16.txt, 14123-master.v17.txt, 14123-master.v18.txt, 
> 14123-master.v19.txt, 14123-master.v2.txt, 14123-master.v20.txt, 
> 14123-master.v21.txt, 14123-master.v24.txt, 14123-master.v25.txt, 
> 14123-master.v27.txt, 14123-master.v28.txt, 14123-master.v29.full.txt, 
> 14123-master.v3.txt, 14123-master.v30.txt, 14123-master.v31.txt, 
> 14123-master.v32.txt, 14123-master.v33.txt, 14123-master.v34.txt, 
> 14123-master.v35.txt, 14123-master.v36.txt, 14123-master.v37.txt, 
> 14123-master.v38.txt, 14123-master.v5.txt, 14123-master.v6.txt, 
> 14123-master.v7.txt, 14123-master.v8.txt, 14123-master.v9.txt, 14123-v14.txt, 
> 14123.master.v39.patch, 14123.master.v40.patch, 14123.master.v41.patch, 
> 14123.master.v42.patch, 14123.master.v44.patch, 
> HBASE-14123-for-7912-v1.patch, HBASE-14123-for-7912-v6.patch, 
> HBASE-14123-v1.patch, HBASE-14123-v10.patch, HBASE-14123-v11.patch, 
> HBASE-14123-v12.patch, HBASE-14123-v13.patch, HBASE-14123-v15.patch, 
> HBASE-14123-v16.patch, HBASE-14123-v2.patch, HBASE-14123-v3.patch, 
> HBASE-14123-v4.patch, HBASE-14123-v5.patch, HBASE-14123-v6.patch, 
> HBASE-14123-v7.patch, HBASE-14123-v9.patch
>
>
> Phase 2 umbrella JIRA. See HBASE-7912 for design document and description. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (HBASE-17257) Add column-aliasing capability to hbase-client

[Daniel Vimont (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15728144#comment-15728144
 ] 

Daniel Vimont edited comment on HBASE-17257 at 12/15/16 5:24 AM:
-

Here are some specifications of what I’ve currently designed and coded for 
column-aliasing (and will soon be submitting as a patch)...


*COLUMN-ALIASING FOR THE END-USER*:
>From the end-user perspective, column-aliasing entails the following two 
>things...


(1) _Environmental configuration to enable aliasing_:
Aliasing makes use of the already-existing {{hbase.client.connection.impl}} 
configuration parameter. The following entry should be added to 
{{hbase-site.xml}}:
{code}
  
hbase.client.connection.impl
org.apache.hadoop.hbase.client.AliasEnabledConnection
  
{code}
Setting this parameter to this value results in 
{{ConnectionFactory#createConnection}} returning Connections of the new 
{{AliasEnabledConnection}} class (subclass of the {{ConnectionImplementation}} 
class).


(2) _Alias-enabling individual column families_:
Aliasing is enabled at the column-family level. When adding a column-descriptor 
(i.e. family) to a Table, the new method {{HColumnDescriptor#setAliasSize}} may 
be invoked to immutably set the fixed size (in bytes) of column-qualifier 
aliases for the column family. The default value of 0 (aliasing disabled) may 
be changed to either 1, 2, or 4.


Other than the above, the end-user-application code should neither require nor 
contain any “awareness” whatsoever that column-aliasing is being utilized for a 
column-family. An end-user-application continues to interact only with the 
standard interfaces of the client API ({{Connection}}, {{Table}}, 
{{BufferedMutator}}, and {{HTableMultiplexer}}).


*COLUMN-ALIASING INTERNALS*:
One of the overriding goals in designing the column-aliasing infrastructure is 
to minimize alterations and insertions into already-existing hbase-client code, 
and to have very-close-to-zero impact on already-existing functionality, 
particularly in those situations in which aliasing will NOT be used. The 
following is a comprehensive list of all new and modified modules, along with 
an explanation as to the role that the new or modified module plays in aliasing.


_Modified classes_:
*HColumnDescriptor*: as described above, the new method {{#setAliasSize}} has 
been added; also corresponding methods, {{#getAliasSize}} and 
{{#isAliasEnabled}} (returns “true” if aliasSize not zero).
*HTableDescriptor*: new method {{#hasAliasEnabledFamily}} has been added 
(returns “true” if one or more of the table’s families are aliasEnabled).
(Corresponding modifications were also made to {{TestHColumnDescriptor}} and 
{{TestHTableDescriptor}}, to test the new methods appropriately.)


_New class_:
*AliasEnabledConnection* (subclass of {{ConnectionImplementation}}): provides 
overrides of {{#getTable}} and {{#getBufferedMutator}} to return objects of the 
{{AliasEnabledTable}} class and the {{AliasEnabledBufferedMutator}} class, 
respectively.


_Modified class_:
*HTableMultiplexer*: new static method added -- 
{{#getAliasEnabledTableMultiplexer}}, returns an {{HTableMultiplexer}} object 
that is actually an instance of the new subclass, 
{{AliasEnabledTableMultiplexer}}.


_New classes_:
*AliasEnabledTable* (subclass of {{HTable}})
*AliasEnabledBufferedMutator* (subclass of {{BufferedMutatorImpl}})
*AliasEnabledTableMultiplexer* (subclass of {{HTableMultiplexer}})
-- all the above contain overrides which allow for {{AliasManager}} methods to 
be invoked when needed -- to perform qualifier-to-alias conversions (for 
{{Get}}, {{Scan}}, and {{Mutation}} objects), and alias-to-qualifier 
conversions (for {{Result}} objects) -- for any Table for which 
{{HTableDescriptor#hasAliasEnabledFamily}} is “true”.


_New class_:
*AliasManager*: performs all alias-oriented conversions -- qualifier-to-alias 
conversions for queries and  mutations, and alias-to-qualifier conversions for 
results. It fully encapsulates all CRUD transactions against the 
{{aliasMappingTable}} (the HBase table in which qualifier-to-alias mappings are 
persisted for each alias-enabled column family). When a {{Mutation}} object 
contains a column-qualifier for which an alias entry does not yet exist, a new 
alias is generated and stored in a qualifier-to-alias mapping entry in the 
{{aliasMappingTable}}. The first time an {{AliasManager}} is instantiated 
against an HBase cluster, the {{aliasMappingTable}} will be created if it does 
not already exist.


_New reserved HBase table_:
*aliasMappingTable*: Each row on the {{aliasMappingTable}} corresponds to an 
aliasEnabled column-family on a user-table. The rowId of each 
{{aliasMappingTable}} row is in the format: {{[fully-qualified-user-table-name 
+ ":" + aliasEnabled-Family]}}. On each {{aliasMappingTable}} row, the column 
with an EMPTY_BYTE_ARRAY column-qualifier 

[jira] [Comment Edited] (HBASE-17257) Add column-aliasing capability to hbase-client

[Daniel Vimont (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15728144#comment-15728144
 ] 

Daniel Vimont edited comment on HBASE-17257 at 12/15/16 5:24 AM:
-

Here are some specifications of what I’ve currently designed and coded for 
column-aliasing (and will soon be submitting as a patch)...


*COLUMN-ALIASING FOR THE END-USER*:
>From the end-user perspective, column-aliasing entails the following two 
>things...


(1) _Environmental configuration to enable aliasing_:
Aliasing makes use of the already-existing {{hbase.client.connection.impl}} 
configuration parameter. The following entry should be added to 
{{hbase-site.xml}}:
{code}
  
hbase.client.connection.impl
org.apache.hadoop.hbase.client.AliasEnabledConnection
  
{code}
Setting this parameter to this value results in 
{{ConnectionFactory#createConnection}} returning Connections of the new 
{{AliasEnabledConnection}} class (subclass of the {{ConnectionImplementation}} 
class).


(2) _Alias-enabling individual column families_:
Aliasing is enabled at the column-family level. When adding a column-descriptor 
(i.e. family) to a Table, the new method {{HColumnDescriptor#setAliasSize}} may 
be invoked to immutably set the fixed size (in bytes) of column-qualifier 
aliases for the column family. The default value of 0 (aliasing disabled) may 
be changed to either 1, 2, or 4.


Other than the above, the end-user-application code should neither require nor 
contain any “awareness” whatsoever that column-aliasing is being utilized for a 
column-family. An end-user-application continues to interact only with the 
standard interfaces of the client API ({{Connection}}, {{Table}}, 
{{BufferedMutator}}, and {{HTableMultiplexer}}).


*COLUMN-ALIASING INTERNALS*:
One of the overriding goals in designing the column-aliasing infrastructure is 
to minimize alterations and insertions into already-existing hbase-client code, 
and to have very-close-to-zero impact on already-existing functionality, 
particularly in those situations in which aliasing will NOT be used. The 
following is a comprehensive list of all new and modified modules, along with 
an explanation as to the role that the new or modified module plays in aliasing.


_Modified classes_:
*HColumnDescriptor*: as described above, the new method {{#setAliasSize}} has 
been added; also corresponding methods, {{#getAliasSize}} and 
{{#isAliasEnabled}} (returns “true” if aliasSize not zero).
*HTableDescriptor*: new method {{#hasAliasEnabledFamily}} has been added 
(returns “true” if one or more of the table’s families are aliasEnabled).
(Corresponding modifications were also made to {{TestHColumnDescriptor}} and 
{{TestHTableDescriptor}}, to test the new methods appropriately.)


_New class_:
*AliasEnabledConnection* (subclass of {{ConnectionImplementation}}): provides 
overrides of {{#getTable}} and {{#getBufferedMutator}} to return objects of the 
{{AliasEnabledTable}} class and the {{AliasEnabledBufferedMutator}} class, 
respectively.


_Modified class_:
*HTableMultiplexer*: new static method added -- 
{{#getAliasEnabledTableMultiplexer}}, returns an {{HTableMultiplexer}} object 
that is actually an instance of the new subclass, 
{{AliasEnabledTableMultiplexer}}.


_New classes_:
*AliasEnabledTable* (subclass of {{HTable}})
*AliasEnabledBufferedMutator* (subclass of {{BufferedMutatorImpl}})
*AliasEnabledTableMultiplexer* (subclass of {{HTableMultiplexer}})
-- all the above contain overrides which allow for {{AliasManager}} methods to 
be invoked when needed -- to perform qualifier-to-alias conversions (for 
{{Get}}, {{Scan}}, and {{Mutation}} objects), and alias-to-qualifier 
conversions (for {{Result}} objects) -- for any Table for which 
{{HTableDescriptor#hasAliasEnabledFamily}} is “true”.


_New class_:
*AliasManager*: performs all alias-oriented conversions -- qualifier-to-alias 
conversions for queries and  mutations, and alias-to-qualifier conversions for 
results. It fully encapsulates all CRUD transactions against the 
{{aliasMappingTable}} (the HBase table in which qualifier-to-alias mappings are 
persisted for each alias-enabled column family). When a {{Mutation}} object 
contains a column-qualifier for which an alias entry does not yet exist, a new 
alias is generated and stored in a qualifier-to-alias mapping entry in the 
{{aliasMappingTable}}. The first time an {{AliasManager}} is instantiated 
against an HBase cluster, the {{aliasMappingTable}} will be created if it does 
not already exist.


_New reserved HBase table_:
*aliasMappingTable*: Each row on the {{aliasMappingTable}} corresponds to an 
aliasEnabled column-family on a user-table. The rowId of each 
{{aliasMappingTable}} row is in the format: {{[fully-qualified-user-table-name 
+ ":" + aliasEnabled-Family]}}. On each {{aliasMappingTable}} row, the column 
with an EMPTY_BYTE_ARRAY column-qualifier 

[jira] [Commented] (HBASE-17257) Add column-aliasing capability to hbase-client

[Daniel Vimont (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750429#comment-15750429
 ] 

Daniel Vimont commented on HBASE-17257:
---

Here is a partial list of outstanding questions regarding this JIRA issue (to 
add a column-aliasing capability to HBase):

*META-ASSESSMENT*: Is this proposed feature (i.e. column-aliasing) of 
sufficient potential importance/usefulness to the HBase ecosystem to justify 
people taking the time to assess its technical viability via this JIRA entry? 
([~aoxiang] commented above "I think it is a very useful feature." This is 
helpful, and others have anecdotally given me similar comments in other venues 
in the past few months, but input from others would still be welcome, 
particularly from those who might take a contrary viewpoint.)

*QUESTION 1* -- Aside from the question of where to persist alias-mapping data 
(that question is below) -- _Is the overall architecture for aliasing as 
presented in patch-v3 (and as outlined in the specifications above)..._
(a) ... viable?
(b) ... optimal?

*QUESTION 2a* -- _Where should alias-mapping data (referred to hereafter as 
"hbase:alias") be persisted?_
I currently see three options:
(1) An existing system table (hbase:meta has been suggested). This might entail 
scalability problems, since system tables apparently cannot be split into more 
than one region.
(2) A separate table in the "hbase" namespace, but created/administered as a 
"normal" HBase table on RegionServer(s). [This is currently what is offered in 
patch-v3: "hbase:alias" is in the system namespace, but it is a standard HBase 
table created on a "normal" RegionServer. In my original design-phase, I went 
with this option after realizing the scalability limitations inherent in Option 
1, above.]
(3) A separate aliasMapping table for every alias-enabled Table.

*QUESTION 2b* -- _How to increase/improve fault-tolerance of hbase:alias table:_
Given that read/write access to all alias-enabled column-families in a cluster 
is completely dependent upon unfettered read/write access to hbase:alias, are 
there heightened "fault tolerance" options available to decrease the chances of 
an unscheduled unavailability of hbase:alias? (Concerns of this have been 
relayed to me in offline discussions.)

*QUESTION 3* -- _Does basic read/write access to alias-enabled column-families 
need to be guaranteed when/if the Master server becomes unavailable?_
It seems to be a publicized feature of HBase that basic read/write access to 
HBase tables (i.e. to RegionServers) can continue uninterrupted even when the 
Master server becomes unavailable. However, in the current, patch-v3 
architecture, column-aliasing requires access to the Master server (to fulfill 
#getTableDescriptor RPC invocations to look-up aliasing metadata for a table 
and its families). I have experimented with a potential "patch-v4" architecture 
in which all required metadata for a column-family is stored in hbase:alias, 
negating the need for invocation of #getTableDescriptor, and thus making 
alias-enabled tables still ostensibly accessible in the event of an outage of 
the Master server. I can finalize and submit this architectural variation upon 
request.

*THANKS TO ALL WHO HAVE READ TO THIS POINT!! I VERY MUCH APPRECIATE YOUR 
TIME!!!*

> Add column-aliasing capability to hbase-client
> --
>
> Key: HBASE-17257
> URL: https://issues.apache.org/jira/browse/HBASE-17257
> Project: HBase
>  Issue Type: New Feature
>  Components: Client
>Affects Versions: 2.0.0
>Reporter: Daniel Vimont
>Assignee: Daniel Vimont
>  Labels: features
> Attachments: HBASE-17257-v2.patch, HBASE-17257-v3.patch, 
> HBASE-17257.patch
>
>
> Review Board link: https://reviews.apache.org/r/54635/
> Column aliasing will provide the option for a 1, 2, or 4 byte alias value to 
> be stored in each cell of an "alias enabled" column-family, in place of the 
> full-length column-qualifier. Aliasing is intended to operate completely 
> invisibly to the end-user developer, with absolutely no "awareness" of 
> aliasing required to be coded into a front-end application. No new public 
> hbase-client interfaces are to be introduced, and only a few new public 
> methods should need to be added to existing interfaces, primarily to allow an 
> administrator to designate that a new column-family is to be alias-enabled by 
> setting its aliasSize attribute to 1, 2, or 4.
> To facilitate such functionality, new subclasses of HTable, 
> BufferedMutatorImpl, and HTableMultiplexer are to be provided. The overriding 
> methods of these new subclasses will invoke methods of the new AliasManager 
> class to facilitate qualifier-to-alias conversions (for user-submitted Gets, 
> Scans, and Mutations) and alias-to-qualifier 

[jira] [Updated] (HBASE-17292) Add observer notification before bulk loaded hfile is moved to region directory

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-17292:
---
Attachment: 17292.v3.txt

Patch v3 moves the call to postBulkLoadHFile() into finally block.

> Add observer notification before bulk loaded hfile is moved to region 
> directory
> ---
>
> Key: HBASE-17292
> URL: https://issues.apache.org/jira/browse/HBASE-17292
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0
>
> Attachments: 17292.v1.txt, 17292.v2.txt, 17292.v3.txt
>
>
> Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded 
> hfiles.
> However, if bulk load fails after hfile is moved to region directory but 
> before postBulkLoadHFile() hook is called, there is no way for pluggable 
> components (replication - see HBASE-17290, backup / restore) to know which 
> hfile(s) have been moved to region directory.
> This issue adds a preCommitStoreFile() hook which notifies path of to be 
> committed hfile before bulk loaded hfile is moved to region directory.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17318) Increment does not add new column if the increment amount is zero at first time writing

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750339#comment-15750339
 ] 

Ted Yu commented on HBASE-17318:


+1

> Increment does not add new column if the increment amount is zero at first 
> time writing
> ---
>
> Key: HBASE-17318
> URL: https://issues.apache.org/jira/browse/HBASE-17318
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0, 0.98.23, 1.2.4
>Reporter: Guangxu Cheng
> Attachments: HBASE-17318-branch-1.2-v1.patch, 
> HBASE-17318-branch-1.2-v2.patch, HBASE-17318-master-v1.patch
>
>
> When the data written for the first time is 0, no new columns are added.
> Iterate the input columns and update existing values if they were found, 
> otherwise add new column initialized to the increment amount.
> Does not add new column if the increment amount is zero at first time 
> writting.
> It is necessary to add a new column at the first write to 0. 
> If not, the result of using the phoenix is NULL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17257) Add column-aliasing capability to hbase-client

[Daniel Vimont (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750330#comment-15750330
 ] 

Daniel Vimont commented on HBASE-17257:
---

Here is my summary of the progress on this task so far:

(1) *THE FIRST PATCH* encountered Jenkins pre-commit errors/failures due to the 
following: new validation logic which included invocations of 
Admin#getTableDescriptor (to determine whether a Table has any aliasEnabled 
families) was causing errors/failures in existing Test modules in which the 
Master server is incidentally or intentionally not available. This was resolved 
by restructuring things such that Admin#getTableDescriptor calls are no longer 
required to determine whether a user-table has any alias-enabled families 
(replaced instead by a lookup by AliasManager to see whether row(s) exist for 
the user-table in the alias-mapping table).
(2) *THE SECOND PATCH* encountered problems when some existing Test modules did 
not receive the specific types of Exceptions they were expecting due to new 
aliasing logic inappropriately catching them. Preparation of the third patch 
also included several improvements/modifications recommended by Ted (listed in 
his comments above).
(3) *THE THIRD PATCH* passed all Jenkins pre-commit testing, and its code was 
posted to Review Board: https://reviews.apache.org/r/54635/

This leads me to produce a partial list of outstanding questions and 
discussion-points regarding all of this, which I'll post as a separate comment.

> Add column-aliasing capability to hbase-client
> --
>
> Key: HBASE-17257
> URL: https://issues.apache.org/jira/browse/HBASE-17257
> Project: HBase
>  Issue Type: New Feature
>  Components: Client
>Affects Versions: 2.0.0
>Reporter: Daniel Vimont
>Assignee: Daniel Vimont
>  Labels: features
> Attachments: HBASE-17257-v2.patch, HBASE-17257-v3.patch, 
> HBASE-17257.patch
>
>
> Review Board link: https://reviews.apache.org/r/54635/
> Column aliasing will provide the option for a 1, 2, or 4 byte alias value to 
> be stored in each cell of an "alias enabled" column-family, in place of the 
> full-length column-qualifier. Aliasing is intended to operate completely 
> invisibly to the end-user developer, with absolutely no "awareness" of 
> aliasing required to be coded into a front-end application. No new public 
> hbase-client interfaces are to be introduced, and only a few new public 
> methods should need to be added to existing interfaces, primarily to allow an 
> administrator to designate that a new column-family is to be alias-enabled by 
> setting its aliasSize attribute to 1, 2, or 4.
> To facilitate such functionality, new subclasses of HTable, 
> BufferedMutatorImpl, and HTableMultiplexer are to be provided. The overriding 
> methods of these new subclasses will invoke methods of the new AliasManager 
> class to facilitate qualifier-to-alias conversions (for user-submitted Gets, 
> Scans, and Mutations) and alias-to-qualifier conversions (for Results 
> returned from HBase) for any Table that has one or more alias-enabled column 
> families. All conversion logic will be encapsulated in the new AliasManager 
> class, and all qualifier-to-alias mappings will be persisted in a new 
> aliasMappingTable in a new, reserved namespace.
> An informal polling of HBase users at HBaseCon East and at the 
> Strata/Hadoop-World conference in Sept. 2016 showed that Column Aliasing 
> could be a popular enhancement to standard HBase functionality, due to the 
> fact that full column-qualifiers are stored in each cell, and reducing this 
> qualifier storage requirement down to 1, 2, or 4 bytes per cell could prove 
> beneficial in terms of reduced storage and bandwidth needs. Aliasing is 
> intended chiefly for column-families which are of the "narrow and tall" 
> variety (i.e., that are designed to use relatively few distinct 
> column-qualifiers throughout a large number of rows, throughout the lifespan 
> of the column-family). A column-family that is set up with an alias-size of 1 
> byte can contain up to 255 unique column-qualifiers; a 2 byte alias-size 
> allows for up to 65,535 unique column-qualifiers; and a 4 byte alias-size 
> allows for up to 4,294,967,295 unique column-qualifiers.
> Fuller specifications will be entered into the comments section below. Note 
> that it may well not be viable to add aliasing support in the new "async" 
> classes that appear to be currently under development.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (HBASE-16398) optimize HRegion computeHDFSBlocksDistribution

[binlijin (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750312#comment-15750312
 ] 

binlijin edited comment on HBASE-16398 at 12/15/16 4:05 AM:


With two+1, if there is no objections, will commit is soon.


was (Author: aoxiang):
With two+1, if there is no objects, will commit is soon.

> optimize HRegion computeHDFSBlocksDistribution
> --
>
> Key: HBASE-16398
> URL: https://issues.apache.org/jira/browse/HBASE-16398
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0
>
> Attachments: HBASE-16398.patch, HBASE-16398_v2.patch, 
> HBASE-16398_v3.patch, HBASE-16398_v4.patch, HBASE-16398_v5.patch, 
> LocatedBlockStatusComparison.java
>
>
> First i assume there is no reference and link in a region family's directory. 
> Without the patch to computeHDFSBlocksDistribution for a region family, there 
> is 1+2*N rpc call, N is hfile numbers, The first rpc call is to 
> DistributedFileSystem#listStatus to get hfiles, for every hfile there is two 
> rpc call DistributedFileSystem#getFileStatus(path) and then 
> DistributedFileSystem#getFileBlockLocations(status, start, length).
> With the patch to computeHDFSBlocksDistribution for a region family, there is 
> 2 rpc call, they are DistributedFileSystem#getFileStatus(path) and  
> DistributedFileSystem#listLocatedStatus(final Path p, final PathFilter 
> filter).
> So if there is at least one hfile, with the patch, the rpc call will less.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16398) optimize HRegion computeHDFSBlocksDistribution

[binlijin (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750312#comment-15750312
 ] 

binlijin commented on HBASE-16398:
--

With two+1, if there is no objects, will commit is soon.

> optimize HRegion computeHDFSBlocksDistribution
> --
>
> Key: HBASE-16398
> URL: https://issues.apache.org/jira/browse/HBASE-16398
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0
>
> Attachments: HBASE-16398.patch, HBASE-16398_v2.patch, 
> HBASE-16398_v3.patch, HBASE-16398_v4.patch, HBASE-16398_v5.patch, 
> LocatedBlockStatusComparison.java
>
>
> First i assume there is no reference and link in a region family's directory. 
> Without the patch to computeHDFSBlocksDistribution for a region family, there 
> is 1+2*N rpc call, N is hfile numbers, The first rpc call is to 
> DistributedFileSystem#listStatus to get hfiles, for every hfile there is two 
> rpc call DistributedFileSystem#getFileStatus(path) and then 
> DistributedFileSystem#getFileBlockLocations(status, start, length).
> With the patch to computeHDFSBlocksDistribution for a region family, there is 
> 2 rpc call, they are DistributedFileSystem#getFileStatus(path) and  
> DistributedFileSystem#listLocatedStatus(final Path p, final PathFilter 
> filter).
> So if there is at least one hfile, with the patch, the rpc call will less.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-17309) Fix connection leaks in TestAcidGuarantees

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-17309:
---
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 2.0.0
   Status: Resolved  (was: Patch Available)

Thanks for the patch, huaxiang

> Fix connection leaks in TestAcidGuarantees
> --
>
> Key: HBASE-17309
> URL: https://issues.apache.org/jira/browse/HBASE-17309
> Project: HBase
>  Issue Type: Bug
>  Components: integration tests
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-17309-master-001.patch, 
> HBASE-17309-master-002.patch
>
>
> Connections are not closed in some of TestAcidGuarantees's classes, this 
> makes integration test fail in some cases.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17318) Increment does not add new column if the increment amount is zero at first time writing

[Guangxu Cheng (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750313#comment-15750313
 ] 

Guangxu Cheng commented on HBASE-17318:
---

[~tedyu] Add a unit test and update patch for master and branch-1.2. Thanks


> Increment does not add new column if the increment amount is zero at first 
> time writing
> ---
>
> Key: HBASE-17318
> URL: https://issues.apache.org/jira/browse/HBASE-17318
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0, 0.98.23, 1.2.4
>Reporter: Guangxu Cheng
> Attachments: HBASE-17318-branch-1.2-v1.patch, 
> HBASE-17318-branch-1.2-v2.patch, HBASE-17318-master-v1.patch
>
>
> When the data written for the first time is 0, no new columns are added.
> Iterate the input columns and update existing values if they were found, 
> otherwise add new column initialized to the increment amount.
> Does not add new column if the increment amount is zero at first time 
> writting.
> It is necessary to add a new column at the first write to 0. 
> If not, the result of using the phoenix is NULL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-17318) Increment does not add new column if the increment amount is zero at first time writing

[Guangxu Cheng (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guangxu Cheng updated HBASE-17318:
--
Attachment: HBASE-17318-branch-1.2-v2.patch
HBASE-17318-master-v1.patch

> Increment does not add new column if the increment amount is zero at first 
> time writing
> ---
>
> Key: HBASE-17318
> URL: https://issues.apache.org/jira/browse/HBASE-17318
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0, 0.98.23, 1.2.4
>Reporter: Guangxu Cheng
> Attachments: HBASE-17318-branch-1.2-v1.patch, 
> HBASE-17318-branch-1.2-v2.patch, HBASE-17318-master-v1.patch
>
>
> When the data written for the first time is 0, no new columns are added.
> Iterate the input columns and update existing values if they were found, 
> otherwise add new column initialized to the increment amount.
> Does not add new column if the increment amount is zero at first time 
> writting.
> It is necessary to add a new column at the first write to 0. 
> If not, the result of using the phoenix is NULL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16398) optimize HRegion computeHDFSBlocksDistribution

[Yu Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750296#comment-15750296
 ] 

Yu Li commented on HBASE-16398:
---

Patch v5 lgtm, +1

> optimize HRegion computeHDFSBlocksDistribution
> --
>
> Key: HBASE-16398
> URL: https://issues.apache.org/jira/browse/HBASE-16398
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver
>Affects Versions: 2.0.0
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0
>
> Attachments: HBASE-16398.patch, HBASE-16398_v2.patch, 
> HBASE-16398_v3.patch, HBASE-16398_v4.patch, HBASE-16398_v5.patch, 
> LocatedBlockStatusComparison.java
>
>
> First i assume there is no reference and link in a region family's directory. 
> Without the patch to computeHDFSBlocksDistribution for a region family, there 
> is 1+2*N rpc call, N is hfile numbers, The first rpc call is to 
> DistributedFileSystem#listStatus to get hfiles, for every hfile there is two 
> rpc call DistributedFileSystem#getFileStatus(path) and then 
> DistributedFileSystem#getFileBlockLocations(status, start, length).
> With the patch to computeHDFSBlocksDistribution for a region family, there is 
> 2 rpc call, they are DistributedFileSystem#getFileStatus(path) and  
> DistributedFileSystem#listLocatedStatus(final Path p, final PathFilter 
> filter).
> So if there is at least one hfile, with the patch, the rpc call will less.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17142) Implement multi get

[Yu Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750294#comment-15750294
 ] 

Yu Li commented on HBASE-17142:
---

bq. For example, maxRetries is 10, and there are 10 gets and we have 2 groups 
for the gets, then you could retry 20 times for the batch as each group could 
retry 10 times
I see, so it's for each *initial* get *group by server*, makes sense. "Each 
get" is a little bit ambiguous and might be regarded as "each single get in the 
batch", so make it clear in javadoc might be a good idea (smile)

bq. But there is a problem which is the support of replica read in the future 
as we will only introduce replica read for get...
Ok, see the point there, thanks for clarification.

> Implement multi get
> ---
>
> Key: HBASE-17142
> URL: https://issues.apache.org/jira/browse/HBASE-17142
> Project: HBase
>  Issue Type: Sub-task
>  Components: asyncclient, Client
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0
>
> Attachments: HBASE-17142.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17309) Fix connection leaks in TestAcidGuarantees

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750080#comment-15750080
 ] 

Hadoop QA commented on HBASE-17309:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 
3s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
25m 11s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
53s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 95m 49s 
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
15s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 132m 58s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12843319/HBASE-17309-master-002.patch
 |
| JIRA Issue | HBASE-17309 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux bd96e4879dee 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 
10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 691f266 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4923/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4923/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Fix connection leaks in TestAcidGuarantees
> --
>
> Key: HBASE-17309
> URL: https://issues.apache.org/jira/browse/HBASE-17309
> Project: HBase
>  Issue Type: Bug
>  Components: integration tests
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-17309-master-001.patch, 

[jira] [Commented] (HBASE-17296) Provide per peer throttling for replication

[Guanghao Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750075#comment-15750075
 ] 

Guanghao Zhang commented on HBASE-17296:


Can you help to commit the patch for branch-1? Then I can rebase the latest 
code to resolve HBASE-17317. Thanks. 

> Provide per peer throttling for replication
> ---
>
> Key: HBASE-17296
> URL: https://issues.apache.org/jira/browse/HBASE-17296
> Project: HBase
>  Issue Type: Improvement
>  Components: Replication
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-17296-branch-1.patch, HBASE-17296.patch
>
>
> HBASE-9501 added a config to provide throttling for replication. And each 
> peer has same bandwidth up limit. In our use case, one cluster may have 
> several peers and several slave clusters. Each slave cluster may have 
> different scales and need different bandwidth up limit for each peer. So We 
> add bandwidth to replication peer config and provide a shell cmd set_peer 
> bandwidth to update the bandwidth in need. It has been used for a long time 
> on our clusters.  Any suggestions are welcomed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11392) add/remove peer requests should be routed through master

[Guanghao Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-11392:
---
Description: 
ReplicationAdmin directly operates over the zookeeper data for replication 
setup. We should move these operations to be routed through master for two 
reasons: 
 - Replication implementation details are exposed to client. We should move 
most of the replication related classes to hbase-server package. 
 - Routing the requests through master is the standard practice for all other 
operations. It allows for decoupling implementation details from the client and 
code.

Review board: https://reviews.apache.org/r/54730/

  was:
ReplicationAdmin directly operates over the zookeeper data for replication 
setup. We should move these operations to be routed through master for two 
reasons: 
 - Replication implementation details are exposed to client. We should move 
most of the replication related classes to hbase-server package. 
 - Routing the requests through master is the standard practice for all other 
operations. It allows for decoupling implementation details from the client and 
code.


> add/remove peer requests should be routed through master
> 
>
> Key: HBASE-11392
> URL: https://issues.apache.org/jira/browse/HBASE-11392
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Guanghao Zhang
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-11392-v1.patch, HBASE-11392-v2.patch
>
>
> ReplicationAdmin directly operates over the zookeeper data for replication 
> setup. We should move these operations to be routed through master for two 
> reasons: 
>  - Replication implementation details are exposed to client. We should move 
> most of the replication related classes to hbase-server package. 
>  - Routing the requests through master is the standard practice for all other 
> operations. It allows for decoupling implementation details from the client 
> and code.
> Review board: https://reviews.apache.org/r/54730/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-17296) Provide per peer throttling for replication

[Guanghao Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-17296:
---
Attachment: (was: HBASE-17296-branch-1.patch)

> Provide per peer throttling for replication
> ---
>
> Key: HBASE-17296
> URL: https://issues.apache.org/jira/browse/HBASE-17296
> Project: HBase
>  Issue Type: Improvement
>  Components: Replication
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-17296-branch-1.patch, HBASE-17296.patch
>
>
> HBASE-9501 added a config to provide throttling for replication. And each 
> peer has same bandwidth up limit. In our use case, one cluster may have 
> several peers and several slave clusters. Each slave cluster may have 
> different scales and need different bandwidth up limit for each peer. So We 
> add bandwidth to replication peer config and provide a shell cmd set_peer 
> bandwidth to update the bandwidth in need. It has been used for a long time 
> on our clusters.  Any suggestions are welcomed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-17296) Provide per peer throttling for replication

[Guanghao Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-17296:
---
Attachment: (was: HBASE-17296-branch-1.patch)

> Provide per peer throttling for replication
> ---
>
> Key: HBASE-17296
> URL: https://issues.apache.org/jira/browse/HBASE-17296
> Project: HBase
>  Issue Type: Improvement
>  Components: Replication
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-17296-branch-1.patch, HBASE-17296.patch
>
>
> HBASE-9501 added a config to provide throttling for replication. And each 
> peer has same bandwidth up limit. In our use case, one cluster may have 
> several peers and several slave clusters. Each slave cluster may have 
> different scales and need different bandwidth up limit for each peer. So We 
> add bandwidth to replication peer config and provide a shell cmd set_peer 
> bandwidth to update the bandwidth in need. It has been used for a long time 
> on our clusters.  Any suggestions are welcomed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16261) MultiHFileOutputFormat Enhancement

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749998#comment-15749998
 ] 

Hadoop QA commented on HBASE-16261:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 40s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 
9s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
41s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
40s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
26m 24s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 90m 23s 
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
17s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 129m 10s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12843312/HBase-16261-V7.patch |
| JIRA Issue | HBASE-16261 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux a6fa3167d022 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 691f266 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4921/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4921/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



>  MultiHFileOutputFormat Enhancement 
> 
>
> Key: HBASE-16261
> URL: https://issues.apache.org/jira/browse/HBASE-16261
> Project: HBase
>  Issue Type: Sub-task
>  Components: hbase, mapreduce
>Affects Versions: 2.0.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16261-V1.patch, 

[jira] [Commented] (HBASE-16261) MultiHFileOutputFormat Enhancement

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749948#comment-15749948
 ] 

Hadoop QA commented on HBASE-16261:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 41s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
27m 14s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 93m 19s 
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
15s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 132m 44s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12843307/HBase-16261-V7.patch |
| JIRA Issue | HBASE-16261 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux c241cd105f23 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 691f266 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4920/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4920/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



>  MultiHFileOutputFormat Enhancement 
> 
>
> Key: HBASE-16261
> URL: https://issues.apache.org/jira/browse/HBASE-16261
> Project: HBase
>  Issue Type: Sub-task
>  Components: hbase, mapreduce
>Affects Versions: 2.0.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16261-V1.patch, 

[jira] [Commented] (HBASE-17311) Refactoring: move backup system table out of 'hbase' namespace

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749911#comment-15749911
 ] 

Ted Yu commented on HBASE-17311:


When 'BACKUP.SYSTEM' table exists at cluster startup, backup feature needs to 
be turned off and ERROR should be logged, asking user to configure a different 
table as backup table.

> Refactoring: move backup system table out of 'hbase' namespace
> --
>
> Key: HBASE-17311
> URL: https://issues.apache.org/jira/browse/HBASE-17311
> Project: HBase
>  Issue Type: Bug
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-17311.HBASE-7912.v1.patch
>
>
> In preparation to big refactoring (separate project for backup) we need to 
> move  'hbase:system' table out of HBase system namespace.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17309) Fix connection leaks in TestAcidGuarantees

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749849#comment-15749849
 ] 

Ted Yu commented on HBASE-17309:


+1, if test passes.

> Fix connection leaks in TestAcidGuarantees
> --
>
> Key: HBASE-17309
> URL: https://issues.apache.org/jira/browse/HBASE-17309
> Project: HBase
>  Issue Type: Bug
>  Components: integration tests
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-17309-master-001.patch, 
> HBASE-17309-master-002.patch
>
>
> Connections are not closed in some of TestAcidGuarantees's classes, this 
> makes integration test fail in some cases.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17292) Add observer notification before bulk loaded hfile is moved to region directory

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749848#comment-15749848
 ] 

Hadoop QA commented on HBASE-17292:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 2s 
{color} | {color:blue} The patch file was not named according to hbase's naming 
conventions. Please see 
https://yetus.apache.org/documentation/0.3.0/precommit-patchnames for 
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s 
{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 
34s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
39s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
35s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
35s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 32s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
23m 11s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 87m 1s 
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
16s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 121m 1s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12843297/17292.v2.txt |
| JIRA Issue | HBASE-17292 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux 1809d5831c98 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 691f266 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4919/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4919/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Add observer notification before bulk loaded hfile is moved to region 
> directory
> 

[jira] [Commented] (HBASE-17309) Fix connection leaks in TestAcidGuarantees

[huaxiang sun (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749843#comment-15749843
 ] 

huaxiang sun commented on HBASE-17309:
--

New patch to address Ted's comments.

> Fix connection leaks in TestAcidGuarantees
> --
>
> Key: HBASE-17309
> URL: https://issues.apache.org/jira/browse/HBASE-17309
> Project: HBase
>  Issue Type: Bug
>  Components: integration tests
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-17309-master-001.patch, 
> HBASE-17309-master-002.patch
>
>
> Connections are not closed in some of TestAcidGuarantees's classes, this 
> makes integration test fail in some cases.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-17309) Fix connection leaks in TestAcidGuarantees

[huaxiang sun (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

huaxiang sun updated HBASE-17309:
-
Attachment: HBASE-17309-master-002.patch

> Fix connection leaks in TestAcidGuarantees
> --
>
> Key: HBASE-17309
> URL: https://issues.apache.org/jira/browse/HBASE-17309
> Project: HBase
>  Issue Type: Bug
>  Components: integration tests
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-17309-master-001.patch, 
> HBASE-17309-master-002.patch
>
>
> Connections are not closed in some of TestAcidGuarantees's classes, this 
> makes integration test fail in some cases.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16998) [Master] Analyze table use reports and update quota violations

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749777#comment-15749777
 ] 

Hadoop QA commented on HBASE-16998:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 0m 14s 
{color} | {color:red} Docker failed to build yetus/hbase:8d52d23. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12843313/HBASE-16998.002.HBASE-16961.patch
 |
| JIRA Issue | HBASE-16998 |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4922/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> [Master] Analyze table use reports and update quota violations
> --
>
> Key: HBASE-16998
> URL: https://issues.apache.org/jira/browse/HBASE-16998
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 2.0.0
>
> Attachments: HBASE-16998.001.patch, HBASE-16998.002.HBASE-16961.patch
>
>
> Given the collected table usage reports from RegionServers, the Master needs 
> to inspect all filesystem-use quotas and determine which need to move into 
> violation and which need to move out of violation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16998) [Master] Analyze table use reports and update quota violations

[Josh Elser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser updated HBASE-16998:
---
Attachment: HBASE-16998.002.HBASE-16961.patch

.002 is .001 rebased on top of HBASE-16961.

> [Master] Analyze table use reports and update quota violations
> --
>
> Key: HBASE-16998
> URL: https://issues.apache.org/jira/browse/HBASE-16998
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 2.0.0
>
> Attachments: HBASE-16998.001.patch, HBASE-16998.002.HBASE-16961.patch
>
>
> Given the collected table usage reports from RegionServers, the Master needs 
> to inspect all filesystem-use quotas and determine which need to move into 
> violation and which need to move out of violation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16261) MultiHFileOutputFormat Enhancement

[Yi Liang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yi Liang updated HBASE-16261:
-
Attachment: (was: HBase-16261-V7.patch)

>  MultiHFileOutputFormat Enhancement 
> 
>
> Key: HBASE-16261
> URL: https://issues.apache.org/jira/browse/HBASE-16261
> Project: HBase
>  Issue Type: Sub-task
>  Components: hbase, mapreduce
>Affects Versions: 2.0.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16261-V1.patch, HBASE-16261-V2.patch, 
> HBASE-16261-V3.patch, HBASE-16261-V4.patch, HBASE-16261-V5.patch, 
> HBase-16261-V6.patch, HBase-16261-V7.patch
>
>
> MultiHFileOutputFormat follow HFileOutputFormat2
> (1) HFileOutputFormat2 can read one table's region split keys. and then 
> output multiple hfiles for one family, and each hfile map to one region. We 
> can add partitioner in MultiHFileOutputFormat to make it support this feature.
> (2) HFileOutputFormat2 support Customized Compression algorithm for column 
> family and BloomFilter, also support customized DataBlockEncoding for the 
> output hfiles. We can also make MultiHFileOutputFormat to support these 
> features.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16261) MultiHFileOutputFormat Enhancement

[Yi Liang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yi Liang updated HBASE-16261:
-
Attachment: HBase-16261-V7.patch

>  MultiHFileOutputFormat Enhancement 
> 
>
> Key: HBASE-16261
> URL: https://issues.apache.org/jira/browse/HBASE-16261
> Project: HBase
>  Issue Type: Sub-task
>  Components: hbase, mapreduce
>Affects Versions: 2.0.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16261-V1.patch, HBASE-16261-V2.patch, 
> HBASE-16261-V3.patch, HBASE-16261-V4.patch, HBASE-16261-V5.patch, 
> HBase-16261-V6.patch, HBase-16261-V7.patch
>
>
> MultiHFileOutputFormat follow HFileOutputFormat2
> (1) HFileOutputFormat2 can read one table's region split keys. and then 
> output multiple hfiles for one family, and each hfile map to one region. We 
> can add partitioner in MultiHFileOutputFormat to make it support this feature.
> (2) HFileOutputFormat2 support Customized Compression algorithm for column 
> family and BloomFilter, also support customized DataBlockEncoding for the 
> output hfiles. We can also make MultiHFileOutputFormat to support these 
> features.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15222) Use less contended classes for metrics

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749760#comment-15749760
 ] 

stack commented on HBASE-15222:
---

[~ghelmling] You fellows running this in prod?

> Use less contended classes for metrics
> --
>
> Key: HBASE-15222
> URL: https://issues.apache.org/jira/browse/HBASE-15222
> Project: HBase
>  Issue Type: Improvement
>  Components: metrics
>Affects Versions: 2.0.0, 1.2.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Critical
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-15222-ADD-0.patch, HBASE-15222-v1.patch, 
> HBASE-15222-v10.patch, HBASE-15222-v11.patch, HBASE-15222-v12.patch, 
> HBASE-15222-v13.patch, HBASE-15222-v2.patch, HBASE-15222-v3.patch, 
> HBASE-15222-v5.patch, HBASE-15222-v6.patch, HBASE-15222-v8.patch, 
> HBASE-15222-v9.patch, HBASE-15222.patch
>
>
> Running the benchmarks now, but it looks like the results are pretty extreme. 
> The locking in our histograms is pretty extreme.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15222) Use less contended classes for metrics

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749737#comment-15749737
 ] 

stack commented on HBASE-15222:
---

Not sure about risk/downside.  It is a big patch. Maybe too risky then 
[~busbey]. The original thread dump here looks to be clearing of thread local 
reference q. Maybe letting in the two thread local changes would be enough late 
in 1.2 cycle.

> Use less contended classes for metrics
> --
>
> Key: HBASE-15222
> URL: https://issues.apache.org/jira/browse/HBASE-15222
> Project: HBase
>  Issue Type: Improvement
>  Components: metrics
>Affects Versions: 2.0.0, 1.2.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Critical
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-15222-ADD-0.patch, HBASE-15222-v1.patch, 
> HBASE-15222-v10.patch, HBASE-15222-v11.patch, HBASE-15222-v12.patch, 
> HBASE-15222-v13.patch, HBASE-15222-v2.patch, HBASE-15222-v3.patch, 
> HBASE-15222-v5.patch, HBASE-15222-v6.patch, HBASE-15222-v8.patch, 
> HBASE-15222-v9.patch, HBASE-15222.patch
>
>
> Running the benchmarks now, but it looks like the results are pretty extreme. 
> The locking in our histograms is pretty extreme.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-17000) [RegionServer] Compute region filesystem space use and report to Master

[Josh Elser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser updated HBASE-17000:
---
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Thanks for the review cycles, Ted and Stephen! Applied .005 to HBASE-16961.

> [RegionServer] Compute region filesystem space use and report to Master
> ---
>
> Key: HBASE-17000
> URL: https://issues.apache.org/jira/browse/HBASE-17000
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 2.0.0
>
> Attachments: HBASE-17000.001.patch, 
> HBASE-17000.002.HBASE-16961.patch, HBASE-17000.004.HBASE-16961.patch, 
> HBASE-17000.005.HBASE-16961.patch
>
>
> Each RegionServer needs to track how much space a Region takes up and roll 
> this up to the table level.
> Aggregation of this information in the Master will be covered by HBASE-16997.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17000) [RegionServer] Compute region filesystem space use and report to Master

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749698#comment-15749698
 ] 

Ted Yu commented on HBASE-17000:


I ran the following locally:
{code}
Running org.apache.hadoop.hbase.quotas.TestRegionSizeUse
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.931 sec - in 
org.apache.hadoop.hbase.quotas.TestRegionSizeUse
Running org.apache.hadoop.hbase.quotas.TestFileSystemUtilizationChore
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.443 sec - in 
org.apache.hadoop.hbase.quotas.TestFileSystemUtilizationChore
Running 
org.apache.hadoop.hbase.regionserver.TestRegionServerRegionSpaceUseReport
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.097 sec - in 
org.apache.hadoop.hbase.regionserver.TestRegionServerRegionSpaceUseReport
{code}
Go ahead with commit.

> [RegionServer] Compute region filesystem space use and report to Master
> ---
>
> Key: HBASE-17000
> URL: https://issues.apache.org/jira/browse/HBASE-17000
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 2.0.0
>
> Attachments: HBASE-17000.001.patch, 
> HBASE-17000.002.HBASE-16961.patch, HBASE-17000.004.HBASE-16961.patch, 
> HBASE-17000.005.HBASE-16961.patch
>
>
> Each RegionServer needs to track how much space a Region takes up and roll 
> this up to the table level.
> Aggregation of this information in the Master will be covered by HBASE-16997.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16261) MultiHFileOutputFormat Enhancement

[Yi Liang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yi Liang updated HBASE-16261:
-
Attachment: HBase-16261-V7.patch

>  MultiHFileOutputFormat Enhancement 
> 
>
> Key: HBASE-16261
> URL: https://issues.apache.org/jira/browse/HBASE-16261
> Project: HBase
>  Issue Type: Sub-task
>  Components: hbase, mapreduce
>Affects Versions: 2.0.0
>Reporter: Yi Liang
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16261-V1.patch, HBASE-16261-V2.patch, 
> HBASE-16261-V3.patch, HBASE-16261-V4.patch, HBASE-16261-V5.patch, 
> HBase-16261-V6.patch, HBase-16261-V7.patch
>
>
> MultiHFileOutputFormat follow HFileOutputFormat2
> (1) HFileOutputFormat2 can read one table's region split keys. and then 
> output multiple hfiles for one family, and each hfile map to one region. We 
> can add partitioner in MultiHFileOutputFormat to make it support this feature.
> (2) HFileOutputFormat2 support Customized Compression algorithm for column 
> family and BloomFilter, also support customized DataBlockEncoding for the 
> output hfiles. We can also make MultiHFileOutputFormat to support these 
> features.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17307) Fix pom.xml (distcp version)

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749669#comment-15749669
 ] 

Ted Yu commented on HBASE-17307:


In hbase-server/pom.xml :
{code}

  org.apache.hadoop
  hadoop-distcp
  ${hadoop-two.version}

{code}
There is no need to add hadoop-two.version , right ?
I don't see such for the other neighboring dependencies.

> Fix pom.xml (distcp version)
> 
>
> Key: HBASE-17307
> URL: https://issues.apache.org/jira/browse/HBASE-17307
> Project: HBase
>  Issue Type: Bug
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-17307.HBASE-7912.v1.patch
>
>
> Currently, hbase-server/pom.xml has hardcoded version for distcp package, 
> which is hadoop-two.version. This won't work for hadoop-three.version profile.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17000) [RegionServer] Compute region filesystem space use and report to Master

[Josh Elser (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749668#comment-15749668
 ] 

Josh Elser commented on HBASE-17000:


bq.  The patch has 11 line(s) that end in whitespace. Use git apply 
--whitespace=fix. 

This is in generated code. Don't think it should be fixed.

bq.  Patch generated 1 new protoc errors in hbase-server. 

This is a false positive on the following line:

{{\[DEBUG\] adding entry 
org/apache/hadoop/hbase/util/HBaseFsck$ErrorReporter$ERROR_CODE.class}}

bq. The patch causes 270 errors with Hadoop ...

These are expected as HBASE-16961 isn't treated like the master branch (which 
it should be).

[~tedyu], shall I go ahead and commit this to HBASE-16961?

> [RegionServer] Compute region filesystem space use and report to Master
> ---
>
> Key: HBASE-17000
> URL: https://issues.apache.org/jira/browse/HBASE-17000
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 2.0.0
>
> Attachments: HBASE-17000.001.patch, 
> HBASE-17000.002.HBASE-16961.patch, HBASE-17000.004.HBASE-16961.patch, 
> HBASE-17000.005.HBASE-16961.patch
>
>
> Each RegionServer needs to track how much space a Region takes up and roll 
> this up to the table level.
> Aggregation of this information in the Master will be covered by HBASE-16997.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15222) Use less contended classes for metrics

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749628#comment-15749628
 ] 

Sean Busbey commented on HBASE-15222:
-

this would be a backport for HBASE-15222, HBASE-12133, and HBASE-16146, right?

what's the risk / downside? Have we gotten some operational feedback written 
down that shows metrics as a pain point? (I've heard through the grapevine a 
bit, but it's hard to rely on that as a matter of transparency.)

> Use less contended classes for metrics
> --
>
> Key: HBASE-15222
> URL: https://issues.apache.org/jira/browse/HBASE-15222
> Project: HBase
>  Issue Type: Improvement
>  Components: metrics
>Affects Versions: 2.0.0, 1.2.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Critical
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-15222-ADD-0.patch, HBASE-15222-v1.patch, 
> HBASE-15222-v10.patch, HBASE-15222-v11.patch, HBASE-15222-v12.patch, 
> HBASE-15222-v13.patch, HBASE-15222-v2.patch, HBASE-15222-v3.patch, 
> HBASE-15222-v5.patch, HBASE-15222-v6.patch, HBASE-15222-v8.patch, 
> HBASE-15222-v9.patch, HBASE-15222.patch
>
>
> Running the benchmarks now, but it looks like the results are pretty extreme. 
> The locking in our histograms is pretty extreme.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17000) [RegionServer] Compute region filesystem space use and report to Master

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749622#comment-15749622
 ] 

Hadoop QA commented on HBASE-17000:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 3 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 4m 21s 
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m 
2s {color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s 
{color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 7m 
17s {color} | {color:green} HBASE-16961 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m 
0s {color} | {color:green} HBASE-16961 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 50s 
{color} | {color:red} hbase-protocol-shaded in HBASE-16961 has 24 extant 
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 45s 
{color} | {color:green} HBASE-16961 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s 
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 1s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 1s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m 
53s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
20s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s 
{color} | {color:red} The patch has 11 line(s) that end in whitespace. Use git 
apply --whitespace=fix. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m 2s 
{color} | {color:red} The patch causes 270 errors with Hadoop v2.4.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m 4s 
{color} | {color:red} The patch causes 270 errors with Hadoop v2.4.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m 6s 
{color} | {color:red} The patch causes 270 errors with Hadoop v2.5.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m 7s 
{color} | {color:red} The patch causes 270 errors with Hadoop v2.5.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m 9s 
{color} | {color:red} The patch causes 270 errors with Hadoop v2.5.2. {color} |
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m 23s 
{color} | {color:red} Patch generated 1 new protoc errors in hbase-server. 
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 5s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 26s 
{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 87m 17s 
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
33s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 160m 49s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL | 

[jira] [Updated] (HBASE-17308) BackupInfo getShortDescription output format

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-17308:
---
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 2.0.0
   Status: Resolved  (was: Patch Available)

Integrated to HBASE-7912 branch.

> BackupInfo getShortDescription output format
> 
>
> Key: HBASE-17308
> URL: https://issues.apache.org/jira/browse/HBASE-17308
> Project: HBase
>  Issue Type: Bug
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Trivial
> Fix For: 2.0.0
>
> Attachments: HBASE-17308.HBASE-7912.v1.patch
>
>
> Currently its multiline and is hard to parse, convert it to a single line 
> output.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16146) Counters are expensive...

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-16146:

Component/s: metrics

> Counters are expensive...
> -
>
> Key: HBASE-16146
> URL: https://issues.apache.org/jira/browse/HBASE-16146
> Project: HBase
>  Issue Type: Sub-task
>  Components: metrics
>Reporter: stack
>Assignee: Gary Helmling
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-16146.001.patch, HBASE-16146.branch-1.001.patch, 
> HBASE-16146.branch-1.3.001.patch, counters.patch, less_and_less_counters.png
>
>
> Doing workloadc, perf shows 10%+ of CPU being spent on counter#add. If I 
> disable some of the hot ones -- see patch -- I can get 10% more throughput 
> (390k to 440k). Figure something better.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-17310) Refactoring BackupAdmin: remove public access modifiers

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-17310:
---
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Integrated to HBASE-7912 branch

> Refactoring BackupAdmin: remove public access modifiers
> ---
>
> Key: HBASE-17310
> URL: https://issues.apache.org/jira/browse/HBASE-17310
> Project: HBase
>  Issue Type: Bug
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Trivial
> Attachments: HBASE-17310.HBASE-7912.v1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15222) Use less contended classes for metrics

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749588#comment-15749588
 ] 

stack commented on HBASE-15222:
---

[~busbey] ok to backport to 1.2?

> Use less contended classes for metrics
> --
>
> Key: HBASE-15222
> URL: https://issues.apache.org/jira/browse/HBASE-15222
> Project: HBase
>  Issue Type: Improvement
>  Components: metrics
>Affects Versions: 2.0.0, 1.2.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Critical
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-15222-ADD-0.patch, HBASE-15222-v1.patch, 
> HBASE-15222-v10.patch, HBASE-15222-v11.patch, HBASE-15222-v12.patch, 
> HBASE-15222-v13.patch, HBASE-15222-v2.patch, HBASE-15222-v3.patch, 
> HBASE-15222-v5.patch, HBASE-15222-v6.patch, HBASE-15222-v8.patch, 
> HBASE-15222-v9.patch, HBASE-15222.patch
>
>
> Running the benchmarks now, but it looks like the results are pretty extreme. 
> The locking in our histograms is pretty extreme.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-17292) Add observer notification before bulk loaded hfile is moved to region directory

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-17292:
---
 Hadoop Flags: Reviewed
Fix Version/s: 2.0.0

> Add observer notification before bulk loaded hfile is moved to region 
> directory
> ---
>
> Key: HBASE-17292
> URL: https://issues.apache.org/jira/browse/HBASE-17292
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0
>
> Attachments: 17292.v1.txt, 17292.v2.txt
>
>
> Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded 
> hfiles.
> However, if bulk load fails after hfile is moved to region directory but 
> before postBulkLoadHFile() hook is called, there is no way for pluggable 
> components (replication - see HBASE-17290, backup / restore) to know which 
> hfile(s) have been moved to region directory.
> This issue adds a preCommitStoreFile() hook which notifies path of to be 
> committed hfile before bulk loaded hfile is moved to region directory.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17072) CPU usage starts to climb up to 90-100% when using G1GC; purge ThreadLocal usage

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749575#comment-15749575
 ] 

stack commented on HBASE-17072:
---

Ok to backport this [~busbey]

> CPU usage starts to climb up to 90-100% when using G1GC; purge ThreadLocal 
> usage
> 
>
> Key: HBASE-17072
> URL: https://issues.apache.org/jira/browse/HBASE-17072
> Project: HBase
>  Issue Type: Bug
>  Components: Performance, regionserver
>Affects Versions: 1.0.0, 2.0.0, 1.2.0
>Reporter: Eiichi Sato
>Assignee: Eiichi Sato
>Priority: Critical
> Fix For: 2.0.0, 1.4.0, 0.98.24
>
> Attachments: HBASE-17072-0.98.patch, HBASE-17072.branch-1.001.patch, 
> HBASE-17072.master.001.patch, HBASE-17072.master.002.patch, 
> HBASE-17072.master.003.patch, HBASE-17072.master.004.patch, 
> HBASE-17072.master.005.patch, HBASE-17072.master.005.patch, 
> disable-block-header-cache.patch, mat-threadlocals.png, mat-threads.png, 
> metrics.png, slave1.svg, slave2.svg, slave3.svg, slave4.svg
>
>
> h5. Problem
> CPU usage of a region server in our CDH 5.4.5 cluster, at some point, starts 
> to gradually get higher up to nearly 90-100% when using G1GC.  We've also run 
> into this problem on CDH 5.7.3 and CDH 5.8.2.
> In our production cluster, it normally takes a few weeks for this to happen 
> after restarting a RS.  We reproduced this on our test cluster and attached 
> the results.  Please note that, to make it easy to reproduce, we did some 
> "anti-tuning" on a table when running tests.
> In metrics.png, soon after we started running some workloads against a test 
> cluster (CDH 5.8.2) at about 7 p.m. CPU usage of the two RSs started to rise. 
>  Flame Graphs (slave1.svg to slave4.svg) are generated from jstack dumps of 
> each RS process around 10:30 a.m. the next day.
> After investigating heapdumps from another occurrence on a test cluster 
> running CDH 5.7.3, we found that the ThreadLocalMap contain a lot of 
> contiguous entries of {{HFileBlock$PrefetchedHeader}} probably due to primary 
> clustering.  This caused more loops in 
> {{ThreadLocalMap#expungeStaleEntries()}}, consuming a certain amount of CPU 
> time.  What is worse is that the method is called from RPC metrics code, 
> which means even a small amount of per-RPC time soon adds up to a huge amount 
> of CPU time.
> This is very similar to the issue in HBASE-16616, but we have many 
> {{HFileBlock$PrefetchedHeader}} not only {{Counter$IndexHolder}} instances.  
> Here are some OQL counts from Eclipse Memory Analyzer (MAT).  This shows a 
> number of ThreadLocal instances in the ThreadLocalMap of a single handler 
> thread.
> {code}
> SELECT *
> FROM OBJECTS (SELECT AS RETAINED SET OBJECTS value
> FROM OBJECTS 0x4ee380430) obj
> WHERE obj.@clazz.@name = 
> "org.apache.hadoop.hbase.io.hfile.HFileBlock$PrefetchedHeader"
> #=> 10980 instances
> {code}
> {code}
> SELECT *
> FROM OBJECTS (SELECT AS RETAINED SET OBJECTS value
> FROM OBJECTS 0x4ee380430) obj
> WHERE obj.@clazz.@name = "org.apache.hadoop.hbase.util.Counter$IndexHolder"
> #=> 2052 instances
> {code}
> Although as described in HBASE-16616 this somewhat seems to be an issue in 
> G1GC side regarding weakly-reachable objects, we should keep ThreadLocal 
> usage minimal and avoid creating an indefinite number (in this case, a number 
> of HFiles) of ThreadLocal instances.
> HBASE-16146 removes ThreadLocals from the RPC metrics code.  That may solve 
> the issue (I just saw the patch, never tested it at all), but the 
> {{HFileBlock$PrefetchedHeader}} are still there in the ThreadLocalMap, which 
> may cause issues in the future again.
> h5. Our Solution
> We simply removed the whole {{HFileBlock$PrefetchedHeader}} caching and 
> fortunately we didn't notice any performance degradation for our production 
> workloads.
> Because the PrefetchedHeader caching uses ThreadLocal and because RPCs are 
> handled randomly in any of the handlers, small Get or small Scan RPCs do not 
> benefit from the caching (See HBASE-10676 and HBASE-11402 for the details).  
> Probably, we need to see how well reads are saved by the caching for large 
> Scan or Get RPCs and especially for compactions if we really remove the 
> caching. It's probably better if we can remove ThreadLocals without breaking 
> the current caching behavior.
> FWIW, I'm attaching the patch we applied. It's for CDH 5.4.5.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16146) Counters are expensive...

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749571#comment-15749571
 ] 

stack commented on HBASE-16146:
---

Ok I backport this one [~busbey]?

> Counters are expensive...
> -
>
> Key: HBASE-16146
> URL: https://issues.apache.org/jira/browse/HBASE-16146
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Assignee: Gary Helmling
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-16146.001.patch, HBASE-16146.branch-1.001.patch, 
> HBASE-16146.branch-1.3.001.patch, counters.patch, less_and_less_counters.png
>
>
> Doing workloadc, perf shows 10%+ of CPU being spent on counter#add. If I 
> disable some of the hot ones -- see patch -- I can get 10% more throughput 
> (390k to 440k). Figure something better.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-17292) Add observer notification before bulk loaded hfile is moved to region directory

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-17292:
---
Attachment: 17292.v2.txt

Patch v2 addresses Vlad's comments

> Add observer notification before bulk loaded hfile is moved to region 
> directory
> ---
>
> Key: HBASE-17292
> URL: https://issues.apache.org/jira/browse/HBASE-17292
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 17292.v1.txt, 17292.v2.txt
>
>
> Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded 
> hfiles.
> However, if bulk load fails after hfile is moved to region directory but 
> before postBulkLoadHFile() hook is called, there is no way for pluggable 
> components (replication - see HBASE-17290, backup / restore) to know which 
> hfile(s) have been moved to region directory.
> This issue adds a preCommitStoreFile() hook which notifies path of to be 
> committed hfile before bulk loaded hfile is moved to region directory.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17149) Procedure v2 - Fix nonce submission

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749539#comment-15749539
 ] 

Hadoop QA commented on HBASE-17149:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 22 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s 
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 
0s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 42s 
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s 
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 46s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 46s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
26m 56s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 30s 
{color} | {color:red} hbase-procedure generated 1 new + 0 unchanged - 0 fixed = 
1 total (was 0) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 28s 
{color} | {color:red} hbase-server generated 1 new + 1 unchanged - 0 fixed = 2 
total (was 1) {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 30s 
{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 105m 20s 
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
26s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 149m 12s {color} 
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-procedure |
|  |  Boxed value is unboxed and then immediately reboxed in 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.setFailureResultForNonce(NonceKey,
 String, User, IOException)  At ProcedureExecutor.java:then immediately reboxed 
in 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.setFailureResultForNonce(NonceKey,
 String, User, IOException)  At ProcedureExecutor.java:[line 745] |
| Failed junit tests | 
hadoop.hbase.security.access.TestCoprocessorWhitelistMasterObserver |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12843272/nonce.patch |
| JIRA Issue | HBASE-17149 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux e6bfa35f376d 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 
21:21:05 UTC 2016 x86_64 x86_64 

[jira] [Commented] (HBASE-17309) Fix connection leaks in TestAcidGuarantees

[huaxiang sun (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749494#comment-15749494
 ] 

huaxiang sun commented on HBASE-17309:
--

Thanks Ted, will change and upload a new patch.

> Fix connection leaks in TestAcidGuarantees
> --
>
> Key: HBASE-17309
> URL: https://issues.apache.org/jira/browse/HBASE-17309
> Project: HBase
>  Issue Type: Bug
>  Components: integration tests
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-17309-master-001.patch
>
>
> Connections are not closed in some of TestAcidGuarantees's classes, this 
> makes integration test fail in some cases.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17316) Addendum to HBASE-17294, 'External Configuration for Memory Compaction'

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749370#comment-15749370
 ] 

Hudson commented on HBASE-17316:


SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2132 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/2132/])
HBASE-17316: Addendum of HBASE-17294 (stack: rev 
691f266fc2f2b3d7bd9b3108f795f12b35e334c3)
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromClient.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java


> Addendum to HBASE-17294, 'External Configuration for Memory Compaction'
> ---
>
> Key: HBASE-17316
> URL: https://issues.apache.org/jira/browse/HBASE-17316
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
> Fix For: 2.0.0
>
> Attachments: HBASE-17316.patch
>
>
> Updating 2 tests that failed during the commit of HBASE-17294



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17313) Add BufferedMutatorParams#clone method

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749373#comment-15749373
 ] 

Hudson commented on HBASE-17313:


SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2132 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/2132/])
HBASE-17313 Add BufferedMutatorParams#clone method (Joep Rottinghuis); (stack: 
rev a5a48dbd23f3dab0f72635b2422f295a667b9ca4)
* (edit) 
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestBufferedMutatorParams.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/BufferedMutatorParams.java


> Add BufferedMutatorParams#clone method
> --
>
> Key: HBASE-17313
> URL: https://issues.apache.org/jira/browse/HBASE-17313
> Project: HBase
>  Issue Type: Sub-task
>  Components: Client
>Reporter: Joep Rottinghuis
>Assignee: Joep Rottinghuis
> Fix For: 2.0.0
>
> Attachments: HBASE-17313.master.001.patch, 
> HBASE-17313.master.002.patch
>
>
> In HBASE-17277 we allow an alternate BufferedMutator implementation. If we 
> want to use that and not make BufferedMutatorImpl public, we need to be able 
> to ask the connection to create us one. In that case, we need to clone the 
> params (and then strip one argument off).
> This jira is to add a clone method for convenience.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17294) External Configuration for Memory Compaction

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749367#comment-15749367
 ] 

Hudson commented on HBASE-17294:


SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2132 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/2132/])
HBASE-17316: Addendum of HBASE-17294 (stack: rev 
691f266fc2f2b3d7bd9b3108f795f12b35e334c3)
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromClient.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java


> External Configuration for Memory Compaction 
> -
>
> Key: HBASE-17294
> URL: https://issues.apache.org/jira/browse/HBASE-17294
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
> Fix For: 2.0.0
>
> Attachments: HBASE-17294-V01.patch, HBASE-17294-V02.patch, 
> HBASE-17294-V03.patch
>
>
> We would like to have a single external knob to control memstore compaction.
> Possible memstore compaction policies are none, basic, and eager.
> This sub-task allows to set this property at the column family level at table 
> creation time:
> {code}
> create ‘’,
>{NAME => ‘’, 
> IN_MEMORY_COMPACTION => ‘’}
> {code}
> or to set this at the global configuration level by setting the property in 
> hbase-site.xml, with BASIC being the default value:
> {code}
> 
>   hbase.hregion.compacting.memstore.type
>   
> 
> {code}
> The values used in this property can change as memstore compaction policies 
> evolve over time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17292) Add observer notification before bulk loaded hfile is moved to region directory

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749297#comment-15749297
 ] 

Vladimir Rodionov commented on HBASE-17292:
---

Yes, add postCommit hook and fix . lgtm after that. 

> Add observer notification before bulk loaded hfile is moved to region 
> directory
> ---
>
> Key: HBASE-17292
> URL: https://issues.apache.org/jira/browse/HBASE-17292
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 17292.v1.txt
>
>
> Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded 
> hfiles.
> However, if bulk load fails after hfile is moved to region directory but 
> before postBulkLoadHFile() hook is called, there is no way for pluggable 
> components (replication - see HBASE-17290, backup / restore) to know which 
> hfile(s) have been moved to region directory.
> This issue adds a preCommitStoreFile() hook which notifies path of to be 
> committed hfile before bulk loaded hfile is moved to region directory.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-17292) Add observer notification before bulk loaded hfile is moved to region directory

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-17292:
---
Description: 
Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded 
hfiles.
However, if bulk load fails after hfile is moved to region directory but before 
postBulkLoadHFile() hook is called, there is no way for pluggable components 
(replication - see HBASE-17290, backup / restore) to know which hfile(s) have 
been moved to region directory.

This issue adds a preCommitStoreFile() hook which notifies path of to be 
committed hfile before bulk loaded hfile is moved to region directory.

  was:
Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded 
hfiles.
However, if bulk load fails after hfile is moved to region directory but before 
postBulkLoadHFile() hook is called, there is no way for pluggable components 
(replication, backup / restore) to know which hfile(s) have been moved to 
region directory.

This issue adds a preCommitStoreFile() hook which notifies path of to be 
committed hfile before bulk loaded hfile is moved to region directory.


> Add observer notification before bulk loaded hfile is moved to region 
> directory
> ---
>
> Key: HBASE-17292
> URL: https://issues.apache.org/jira/browse/HBASE-17292
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 17292.v1.txt
>
>
> Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded 
> hfiles.
> However, if bulk load fails after hfile is moved to region directory but 
> before postBulkLoadHFile() hook is called, there is no way for pluggable 
> components (replication - see HBASE-17290, backup / restore) to know which 
> hfile(s) have been moved to region directory.
> This issue adds a preCommitStoreFile() hook which notifies path of to be 
> committed hfile before bulk loaded hfile is moved to region directory.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17292) Add observer notification before bulk loaded hfile is moved to region directory

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749244#comment-15749244
 ] 

Ted Yu commented on HBASE-17292:


bq. Hooks in RegionObserver usually comes in pairs: pre- and post-
Just want to confirm: you're suggesting adding postCommitStoreFile() in 
HStore#bulkLoadHFile() after fs.commitStoreFile() call:
{code}
  public Path bulkLoadHFile(String srcPathStr, Path dstPath) throws IOException 
{
Path srcPath = new Path(srcPathStr);
fs.commitStoreFile(srcPath, dstPath);
{code}

bq. Who else is going to use this hook

Once this hook is added, bulk load support for replication would be able to use 
it as well.

> Add observer notification before bulk loaded hfile is moved to region 
> directory
> ---
>
> Key: HBASE-17292
> URL: https://issues.apache.org/jira/browse/HBASE-17292
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 17292.v1.txt
>
>
> Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded 
> hfiles.
> However, if bulk load fails after hfile is moved to region directory but 
> before postBulkLoadHFile() hook is called, there is no way for pluggable 
> components (replication, backup / restore) to know which hfile(s) have been 
> moved to region directory.
> This issue adds a preCommitStoreFile() hook which notifies path of to be 
> committed hfile before bulk loaded hfile is moved to region directory.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17292) Add observer notification before bulk loaded hfile is moved to region directory

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749233#comment-15749233
 ] 

Vladimir Rodionov commented on HBASE-17292:
---

Another note:

For your specific problem you extend public API (RegionObserver). Make sure you 
have other use cases for new API and you can convince community in necessity of 
this decision. 

> Add observer notification before bulk loaded hfile is moved to region 
> directory
> ---
>
> Key: HBASE-17292
> URL: https://issues.apache.org/jira/browse/HBASE-17292
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 17292.v1.txt
>
>
> Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded 
> hfiles.
> However, if bulk load fails after hfile is moved to region directory but 
> before postBulkLoadHFile() hook is called, there is no way for pluggable 
> components (replication, backup / restore) to know which hfile(s) have been 
> moved to region directory.
> This issue adds a preCommitStoreFile() hook which notifies path of to be 
> committed hfile before bulk loaded hfile is moved to region directory.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17292) Add observer notification before bulk loaded hfile is moved to region directory

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749199#comment-15749199
 ] 

Vladimir Rodionov commented on HBASE-17292:
---

Ted, couple notes:

# Hooks in RegionObserver usually comes in pairs: pre- and post-. You introduce 
only one.
# Who else is going to use this hook in your opinion?  

{code}
   /**
   * Called before moving bulk loaded hfile to region directory.
   *
   * @param ctx
   * @param family column family
   * @param pairs List of pairs of { HFile location in staging dir, HFile path 
in region dir }
   * Each pair are for the same hfile.
   * @throws IOException
   */
  void preCommitStoreFile(final ObserverContext 
ctx,
  final byte[] family, final List> pairs) throws 
IOException;
 
{code}

You have String and Path for file locations. Make them both Path.

> Add observer notification before bulk loaded hfile is moved to region 
> directory
> ---
>
> Key: HBASE-17292
> URL: https://issues.apache.org/jira/browse/HBASE-17292
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 17292.v1.txt
>
>
> Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded 
> hfiles.
> However, if bulk load fails after hfile is moved to region directory but 
> before postBulkLoadHFile() hook is called, there is no way for pluggable 
> components (replication, backup / restore) to know which hfile(s) have been 
> moved to region directory.
> This issue adds a preCommitStoreFile() hook which notifies path of to be 
> committed hfile before bulk loaded hfile is moved to region directory.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-17000) [RegionServer] Compute region filesystem space use and report to Master

[Josh Elser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser updated HBASE-17000:
---
Attachment: HBASE-17000.005.HBASE-16961.patch

.005 encompasses all suggestions from reviewboard.

> [RegionServer] Compute region filesystem space use and report to Master
> ---
>
> Key: HBASE-17000
> URL: https://issues.apache.org/jira/browse/HBASE-17000
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 2.0.0
>
> Attachments: HBASE-17000.001.patch, 
> HBASE-17000.002.HBASE-16961.patch, HBASE-17000.004.HBASE-16961.patch, 
> HBASE-17000.005.HBASE-16961.patch
>
>
> Each RegionServer needs to track how much space a Region takes up and roll 
> this up to the table level.
> Aggregation of this information in the Master will be covered by HBASE-16997.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-17149) Procedure v2 - Fix nonce submission

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-17149:
--
Status: Patch Available  (was: Open)

> Procedure v2 - Fix nonce submission
> ---
>
> Key: HBASE-17149
> URL: https://issues.apache.org/jira/browse/HBASE-17149
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Affects Versions: 1.2.4, 1.1.7, 2.0.0, 1.3.0, 1.4.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Attachments: nonce.patch
>
>
> instead of having all the logic in submitProcedure(), split in 
> registerNonce() + submitProcedure().
> In this case we can avoid calling the coprocessor twice and having a clean 
> submit logic knowing that there will only be one submission.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-17149) Procedure v2 - Fix nonce submission

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-17149:
--
Attachment: nonce.patch

Matteo's patch

> Procedure v2 - Fix nonce submission
> ---
>
> Key: HBASE-17149
> URL: https://issues.apache.org/jira/browse/HBASE-17149
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.1.7, 1.2.4
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Attachments: nonce.patch
>
>
> instead of having all the logic in submitProcedure(), split in 
> registerNonce() + submitProcedure().
> In this case we can avoid calling the coprocessor twice and having a clean 
> submit logic knowing that there will only be one submission.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16542) Skip full backup in selected backup tests

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749099#comment-15749099
 ] 

Ted Yu commented on HBASE-16542:


Take TestIncrementalBackupDeleteTable as an example, skipping the restore of 
full backup reduced runtime by 1 minute.

> Skip full backup in selected backup tests
> -
>
> Key: HBASE-16542
> URL: https://issues.apache.org/jira/browse/HBASE-16542
> Project: HBase
>  Issue Type: Test
>Reporter: Ted Yu
>Priority: Minor
>  Labels: backup
>
> Since automatic mode is always used (HBASE-16037), some tests take longer 
> time to run:
> 1. restore full backup
> 2. restore incremental backup
> Action 2 would execute action 1 again.
> We can selectively skip full backup in backup / restore tests where 
> incremental backup is involved.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-16579) Concurrent backup / restore support

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu resolved HBASE-16579.

Resolution: Later

With dramatic refactoring, this may require significant amount of work.

> Concurrent backup / restore support
> ---
>
> Key: HBASE-16579
> URL: https://issues.apache.org/jira/browse/HBASE-16579
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>  Labels: backup
>
> Currently we take exclusive lock on hbase:backup table in backup / restore 
> procedures.
> There is only one backup or restore operation which can run at given time.
> We should consider using shared lock on hbase:backup table in the procedures 
> so that concurrent backup / restore operations can proceed.
> The concurrency is useful when table sets don't overlap.
> We may devise other mechanism(s) when there is overlap among the table sets.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17081) Flush the entire CompactingMemStore content to disk

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749072#comment-15749072
 ] 

stack commented on HBASE-17081:
---

Apache blog and refguide would be great places for blog and operator/dev doc.

> Flush the entire CompactingMemStore content to disk
> ---
>
> Key: HBASE-17081
> URL: https://issues.apache.org/jira/browse/HBASE-17081
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Anastasia Braginsky
>Assignee: Anastasia Braginsky
> Attachments: HBASE-15787_8.patch, HBASE-17081-V01.patch, 
> HBASE-17081-V02.patch, HBASE-17081-V03.patch, HBASE-17081-V04.patch, 
> HBASE-17081-V05.patch, HBASE-17081-V06.patch, HBASE-17081-V06.patch, 
> HBaseMeetupDecember2016-V02.pptx, Pipelinememstore_fortrunk_3.patch
>
>
> Part of CompactingMemStore's memory is held by an active segment, and another 
> part is divided between immutable segments in the compacting pipeline. Upon 
> flush-to-disk request we want to flush all of it to disk, in contrast to 
> flushing only tail of the compacting pipeline.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17318) Increment does not add new column if the increment amount is zero at first time writing

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749019#comment-15749019
 ] 

Ted Yu commented on HBASE-17318:


Failure of TestDurability might be related to the patch.

> Increment does not add new column if the increment amount is zero at first 
> time writing
> ---
>
> Key: HBASE-17318
> URL: https://issues.apache.org/jira/browse/HBASE-17318
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0, 0.98.23, 1.2.4
>Reporter: Guangxu Cheng
> Attachments: HBASE-17318-branch-1.2-v1.patch
>
>
> When the data written for the first time is 0, no new columns are added.
> Iterate the input columns and update existing values if they were found, 
> otherwise add new column initialized to the increment amount.
> Does not add new column if the increment amount is zero at first time 
> writting.
> It is necessary to add a new column at the first write to 0. 
> If not, the result of using the phoenix is NULL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17318) Increment does not add new column if the increment amount is zero at first time writing

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15748877#comment-15748877
 ] 

Hadoop QA commented on HBASE-17318:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s 
{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 
38s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s 
{color} | {color:green} branch-1.2 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s 
{color} | {color:green} branch-1.2 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 
6s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
26s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
51s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s 
{color} | {color:green} branch-1.2 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s 
{color} | {color:green} branch-1.2 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s 
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 31s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
54s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
12m 44s {color} | {color:green} The patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m 
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 2s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s 
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 74m 44s {color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
20s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 108m 7s {color} 
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.wal.TestDurability |
| Timed out junit tests | 
org.apache.hadoop.hbase.snapshot.TestSecureExportSnapshot |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:d6626eb |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12843220/HBASE-17318-branch-1.2-v1.patch
 |
| JIRA Issue | HBASE-17318 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux f0bef923c65d 3.13.0-93-generic 

[jira] [Commented] (HBASE-17018) Spooling BufferedMutator

[Joep Rottinghuis (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15748827#comment-15748827
 ] 

Joep Rottinghuis commented on HBASE-17018:
--

If I can make it so that the SpoolingBufferedMutator doesn't even connect to an 
actual BufferedMutatorImpl, this setup could help in some tests where you want 
to be able to send mutations but just have them appear in a file.

> Spooling BufferedMutator
> 
>
> Key: HBASE-17018
> URL: https://issues.apache.org/jira/browse/HBASE-17018
> Project: HBase
>  Issue Type: New Feature
>Reporter: Joep Rottinghuis
> Attachments: HBASE-17018.master.001.patch, 
> HBASE-17018SpoolingBufferedMutatorDesign-v1.pdf, YARN-4061 HBase requirements 
> for fault tolerant writer.pdf
>
>
> For Yarn Timeline Service v2 we use HBase as a backing store.
> A big concern we would like to address is what to do if HBase is 
> (temporarily) down, for example in case of an HBase upgrade.
> Most of the high volume writes will be mostly on a best-effort basis, but 
> occasionally we do a flush. Mainly during application lifecycle events, 
> clients will call a flush on the timeline service API. In order to handle the 
> volume of writes we use a BufferedMutator. When flush gets called on our API, 
> we in turn call flush on the BufferedMutator.
> We would like our interface to HBase be able to spool the mutations to a 
> filesystems in case of HBase errors. If we use the Hadoop filesystem 
> interface, this can then be HDFS, gcs, s3, or any other distributed storage. 
> The mutations can then later be re-played, for example through a MapReduce 
> job.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-17316) Addendum to HBASE-17294, 'External Configuration for Memory Compaction'

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-17316:
--
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 2.0.0
   Status: Resolved  (was: Patch Available)

Pushed to master. Thanks @eshcar for keeping an eye on builds fixing my 
breaking commit.

> Addendum to HBASE-17294, 'External Configuration for Memory Compaction'
> ---
>
> Key: HBASE-17316
> URL: https://issues.apache.org/jira/browse/HBASE-17316
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
> Fix For: 2.0.0
>
> Attachments: HBASE-17316.patch
>
>
> Updating 2 tests that failed during the commit of HBASE-17294



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-17316) Addendum to HBASE-17294, 'External Configuration for Memory Compaction'

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-17316:
--
Summary: Addendum to HBASE-17294, 'External Configuration for Memory 
Compaction'  (was: Addendum to HBASE-17294)

> Addendum to HBASE-17294, 'External Configuration for Memory Compaction'
> ---
>
> Key: HBASE-17316
> URL: https://issues.apache.org/jira/browse/HBASE-17316
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
> Attachments: HBASE-17316.patch
>
>
> Updating 2 tests that failed during the commit of HBASE-17294



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17313) Add BufferedMutatorParams#clone method

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15748738#comment-15748738
 ] 

stack commented on HBASE-17313:
---

Thanks [~jrottinghuis]. I pushed the amendment HBASE-17313.master.002.patch

> Add BufferedMutatorParams#clone method
> --
>
> Key: HBASE-17313
> URL: https://issues.apache.org/jira/browse/HBASE-17313
> Project: HBase
>  Issue Type: Sub-task
>  Components: Client
>Reporter: Joep Rottinghuis
>Assignee: Joep Rottinghuis
> Fix For: 2.0.0
>
> Attachments: HBASE-17313.master.001.patch, 
> HBASE-17313.master.002.patch
>
>
> In HBASE-17277 we allow an alternate BufferedMutator implementation. If we 
> want to use that and not make BufferedMutatorImpl public, we need to be able 
> to ask the connection to create us one. In that case, we need to clone the 
> params (and then strip one argument off).
> This jira is to add a clone method for convenience.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17309) Fix connection leaks in TestAcidGuarantees

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15748619#comment-15748619
 ] 

Ted Yu commented on HBASE-17309:


{code}
148   table.close();
149   connection.close();
{code}
Please enclose connection.close() in finally block.

> Fix connection leaks in TestAcidGuarantees
> --
>
> Key: HBASE-17309
> URL: https://issues.apache.org/jira/browse/HBASE-17309
> Project: HBase
>  Issue Type: Bug
>  Components: integration tests
>Affects Versions: 2.0.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-17309-master-001.patch
>
>
> Connections are not closed in some of TestAcidGuarantees's classes, this 
> makes integration test fail in some cases.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17318) Increment does not add new column if the increment amount is zero at first time writing

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15748594#comment-15748594
 ] 

Ted Yu commented on HBASE-17318:


Can you add a unit test to prevent regression ?

Please attach patch for master branch.

> Increment does not add new column if the increment amount is zero at first 
> time writing
> ---
>
> Key: HBASE-17318
> URL: https://issues.apache.org/jira/browse/HBASE-17318
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0, 0.98.23, 1.2.4
>Reporter: Guangxu Cheng
> Attachments: HBASE-17318-branch-1.2-v1.patch
>
>
> When the data written for the first time is 0, no new columns are added.
> Iterate the input columns and update existing values if they were found, 
> otherwise add new column initialized to the increment amount.
> Does not add new column if the increment amount is zero at first time 
> writting.
> It is necessary to add a new column at the first write to 0. 
> If not, the result of using the phoenix is NULL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-17318) Increment does not add new column if the increment amount is zero at first time writing

[Guangxu Cheng (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guangxu Cheng updated HBASE-17318:
--
Status: Patch Available  (was: Open)

> Increment does not add new column if the increment amount is zero at first 
> time writing
> ---
>
> Key: HBASE-17318
> URL: https://issues.apache.org/jira/browse/HBASE-17318
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 1.2.4, 0.98.23, 2.0.0
>Reporter: Guangxu Cheng
> Attachments: HBASE-17318-branch-1.2-v1.patch
>
>
> When the data written for the first time is 0, no new columns are added.
> Iterate the input columns and update existing values if they were found, 
> otherwise add new column initialized to the increment amount.
> Does not add new column if the increment amount is zero at first time 
> writting.
> It is necessary to add a new column at the first write to 0. 
> If not, the result of using the phoenix is NULL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17296) Provide per peer throttling for replication

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15748320#comment-15748320
 ] 

Hadoop QA commented on HBASE-17296:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 1s 
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 1s 
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 18s 
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 
47s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 20s 
{color} | {color:green} branch-1 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 31s 
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 9m 
2s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
51s {color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 59s 
{color} | {color:red} hbase-server in branch-1 has 2 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s 
{color} | {color:green} branch-1 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 11s 
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s 
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 19s 
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 19s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 19s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 36s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 36s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 36s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 9m 
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
17m 12s {color} | {color:green} The patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 
54s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s 
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 17s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 23s 
{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 59s 
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 113m 42s 
{color} | {color:red} hbase-server in the patch failed. {color} |
| 

[jira] [Updated] (HBASE-17318) Increment does not add new column if the increment amount is zero at first time writing

[Guangxu Cheng (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guangxu Cheng updated HBASE-17318:
--
Attachment: HBASE-17318-branch-1.2-v1.patch

> Increment does not add new column if the increment amount is zero at first 
> time writing
> ---
>
> Key: HBASE-17318
> URL: https://issues.apache.org/jira/browse/HBASE-17318
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0, 0.98.23, 1.2.4
>Reporter: Guangxu Cheng
> Attachments: HBASE-17318-branch-1.2-v1.patch
>
>
> When the data written for the first time is 0, no new columns are added.
> Iterate the input columns and update existing values if they were found, 
> otherwise add new column initialized to the increment amount.
> Does not add new column if the increment amount is zero at first time 
> writting.
> It is necessary to add a new column at the first write to 0. 
> If not, the result of using the phoenix is NULL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-17318) Increment does not add new column if the increment amount is zero at first time writing

[Guangxu Cheng (JIRA)]

Guangxu Cheng created HBASE-17318:
-

 Summary: Increment does not add new column if the increment amount 
is zero at first time writing
 Key: HBASE-17318
 URL: https://issues.apache.org/jira/browse/HBASE-17318
 Project: HBase
  Issue Type: Improvement
Affects Versions: 1.2.4, 0.98.23, 2.0.0
Reporter: Guangxu Cheng


When the data written for the first time is 0, no new columns are added.

Iterate the input columns and update existing values if they were found, 
otherwise add new column initialized to the increment amount.

Does not add new column if the increment amount is zero at first time writting.

It is necessary to add a new column at the first write to 0. 
If not, the result of using the phoenix is NULL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17296) Provide per peer throttling for replication

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15748224#comment-15748224
 ] 

Hadoop QA commented on HBASE-17296:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 11m 59s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 0s 
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 0s 
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 14s 
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 
55s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s 
{color} | {color:green} branch-1 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 25s 
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 9m 
5s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m 
3s {color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 52s 
{color} | {color:red} hbase-server in branch-1 has 2 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s 
{color} | {color:green} branch-1 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s 
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s 
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 
36s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s 
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 15s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 15s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 26s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 26s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 26s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 9m 
0s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
14m 56s {color} | {color:green} The patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s 
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 10s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 20s 
{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 49s 
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 116m 24s 
{color} | {color:red} hbase-server in the patch failed. {color} |
| 

[jira] [Commented] (HBASE-17277) Allow alternate BufferedMutator implementation

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15748221#comment-15748221
 ] 

Hudson commented on HBASE-17277:


SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2130 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/2130/])
HBASE-17277 Allow alternate BufferedMutator implemenation Specify the (stack: 
rev 68ce3f1e3b22e8a8c6b17072738be6a08489dfca)
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/BufferedMutator.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/BufferedMutatorImpl.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/BufferedMutatorParams.java
* (add) 
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestBufferedMutator.java


> Allow alternate BufferedMutator implementation
> --
>
> Key: HBASE-17277
> URL: https://issues.apache.org/jira/browse/HBASE-17277
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0
>
> Attachments: HBASE-17277.master.001.patch, 
> HBASE-17277.master.002.patch
>
>
> Allow being able to supply alternate BufferedMutator implementation. For 
> example, the parent issue would like to spool writes to the filesystem if 
> hbase is down.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17313) Add BufferedMutatorParams#clone method

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15748219#comment-15748219
 ] 

Hudson commented on HBASE-17313:


SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2130 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/2130/])
HBASE-17313 Add BufferedMutatorParams#clone method (Joep Rottinghuis) (stack: 
rev a73b0b3e6d42c3f7fd0c3a0d9f80c2763a6cf069)
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/BufferedMutatorParams.java
* (add) 
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestBufferedMutatorParams.java


> Add BufferedMutatorParams#clone method
> --
>
> Key: HBASE-17313
> URL: https://issues.apache.org/jira/browse/HBASE-17313
> Project: HBase
>  Issue Type: Sub-task
>  Components: Client
>Reporter: Joep Rottinghuis
>Assignee: Joep Rottinghuis
> Fix For: 2.0.0
>
> Attachments: HBASE-17313.master.001.patch, 
> HBASE-17313.master.002.patch
>
>
> In HBASE-17277 we allow an alternate BufferedMutator implementation. If we 
> want to use that and not make BufferedMutatorImpl public, we need to be able 
> to ask the connection to create us one. In that case, we need to clone the 
> params (and then strip one argument off).
> This jira is to add a clone method for convenience.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15787) Change the flush related heuristics to work with offheap size configured

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15748174#comment-15748174
 ] 

Hadoop QA commented on HBASE-15787:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 4 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 
54s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
41s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 37s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
25m 34s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 88m 52s 
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
14s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 126m 10s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12843197/HBASE-15787_8.patch |
| JIRA Issue | HBASE-15787 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux 7267434fa456 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
 |
| git revision | master / a73b0b3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4914/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4914/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Change the flush related heuristics to work with offheap size configured
> 
>
> Key: HBASE-15787
> URL: https://issues.apache.org/jira/browse/HBASE-15787
> Project: HBase
>  Issue Type: Sub-task
>  Components: regionserver
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> 

[jira] [Commented] (HBASE-16421) Introducing the CellChunkMap as a new additional index variant in the MemStore

[ramkrishna.s.vasudevan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15748154#comment-15748154
 ] 

ramkrishna.s.vasudevan commented on HBASE-16421:


I have not done that. But let me get back to you before the end of this year 
with my findings. I will complete that AR ASAP.

> Introducing the CellChunkMap as a new additional index variant in the MemStore
> --
>
> Key: HBASE-16421
> URL: https://issues.apache.org/jira/browse/HBASE-16421
> Project: HBase
>  Issue Type: Umbrella
>Reporter: Anastasia Braginsky
> Attachments: CellChunkMapRevived.pdf, 
> IntroductiontoNewFlatandCompactMemStore.pdf
>
>
> Follow up for HBASE-14921. This is going to be the umbrella JIRA to include 
> all the parts of integration of the CellChunkMap to the MemStore.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17296) Provide per peer throttling for replication

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15748115#comment-15748115
 ] 

Hadoop QA commented on HBASE-17296:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 10m 38s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 1s 
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 1s 
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 13s 
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 
55s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s 
{color} | {color:green} branch-1 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 27s 
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 8m 
48s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m 
4s {color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 51s 
{color} | {color:red} hbase-server in branch-1 has 2 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s 
{color} | {color:green} branch-1 passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s 
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s 
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 
36s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 13s 
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 13s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 13s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 26s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 26s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 26s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 8m 
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
14m 53s {color} | {color:green} The patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 5s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s 
{color} | {color:green} the patch passed with JDK v1.8.0_111 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 9s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 20s 
{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 48s 
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 85m 51s 
{color} | {color:green} hbase-server in the patch passed. {color} 

[jira] [Commented] (HBASE-16421) Introducing the CellChunkMap as a new additional index variant in the MemStore

[Anastasia Braginsky (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15748066#comment-15748066
 ] 

Anastasia Braginsky commented on HBASE-16421:
-

Returning to this issue after a while.
[~anoop.hbase], [~ram_krish], do you (by any chance) have some results about 
running scans on top of CellChunkMaps? (related to the possible GC problem with 
scan's performance as discussed earlier)

> Introducing the CellChunkMap as a new additional index variant in the MemStore
> --
>
> Key: HBASE-16421
> URL: https://issues.apache.org/jira/browse/HBASE-16421
> Project: HBase
>  Issue Type: Umbrella
>Reporter: Anastasia Braginsky
> Attachments: CellChunkMapRevived.pdf, 
> IntroductiontoNewFlatandCompactMemStore.pdf
>
>
> Follow up for HBASE-14921. This is going to be the umbrella JIRA to include 
> all the parts of integration of the CellChunkMap to the MemStore.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17316) Addendum to HBASE-17294

[Eshcar Hillel (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15748035#comment-15748035
 ] 

Eshcar Hillel commented on HBASE-17316:
---

QA passed.
[~stack] please consider committing this addendum as well.

> Addendum to HBASE-17294
> ---
>
> Key: HBASE-17316
> URL: https://issues.apache.org/jira/browse/HBASE-17316
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
> Attachments: HBASE-17316.patch
>
>
> Updating 2 tests that failed during the commit of HBASE-17294



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17316) Addendum to HBASE-17294

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15748029#comment-15748029
 ] 

Hadoop QA commented on HBASE-17316:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 
59s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
41s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 37s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
23m 40s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 89m 27s 
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
13s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 124m 38s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12843190/HBASE-17316.patch |
| JIRA Issue | HBASE-17316 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux 5254148747f5 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / a73b0b3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4912/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/4912/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Addendum to HBASE-17294
> ---
>
> Key: HBASE-17316
> URL: https://issues.apache.org/jira/browse/HBASE-17316
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
> Attachments: HBASE-17316.patch
>
>
> Updating 2 tests that failed during the commit of HBASE-17294



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-9774) Provide a way for coprocessors to register and report custom metrics

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15748020#comment-15748020
 ] 

Hadoop QA commented on HBASE-9774:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 12m 55s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 13 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 29s 
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 42s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m 
59s {color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s 
{color} | {color:blue} Skipped patched modules with no Java source: . 
hbase-assembly {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 
52s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 58s 
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 5s 
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m 
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 1s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m 1s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m 
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m 
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 9s 
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
24m 10s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s 
{color} | {color:blue} Skipped patched modules with no Java source: . 
hbase-assembly {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 23s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 48s 
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 9s 
{color} | {color:green} hbase-metrics-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 16s 
{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 91m 56s {color} 
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 13s 
{color} | {color:green} hbase-assembly in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 18s 
{color} | {color:green} hbase-examples in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 25s 
{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 19s 
{color} | {color:green} hbase-metrics in the patch passed. 

[jira] [Commented] (HBASE-11392) add/remove peer requests should be routed through master

[Guanghao Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-11392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15747966#comment-15747966
 ] 

Guanghao Zhang commented on HBASE-11392:


The hbase-client findbugs warning 
"org.apache.hadoop.hbase.client.BufferedMutatorParams defines clone() but 
doesn't implement Cloneable" is not related to this patch.

> add/remove peer requests should be routed through master
> 
>
> Key: HBASE-11392
> URL: https://issues.apache.org/jira/browse/HBASE-11392
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Guanghao Zhang
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-11392-v1.patch, HBASE-11392-v2.patch
>
>
> ReplicationAdmin directly operates over the zookeeper data for replication 
> setup. We should move these operations to be routed through master for two 
> reasons: 
>  - Replication implementation details are exposed to client. We should move 
> most of the replication related classes to hbase-server package. 
>  - Routing the requests through master is the standard practice for all other 
> operations. It allows for decoupling implementation details from the client 
> and code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-17296) Provide per peer throttling for replication

[Guanghao Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15747938#comment-15747938
 ] 

Guanghao Zhang commented on HBASE-17296:


Open a new issue HBASE-17317 for the bug.

> Provide per peer throttling for replication
> ---
>
> Key: HBASE-17296
> URL: https://issues.apache.org/jira/browse/HBASE-17296
> Project: HBase
>  Issue Type: Improvement
>  Components: Replication
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-17296-branch-1.patch, HBASE-17296-branch-1.patch, 
> HBASE-17296-branch-1.patch, HBASE-17296.patch
>
>
> HBASE-9501 added a config to provide throttling for replication. And each 
> peer has same bandwidth up limit. In our use case, one cluster may have 
> several peers and several slave clusters. Each slave cluster may have 
> different scales and need different bandwidth up limit for each peer. So We 
> add bandwidth to replication peer config and provide a shell cmd set_peer 
> bandwidth to update the bandwidth in need. It has been used for a long time 
> on our clusters.  Any suggestions are welcomed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)