date:20170721


[ 
https://issues.apache.org/jira/browse/HBASE-18432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097120#comment-16097120
 ] 

Hadoop QA commented on HBASE-18432:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  2m 
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
 1s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
53s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
43s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
36s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  2m 
39s{color} | {color:red} hbase-server in HBASE-14070.HLC has 9 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
45s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
 0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
29m  3s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
23s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m 
15s{color} | {color:red} hbase-common generated 1 new + 0 unchanged - 0 fixed = 
1 total (was 0) {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  2m 14s{color} 
| {color:red} hbase-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 71m 14s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
58s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}125m  5s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.TestTimestampType |
| Timed out junit tests | 
org.apache.hadoop.hbase.master.procedure.TestDisableTableProcedure |
|   | org.apache.hadoop.hbase.regionserver.wal.TestSecureWALReplay |
|   | org.apache.hadoop.hbase.master.procedure.TestModifyTableProcedure |
|   | org.apache.hadoop.hbase.master.procedure.TestCreateTableProcedure |
|   | org.apache.hadoop.hbase.master.procedure.TestEnableTableProcedure |
|   | org.apache.hadoop.hbase.master.procedure.TestServerCrashProcedure |
|   | org.apache.hadoop.hbase.master.procedure.TestDeleteTableProcedure |
|   | org.apache.hadoop.hbase.regionserver.TestRowTooBig |
|   | org.apache.hadoop.hbase.regionserver.TestSplitLogWorker |
|   | org.apache.hadoop.hbase.regionserver.wal.TestAsyncWALReplay |
|   | org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence |
|   | org.apache.hadoop.hbase.coprocessor.TestHTableWrapper |
|   |

[jira] [Commented] (HBASE-18389) Remove byte[] from formal parameter of sizeOf() of ClassSize, ClassSize.MemoryLayout and ClassSize.UnsafeLayout

2017-07-21 Thread Xiang Li (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097114#comment-16097114
 ] 

Xiang Li commented on HBASE-18389:
--

Thanks [~chia7712] and [~anoop.hbase] very much for the review! 

> Remove byte[] from formal parameter of sizeOf() of ClassSize, 
> ClassSize.MemoryLayout and ClassSize.UnsafeLayout
> ---
>
> Key: HBASE-18389
> URL: https://issues.apache.org/jira/browse/HBASE-18389
> Project: HBase
>  Issue Type: Improvement
>  Components: util
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18389.master.000.patch, 
> HBASE-18389.master.001.patch, HBASE-18389.master.002.patch, 
> HBASE-18389.master.003.patch
>
>
> In ClassSize class and its internal static class, sizeOf() function has 2 
> formal parameters: byte[] b and int len. But the function's internal logic 
> does not use or refer to byte[] b. Could be removed.
> {code:title=hbase-common/src/main/java/org/apache/hadoop/hbase/util/ClassSize.java|borderStyle=solid}
> // Class of ClassSize
> public static long sizeOf(byte[] b, int len) {
>   return memoryLayout.sizeOf(b, len);
> }
> // Class of ClassSize.MemoryLayout
> long sizeOf(byte[] b, int len) {
>   return align(arrayHeaderSize() + len);
> }
> // Class of ClassSize.UnsafeLayout
> long sizeOf(byte[] b, int len) {
>   return align(arrayHeaderSize() + len * 
> UnsafeAccess.theUnsafe.ARRAY_BYTE_INDEX_SCALE);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18323) Remove multiple ACLs for the same user in kerberos


[ 
https://issues.apache.org/jira/browse/HBASE-18323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097111#comment-16097111
 ] 

Hadoop QA commented on HBASE-18323:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
 9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
29m 18s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
32s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
 7s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 10s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18323 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12878450/HBASE-18323-V4.patch |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux d201832e356f 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / ec3cb19 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7761/testReport/ |
| modules | C: hbase-client U: hbase-client |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7761/console |
| Powered by | Apache Yetus 0.4.0   http://yetus.apache.org |


This message was automatically generated.



> Remove multiple ACLs for the same user in kerberos
> --
>
> Key: HBASE-18323
> URL: https://issues.apache.org/jira/browse/HBASE-18323
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.2.0, 3.0.0
>Reporter: Shibin Zhang
>Assignee: Shibin Zhang
>Priority: Minor
> Attachments: HBASE-18323.patch, HBASE-18323-V2.patch, 
>

[jira] [Commented] (HBASE-18323) Remove multiple ACLs for the same user in kerberos

2017-07-21 Thread Shibin Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097105#comment-16097105
 ] 

Shibin Zhang commented on HBASE-18323:
--

[~elserj] thanks for your advice and review , i also submit  a patch  with test
  
https://issues.apache.org/jira/secure/attachment/12878450/HBASE-18323-V4.patch

> Remove multiple ACLs for the same user in kerberos
> --
>
> Key: HBASE-18323
> URL: https://issues.apache.org/jira/browse/HBASE-18323
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.2.0, 3.0.0
>Reporter: Shibin Zhang
>Assignee: Shibin Zhang
>Priority: Minor
> Attachments: HBASE-18323.patch, HBASE-18323-V2.patch, 
> HBASE-18323-V3.patch, HBASE-18323-V4.patch
>
>
> When deploy hbase in kerberos way ,there will be multiple acls in znode :
> 'world,'anyone
> : r
> 'sasl,'hbase
> : cdrwa
> 'sasl,'hbase
> : cdrwa
> I also see the related issue and apply the patch, like  
> https://issues.apache.org/jira/browse/HBASE-17717 
> but in my environment ,this situation still appear,
> After dig into the code , i found the reason in source code  ZKUtil.createAcl 
>  is
>  if (zkw.isClientReadable(node)) {
> LOG.error("isSecureZooKeeper user: clientReadable");
> acls.addAll(Ids.CREATOR_ALL_ACL);
> acls.addAll(Ids.READ_ACL_UNSAFE);
>   } else {
> LOG.error("isSecureZooKeeper user: clientReadable no");
> acls.addAll(Ids.CREATOR_ALL_ACL);
>   } 
>   acls.addAll(Ids.CREATOR_ALL_ACL);   
>   
>   Id AUTH_IDS = new Id("auth", "");
> ArrayList CREATOR_ALL_ACL = new ArrayList(Collections.singletonList(new 
> ACL(31, AUTH_IDS)));
> AUTH_IDS   with  "auth " will result  current connection auth user  add to 
> znode acl ,
> so it will appear multiple acls for same users.
> I think this line of code  we can remove  :  
> acls.addAll(Ids.CREATOR_ALL_ACL);   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18323) Remove multiple ACLs for the same user in kerberos

2017-07-21 Thread Shibin Zhang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-18323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shibin Zhang updated HBASE-18323:
-
Attachment: HBASE-18323-V4.patch

> Remove multiple ACLs for the same user in kerberos
> --
>
> Key: HBASE-18323
> URL: https://issues.apache.org/jira/browse/HBASE-18323
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.2.0, 3.0.0
>Reporter: Shibin Zhang
>Assignee: Shibin Zhang
>Priority: Minor
> Attachments: HBASE-18323.patch, HBASE-18323-V2.patch, 
> HBASE-18323-V3.patch, HBASE-18323-V4.patch
>
>
> When deploy hbase in kerberos way ,there will be multiple acls in znode :
> 'world,'anyone
> : r
> 'sasl,'hbase
> : cdrwa
> 'sasl,'hbase
> : cdrwa
> I also see the related issue and apply the patch, like  
> https://issues.apache.org/jira/browse/HBASE-17717 
> but in my environment ,this situation still appear,
> After dig into the code , i found the reason in source code  ZKUtil.createAcl 
>  is
>  if (zkw.isClientReadable(node)) {
> LOG.error("isSecureZooKeeper user: clientReadable");
> acls.addAll(Ids.CREATOR_ALL_ACL);
> acls.addAll(Ids.READ_ACL_UNSAFE);
>   } else {
> LOG.error("isSecureZooKeeper user: clientReadable no");
> acls.addAll(Ids.CREATOR_ALL_ACL);
>   } 
>   acls.addAll(Ids.CREATOR_ALL_ACL);   
>   
>   Id AUTH_IDS = new Id("auth", "");
> ArrayList CREATOR_ALL_ACL = new ArrayList(Collections.singletonList(new 
> ACL(31, AUTH_IDS)));
> AUTH_IDS   with  "auth " will result  current connection auth user  add to 
> znode acl ,
> so it will appear multiple acls for same users.
> I think this line of code  we can remove  :  
> acls.addAll(Ids.CREATOR_ALL_ACL);   



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18238) Address ruby static analysis for bin directory


[ 
https://issues.apache.org/jira/browse/HBASE-18238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097098#comment-16097098
 ] 

Mike Drob commented on HBASE-18238:
---

now that we can see rubocop output in precommit, we can claim that we are 
making progress. this patch is not meant to fix everything, but to get a big 
chunk out of the way.

Not sure what the new warnings are or why the auto-correct mode would be 
generating new warnings. I don't think that should block commit here though.

> Address ruby static analysis for bin directory
> --
>
> Key: HBASE-18238
> URL: https://issues.apache.org/jira/browse/HBASE-18238
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Mike Drob
>Assignee: Mike Drob
> Fix For: 2.0.0
>
> Attachments: HBASE-18238.patch, HBASE-18238.v2.patch, report.txt
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18433) Create convenience methods for getting ColumnDescriptors without using builders


 [ 
https://issues.apache.org/jira/browse/HBASE-18433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob updated HBASE-18433:
--
Status: Patch Available  (was: Open)

> Create convenience methods for getting ColumnDescriptors without using 
> builders
> ---
>
> Key: HBASE-18433
> URL: https://issues.apache.org/jira/browse/HBASE-18433
> Project: HBase
>  Issue Type: Bug
>  Components: Client
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
>Assignee: Mike Drob
> Attachments: HBASE-18433.patch
>
>
> The old {{HColumnDescriptor(String)}} is deprecated and going away. The most 
> straightforward replacement for it is very wordy: 
> {{ColumnFamilyDescriptorBuilder.newBuilder(byte \[]).build()}}
> We can provide a more inline replacement like 
> {{ColumnFamilyDescriptor.for(String)}} that will be useful if we do not need 
> to actually do any modifications on it, only pass it as an argument to 
> something (like table descriptor operations).
> We can do the same kind of improvement for TableDescriptor, but I do not 
> think the use case is as helpful because usually we will want to do something 
> to the modifiable TableDescriptor, and I have not seen many use cases where 
> it is passed as an argument unadorned.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18433) Create convenience methods for getting ColumnDescriptors without using builders


 [ 
https://issues.apache.org/jira/browse/HBASE-18433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob updated HBASE-18433:
--
Attachment: HBASE-18433.patch

* Create {{ColumnFamilyDescriptorBuilder.of}} methods.
* Add javadoc pointers on old HColumnDescriptor constructors to new convenience 
methods
* Update usages in the code to use the new convenience method where meaningful.

> Create convenience methods for getting ColumnDescriptors without using 
> builders
> ---
>
> Key: HBASE-18433
> URL: https://issues.apache.org/jira/browse/HBASE-18433
> Project: HBase
>  Issue Type: Bug
>  Components: Client
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
>Assignee: Mike Drob
> Attachments: HBASE-18433.patch
>
>
> The old {{HColumnDescriptor(String)}} is deprecated and going away. The most 
> straightforward replacement for it is very wordy: 
> {{ColumnFamilyDescriptorBuilder.newBuilder(byte \[]).build()}}
> We can provide a more inline replacement like 
> {{ColumnFamilyDescriptor.for(String)}} that will be useful if we do not need 
> to actually do any modifications on it, only pass it as an argument to 
> something (like table descriptor operations).
> We can do the same kind of improvement for TableDescriptor, but I do not 
> think the use case is as helpful because usually we will want to do something 
> to the modifiable TableDescriptor, and I have not seen many use cases where 
> it is passed as an argument unadorned.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18433) Create convenience methods for getting ColumnDescriptors without using builders

Mike Drob created HBASE-18433:
-

 Summary: Create convenience methods for getting ColumnDescriptors 
without using builders
 Key: HBASE-18433
 URL: https://issues.apache.org/jira/browse/HBASE-18433
 Project: HBase
  Issue Type: Bug
  Components: Client
Affects Versions: 2.0.0-alpha-1
Reporter: Mike Drob
Assignee: Mike Drob


The old {{HColumnDescriptor(String)}} is deprecated and going away. The most 
straightforward replacement for it is very wordy: 
{{ColumnFamilyDescriptorBuilder.newBuilder(byte \[]).build()}}

We can provide a more inline replacement like 
{{ColumnFamilyDescriptor.for(String)}} that will be useful if we do not need to 
actually do any modifications on it, only pass it as an argument to something 
(like table descriptor operations).

We can do the same kind of improvement for TableDescriptor, but I do not think 
the use case is as helpful because usually we will want to do something to the 
modifiable TableDescriptor, and I have not seen many use cases where it is 
passed as an argument unadorned.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18432) Prevent clock from getting stuck after update()


 [ 
https://issues.apache.org/jira/browse/HBASE-18432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Appy updated HBASE-18432:
-
Description: 
There were a [bunch of 
problems|https://issues.apache.org/jira/browse/HBASE-14070?focusedCommentId=16094013=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16094013]
 (also copied below) with clock getting stuck after call to update() until it's 
own system time caught up.

PT = physical time, LT = logical time, ST = system time, X = don't care terms

Core issue:
- Note that in current implementation, we are passing master clock to RS in 
open/close region request and RS clock to master in the responses. And they 
both update their own time on receiving these request/response.
- On receiving a clock ahead of its own, they update their own clock to its 
PT+LT, and keep increasing LT till their own ST catches that PT.

Proposed solution:
Keep track of skew in clock. And instead of keeping track of physical time, 
always compute it by adding system time and skew.
On update(), recalculate skew and validate if it's greater than max_skew.
On toTimestamp(), calculate PT = ST+skew.
-
-
Issues with current approach:

Problem 1: Logical time window too small.
RS clock (10, X)
Master clock (20, X)
Master --request-> RS
RS clock (20, X)
While RS's physical java clock (which is backing up physical component of hlc 
clock) will still take 10 sec to catch up, we'll keep incrementing logical 
component. That means, in worst case, our logical clock window should be big 
enough to support all the events that can happen in max skew time.
The problem is, that doesn't seem to be the case. Our logical window is 1M 
events (20bits) and max skew time is 30 sec, that results in 33k max write qps, 
which is quite low. We can easily see 150k update qps per beefy server with 1k 
values.
Even 22 bits won't be enough. We'll need minimum of 23 bits and 20 sec max skew 
time to support ~420k max events per second in worst case clock skew.

Problem 2: Cascading logical time increment.
When more RS are involved say - 3 RS and 1 master. Let's say max skew is 30 sec.
HLC Clocks (physical time, logical time): X = don't care
RS1: (50, 100k)
Master: (40, X)
RS2: (30, X)
RS3: (20, X) 
[RS3's ST behind RS1's by 30 sec.]
RS1 replies to master, sends it's clock (50,X).
Master's clock (50, X). It'll be another 10 sec before it's own physical clock 
reaches 50, so HLC's PT will remain 50 for next 10 sec.
Master --> RS2
RS2's clock = (50, X).
RS2 keeps incrementing LT on writes (since it's own PT is behind) for few 
seconds before it replies back to master with (50, X+ few 100k).
Master's clock = (50, X+ few 100k) [Since master's physical clock hasn't caught 
up yet, note that it was 10 seconds behind, PT remains 50.].
Master --> RS3
RS3's clock (50, X+few 100k) 
But RS3's ST is behind RS1's ST by 30 sec, which means it'll keep incrementing 
LT for next 30 sec (unless it gets a newer clock from master).
But the problem is, RS3 has much smaller LT window than actual 1M!!
—
Problem 3: Single bad RS clock crashing the cluster:
If a single RS's clock is bad and a bit faster, it'll catch time and keep 
pulling master's PT with it. If 'real time' is say 20, max skew time is 10, and 
bad RS is at time 29.9, it'll pull master to 29.9 (via next response), and then 
any RS less than 19.9, i.e. just 0.1 sec away from real time will die due to 
higher than max skew.
This can bring whole clusters down!
—
Problem 4: Time jumps (not a bug, but more of a nuisance)
Say a RS is behind master by 20 sec. On each communication from master, RS will 
update its own PT to master's PT, and it'll remain that till RS's ST catches 
up. If there are frequent communication from master, ST might never catch up 
and RS's PT will actually look like discrete time jumps rather than continuous 
time.
For eg. If master communicated with RS at times 30, 40, 50 (RSs corresponding 
times are 10, 20, 30), than all events on RS between time [10, 50] will be 
timestamped with either 30, 40 or 50.
—

  was:
There were a [bunch of 
problems|https://issues.apache.org/jira/browse/HBASE-14070?focusedCommentId=16094013=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16094013]
 (also copied below) with clock getting stuck after call to update() until it's 
own system time caught up.
Proposed solution is, keeping track of skew separately.
-
PT = physical time, LT = logical time, ST = system time, X = don't care terms
Note that in current implementation, we are passing master clock to RS in 
open/close region request and RS clock to master in the responses. And they 
both update their own time on receiving these request/response.
 
Also, on receiving a clock ahead of its own, they update their own clock to its 
PT+LT, and keep increasing LT till their own ST catches that PT.

Problem 1: Logical time window too small.
RS

[jira] [Updated] (HBASE-18432) Prevent clock from getting stuck after update()


 [ 
https://issues.apache.org/jira/browse/HBASE-18432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Appy updated HBASE-18432:
-
Description: 
There were a [bunch of 
problems|https://issues.apache.org/jira/browse/HBASE-14070?focusedCommentId=16094013=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16094013]
 (also copied below) with clock getting stuck after call to update() until it's 
own system time caught up.

PT = physical time, LT = logical time, ST = system time, X = don't care terms

Core issue:
- Note that in current implementation, we are passing master clock to RS in 
open/close region request and RS clock to master in the responses. And they 
both update their own time on receiving these request/response.
- On receiving a clock ahead of its own, they update their own clock to its 
PT+LT, and keep increasing LT till their own ST catches that PT.


Proposed solution:
Keep track of skew in clock. And instead of keeping track of physical time, 
always compute it by adding system time and skew.
On update(), recalculate skew and validate if it's greater than max_skew.
On toTimestamp(), calculate PT = ST+skew.
-
-
Issues with current approach:

Problem 1: Logical time window too small.
RS clock (10, X)
Master clock (20, X)
Master --request-> RS
RS clock (20, X)
While RS's physical java clock (which is backing up physical component of hlc 
clock) will still take 10 sec to catch up, we'll keep incrementing logical 
component. That means, in worst case, our logical clock window should be big 
enough to support all the events that can happen in max skew time.
The problem is, that doesn't seem to be the case. Our logical window is 1M 
events (20bits) and max skew time is 30 sec, that results in 33k max write qps, 
which is quite low. We can easily see 150k update qps per beefy server with 1k 
values.
Even 22 bits won't be enough. We'll need minimum of 23 bits and 20 sec max skew 
time to support ~420k max events per second in worst case clock skew.

Problem 2: Cascading logical time increment.
When more RS are involved say - 3 RS and 1 master. Let's say max skew is 30 sec.
HLC Clocks (physical time, logical time): X = don't care
RS1: (50, 100k)
Master: (40, X)
RS2: (30, X)
RS3: (20, X) 
[RS3's ST behind RS1's by 30 sec.]
RS1 replies to master, sends it's clock (50,X).
Master's clock (50, X). It'll be another 10 sec before it's own physical clock 
reaches 50, so HLC's PT will remain 50 for next 10 sec.
Master --> RS2
RS2's clock = (50, X).
RS2 keeps incrementing LT on writes (since it's own PT is behind) for few 
seconds before it replies back to master with (50, X+ few 100k).
Master's clock = (50, X+ few 100k) [Since master's physical clock hasn't caught 
up yet, note that it was 10 seconds behind, PT remains 50.].
Master --> RS3
RS3's clock (50, X+few 100k) 
But RS3's ST is behind RS1's ST by 30 sec, which means it'll keep incrementing 
LT for next 30 sec (unless it gets a newer clock from master).
But the problem is, RS3 has much smaller LT window than actual 1M!!
—
Problem 3: Single bad RS clock crashing the cluster:
If a single RS's clock is bad and a bit faster, it'll catch time and keep 
pulling master's PT with it. If 'real time' is say 20, max skew time is 10, and 
bad RS is at time 29.9, it'll pull master to 29.9 (via next response), and then 
any RS less than 19.9, i.e. just 0.1 sec away from real time will die due to 
higher than max skew.
This can bring whole clusters down!
—
Problem 4: Time jumps (not a bug, but more of a nuisance)
Say a RS is behind master by 20 sec. On each communication from master, RS will 
update its own PT to master's PT, and it'll remain that till RS's ST catches 
up. If there are frequent communication from master, ST might never catch up 
and RS's PT will actually look like discrete time jumps rather than continuous 
time.
For eg. If master communicated with RS at times 30, 40, 50 (RSs corresponding 
times are 10, 20, 30), than all events on RS between time [10, 50] will be 
timestamped with either 30, 40 or 50.
—

  was:
There were a [bunch of 
problems|https://issues.apache.org/jira/browse/HBASE-14070?focusedCommentId=16094013=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16094013]
 (also copied below) with clock getting stuck after call to update() until it's 
own system time caught up.

PT = physical time, LT = logical time, ST = system time, X = don't care terms

Core issue:
- Note that in current implementation, we are passing master clock to RS in 
open/close region request and RS clock to master in the responses. And they 
both update their own time on receiving these request/response.
- On receiving a clock ahead of its own, they update their own clock to its 
PT+LT, and keep increasing LT till their own ST catches that PT.

Proposed solution:
Keep track of skew in clock. And instead of keeping track of physical

[jira] [Updated] (HBASE-18432) Prevent clock from getting stuck after update()


 [ 
https://issues.apache.org/jira/browse/HBASE-18432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Appy updated HBASE-18432:
-
Attachment: HBASE-18432.HBASE-14070.HLC.001.patch

> Prevent clock from getting stuck after update()
> ---
>
> Key: HBASE-18432
> URL: https://issues.apache.org/jira/browse/HBASE-18432
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-18432.HBASE-14070.HLC.001.patch
>
>
> There were a [bunch of 
> problems|https://issues.apache.org/jira/browse/HBASE-14070?focusedCommentId=16094013=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16094013]
>  (also copied below) with clock getting stuck after call to update() until 
> it's own system time caught up.
> Proposed solution is, keeping track of skew separately.
> -
> PT = physical time, LT = logical time, ST = system time, X = don't care terms
> Note that in current implementation, we are passing master clock to RS in 
> open/close region request and RS clock to master in the responses. And they 
> both update their own time on receiving these request/response.
>  
> Also, on receiving a clock ahead of its own, they update their own clock to 
> its PT+LT, and keep increasing LT till their own ST catches that PT.
> 
> Problem 1: Logical time window too small.
> RS clock (10, X)
> Master clock (20, X)
> Master --request-> RS
> RS clock (20, X)
> While RS's physical java clock (which is backing up physical component of hlc 
> clock) will still take 10 sec to catch up, we'll keep incrementing logical 
> component. That means, in worst case, our logical clock window should be big 
> enough to support all the events that can happen in max skew time.
> The problem is, that doesn't seem to be the case. Our logical window is 1M 
> events (20bits) and max skew time is 30 sec, that results in 33k max write 
> qps, which is quite low. We can easily see 150k update qps per beefy server 
> with 1k values.
> Even 22 bits won't be enough. We'll need minimum of 23 bits and 20 sec max 
> skew time to support ~420k max events per second in worst case clock skew.
> 
> Problem 2: Cascading logical time increment.
> When more RS are involved say - 3 RS and 1 master. Let's say max skew is 30 
> sec.
> HLC Clocks (physical time, logical time): X = don't care
> RS1: (50, 100k)
> Master: (40, X)
> RS2: (30, X)
> RS3: (20, X) 
> [RS3's ST behind RS1's by 30 sec.]
> RS1 replies to master, sends it's clock (50,X).
> Master's clock (50, X). It'll be another 10 sec before it's own physical 
> clock reaches 50, so HLC's PT will remain 50 for next 10 sec.
> Master --> RS2
> RS2's clock = (50, X).
> RS2 keeps incrementing LT on writes (since it's own PT is behind) for few 
> seconds before it replies back to master with (50, X+ few 100k).
> Master's clock = (50, X+ few 100k) [Since master's physical clock hasn't 
> caught up yet, note that it was 10 seconds behind, PT remains 50.].
> Master --> RS3
> RS3's clock (50, X+few 100k) 
> But RS3's ST is behind RS1's ST by 30 sec, which means it'll keep 
> incrementing LT for next 30 sec (unless it gets a newer clock from master).
> But the problem is, RS3 has much smaller LT window than actual 1M!!
> —
> Problem 3: Single bad RS clock crashing the cluster:
> If a single RS's clock is bad and a bit faster, it'll catch time and keep 
> pulling master's PT with it. If 'real time' is say 20, max skew time is 10, 
> and bad RS is at time 29.9, it'll pull master to 29.9 (via next response), 
> and then any RS less than 19.9, i.e. just 0.1 sec away from real time will 
> die due to higher than max skew.
> This can bring whole clusters down!
> —
> Problem 4: Time jumps (not a bug, but more of a nuisance)
> Say a RS is behind master by 20 sec. On each communication from master, RS 
> will update its own PT to master's PT, and it'll remain that till RS's ST 
> catches up. If there are frequent communication from master, ST might never 
> catch up and RS's PT will actually look like discrete time jumps rather than 
> continuous time.
> For eg. If master communicated with RS at times 30, 40, 50 (RSs corresponding 
> times are 10, 20, 30), than all events on RS between time [10, 50] will be 
> timestamped with either 30, 40 or 50.
> —



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18432) Prevent clock from getting stuck after update()


 [ 
https://issues.apache.org/jira/browse/HBASE-18432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Appy updated HBASE-18432:
-
Status: Patch Available  (was: Open)

> Prevent clock from getting stuck after update()
> ---
>
> Key: HBASE-18432
> URL: https://issues.apache.org/jira/browse/HBASE-18432
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-18432.HBASE-14070.HLC.001.patch
>
>
> There were a [bunch of 
> problems|https://issues.apache.org/jira/browse/HBASE-14070?focusedCommentId=16094013=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16094013]
>  (also copied below) with clock getting stuck after call to update() until 
> it's own system time caught up.
> Proposed solution is, keeping track of skew separately.
> -
> PT = physical time, LT = logical time, ST = system time, X = don't care terms
> Note that in current implementation, we are passing master clock to RS in 
> open/close region request and RS clock to master in the responses. And they 
> both update their own time on receiving these request/response.
>  
> Also, on receiving a clock ahead of its own, they update their own clock to 
> its PT+LT, and keep increasing LT till their own ST catches that PT.
> 
> Problem 1: Logical time window too small.
> RS clock (10, X)
> Master clock (20, X)
> Master --request-> RS
> RS clock (20, X)
> While RS's physical java clock (which is backing up physical component of hlc 
> clock) will still take 10 sec to catch up, we'll keep incrementing logical 
> component. That means, in worst case, our logical clock window should be big 
> enough to support all the events that can happen in max skew time.
> The problem is, that doesn't seem to be the case. Our logical window is 1M 
> events (20bits) and max skew time is 30 sec, that results in 33k max write 
> qps, which is quite low. We can easily see 150k update qps per beefy server 
> with 1k values.
> Even 22 bits won't be enough. We'll need minimum of 23 bits and 20 sec max 
> skew time to support ~420k max events per second in worst case clock skew.
> 
> Problem 2: Cascading logical time increment.
> When more RS are involved say - 3 RS and 1 master. Let's say max skew is 30 
> sec.
> HLC Clocks (physical time, logical time): X = don't care
> RS1: (50, 100k)
> Master: (40, X)
> RS2: (30, X)
> RS3: (20, X) 
> [RS3's ST behind RS1's by 30 sec.]
> RS1 replies to master, sends it's clock (50,X).
> Master's clock (50, X). It'll be another 10 sec before it's own physical 
> clock reaches 50, so HLC's PT will remain 50 for next 10 sec.
> Master --> RS2
> RS2's clock = (50, X).
> RS2 keeps incrementing LT on writes (since it's own PT is behind) for few 
> seconds before it replies back to master with (50, X+ few 100k).
> Master's clock = (50, X+ few 100k) [Since master's physical clock hasn't 
> caught up yet, note that it was 10 seconds behind, PT remains 50.].
> Master --> RS3
> RS3's clock (50, X+few 100k) 
> But RS3's ST is behind RS1's ST by 30 sec, which means it'll keep 
> incrementing LT for next 30 sec (unless it gets a newer clock from master).
> But the problem is, RS3 has much smaller LT window than actual 1M!!
> —
> Problem 3: Single bad RS clock crashing the cluster:
> If a single RS's clock is bad and a bit faster, it'll catch time and keep 
> pulling master's PT with it. If 'real time' is say 20, max skew time is 10, 
> and bad RS is at time 29.9, it'll pull master to 29.9 (via next response), 
> and then any RS less than 19.9, i.e. just 0.1 sec away from real time will 
> die due to higher than max skew.
> This can bring whole clusters down!
> —
> Problem 4: Time jumps (not a bug, but more of a nuisance)
> Say a RS is behind master by 20 sec. On each communication from master, RS 
> will update its own PT to master's PT, and it'll remain that till RS's ST 
> catches up. If there are frequent communication from master, ST might never 
> catch up and RS's PT will actually look like discrete time jumps rather than 
> continuous time.
> For eg. If master communicated with RS at times 30, 40, 50 (RSs corresponding 
> times are 10, 20, 30), than all events on RS between time [10, 50] will be 
> timestamped with either 30, 40 or 50.
> —



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (HBASE-14070) Hybrid Logical Clocks for HBase


[ 
https://issues.apache.org/jira/browse/HBASE-14070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097089#comment-16097089
 ] 

Appy edited comment on HBASE-14070 at 7/22/17 3:19 AM:
---

I think a simple solution here is, keep track of skew in clock. And instead of 
keeping track of physical time, always compute it by ST + skew.
On update(), recalculate {{skew}} and validate if it's greater than max_skew.
On toTimestamp(), calculate PT = ST+skew.
The biggest advantage is, PT will keep moving forward. It'll fix most problems 
mentioned earlier which arise because PT gets stuck.
I'll make a patch to show what i mean.

Here's the jira which has the patch: 
https://issues.apache.org/jira/browse/HBASE-18432


was (Author: appy):
I think a simple solution here is, keep track of skew in clock. And instead of 
keeping track of physical time, always compute it by ST + skew.
On update(), recalculate {{skew}} and validate if it's greater than max_skew.
On toTimestamp(), calculate PT = ST+skew.
The biggest advantage is, PT will keep moving forward. It'll fix most problems 
mentioned earlier which arise because PT gets stuck.
I'll make a patch to show what i mean.

> Hybrid Logical Clocks for HBase
> ---
>
> Key: HBASE-14070
> URL: https://issues.apache.org/jira/browse/HBASE-14070
> Project: HBase
>  Issue Type: New Feature
>Reporter: Enis Soztutar
>Assignee: Amit Patel
> Attachments: HBASE-14070.master.001.patch, 
> HybridLogicalClocksforHBaseandPhoenix.docx, 
> HybridLogicalClocksforHBaseandPhoenix.pdf
>
>
> HBase and Phoenix uses systems physical clock (PT) to give timestamps to 
> events (read and writes). This works mostly when the system clock is strictly 
> monotonically increasing and there is no cross-dependency between servers 
> clocks. However we know that leap seconds, general clock skew and clock drift 
> are in fact real. 
> This jira proposes using Hybrid Logical Clocks (HLC) as an implementation of 
> hybrid physical clock + a logical clock. HLC is best of both worlds where it 
> keeps causality relationship similar to logical clocks, but still is 
> compatible with NTP based physical system clock. HLC can be represented in 
> 64bits. 
> A design document is attached and also can be found here: 
> https://docs.google.com/document/d/1LL2GAodiYi0waBz5ODGL4LDT4e_bXy8P9h6kWC05Bhw/edit#



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18432) Prevent clock from getting stuck after update()

Appy created HBASE-18432:


 Summary: Prevent clock from getting stuck after update()
 Key: HBASE-18432
 URL: https://issues.apache.org/jira/browse/HBASE-18432
 Project: HBase
  Issue Type: Sub-task
Reporter: Appy
Assignee: Appy


There were a [bunch of 
problems|https://issues.apache.org/jira/browse/HBASE-14070?focusedCommentId=16094013=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16094013]
 (also copied below) with clock getting stuck after call to update() until it's 
own system time caught up.
Proposed solution is, keeping track of skew separately.
-
PT = physical time, LT = logical time, ST = system time, X = don't care terms
Note that in current implementation, we are passing master clock to RS in 
open/close region request and RS clock to master in the responses. And they 
both update their own time on receiving these request/response.
 
Also, on receiving a clock ahead of its own, they update their own clock to its 
PT+LT, and keep increasing LT till their own ST catches that PT.

Problem 1: Logical time window too small.
RS clock (10, X)
Master clock (20, X)
Master --request-> RS
RS clock (20, X)
While RS's physical java clock (which is backing up physical component of hlc 
clock) will still take 10 sec to catch up, we'll keep incrementing logical 
component. That means, in worst case, our logical clock window should be big 
enough to support all the events that can happen in max skew time.
The problem is, that doesn't seem to be the case. Our logical window is 1M 
events (20bits) and max skew time is 30 sec, that results in 33k max write qps, 
which is quite low. We can easily see 150k update qps per beefy server with 1k 
values.
Even 22 bits won't be enough. We'll need minimum of 23 bits and 20 sec max skew 
time to support ~420k max events per second in worst case clock skew.

Problem 2: Cascading logical time increment.
When more RS are involved say - 3 RS and 1 master. Let's say max skew is 30 sec.
HLC Clocks (physical time, logical time): X = don't care
RS1: (50, 100k)
Master: (40, X)
RS2: (30, X)
RS3: (20, X) 
[RS3's ST behind RS1's by 30 sec.]
RS1 replies to master, sends it's clock (50,X).
Master's clock (50, X). It'll be another 10 sec before it's own physical clock 
reaches 50, so HLC's PT will remain 50 for next 10 sec.
Master --> RS2
RS2's clock = (50, X).
RS2 keeps incrementing LT on writes (since it's own PT is behind) for few 
seconds before it replies back to master with (50, X+ few 100k).
Master's clock = (50, X+ few 100k) [Since master's physical clock hasn't caught 
up yet, note that it was 10 seconds behind, PT remains 50.].
Master --> RS3
RS3's clock (50, X+few 100k) 
But RS3's ST is behind RS1's ST by 30 sec, which means it'll keep incrementing 
LT for next 30 sec (unless it gets a newer clock from master).
But the problem is, RS3 has much smaller LT window than actual 1M!!
—
Problem 3: Single bad RS clock crashing the cluster:
If a single RS's clock is bad and a bit faster, it'll catch time and keep 
pulling master's PT with it. If 'real time' is say 20, max skew time is 10, and 
bad RS is at time 29.9, it'll pull master to 29.9 (via next response), and then 
any RS less than 19.9, i.e. just 0.1 sec away from real time will die due to 
higher than max skew.
This can bring whole clusters down!
—
Problem 4: Time jumps (not a bug, but more of a nuisance)
Say a RS is behind master by 20 sec. On each communication from master, RS will 
update its own PT to master's PT, and it'll remain that till RS's ST catches 
up. If there are frequent communication from master, ST might never catch up 
and RS's PT will actually look like discrete time jumps rather than continuous 
time.
For eg. If master communicated with RS at times 30, 40, 50 (RSs corresponding 
times are 10, 20, 30), than all events on RS between time [10, 50] will be 
timestamped with either 30, 40 or 50.
—



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18430) Typo in "contributing to documentation" page


[ 
https://issues.apache.org/jira/browse/HBASE-18430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097092#comment-16097092
 ] 

Hudson commented on HBASE-18430:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3415 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/3415/])
HBASE-18430 fixed typo (misty: rev 70a357dc5cc74ae6a354c907959f644f563aeee4)
* (edit) src/main/asciidoc/_chapters/appendix_contributing_to_documentation.adoc


> Typo in "contributing to documentation" page
> 
>
> Key: HBASE-18430
> URL: https://issues.apache.org/jira/browse/HBASE-18430
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: Coral
>Assignee: Coral
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18430.patch
>
>
> Be sure to set the component to Documentation in addition any other involved 
> components.
> Missing "to." Should be: 
> Be sure to set the component to Documentation in addition *to* any other 
> involved components.
> http://hbase.apache.org/book.html#appendix_contributing_to_documentation



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-14070) Hybrid Logical Clocks for HBase


[ 
https://issues.apache.org/jira/browse/HBASE-14070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097089#comment-16097089
 ] 

Appy commented on HBASE-14070:
--

I think a simple solution here is, keep track of skew in clock. And instead of 
keeping track of physical time, always compute it by ST + skew.
On update(), recalculate {{skew}} and validate if it's greater than max_skew.
On toTimestamp(), calculate PT = ST+skew.
The biggest advantage is, PT will keep moving forward. It'll fix most problems 
mentioned earlier which arise because PT gets stuck.
I'll make a patch to show what i mean.

> Hybrid Logical Clocks for HBase
> ---
>
> Key: HBASE-14070
> URL: https://issues.apache.org/jira/browse/HBASE-14070
> Project: HBase
>  Issue Type: New Feature
>Reporter: Enis Soztutar
>Assignee: Amit Patel
> Attachments: HBASE-14070.master.001.patch, 
> HybridLogicalClocksforHBaseandPhoenix.docx, 
> HybridLogicalClocksforHBaseandPhoenix.pdf
>
>
> HBase and Phoenix uses systems physical clock (PT) to give timestamps to 
> events (read and writes). This works mostly when the system clock is strictly 
> monotonically increasing and there is no cross-dependency between servers 
> clocks. However we know that leap seconds, general clock skew and clock drift 
> are in fact real. 
> This jira proposes using Hybrid Logical Clocks (HLC) as an implementation of 
> hybrid physical clock + a logical clock. HLC is best of both worlds where it 
> keeps causality relationship similar to logical clocks, but still is 
> compatible with NTP based physical system clock. HLC can be represented in 
> 64bits. 
> A design document is attached and also can be found here: 
> https://docs.google.com/document/d/1LL2GAodiYi0waBz5ODGL4LDT4e_bXy8P9h6kWC05Bhw/edit#



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-15816) Provide client with ability to set priority on Operations


[ 
https://issues.apache.org/jira/browse/HBASE-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097086#comment-16097086
 ] 

Hudson commented on HBASE-15816:


FAILURE: Integrated in Jenkins build HBase-1.4 #816 (See 
[https://builds.apache.org/job/HBase-1.4/816/])
HBASE-15816 Provide client with ability to set priority on Operations 
(apurtell: rev 26247996d25dad38678fed2e2a1b8f0d383df082)
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Append.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/PayloadCarryingServerCallable.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Scan.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/IPCUtil.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RegionCoprocessorRpcChannel.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestRpcControllerFactory.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Mutation.java
* (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Get.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MultiServerCallable.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRpcController.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/TestHeapSize.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Action.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionServerCallable.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/OperationWithAttributes.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRpcControllerImpl.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MultiAction.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/Increment.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RowMutations.java


> Provide client with ability to set priority on Operations 
> --
>
> Key: HBASE-15816
> URL: https://issues.apache.org/jira/browse/HBASE-15816
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: churro morales
>Assignee: churro morales
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0
>
> Attachments: HBASE-15816.patch, HBASE-15816.v1.branch-1.patch, 
> HBASE-15816-v1.patch, HBASE-15816.v2.patch
>
>
> First round will just be to expose the ability to set priorities for client 
> operations.  For more background: 
> http://mail-archives.apache.org/mod_mbox/hbase-dev/201604.mbox/%3CCA+RK=_BG_o=q8HMptcP2WauAinmEsL+15f3YEJuz=qbpcya...@mail.gmail.com%3E
> Next step would be to remove AnnotationReadingPriorityFunction and have the 
> client send priorities explicitly.  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18370) Port HBASE-16209 to branch-1.3


[ 
https://issues.apache.org/jira/browse/HBASE-18370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097071#comment-16097071
 ] 

Hadoop QA commented on HBASE-18370:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
 1s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
31s{color} | {color:green} branch-1.3 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
34s{color} | {color:green} branch-1.3 passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
57s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
16s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
51s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
24s{color} | {color:green} branch-1.3 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
32s{color} | {color:green} branch-1.3 passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
30s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
32s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
15m 37s{color} | {color:green} The patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m  
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
24s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
33s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 80m 
17s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
21s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}110m  5s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:b3a2a00 |
| JIRA Issue | HBASE-18370 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12878436/HBASE-18370-branch-1.3.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux 509b77104a81 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |

[jira] [Commented] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop

2017-07-21 Thread Dima Spivak (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097064#comment-16097064
 ] 

Dima Spivak commented on HBASE-18020:
-

Sorry for disappearing, [~awleblang]; been swamped at work. If it's okay, I'll 
take a look at this first thing next week.

> Update API Compliance Checker to Incorporate Improvements Done in Hadoop
> 
>
> Key: HBASE-18020
> URL: https://issues.apache.org/jira/browse/HBASE-18020
> Project: HBase
>  Issue Type: Improvement
>  Components: API, community
>Reporter: Alex Leblang
>Assignee: Alex Leblang
> Fix For: 2.0.0
>
> Attachments: HBASE-18020.0.patch, HBASE-18020.branch-1.2.001.patch, 
> HBASE-18020.branch-1.2.002.patch, HBASE-18020.branch-1.2.003.patch, 
> HBASE-18020.branch-1.2.004.patch
>
>
> Recently the Hadoop community has made a number of improvements in their api 
> compliance checker based on feedback from the hbase and kudu community. We 
> should adopt these changes ourselves.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-10092) Move up on to log4j2


 [ 
https://issues.apache.org/jira/browse/HBASE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-10092:

Priority: Critical  (was: Major)

> Move up on to log4j2
> 
>
> Key: HBASE-10092
> URL: https://issues.apache.org/jira/browse/HBASE-10092
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Assignee: Alex Newman
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 10092.txt, 10092v2.txt, HBASE-10092.patch, 
> HBASE-10092-preview-v0.patch
>
>
> Allows logging with less friction.  See http://logging.apache.org/log4j/2.x/  
> This rather radical transition can be done w/ minor change given they have an 
> adapter for apache's logging, the one we use.  They also have and adapter for 
> slf4j so we likely can remove at least some of the 4 versions of this module 
> our dependencies make use of.
> I made a start in attached patch but am currently stuck in maven dependency 
> resolve hell courtesy of our slf4j.  Fixing will take some concentration and 
> a good net connection, an item I currently lack.  Other TODOs are that will 
> need to fix our little log level setting jsp page -- will likely have to undo 
> our use of hadoop's tool here -- and the config system changes a little.
> I will return to this project soon.  Will bring numbers.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-10092) Move up on to log4j2


[ 
https://issues.apache.org/jira/browse/HBASE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097039#comment-16097039
 ] 

Sean Busbey commented on HBASE-10092:
-

bq. We are working on supporting Log4j 1 log4j.properties configuration files. 
The plan is to have that ready in the next release of Log4j (2.7).

Any update here [~mikaelstaldal]?

> Move up on to log4j2
> 
>
> Key: HBASE-10092
> URL: https://issues.apache.org/jira/browse/HBASE-10092
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Assignee: Alex Newman
> Fix For: 2.0.0
>
> Attachments: 10092.txt, 10092v2.txt, HBASE-10092.patch, 
> HBASE-10092-preview-v0.patch
>
>
> Allows logging with less friction.  See http://logging.apache.org/log4j/2.x/  
> This rather radical transition can be done w/ minor change given they have an 
> adapter for apache's logging, the one we use.  They also have and adapter for 
> slf4j so we likely can remove at least some of the 4 versions of this module 
> our dependencies make use of.
> I made a start in attached patch but am currently stuck in maven dependency 
> resolve hell courtesy of our slf4j.  Fixing will take some concentration and 
> a good net connection, an item I currently lack.  Other TODOs are that will 
> need to fix our little log level setting jsp page -- will likely have to undo 
> our use of hadoop's tool here -- and the config system changes a little.
> I will return to this project soon.  Will bring numbers.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-15816) Provide client with ability to set priority on Operations


 [ 
https://issues.apache.org/jira/browse/HBASE-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-15816:
---
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 1.5.0
   1.4.0
   3.0.0
   2.0.0
   Status: Resolved  (was: Patch Available)

Pushed to branch-1.4, branch-1, branch-2, master

> Provide client with ability to set priority on Operations 
> --
>
> Key: HBASE-15816
> URL: https://issues.apache.org/jira/browse/HBASE-15816
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: churro morales
>Assignee: churro morales
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0
>
> Attachments: HBASE-15816.patch, HBASE-15816.v1.branch-1.patch, 
> HBASE-15816-v1.patch, HBASE-15816.v2.patch
>
>
> First round will just be to expose the ability to set priorities for client 
> operations.  For more background: 
> http://mail-archives.apache.org/mod_mbox/hbase-dev/201604.mbox/%3CCA+RK=_BG_o=q8HMptcP2WauAinmEsL+15f3YEJuz=qbpcya...@mail.gmail.com%3E
> Next step would be to remove AnnotationReadingPriorityFunction and have the 
> client send priorities explicitly.  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18370) Port HBASE-16209 to branch-1.3


 [ 
https://issues.apache.org/jira/browse/HBASE-18370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-18370:
---
Attachment: HBASE-18370-branch-1.3.patch

> Port HBASE-16209 to branch-1.3
> --
>
> Key: HBASE-18370
> URL: https://issues.apache.org/jira/browse/HBASE-18370
> Project: HBase
>  Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Fix For: 1.3.2
>
> Attachments: HBASE-18370-branch-1.3.patch
>
>
> Currently once a region goes into FAILED_OPEN state this requires operator 
> intervention. With some underlying causes, this is necessary. With others, 
> the master could eventually successfully deploy the region without humans in 
> the loop. The master should optionally attempt automatic resolution of 
> FAILED_OPEN states with a strategy of: delay, unassign, reassign. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18370) Port HBASE-16209 to branch-1.3


[ 
https://issues.apache.org/jira/browse/HBASE-18370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097030#comment-16097030
 ] 

Andrew Purtell commented on HBASE-18370:


Patch

> Port HBASE-16209 to branch-1.3
> --
>
> Key: HBASE-18370
> URL: https://issues.apache.org/jira/browse/HBASE-18370
> Project: HBase
>  Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Fix For: 1.3.2
>
> Attachments: HBASE-18370-branch-1.3.patch
>
>
> Currently once a region goes into FAILED_OPEN state this requires operator 
> intervention. With some underlying causes, this is necessary. With others, 
> the master could eventually successfully deploy the region without humans in 
> the loop. The master should optionally attempt automatic resolution of 
> FAILED_OPEN states with a strategy of: delay, unassign, reassign. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18370) Port HBASE-16209 to branch-1.3


 [ 
https://issues.apache.org/jira/browse/HBASE-18370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-18370:
---
Fix Version/s: 1.3.2
   Status: Patch Available  (was: Open)

> Port HBASE-16209 to branch-1.3
> --
>
> Key: HBASE-18370
> URL: https://issues.apache.org/jira/browse/HBASE-18370
> Project: HBase
>  Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Fix For: 1.3.2
>
> Attachments: HBASE-18370-branch-1.3.patch
>
>
> Currently once a region goes into FAILED_OPEN state this requires operator 
> intervention. With some underlying causes, this is necessary. With others, 
> the master could eventually successfully deploy the region without humans in 
> the loop. The master should optionally attempt automatic resolution of 
> FAILED_OPEN states with a strategy of: delay, unassign, reassign. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18419) Update IntegrationTestIngestWithMOB and Actions to use ColumnFamily builders for modification


[ 
https://issues.apache.org/jira/browse/HBASE-18419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097028#comment-16097028
 ] 

Chia-Ping Tsai commented on HBASE-18419:


bq. If you agree I'll file a new JIRA for it and provide a patch later.
+1 if the helper methods can simplify some works.

> Update IntegrationTestIngestWithMOB and Actions to use ColumnFamily builders 
> for modification
> -
>
> Key: HBASE-18419
> URL: https://issues.apache.org/jira/browse/HBASE-18419
> Project: HBase
>  Issue Type: Sub-task
>  Components: integration tests
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18419.patch, HBASE-18419.v2.patch, 
> HBASE-18419.v3.patch
>
>
> When attempting to run ITIWM against a cluster I get the following error:
> {noformat}
> java.lang.UnsupportedOperationException: HColumnDescriptor is read-only
>   at 
> org.apache.hadoop.hbase.client.ImmutableHColumnDescriptor.getDelegateeForModification(ImmutableHColumnDescriptor.java:44)
>   at 
> org.apache.hadoop.hbase.HColumnDescriptor.setMobEnabled(HColumnDescriptor.java:735)
>   at 
> org.apache.hadoop.hbase.IntegrationTestIngestWithMOB.initTable(IntegrationTestIngestWithMOB.java:122)
>   at 
> org.apache.hadoop.hbase.IntegrationTestIngest.setUpCluster(IntegrationTestIngest.java:92)
>   at 
> org.apache.hadoop.hbase.IntegrationTestBase.setUp(IntegrationTestBase.java:148)
>   at 
> org.apache.hadoop.hbase.IntegrationTestBase.doWork(IntegrationTestBase.java:131)
>   at 
> org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:154)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
>   at 
> org.apache.hadoop.hbase.IntegrationTestIngestWithMOB.main(IntegrationTestIngestWithMOB.java:153)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18086) Create native client which creates load on selected cluster


[ 
https://issues.apache.org/jira/browse/HBASE-18086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097024#comment-16097024
 ] 

Enis Soztutar commented on HBASE-18086:
---

bq. The tool does puts, appends, scans and gets in separate rounds.
Thanks. The logic is much more easier to follow now.  
bq. With shifted region (along with unshifted region), more than one region is 
involved for the multi-get requests.
But it is still 2 regions at a time. Can we please do what I was suggesting 
above. You can do something like this (assuming 10 threads): 
thread 0 reads: row_0, row_10, row_20, row_30
thread 1 reads: row_1, row_11, row_21, ... 
Other than this, looks good. 

> Create native client which creates load on selected cluster
> ---
>
> Key: HBASE-18086
> URL: https://issues.apache.org/jira/browse/HBASE-18086
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18086.v11.txt, 18086.v12.txt, 18086.v14.txt, 
> 18086.v17.txt, 18086.v18.txt, 18086.v1.txt, 18086.v3.txt, 18086.v4.txt, 
> 18086.v5.txt, 18086.v6.txt, 18086.v7.txt, 18086.v8.txt
>
>
> This task is to create a client which uses multiple threads to conduct Puts 
> followed by Gets against selected cluster.
> Default is to run the tool against local cluster.
> This would give us some idea on the characteristics of native client in terms 
> of handling high load.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18431) Mitigate compatibility concerns between branch-1.3 and branch-1.4


[ 
https://issues.apache.org/jira/browse/HBASE-18431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097022#comment-16097022
 ] 

Andrew Purtell commented on HBASE-18431:


Both the Admin and MasterObserver changes can be resolved by moving 
SnapshotDescription back into HBaseProtos.

> Mitigate compatibility concerns between branch-1.3 and branch-1.4
> -
>
> Key: HBASE-18431
> URL: https://issues.apache.org/jira/browse/HBASE-18431
> Project: HBase
>  Issue Type: Bug
>Reporter: Andrew Purtell
>Priority: Blocker
> Fix For: 1.4.0, 1.5.0
>
>
> There are compatibility concerns with branch-1.4. 
> {noformat}
> Library Name  HBase
> Version #11.3.1
> Version #21.4.0-SNAPSHOT
> Subject   Binary Compatibility
> Compatibility - 89.9%
> Added Methods - 305
> Removed Methods - 105
> Problems with Data Types
>   High - 23
>   Medium - 9
>   Low - 21
> {noformat}
> {noformat}
> Library Name  HBase
> Version #11.3.1
> Version #21.4.0-SNAPSHOT
> Subject   Source Compatibility
> Compatibility- 86.5%
> Added Methods - 305
> Removed Methods - 105
> Problems with Data Types
>   High - 88
>   Medium - 0
>   Low - 0
> Other Changes in Data Types- 25
> {noformat}
> This report includes HBASE-15816 which hasn't been committed yet. Otherwise 
> it's current.
> I'm not generally concerned with added methods. 
> The following methods have been added to Public/Evolving interface Table. 
> Pointing them out in case it merits review.
> \\
> * Abstract method Table.getReadRpcTimeout ( ) has been added to this 
> interface.   No effect.
> * Abstract method Table.getWriteRpcTimeout ( ) has been added to this 
> interface.  No effect.
> * Abstract method Table.setReadRpcTimeout ( int ) has been added to this 
> interface.   No effect.
> * Abstract method Table.setWriteRpcTimeout ( int ) has been added to this 
> interface.
> The Public/Evolving interface Admin has some signature changes equating to 
> removed methods. I don't think this is allowed in a minor release.
> \\
> * Abstract method Admin.isSnapshotFinished ( HBaseProtos.SnapshotDescription 
> ) has been removed from Admin.
> *  Abstract method Admin.snapshot ( String, TableName, 
> HBaseProtos.SnapshotDescription.Type ) has been removed from Admin.
> * Abstract method Admin.snapshot ( HBaseProtos.SnapshotDescription ) has been 
> removed from Admin.
> *  Abstract method Admin.takeSnapshotAsync ( HBaseProtos.SnapshotDescription 
> ) has been removed from Admin.
> The LimitedPrivate(CONFIG) interface AsyncRpcClient has been removed. This 
> change is debatable but I think we can allow it.
> \\
> * AsyncRpcClient has been removed
> The Public/Evolving class FastLongHistogram has been removed. I don't believe 
> this change is allowed in a minor release.
> \\
> * FastLongHistogram has been removed
> Method signatures in LimitedPrivate(COPROC) interfaces MasterObserver and 
> RegionObserver have changed, equating to removed methods. The first set of 
> changes is due to move of SnapshotDescription from HBaseProtos to 
> SnapshotProtos:
> \\
> * Abstract method MasterObserver.postCloneSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> * Abstract method MasterObserver.postDeleteSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.postListSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.postRestoreSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> * Abstract method MasterObserver.postSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> * Abstract method MasterObserver.preCloneSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> * Abstract method MasterObserver.preDeleteSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.preListSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.preRestoreSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> * Abstract method MasterObserver.preSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> Here, maybe DeleteTracker moved packages?
> \\
> * Abstract method

[jira] [Commented] (HBASE-18389) Remove byte[] from formal parameter of sizeOf() of ClassSize, ClassSize.MemoryLayout and ClassSize.UnsafeLayout


[ 
https://issues.apache.org/jira/browse/HBASE-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097015#comment-16097015
 ] 

Hudson commented on HBASE-18389:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3414 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/3414/])
HBASE-18389 Remove byte[] from formal parameter of sizeOf() of (chia7712: rev 
31c3edaa29776b63cc69ecd37306ad10836992eb)
* (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java
* (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java
* (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/util/ClassSize.java


> Remove byte[] from formal parameter of sizeOf() of ClassSize, 
> ClassSize.MemoryLayout and ClassSize.UnsafeLayout
> ---
>
> Key: HBASE-18389
> URL: https://issues.apache.org/jira/browse/HBASE-18389
> Project: HBase
>  Issue Type: Improvement
>  Components: util
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18389.master.000.patch, 
> HBASE-18389.master.001.patch, HBASE-18389.master.002.patch, 
> HBASE-18389.master.003.patch
>
>
> In ClassSize class and its internal static class, sizeOf() function has 2 
> formal parameters: byte[] b and int len. But the function's internal logic 
> does not use or refer to byte[] b. Could be removed.
> {code:title=hbase-common/src/main/java/org/apache/hadoop/hbase/util/ClassSize.java|borderStyle=solid}
> // Class of ClassSize
> public static long sizeOf(byte[] b, int len) {
>   return memoryLayout.sizeOf(b, len);
> }
> // Class of ClassSize.MemoryLayout
> long sizeOf(byte[] b, int len) {
>   return align(arrayHeaderSize() + len);
> }
> // Class of ClassSize.UnsafeLayout
> long sizeOf(byte[] b, int len) {
>   return align(arrayHeaderSize() + len * 
> UnsafeAccess.theUnsafe.ARRAY_BYTE_INDEX_SCALE);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18404) Small typo on ACID documentation page


[ 
https://issues.apache.org/jira/browse/HBASE-18404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097014#comment-16097014
 ] 

Hudson commented on HBASE-18404:


FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3414 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/3414/])
HBASE-18404 fixed typo in acid semantics (busbey: rev 
2a0d18928e372b3e976c9c89457390b7afc0aafc)
* (edit) src/main/site/xdoc/acid-semantics.xml


> Small typo on ACID documentation page
> -
>
> Key: HBASE-18404
> URL: https://issues.apache.org/jira/browse/HBASE-18404
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Affects Versions: 1.3.1
>Reporter: Michael Crutcher
>Assignee: Coral
>Priority: Trivial
> Fix For: 3.0.0
>
> Attachments: HBASE-18404.patch
>
>
> I noticed a couple of occurrences of the "word" wholely on the ACID semantics 
> doc page (https://hbase.apache.org/acid-semantics.html)
> This should be "wholly".



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18431) Mitigate compatibility concerns between branch-1.3 and branch-1.4

[
https://issues.apache.org/jira/browse/HBASE-18431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrew Purtell updated HBASE-18431:
---
Description:
There are compatibility concerns with branch-1.4.

{noformat}
Library NameHBase
Version #1 1.3.1
Version #2 1.4.0-SNAPSHOT
Subject Binary Compatibility

Compatibility - 89.9%

Added Methods - 305
Removed Methods - 105
Problems with Data Types
High - 23
Medium - 9
Low - 21
{noformat}

{noformat}
Library NameHBase
Version #1 1.3.1
Version #2 1.4.0-SNAPSHOT
Subject Source Compatibility

Compatibility- 86.5%

Added Methods - 305
Removed Methods - 105
Problems with Data Types
High - 88
Medium - 0
Low - 0
Other Changes in Data Types - 25
{noformat}

This report includes HBASE-15816 which hasn't been committed yet. Otherwise
it's current.

I'm not generally concerned with added methods.

The following methods have been added to Public/Evolving interface Table.
Pointing them out in case it merits review.

\\
* Abstract method Table.getReadRpcTimeout ( ) has been added to this interface.
No effect.
* Abstract method Table.getWriteRpcTimeout ( ) has been added to this
interface.No effect.
* Abstract method Table.setReadRpcTimeout ( int ) has been added to this
interface. No effect.
* Abstract method Table.setWriteRpcTimeout ( int ) has been added to this
interface.

The Public/Evolving interface Admin has some signature changes equating to
removed methods. I don't think this is allowed in a minor release.

\\
* Abstract method Admin.isSnapshotFinished ( HBaseProtos.SnapshotDescription )
has been removed from Admin.
* Abstract method Admin.snapshot ( String, TableName,
HBaseProtos.SnapshotDescription.Type ) has been removed from Admin.
* Abstract method Admin.snapshot ( HBaseProtos.SnapshotDescription ) has been
removed from Admin.
* Abstract method Admin.takeSnapshotAsync ( HBaseProtos.SnapshotDescription )
has been removed from Admin.

The LimitedPrivate(CONFIG) interface AsyncRpcClient has been removed. This
change is debatable but I think we can allow it.

\\
* AsyncRpcClient has been removed

The Public/Evolving class FastLongHistogram has been removed. I don't believe
this change is allowed in a minor release.

\\
* FastLongHistogram has been removed

Method signatures in LimitedPrivate(COPROC) interfaces MasterObserver and
RegionObserver have changed, equating to removed methods. The first set of
changes is due to move of SnapshotDescription from HBaseProtos to
SnapshotProtos:

\\
* Abstract method MasterObserver.postCloneSnapshot (
ObserverContext, HBaseProtos.SnapshotDescription,
HTableDescriptor ) has been removed from MasterObserver.
* Abstract method MasterObserver.postDeleteSnapshot (
ObserverContext, HBaseProtos.SnapshotDescription
) has been removed from MasterObserver.
* Abstract method MasterObserver.postListSnapshot (
ObserverContext, HBaseProtos.SnapshotDescription
) has been removed from MasterObserver.
* Abstract method MasterObserver.postRestoreSnapshot (
ObserverContext, HBaseProtos.SnapshotDescription,
HTableDescriptor ) has been removed from MasterObserver.
* Abstract method MasterObserver.postSnapshot (
ObserverContext, HBaseProtos.SnapshotDescription,
HTableDescriptor ) has been removed from MasterObserver.
* Abstract method MasterObserver.preCloneSnapshot (
ObserverContext, HBaseProtos.SnapshotDescription,
HTableDescriptor ) has been removed from MasterObserver.
* Abstract method MasterObserver.preDeleteSnapshot (
ObserverContext, HBaseProtos.SnapshotDescription
) has been removed from MasterObserver.
* Abstract method MasterObserver.preListSnapshot (
ObserverContext, HBaseProtos.SnapshotDescription
) has been removed from MasterObserver.
* Abstract method MasterObserver.preRestoreSnapshot (
ObserverContext, HBaseProtos.SnapshotDescription,
HTableDescriptor ) has been removed from MasterObserver.
* Abstract method MasterObserver.preSnapshot (
ObserverContext, HBaseProtos.SnapshotDescription,
HTableDescriptor ) has been removed from MasterObserver.

Here, maybe DeleteTracker moved packages?

\\
* Abstract method RegionObserver.postInstantiateDeleteTracker (
ObserverContext, DeleteTracker ) has been removed
from RegionObserver.

The LimitedPrivate(COPROC) interface Store has method signature changes
equating to removed methods. The changes are debatable. I am thinking we can
allow them. Anyone implementing their own Stores?

\\
* Abstract method Store.bulkLoadHFile ( String, long ) has been removed from
Store.
* Abstract method Store.getScanners ( List, boolean, boolean,
boolean, boolean, ScanQueryMatcher, byte[ ], byte[ ], long, boolean ) has been
removed from Store.
* Abstract method Store.getScanners ( boolean, boolean, boolean, boolean,
ScanQueryMatcher, byte[ ], byte[ ], long ) has been removed from Store.
* Abstract method

[jira] [Commented] (HBASE-15816) Provide client with ability to set priority on Operations


[ 
https://issues.apache.org/jira/browse/HBASE-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097003#comment-16097003
 ] 

Andrew Purtell commented on HBASE-15816:


I checked compatibility of this change. This change is fine but I found other 
issues tracked with HBASE-18431. Going to commit shortly.

> Provide client with ability to set priority on Operations 
> --
>
> Key: HBASE-15816
> URL: https://issues.apache.org/jira/browse/HBASE-15816
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: churro morales
>Assignee: churro morales
> Attachments: HBASE-15816.patch, HBASE-15816.v1.branch-1.patch, 
> HBASE-15816-v1.patch, HBASE-15816.v2.patch
>
>
> First round will just be to expose the ability to set priorities for client 
> operations.  For more background: 
> http://mail-archives.apache.org/mod_mbox/hbase-dev/201604.mbox/%3CCA+RK=_BG_o=q8HMptcP2WauAinmEsL+15f3YEJuz=qbpcya...@mail.gmail.com%3E
> Next step would be to remove AnnotationReadingPriorityFunction and have the 
> client send priorities explicitly.  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18431) Mitigate compatibility concerns between branch-1.3 and branch-1.4


 [ 
https://issues.apache.org/jira/browse/HBASE-18431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-18431:
---
Priority: Blocker  (was: Major)

> Mitigate compatibility concerns between branch-1.3 and branch-1.4
> -
>
> Key: HBASE-18431
> URL: https://issues.apache.org/jira/browse/HBASE-18431
> Project: HBase
>  Issue Type: Bug
>Reporter: Andrew Purtell
>Priority: Blocker
> Fix For: 1.4.0, 1.5.0
>
>
> There are compatibility concerns with branch-1.4. 
> {noformat}
> Library Name  HBase
> Version #11.3.1
> Version #21.4.0-SNAPSHOT
> Subject   Binary Compatibility
> Compatibility - 89.9%
> Added Methods - 305
> Removed Methods - 105
> Problems with Data Types
>   High - 23
>   Medium - 9
>   Low - 21
> {noformat}
> {noformat}
> Library Name  HBase
> Version #11.3.1
> Version #21.4.0-SNAPSHOT
> Subject   Source Compatibility
> Compatibility- 86.5%
> Added Methods - 305
> Removed Methods - 105
> Problems with Data Types
>   High - 88
>   Medium - 0
>   Low - 0
> Other Changes in Data Types- 25
> {noformat}
> This report includes HBASE-15816 which hasn't been committed yet. Otherwise 
> it's current.
> I'm not generally concerned with added methods. 
> The Public/Evolving interface Admin has some signature changes equating to 
> removed methods. I don't think this is allowed in a minor release.
> \\
> * Abstract method Admin.isSnapshotFinished ( HBaseProtos.SnapshotDescription 
> ) has been removed from Admin.
> *  Abstract method Admin.snapshot ( String, TableName, 
> HBaseProtos.SnapshotDescription.Type ) has been removed from Admin.
> * Abstract method Admin.snapshot ( HBaseProtos.SnapshotDescription ) has been 
> removed from Admin.
> *  Abstract method Admin.takeSnapshotAsync ( HBaseProtos.SnapshotDescription 
> ) has been removed from Admin.
> The LimitedPrivate(CONFIG) interface AsyncRpcClient has been removed. This 
> change is debatable but I think we can allow it.
> \\
> * AsyncRpcClient has been removed
> The Public/Evolving class FastLongHistogram has been removed. I don't believe 
> this change is allowed in a minor release.
> \\
> * FastLongHistogram has been removed
> Method signatures in LimitedPrivate(COPROC) interfaces MasterObserver and 
> RegionObserver have changed, equating to removed methods. These changes 
> should be mitigated with compatibility shims:
> \\
> * Abstract method MasterObserver.postCloneSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> * Abstract method MasterObserver.postDeleteSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.postListSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.postRestoreSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> * Abstract method MasterObserver.postSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> * Abstract method MasterObserver.preCloneSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> * Abstract method MasterObserver.preDeleteSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.preListSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.preRestoreSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> * Abstract method MasterObserver.preSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> * Abstract method RegionObserver.postInstantiateDeleteTracker ( 
> ObserverContext, DeleteTracker ) has been 
> removed from RegionObserver.
> The LimitedPrivate(COPROC) interface Store has method signature changes 
> equating to removed methods. The changes are debatable. I am thinking we can 
> allow them. Anyone implementing their own Stores?
> \\
> * Abstract method Store.bulkLoadHFile ( String, long ) has been removed from 
> Store.
> * Abstract method Store.getScanners ( List, boolean, boolean, 
> boolean, boolean, ScanQueryMatcher, byte[ ], byte[ ], long, boolean ) has 
> been removed from Store.
> * Abstract method Store.getScanners ( boolean, boolean, boolean, boolean, 
> ScanQueryMatcher, byte[ ], byte[ ], long ) has been removed from Store.
> *

[jira] [Updated] (HBASE-18431) Mitigate compatibility concerns between branch-1.3 and branch-1.4


 [ 
https://issues.apache.org/jira/browse/HBASE-18431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-18431:
---
Fix Version/s: 1.5.0

> Mitigate compatibility concerns between branch-1.3 and branch-1.4
> -
>
> Key: HBASE-18431
> URL: https://issues.apache.org/jira/browse/HBASE-18431
> Project: HBase
>  Issue Type: Bug
>Reporter: Andrew Purtell
>Priority: Blocker
> Fix For: 1.4.0, 1.5.0
>
>
> There are compatibility concerns with branch-1.4. 
> {noformat}
> Library Name  HBase
> Version #11.3.1
> Version #21.4.0-SNAPSHOT
> Subject   Binary Compatibility
> Compatibility - 89.9%
> Added Methods - 305
> Removed Methods - 105
> Problems with Data Types
>   High - 23
>   Medium - 9
>   Low - 21
> {noformat}
> {noformat}
> Library Name  HBase
> Version #11.3.1
> Version #21.4.0-SNAPSHOT
> Subject   Source Compatibility
> Compatibility- 86.5%
> Added Methods - 305
> Removed Methods - 105
> Problems with Data Types
>   High - 88
>   Medium - 0
>   Low - 0
> Other Changes in Data Types- 25
> {noformat}
> This report includes HBASE-15816 which hasn't been committed yet. Otherwise 
> it's current.
> I'm not generally concerned with added methods. 
> The Public/Evolving interface Admin has some signature changes equating to 
> removed methods. I don't think this is allowed in a minor release.
> \\
> * Abstract method Admin.isSnapshotFinished ( HBaseProtos.SnapshotDescription 
> ) has been removed from Admin.
> *  Abstract method Admin.snapshot ( String, TableName, 
> HBaseProtos.SnapshotDescription.Type ) has been removed from Admin.
> * Abstract method Admin.snapshot ( HBaseProtos.SnapshotDescription ) has been 
> removed from Admin.
> *  Abstract method Admin.takeSnapshotAsync ( HBaseProtos.SnapshotDescription 
> ) has been removed from Admin.
> The LimitedPrivate(CONFIG) interface AsyncRpcClient has been removed. This 
> change is debatable but I think we can allow it.
> \\
> * AsyncRpcClient has been removed
> The Public/Evolving class FastLongHistogram has been removed. I don't believe 
> this change is allowed in a minor release.
> \\
> * FastLongHistogram has been removed
> Method signatures in LimitedPrivate(COPROC) interfaces MasterObserver and 
> RegionObserver have changed, equating to removed methods. These changes 
> should be mitigated with compatibility shims:
> \\
> * Abstract method MasterObserver.postCloneSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> * Abstract method MasterObserver.postDeleteSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.postListSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.postRestoreSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> * Abstract method MasterObserver.postSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> * Abstract method MasterObserver.preCloneSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> * Abstract method MasterObserver.preDeleteSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.preListSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription ) has been removed from MasterObserver.
> * Abstract method MasterObserver.preRestoreSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> * Abstract method MasterObserver.preSnapshot ( 
> ObserverContext, 
> HBaseProtos.SnapshotDescription, HTableDescriptor ) has been removed from 
> MasterObserver.
> * Abstract method RegionObserver.postInstantiateDeleteTracker ( 
> ObserverContext, DeleteTracker ) has been 
> removed from RegionObserver.
> The LimitedPrivate(COPROC) interface Store has method signature changes 
> equating to removed methods. The changes are debatable. I am thinking we can 
> allow them. Anyone implementing their own Stores?
> \\
> * Abstract method Store.bulkLoadHFile ( String, long ) has been removed from 
> Store.
> * Abstract method Store.getScanners ( List, boolean, boolean, 
> boolean, boolean, ScanQueryMatcher, byte[ ], byte[ ], long, boolean ) has 
> been removed from Store.
> * Abstract method Store.getScanners ( boolean, boolean, boolean, boolean, 
> ScanQueryMatcher, byte[ ], byte[ ], long ) has been removed from Store.
> * Abstract

[jira] [Updated] (HBASE-18407) [C++] make Configuration::Set/GetBool work for both true/false and 1/0


 [ 
https://issues.apache.org/jira/browse/HBASE-18407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Enis Soztutar updated HBASE-18407:
--
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: HBASE-14850
   Status: Resolved  (was: Patch Available)

I've committed this. Thanks [~xiaobingo]. 

> [C++] make Configuration::Set/GetBool work for both true/false and 1/0
> --
>
> Key: HBASE-18407
> URL: https://issues.apache.org/jira/browse/HBASE-18407
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
> Fix For: HBASE-14850
>
> Attachments: HBASE-18407.000.patch, HBASE-18407-HBASE-14850.000.patch
>
>
> Configuration::SetBool internally converts true/false to 1/0 using 
> boost::lexical_cast(value), this results in runtime exception 
> with 'Unexpected value found while conversion to bool.' in 
> Configuration::GetBool since it only recognizes "true" or "false" as string 
> representation of true or false. 1/0 should be considered in the check in 
> GetBool.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18431) Mitigate compatibility concerns between branch-1.3 and branch-1.4

Andrew Purtell created HBASE-18431:
--

 Summary: Mitigate compatibility concerns between branch-1.3 and 
branch-1.4
 Key: HBASE-18431
 URL: https://issues.apache.org/jira/browse/HBASE-18431
 Project: HBase
  Issue Type: Bug
Reporter: Andrew Purtell
 Fix For: 1.4.0


There are compatibility concerns with branch-1.4. 

{noformat}
Library NameHBase
Version #1  1.3.1
Version #2  1.4.0-SNAPSHOT
Subject Binary Compatibility

Compatibility - 89.9%

Added Methods - 305
Removed Methods - 105
Problems with Data Types
High - 23
Medium - 9
Low - 21
{noformat}

{noformat}
Library NameHBase
Version #1  1.3.1
Version #2  1.4.0-SNAPSHOT
Subject Source Compatibility

Compatibility- 86.5%

Added Methods - 305
Removed Methods - 105
Problems with Data Types
High - 88
Medium - 0
Low - 0
Other Changes in Data Types  - 25
{noformat}

This report includes HBASE-15816 which hasn't been committed yet. Otherwise 
it's current.

I'm not generally concerned with added methods. 

The Public/Evolving interface Admin has some signature changes equating to 
removed methods. I don't think this is allowed in a minor release.

\\
* Abstract method Admin.isSnapshotFinished ( HBaseProtos.SnapshotDescription ) 
has been removed from Admin.
*  Abstract method Admin.snapshot ( String, TableName, 
HBaseProtos.SnapshotDescription.Type ) has been removed from Admin.
* Abstract method Admin.snapshot ( HBaseProtos.SnapshotDescription ) has been 
removed from Admin.
*  Abstract method Admin.takeSnapshotAsync ( HBaseProtos.SnapshotDescription ) 
has been removed from Admin.


The LimitedPrivate(CONFIG) interface AsyncRpcClient has been removed. This 
change is debatable but I think we can allow it.

\\
* AsyncRpcClient has been removed


The Public/Evolving class FastLongHistogram has been removed. I don't believe 
this change is allowed in a minor release.

\\
* FastLongHistogram has been removed


Method signatures in LimitedPrivate(COPROC) interfaces MasterObserver and 
RegionObserver have changed, equating to removed methods. These changes should 
be mitigated with compatibility shims:

\\
* Abstract method MasterObserver.postCloneSnapshot ( 
ObserverContext, HBaseProtos.SnapshotDescription, 
HTableDescriptor ) has been removed from MasterObserver.
* Abstract method MasterObserver.postDeleteSnapshot ( 
ObserverContext, HBaseProtos.SnapshotDescription 
) has been removed from MasterObserver.
* Abstract method MasterObserver.postListSnapshot ( 
ObserverContext, HBaseProtos.SnapshotDescription 
) has been removed from MasterObserver.
* Abstract method MasterObserver.postRestoreSnapshot ( 
ObserverContext, HBaseProtos.SnapshotDescription, 
HTableDescriptor ) has been removed from MasterObserver.
* Abstract method MasterObserver.postSnapshot ( 
ObserverContext, HBaseProtos.SnapshotDescription, 
HTableDescriptor ) has been removed from MasterObserver.
* Abstract method MasterObserver.preCloneSnapshot ( 
ObserverContext, HBaseProtos.SnapshotDescription, 
HTableDescriptor ) has been removed from MasterObserver.
* Abstract method MasterObserver.preDeleteSnapshot ( 
ObserverContext, HBaseProtos.SnapshotDescription 
) has been removed from MasterObserver.
* Abstract method MasterObserver.preListSnapshot ( 
ObserverContext, HBaseProtos.SnapshotDescription 
) has been removed from MasterObserver.
* Abstract method MasterObserver.preRestoreSnapshot ( 
ObserverContext, HBaseProtos.SnapshotDescription, 
HTableDescriptor ) has been removed from MasterObserver.
* Abstract method MasterObserver.preSnapshot ( 
ObserverContext, HBaseProtos.SnapshotDescription, 
HTableDescriptor ) has been removed from MasterObserver.
* Abstract method RegionObserver.postInstantiateDeleteTracker ( 
ObserverContext, DeleteTracker ) has been removed 
from RegionObserver.


The LimitedPrivate(COPROC) interface Store has method signature changes 
equating to removed methods. The changes are debatable. I am thinking we can 
allow them. Anyone implementing their own Stores?

\\
* Abstract method Store.bulkLoadHFile ( String, long ) has been removed from 
Store.
* Abstract method Store.getScanners ( List, boolean, boolean, 
boolean, boolean, ScanQueryMatcher, byte[ ], byte[ ], long, boolean ) has been 
removed from Store.
* Abstract method Store.getScanners ( boolean, boolean, boolean, boolean, 
ScanQueryMatcher, byte[ ], byte[ ], long ) has been removed from Store.
* Abstract method Store.upsert ( Iterable, long ) has been removed from 
Store.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (HBASE-18352) Enable Replica tests that were disabled by Proc-V2 AM in HBASE-14614

2017-07-21 Thread Vladimir Rodionov (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov reassigned HBASE-18352:
-

Assignee: Vladimir Rodionov

> Enable Replica tests that were disabled by Proc-V2 AM in HBASE-14614
> 
>
> Key: HBASE-18352
> URL: https://issues.apache.org/jira/browse/HBASE-18352
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.0.0-alpha-1
>Reporter: Stephen Yuan Jiang
>Assignee: Vladimir Rodionov
>
> The following replica tests were disabled by Core Proc-V2 AM in HBASE-14614:
> - Disabled parts of...testCreateTableWithMultipleReplicas in 
> TestMasterOperationsForRegionReplicas There is an issue w/ assigning more 
> replicas if number of replicas is changed on us. See '/* DISABLED! FOR 
> NOW'.
> - Disabled testRegionReplicasOnMidClusterHighReplication in 
> TestStochasticLoadBalancer2
> - Disabled testFlushAndCompactionsInPrimary in TestRegionReplicas
> This JIRA tracks the work to enable them (or modify/remove if not applicable).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18338) [C++] Implement RpcTestServer


 [ 
https://issues.apache.org/jira/browse/HBASE-18338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Enis Soztutar updated HBASE-18338:
--
   Resolution: Fixed
Fix Version/s: HBASE-14850
   Status: Resolved  (was: Patch Available)

Committed this. Thanks [~xiaobingo] for the patch. 

> [C++] Implement RpcTestServer
> -
>
> Key: HBASE-18338
> URL: https://issues.apache.org/jira/browse/HBASE-18338
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Xiaobing Zhou
>Assignee: Xiaobing Zhou
> Fix For: HBASE-14850
>
> Attachments: HBASE-18338.000.patch, HBASE-18338.001.patch, 
> HBASE-18338.002.patch, HBASE-18338.003.patch, HBASE-18338.004.patch, 
> HBASE-18338.005.patch, HBASE-18338.006.patch
>
>
> This is a spin-off from HBASE-18078. We need RpcTestServer to simulate 
> various communication scenarios, e.g. timeout, connection aborted, long 
> running services and so on.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18371) [C++] Update folly and wangle dependencies


[ 
https://issues.apache.org/jira/browse/HBASE-18371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096967#comment-16096967
 ] 

Enis Soztutar commented on HBASE-18371:
---

I think buck does not re-compile if the dependency libraries change. That's why 
you should remove the buck generated files first I think. Can you please do 
something like 
{code}
rm -rf buck-out/
{code}


> [C++] Update folly and wangle dependencies
> --
>
> Key: HBASE-18371
> URL: https://issues.apache.org/jira/browse/HBASE-18371
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: HBASE-14850
>
> Attachments: hbase-18371_v1.patch, hbase-18371_v2.patch, 
> hbase-18371_v3.patch
>
>
> We need to update folly and wangle dependency versions. Debugging an issue, I 
> realized that we may need a couple of recent patches from wangle. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18025) CatalogJanitor should collect outdated RegionStates from the AM


[ 
https://issues.apache.org/jira/browse/HBASE-18025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096968#comment-16096968
 ] 

Hadoop QA commented on HBASE-18025:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
12s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
36m 36s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  3m 
54s{color} | {color:red} hbase-server generated 2 new + 0 unchanged - 0 fixed = 
2 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}132m 
37s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
20s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}188m 16s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
|  |  Null passed for non-null parameter of 
ServerManager.removeRegion(HRegionInfo) in 
org.apache.hadoop.hbase.master.CatalogJanitor.scan()  Method invoked at 
CatalogJanitor.java:of ServerManager.removeRegion(HRegionInfo) in 
org.apache.hadoop.hbase.master.CatalogJanitor.scan()  Method invoked at 
CatalogJanitor.java:[line 268] |
|  |  Null passed for non-null parameter of 
ServerManager.removeRegion(HRegionInfo) in 
org.apache.hadoop.hbase.master.CatalogJanitor.scan()  Method invoked at 
CatalogJanitor.java:of ServerManager.removeRegion(HRegionInfo) in 
org.apache.hadoop.hbase.master.CatalogJanitor.scan()  Method invoked at 
CatalogJanitor.java:[line 267] |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18025 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12878406/HBASE-18025.001.patch 
|
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux feea8806ed30 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |

[jira] [Commented] (HBASE-18389) Remove byte[] from formal parameter of sizeOf() of ClassSize, ClassSize.MemoryLayout and ClassSize.UnsafeLayout


[ 
https://issues.apache.org/jira/browse/HBASE-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096966#comment-16096966
 ] 

Hudson commented on HBASE-18389:


FAILURE: Integrated in Jenkins build HBase-2.0 #213 (See 
[https://builds.apache.org/job/HBase-2.0/213/])
HBASE-18389 Remove byte[] from formal parameter of sizeOf() of (chia7712: rev 
946289113a1a867a6d15ba2c147b2f05b28fc806)
* (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java
* (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/util/ClassSize.java
* (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java


> Remove byte[] from formal parameter of sizeOf() of ClassSize, 
> ClassSize.MemoryLayout and ClassSize.UnsafeLayout
> ---
>
> Key: HBASE-18389
> URL: https://issues.apache.org/jira/browse/HBASE-18389
> Project: HBase
>  Issue Type: Improvement
>  Components: util
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18389.master.000.patch, 
> HBASE-18389.master.001.patch, HBASE-18389.master.002.patch, 
> HBASE-18389.master.003.patch
>
>
> In ClassSize class and its internal static class, sizeOf() function has 2 
> formal parameters: byte[] b and int len. But the function's internal logic 
> does not use or refer to byte[] b. Could be removed.
> {code:title=hbase-common/src/main/java/org/apache/hadoop/hbase/util/ClassSize.java|borderStyle=solid}
> // Class of ClassSize
> public static long sizeOf(byte[] b, int len) {
>   return memoryLayout.sizeOf(b, len);
> }
> // Class of ClassSize.MemoryLayout
> long sizeOf(byte[] b, int len) {
>   return align(arrayHeaderSize() + len);
> }
> // Class of ClassSize.UnsafeLayout
> long sizeOf(byte[] b, int len) {
>   return align(arrayHeaderSize() + len * 
> UnsafeAccess.theUnsafe.ARRAY_BYTE_INDEX_SCALE);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18424) Fix TestAsyncTableGetMultiThreaded

2017-07-21 Thread Vladimir Rodionov (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096942#comment-16096942
 ] 

Vladimir Rodionov commented on HBASE-18424:
---

So did 100+ other tests. 

> Fix TestAsyncTableGetMultiThreaded
> --
>
> Key: HBASE-18424
> URL: https://issues.apache.org/jira/browse/HBASE-18424
> Project: HBase
>  Issue Type: Sub-task
>  Components: test
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-18424-v1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (HBASE-18204) [C++] Rpc connection close and reconnecting

2017-07-21 Thread Xiaobing Zhou (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096934#comment-16096934
 ] 

Xiaobing Zhou edited comment on HBASE-18204 at 7/21/17 10:18 PM:
-

Here are some points:
# Not all exceptions from reading mean 'Connection closed to server', outgoing 
RPCs shouldn't be treated as same by being cleaned as a result. Should we 
respond based on different underlying errors?
# It actually introduces some bi-directional dependencies, e.g. RpcConnection 
and ConnectionPool, RpcConnection and ClientDispatcher, which make code less 
modular.
# RpcConnection::SendRequest is not thread safe, e.g. when it comes to connect


was (Author: xiaobingo):
Here are some points:
# Not all exceptions from reading mean 'Connection closed to server', outgoing 
RPCs shouldn't be treated as same by being cleaned as a result. Should we 
respond based on different underlying errors?
# It actually introduces some bi-directional dependencies, e.g. RpcConnection 
and ConnectionPool, RpcConnection and ClientDispatcher
# RpcConnection::SendRequest is not thread safe, e.g. when it comes to connect

> [C++] Rpc connection close and reconnecting
> ---
>
> Key: HBASE-18204
> URL: https://issues.apache.org/jira/browse/HBASE-18204
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: hbase-18204_v1.patch, hbase-18204_v2.patch
>
>
> Our client-dispatcher maintains a map of outgoing RPCs per server with the 
> promises. 
> In case the server goes down, or TCP connection is closed, we should complete 
> the outgoing RPCs with exceptions so that higher level waiters can unblock 
> and retry. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18204) [C++] Rpc connection close and reconnecting

2017-07-21 Thread Xiaobing Zhou (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096934#comment-16096934
 ] 

Xiaobing Zhou commented on HBASE-18204:
---

Here are some points:
# Not all exceptions from reading mean 'Connection closed to server', outgoing 
RPCs shouldn't be treated as same by being cleaned as a result. Should we 
respond based on different underlying errors?
# It actually introduces some bi-directional dependencies, e.g. RpcConnection 
and ConnectionPool, RpcConnection and ClientDispatcher
# RpcConnection::SendRequest is not thread safe, e.g. when it comes to connect

> [C++] Rpc connection close and reconnecting
> ---
>
> Key: HBASE-18204
> URL: https://issues.apache.org/jira/browse/HBASE-18204
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: hbase-18204_v1.patch, hbase-18204_v2.patch
>
>
> Our client-dispatcher maintains a map of outgoing RPCs per server with the 
> promises. 
> In case the server goes down, or TCP connection is closed, we should complete 
> the outgoing RPCs with exceptions so that higher level waiters can unblock 
> and retry. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18025) CatalogJanitor should collect outdated RegionStates from the AM


[ 
https://issues.apache.org/jira/browse/HBASE-18025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096931#comment-16096931
 ] 

Hadoop QA commented on HBASE-18025:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m  
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
30m 58s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  3m 
35s{color} | {color:red} hbase-server generated 2 new + 0 unchanged - 0 fixed = 
2 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
32s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 83m 32s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
34s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}131m 11s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
|  |  Null passed for non-null parameter of 
org.apache.hadoop.hbase.master.assignment.RegionStates.deleteRegion(HRegionInfo)
 in org.apache.hadoop.hbase.master.CatalogJanitor.scan()  Method invoked at 
CatalogJanitor.java:of 
org.apache.hadoop.hbase.master.assignment.RegionStates.deleteRegion(HRegionInfo)
 in org.apache.hadoop.hbase.master.CatalogJanitor.scan()  Method invoked at 
CatalogJanitor.java:[line 273] |
|  |  Null passed for non-null parameter of 
org.apache.hadoop.hbase.master.assignment.RegionStates.deleteRegion(HRegionInfo)
 in org.apache.hadoop.hbase.master.CatalogJanitor.scan()  Method invoked at 
CatalogJanitor.java:of 
org.apache.hadoop.hbase.master.assignment.RegionStates.deleteRegion(HRegionInfo)
 in org.apache.hadoop.hbase.master.CatalogJanitor.scan()  Method invoked at 
CatalogJanitor.java:[line 272] |
| Timed out junit tests | org.apache.hadoop.hbase.client.TestReplicasClient |
|   | org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface |
|   | org.apache.hadoop.hbase.mapreduce.TestWALPlayer |
|   | org.apache.hadoop.hbase.coprocessor.TestHTableWrapper |
|   | org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence |
|   | org.apache.hadoop.hbase.mapreduce.TestHRegionPartitioner |
|   | org.apache.hadoop.hbase.mapreduce.TestTableInputFormat |

[jira] [Commented] (HBASE-15816) Provide client with ability to set priority on Operations


[ 
https://issues.apache.org/jira/browse/HBASE-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096930#comment-16096930
 ] 

churro morales commented on HBASE-15816:


Ran all the failed tests above locally and each one of them passed.  

> Provide client with ability to set priority on Operations 
> --
>
> Key: HBASE-15816
> URL: https://issues.apache.org/jira/browse/HBASE-15816
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: churro morales
>Assignee: churro morales
> Attachments: HBASE-15816.patch, HBASE-15816.v1.branch-1.patch, 
> HBASE-15816-v1.patch, HBASE-15816.v2.patch
>
>
> First round will just be to expose the ability to set priorities for client 
> operations.  For more background: 
> http://mail-archives.apache.org/mod_mbox/hbase-dev/201604.mbox/%3CCA+RK=_BG_o=q8HMptcP2WauAinmEsL+15f3YEJuz=qbpcya...@mail.gmail.com%3E
> Next step would be to remove AnnotationReadingPriorityFunction and have the 
> client send priorities explicitly.  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-14070) Hybrid Logical Clocks for HBase


[ 
https://issues.apache.org/jira/browse/HBASE-14070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096927#comment-16096927
 ] 

Appy commented on HBASE-14070:
--

PT = physical time, LT = logical time, ST = system time

Note that in current implementation, master and RSs are updating their own 
clocks on receiving any region close/open request/response.
Also, on receiving a clock ahead of its own, they update their own clock to its 
PT+LT, and keep increasing LT till their own ST catches that PT.

Problem 1: cascading logical time increment
When more RS are involved say - 3 RS and 1 master.  Let's say max skew is 30 
sec.
HLC Clocks (physical time, logical time):  X = don't care
RS1: (50, 100k)
Master: (40, X)
RS2: (30, X)
RS3: (20, X) 
[RS3's ST behind RS1's by 30 sec.]

RS1 replies to master, sends it's clock (50,X).
Master's clock (50, X).  It'll be another 10 sec before it's own physical clock 
reaches 50, so HLC's PT will remain 50 for next 10 sec.
Master --> RS2
RS2's clock = (50, X).
RS2 keeps incrementing LT on writes (since it's own PT is behind) for few 
seconds before it replies back to master with (50, X+ few 100k).
Master's clock = (50, X+ few 100k) [Since master's physical clock hasn't caught 
up yet, note that it was 10 seconds behind, PT remains 50.].
Master --> RS3
RS3's clock (50, X+few 100k) 
But RS3's ST is behind RS1's ST by 30 sec, which means it'll keep incrementing 
LT for next 30 sec (unless it gets a newer clock from master).
But the problem is, RS3 has much smaller LT window than actual 1M!!
---
Problem 2:
Single bad RS clock crashing the cluster:
If a single RS's clock is bad and a bit faster, it'll catch time and keep 
pulling master's PT with it. If 'real time' is say 20, max skew time is 10, and 
bad RS is at time 29.9, it'll pull master to 29.9 (via next response), and then 
any RS less than 19.9, i.e. just  0.1 sec away from real time will die due to 
higher than max skew.
This can bring whole clusters down!
---
Problem 3: Time jumps (not a bug, but more of a nuisance)
Say a RS is behind master by 20 sec. On each communication from master, RS will 
update its own PT to master's PT, and it'll remain that till RS's ST catches 
up. If there are frequent communication from master, ST might never catch up 
and RS's PT will actually look like discrete time jumps rather than continuous 
time.
For eg. If master communicated with RS at times 30, 40, 50 (RSs corresponding 
times are 10, 20, 30), than all events on RS between time [10, 50] will be 
timestamped with either 30, 40 or 50.
---




> Hybrid Logical Clocks for HBase
> ---
>
> Key: HBASE-14070
> URL: https://issues.apache.org/jira/browse/HBASE-14070
> Project: HBase
>  Issue Type: New Feature
>Reporter: Enis Soztutar
>Assignee: Amit Patel
> Attachments: HBASE-14070.master.001.patch, 
> HybridLogicalClocksforHBaseandPhoenix.docx, 
> HybridLogicalClocksforHBaseandPhoenix.pdf
>
>
> HBase and Phoenix uses systems physical clock (PT) to give timestamps to 
> events (read and writes). This works mostly when the system clock is strictly 
> monotonically increasing and there is no cross-dependency between servers 
> clocks. However we know that leap seconds, general clock skew and clock drift 
> are in fact real. 
> This jira proposes using Hybrid Logical Clocks (HLC) as an implementation of 
> hybrid physical clock + a logical clock. HLC is best of both worlds where it 
> keeps causality relationship similar to logical clocks, but still is 
> compatible with NTP based physical system clock. HLC can be represented in 
> 64bits. 
> A design document is attached and also can be found here: 
> https://docs.google.com/document/d/1LL2GAodiYi0waBz5ODGL4LDT4e_bXy8P9h6kWC05Bhw/edit#



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18334) Remove sync client implementation and wrap async client under sync client interface

2017-07-21 Thread Biju Nair (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096924#comment-16096924
 ] 

Biju Nair commented on HBASE-18334:
---

[~yangzhe1991] Thanks for sharing the slide with the PE results. Will you be 
able to share a bit more about the test set-up and test run like the cluster 
config, RS config like heap/cache, total data size etc.

> Remove sync client implementation and wrap async client under sync client 
> interface
> ---
>
> Key: HBASE-18334
> URL: https://issues.apache.org/jira/browse/HBASE-18334
> Project: HBase
>  Issue Type: Task
>Reporter: Phil Yang
> Fix For: 3.0.0
>
>
> Since 2.0 we have async client, now we have two client implementations. We 
> can implement an sync client (Table interface) by using async client, getting 
> a CompletableFuture and then waiting it done directly. This can reduce the 
> maintenance work at client side in the future.
> Async client is done, we tested the performance and it showed it has same 
> performance with sync client. In branch-2 we can keep old sync client 
> implementations and remove it in master branch (since 3.0).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-15816) Provide client with ability to set priority on Operations


[ 
https://issues.apache.org/jira/browse/HBASE-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096923#comment-16096923
 ] 

Hadoop QA commented on HBASE-15816:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
20s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
54s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
19s{color} | {color:green} branch-1 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
32s{color} | {color:green} branch-1 passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
12s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  1m 
19s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  7m 
25s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
55s{color} | {color:green} branch-1 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  2m 
22s{color} | {color:green} branch-1 passed with JDK v1.7.0_131 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
4s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
14s{color} | {color:green} hbase-common-jdk1.8.0_131 with JDK v1.8.0_131 
generated 0 new + 18 unchanged - 18 fixed = 18 total (was 36) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
16s{color} | {color:green} hbase-client in the patch passed with JDK 
v1.8.0_131. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
34s{color} | {color:green} hbase-server-jdk1.8.0_131 with JDK v1.8.0_131 
generated 0 new + 5 unchanged - 5 fixed = 5 total (was 10) {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
9s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
17s{color} | {color:green} hbase-common-jdk1.7.0_131 with JDK v1.7.0_131 
generated 0 new + 18 unchanged - 18 fixed = 18 total (was 36) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
18s{color} | {color:green} hbase-client in the patch passed with JDK 
v1.7.0_131. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
34s{color} | {color:green} hbase-server-jdk1.7.0_131 with JDK v1.7.0_131 
generated 0 new + 5 unchanged - 5 fixed = 5 total (was 10) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
 8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
15m 39s{color} | {color:green} The patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m 
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
14s{color} | {color:green} hbase-common-jdk1.8.0_131 with JDK v1.8.0_131 
generated 0 new + 1 unchanged -

[jira] [Commented] (HBASE-18424) Fix TestAsyncTableGetMultiThreaded

2017-07-21 Thread Devaraj Das (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096918#comment-16096918
 ] 

Devaraj Das commented on HBASE-18424:
-

TestAsyncTableGetMultiThreaded timed out on the hadoopqa run, [~vrodionov] ?

> Fix TestAsyncTableGetMultiThreaded
> --
>
> Key: HBASE-18424
> URL: https://issues.apache.org/jira/browse/HBASE-18424
> Project: HBase
>  Issue Type: Sub-task
>  Components: test
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-18424-v1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18361) CellComparator compare function should compare cell's value along with other attributes

2017-07-21 Thread Biju Nair (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096915#comment-16096915
 ] 

Biju Nair commented on HBASE-18361:
---

>From API usability perspective adding a method in {{CellComparator}} will make 
>it easier for users. 

> CellComparator compare function should compare cell's value along with other 
> attributes
> ---
>
> Key: HBASE-18361
> URL: https://issues.apache.org/jira/browse/HBASE-18361
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.1
>Reporter: Amit Kabra
>Assignee: Amit Kabra
> Attachments: HBASE-18361.patch
>
>
> CellComparator.compare(Cell a, Cell b) should compare a and b's value as well.
> If we create two cells as 
> byte [] row = Bytes.toBytes("row");
> byte [] value = Bytes.toBytes("value");
> byte [] value1 = Bytes.toBytes("value1");
> Cell c1 = CellUtil.createCell(row, value);
> Cell c2 = CellUtil.createCell(row, value1);
> And compare them using CellComparator.compare(c1, c2,true) , it returns 0 i.e 
> it matches them while they are different.CellComparator compares each 
> attribute of cell but value.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18405) Track scope for HBase-Spark module


[ 
https://issues.apache.org/jira/browse/HBASE-18405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096902#comment-16096902
 ] 

Sean Busbey commented on HBASE-18405:
-

Any feedback here folks? Objections to us moving forward with this as a guide?

ping [~easyliangjob], [~jerryhe], [~Weiqing Yang], [~tedyu]

> Track scope for HBase-Spark module
> --
>
> Key: HBASE-18405
> URL: https://issues.apache.org/jira/browse/HBASE-18405
> Project: HBase
>  Issue Type: Task
>  Components: spark
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Fix For: 1.4.0, 2.0.0-beta-1
>
> Attachments: Apache HBase - Apache Spark Integration Scope.pdf, 
> Apache HBase - Apache Spark Integration Scope - update 1.pdf
>
>
> Start with [\[DISCUSS\]  status of and plans for our hbase-spark integration 
> |https://lists.apache.org/thread.html/fd74ef9b9da77abf794664f06ea19c839fb3d543647fb29115081683@%3Cdev.hbase.apache.org%3E]
>  and formalize into a scope document for bringing this feature into a release.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18428) IntegrationTestDDLMasterFailover uses old-style immutable column descriptors


[ 
https://issues.apache.org/jira/browse/HBASE-18428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096885#comment-16096885
 ] 

Hadoop QA commented on HBASE-18428:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m  
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
31m 57s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m  
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
19s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
 8s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m  0s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18428 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12878415/HBASE-18428.patch |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux 624bb9e5d4f7 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
 |
| git revision | master / 31c3eda |
| Default Java | 1.8.0_131 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7756/testReport/ |
| modules | C: hbase-it U: hbase-it |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7756/console |
| Powered by | Apache Yetus 0.4.0   http://yetus.apache.org |


This message was automatically generated.



> IntegrationTestDDLMasterFailover uses old-style immutable column descriptors
> 
>
> Key: HBASE-18428
> URL: https://issues.apache.org/jira/browse/HBASE-18428
> Project: HBase
>  Issue Type: Sub-task
>  Components: integration tests
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
>Assignee: Mike Drob
> Attachments: HBASE-18428.patch
>
>




--

[jira] [Commented] (HBASE-18430) Typo in "contributing to documentation" page


[ 
https://issues.apache.org/jira/browse/HBASE-18430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096862#comment-16096862
 ] 

Coral commented on HBASE-18430:
---

Awesome. Thanks for the help, [~misty]! I appreciate it. 

> Typo in "contributing to documentation" page
> 
>
> Key: HBASE-18430
> URL: https://issues.apache.org/jira/browse/HBASE-18430
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: Coral
>Assignee: Coral
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18430.patch
>
>
> Be sure to set the component to Documentation in addition any other involved 
> components.
> Missing "to." Should be: 
> Be sure to set the component to Documentation in addition *to* any other 
> involved components.
> http://hbase.apache.org/book.html#appendix_contributing_to_documentation



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-14135) HBase Backup/Restore Phase 3: Merge backup images

2017-07-21 Thread Josh Elser (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-14135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096859#comment-16096859
 ] 

Josh Elser commented on HBASE-14135:


bq. We have repair tool, just in case if "client goes away" or some other 
disaster happens. 

This was one question I had on RB. Let me add to it since it was brought up 
here as well.

It looks like there is the potential for files in HDFS to be orphaned (via a 
hard JVM crash where "deleteOnExit" wouldn't fire) and for entries in the 
hbase:backup table to be orphaned. The tooling to clean these up is great -- 
should we have some automation inside of the Master to try to catch these 
scenarios and proactively clean it up for the user?

I am thinking about something very un-aggressive (e.g. cleaning up files in 
hdfs after many hours or days). WDYT?

> HBase Backup/Restore Phase 3: Merge backup images
> -
>
> Key: HBASE-14135
> URL: https://issues.apache.org/jira/browse/HBASE-14135
> Project: HBase
>  Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Blocker
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: HBASE-14135-v3.patch, HBASE-14135-v5.patch, 
> HBASE-14135-v6.patch, HBASE-14135-v7.patch, HBASE-14135-v8.patch
>
>
> User can merge incremental backup images into single incremental backup image.
> # Merge supports only incremental images
> # Merge supports only images for the same backup destinations
> Command:
> {code}
> hbase backup merge image1,image2,..imageK
> {code}
> Example:
> {code}
> hbase backup merge backup_143126764557,backup_143126764456 
> {code}
> When operation is complete, only the most recent backup image will be kept 
> (in above example -  backup_143126764557) as a merged backup image, all other 
> images will be deleted from both: file system and backup system tables, 
> corresponding backup manifest for the merged backup image will be updated to 
> remove dependencies from deleted images. Merged backup image will contains 
> all the data from original image and from deleted images.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18430) Typo in "contributing to documentation" page


[ 
https://issues.apache.org/jira/browse/HBASE-18430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096860#comment-16096860
 ] 

Misty Stanley-Jones commented on HBASE-18430:
-

No worries. The sign-off is not a HBase requirement, just a personal preference 
for some of us. It just adds that little "Signed-off by" line to the commit 
message. It's actually something that happens during the {{git commit}}, not 
the {{git format-patch}}. Your {{git format-patch}} is perfect. If you commit 
and then realize you didn't sign off, you can do {{git commit --amend -s}}. I 
do that all the time. :) You can always look at your existing commit by doing 
{{git log -n1}}.

> Typo in "contributing to documentation" page
> 
>
> Key: HBASE-18430
> URL: https://issues.apache.org/jira/browse/HBASE-18430
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: Coral
>Assignee: Coral
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18430.patch
>
>
> Be sure to set the component to Documentation in addition any other involved 
> components.
> Missing "to." Should be: 
> Be sure to set the component to Documentation in addition *to* any other 
> involved components.
> http://hbase.apache.org/book.html#appendix_contributing_to_documentation



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18430) Typo in "contributing to documentation" page


[ 
https://issues.apache.org/jira/browse/HBASE-18430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096842#comment-16096842
 ] 

Coral commented on HBASE-18430:
---

Works for me. I'll add the sign-off line in the future. 

> Typo in "contributing to documentation" page
> 
>
> Key: HBASE-18430
> URL: https://issues.apache.org/jira/browse/HBASE-18430
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: Coral
>Assignee: Coral
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18430.patch
>
>
> Be sure to set the component to Documentation in addition any other involved 
> components.
> Missing "to." Should be: 
> Be sure to set the component to Documentation in addition *to* any other 
> involved components.
> http://hbase.apache.org/book.html#appendix_contributing_to_documentation



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-15816) Provide client with ability to set priority on Operations


[ 
https://issues.apache.org/jira/browse/HBASE-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096840#comment-16096840
 ] 

Andrew Purtell commented on HBASE-15816:


Ok, let me run the tests locally and then commit these

> Provide client with ability to set priority on Operations 
> --
>
> Key: HBASE-15816
> URL: https://issues.apache.org/jira/browse/HBASE-15816
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: churro morales
>Assignee: churro morales
> Attachments: HBASE-15816.patch, HBASE-15816.v1.branch-1.patch, 
> HBASE-15816-v1.patch, HBASE-15816.v2.patch
>
>
> First round will just be to expose the ability to set priorities for client 
> operations.  For more background: 
> http://mail-archives.apache.org/mod_mbox/hbase-dev/201604.mbox/%3CCA+RK=_BG_o=q8HMptcP2WauAinmEsL+15f3YEJuz=qbpcya...@mail.gmail.com%3E
> Next step would be to remove AnnotationReadingPriorityFunction and have the 
> client send priorities explicitly.  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-15816) Provide client with ability to set priority on Operations


[ 
https://issues.apache.org/jira/browse/HBASE-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096839#comment-16096839
 ] 

Andrew Purtell commented on HBASE-15816:


branch-1 patch lgtm

> Provide client with ability to set priority on Operations 
> --
>
> Key: HBASE-15816
> URL: https://issues.apache.org/jira/browse/HBASE-15816
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: churro morales
>Assignee: churro morales
> Attachments: HBASE-15816.patch, HBASE-15816.v1.branch-1.patch, 
> HBASE-15816-v1.patch, HBASE-15816.v2.patch
>
>
> First round will just be to expose the ability to set priorities for client 
> operations.  For more background: 
> http://mail-archives.apache.org/mod_mbox/hbase-dev/201604.mbox/%3CCA+RK=_BG_o=q8HMptcP2WauAinmEsL+15f3YEJuz=qbpcya...@mail.gmail.com%3E
> Next step would be to remove AnnotationReadingPriorityFunction and have the 
> client send priorities explicitly.  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18430) Typo in "contributing to documentation" page


[ 
https://issues.apache.org/jira/browse/HBASE-18430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096837#comment-16096837
 ] 

Mike Drob commented on HBASE-18430:
---

This JIRA might not be the best venue to hash out git internals, but it looks 
like we already have all the information even without the sign-off line?

{noformat}
mdrob@mdrob-MBP:~/IdeaProjects/hbase$ git log -1 70a357dc5c --format=full
commit 70a357dc5cc74ae6a354c907959f644f563aeee4 (origin/master, origin/HEAD)
Author: coral 
Commit: Misty Stanley-Jones 

HBASE-18430 fixed typo

Signed-off-by: Misty Stanley-Jones 
{noformat}

> Typo in "contributing to documentation" page
> 
>
> Key: HBASE-18430
> URL: https://issues.apache.org/jira/browse/HBASE-18430
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: Coral
>Assignee: Coral
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18430.patch
>
>
> Be sure to set the component to Documentation in addition any other involved 
> components.
> Missing "to." Should be: 
> Be sure to set the component to Documentation in addition *to* any other 
> involved components.
> http://hbase.apache.org/book.html#appendix_contributing_to_documentation



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18430) Typo in "contributing to documentation" page


 [ 
https://issues.apache.org/jira/browse/HBASE-18430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Misty Stanley-Jones updated HBASE-18430:

   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 2.0.0-alpha-2
   3.0.0
 Release Note: Pushed to {{master}}. Thanks, Coral! Congratulations on your 
first Apache HBase commit!
   Status: Resolved  (was: Patch Available)

> Typo in "contributing to documentation" page
> 
>
> Key: HBASE-18430
> URL: https://issues.apache.org/jira/browse/HBASE-18430
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: Coral
>Assignee: Coral
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18430.patch
>
>
> Be sure to set the component to Documentation in addition any other involved 
> components.
> Missing "to." Should be: 
> Be sure to set the component to Documentation in addition *to* any other 
> involved components.
> http://hbase.apache.org/book.html#appendix_contributing_to_documentation



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18430) Typo in "contributing to documentation" page


[ 
https://issues.apache.org/jira/browse/HBASE-18430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096828#comment-16096828
 ] 

Misty Stanley-Jones commented on HBASE-18430:
-

With that having been said, this LGTM and I will commit it. Thanks [~coral.w]!

> Typo in "contributing to documentation" page
> 
>
> Key: HBASE-18430
> URL: https://issues.apache.org/jira/browse/HBASE-18430
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: Coral
>Assignee: Coral
> Attachments: HBASE-18430.patch
>
>
> Be sure to set the component to Documentation in addition any other involved 
> components.
> Missing "to." Should be: 
> Be sure to set the component to Documentation in addition *to* any other 
> involved components.
> http://hbase.apache.org/book.html#appendix_contributing_to_documentation



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18430) Typo in "contributing to documentation" page


[ 
https://issues.apache.org/jira/browse/HBASE-18430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096829#comment-16096829
 ] 

Coral commented on HBASE-18430:
---

Should I be using a different command to create the patch? I used this command 
that I got from the doc: 

{code:java}
git format-patch --stdout origin/master > HBASE-123456.patch
{code}

I'm new to git, so thanks for the help.  :)

> Typo in "contributing to documentation" page
> 
>
> Key: HBASE-18430
> URL: https://issues.apache.org/jira/browse/HBASE-18430
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: Coral
>Assignee: Coral
> Attachments: HBASE-18430.patch
>
>
> Be sure to set the component to Documentation in addition any other involved 
> components.
> Missing "to." Should be: 
> Be sure to set the component to Documentation in addition *to* any other 
> involved components.
> http://hbase.apache.org/book.html#appendix_contributing_to_documentation



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18430) Typo in "contributing to documentation" page


[ 
https://issues.apache.org/jira/browse/HBASE-18430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096826#comment-16096826
 ] 

Misty Stanley-Jones commented on HBASE-18430:
-

No, {{-s}} is not cryptographic signing. You use {{-S}} for that. Gotta love 
{{git}}. I didn't realize that about the subject. I don't recall seeing it in 
other patches created using {{git format-patch}}, but when I look back at my 
old patches it is there. Sorry about that!

The {{-s}} is useful for committers who are committing others' patches. It adds 
a sign-off line for you beneath the sign-off line for the contributor. It's not 
a current requirement, and I forget to do it half the time myself.

> Typo in "contributing to documentation" page
> 
>
> Key: HBASE-18430
> URL: https://issues.apache.org/jira/browse/HBASE-18430
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: Coral
>Assignee: Coral
> Attachments: HBASE-18430.patch
>
>
> Be sure to set the component to Documentation in addition any other involved 
> components.
> Missing "to." Should be: 
> Be sure to set the component to Documentation in addition *to* any other 
> involved components.
> http://hbase.apache.org/book.html#appendix_contributing_to_documentation



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (HBASE-18430) Typo in "contributing to documentation" page


[ 
https://issues.apache.org/jira/browse/HBASE-18430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096820#comment-16096820
 ] 

Mike Drob edited comment on HBASE-18430 at 7/21/17 8:43 PM:


[~misty] the \[PATCH] is auto generated by {{git format-patch}} and gets 
stripped if you use {{git am}}.

-Also, I didn't know we cryptographically signed our commits. We should update 
the docs and {{dev-support/submit-patch.py}} for that.- Nevermind I confused 
the flags for commit and tag.


was (Author: mdrob):
[~misty] the \[PATCH] is auto generated by {{git format-patch}} and gets 
stripped if you use {{git am}}.

Also, I didn't know we cryptographically signed our commits. We should update 
the docs and {{dev-support/submit-patch.py}} for that.

> Typo in "contributing to documentation" page
> 
>
> Key: HBASE-18430
> URL: https://issues.apache.org/jira/browse/HBASE-18430
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: Coral
>Assignee: Coral
> Attachments: HBASE-18430.patch
>
>
> Be sure to set the component to Documentation in addition any other involved 
> components.
> Missing "to." Should be: 
> Be sure to set the component to Documentation in addition *to* any other 
> involved components.
> http://hbase.apache.org/book.html#appendix_contributing_to_documentation



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18430) Typo in "contributing to documentation" page


[ 
https://issues.apache.org/jira/browse/HBASE-18430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096820#comment-16096820
 ] 

Mike Drob commented on HBASE-18430:
---

[~misty] the \[PATCH] is auto generated by {{git format-patch}} and gets 
stripped if you use {{git am}}.

Also, I didn't know we cryptographically signed our commits. We should update 
the docs and {{dev-support/submit-patch.py}} for that.

> Typo in "contributing to documentation" page
> 
>
> Key: HBASE-18430
> URL: https://issues.apache.org/jira/browse/HBASE-18430
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: Coral
>Assignee: Coral
> Attachments: HBASE-18430.patch
>
>
> Be sure to set the component to Documentation in addition any other involved 
> components.
> Missing "to." Should be: 
> Be sure to set the component to Documentation in addition *to* any other 
> involved components.
> http://hbase.apache.org/book.html#appendix_contributing_to_documentation



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18428) IntegrationTestDDLMasterFailover uses old-style immutable column descriptors


 [ 
https://issues.apache.org/jira/browse/HBASE-18428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob updated HBASE-18428:
--
Assignee: Mike Drob
  Status: Patch Available  (was: Open)

> IntegrationTestDDLMasterFailover uses old-style immutable column descriptors
> 
>
> Key: HBASE-18428
> URL: https://issues.apache.org/jira/browse/HBASE-18428
> Project: HBase
>  Issue Type: Sub-task
>  Components: integration tests
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
>Assignee: Mike Drob
> Attachments: HBASE-18428.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18428) IntegrationTestDDLMasterFailover uses old-style immutable column descriptors


 [ 
https://issues.apache.org/jira/browse/HBASE-18428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob updated HBASE-18428:
--
Attachment: HBASE-18428.patch

Tried testing this locally, it looked like it worked for a while but then my 
server fell over (OOM I think?).

I think the API changes are good, and my laptop isn't big enough.

> IntegrationTestDDLMasterFailover uses old-style immutable column descriptors
> 
>
> Key: HBASE-18428
> URL: https://issues.apache.org/jira/browse/HBASE-18428
> Project: HBase
>  Issue Type: Sub-task
>  Components: integration tests
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
> Attachments: HBASE-18428.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18419) Update IntegrationTestIngestWithMOB and Actions to use ColumnFamily builders for modification


[ 
https://issues.apache.org/jira/browse/HBASE-18419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096812#comment-16096812
 ] 

Mike Drob commented on HBASE-18419:
---

[~chia7712] - With the new immutable Descriptors and the builder pattern, I 
find myself missing a lot of the old style {{new TableDescriptor(tableName)}} 
methods for their simplicity.

The equivalent call chain now would be 
{{TableDescriptorBuilder.newBuilder(tableName).build()}}
Similar for CFD: 
{{ColumnFamilyDescriptorBuilder.newBuilder(columnName).build()}}

What do you think of having {{ColumnFamilyDescriptor.forColumn(String)}} and 
{{.forColumn(byte[])}}, etc.? If you agree I'll file a new JIRA for it and 
provide a patch later.

> Update IntegrationTestIngestWithMOB and Actions to use ColumnFamily builders 
> for modification
> -
>
> Key: HBASE-18419
> URL: https://issues.apache.org/jira/browse/HBASE-18419
> Project: HBase
>  Issue Type: Sub-task
>  Components: integration tests
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18419.patch, HBASE-18419.v2.patch, 
> HBASE-18419.v3.patch
>
>
> When attempting to run ITIWM against a cluster I get the following error:
> {noformat}
> java.lang.UnsupportedOperationException: HColumnDescriptor is read-only
>   at 
> org.apache.hadoop.hbase.client.ImmutableHColumnDescriptor.getDelegateeForModification(ImmutableHColumnDescriptor.java:44)
>   at 
> org.apache.hadoop.hbase.HColumnDescriptor.setMobEnabled(HColumnDescriptor.java:735)
>   at 
> org.apache.hadoop.hbase.IntegrationTestIngestWithMOB.initTable(IntegrationTestIngestWithMOB.java:122)
>   at 
> org.apache.hadoop.hbase.IntegrationTestIngest.setUpCluster(IntegrationTestIngest.java:92)
>   at 
> org.apache.hadoop.hbase.IntegrationTestBase.setUp(IntegrationTestBase.java:148)
>   at 
> org.apache.hadoop.hbase.IntegrationTestBase.doWork(IntegrationTestBase.java:131)
>   at 
> org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:154)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
>   at 
> org.apache.hadoop.hbase.IntegrationTestIngestWithMOB.main(IntegrationTestIngestWithMOB.java:153)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18430) Typo in "contributing to documentation" page


[ 
https://issues.apache.org/jira/browse/HBASE-18430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096810#comment-16096810
 ] 

Misty Stanley-Jones commented on HBASE-18430:
-

Looks great. Please amend the commit message to remove the {{[patch]}} prefix, 
as we don't use that. Also consider signing your commit by adding the {{-s}} 
flag to the git commit message. To do both things at once, do a command like:
{code}
git commit --amend -s
{code}

> Typo in "contributing to documentation" page
> 
>
> Key: HBASE-18430
> URL: https://issues.apache.org/jira/browse/HBASE-18430
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: Coral
>Assignee: Coral
> Attachments: HBASE-18430.patch
>
>
> Be sure to set the component to Documentation in addition any other involved 
> components.
> Missing "to." Should be: 
> Be sure to set the component to Documentation in addition *to* any other 
> involved components.
> http://hbase.apache.org/book.html#appendix_contributing_to_documentation



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17819) Reduce the heap overhead for BucketCache

2017-07-21 Thread Ted Yu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-17819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096807#comment-16096807
 ] 

Ted Yu commented on HBASE-17819:


{code}
+  // We would like to reduce the head overhead per object of this type as much 
as possible.
{code}
Remove "head"
{code}
+public class SharedMemoryBucketEntry extends BucketEntry {
{code}
Add short javadoc for the above class.

Please check the two failed tests.

> Reduce the heap overhead for BucketCache
> 
>
> Key: HBASE-17819
> URL: https://issues.apache.org/jira/browse/HBASE-17819
> Project: HBase
>  Issue Type: Sub-task
>  Components: BucketCache
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Fix For: 2.0.0
>
> Attachments: HBASE-17819_V1.patch, HBASE-17819_V2.patch
>
>
> We keep Bucket entry map in BucketCache.  Below is the math for heapSize for 
> the key , value into this map.
> BlockCacheKey
> ---
> String hfileName  -  Ref  - 4
> long offset  - 8
> BlockType blockType  - Ref  - 4
> boolean isPrimaryReplicaBlock  - 1
> Total  =  12 (Object) + 17 = 29
> BucketEntry
> 
> int offsetBase  -  4
> int length  - 4
> byte offset1  -  1
> byte deserialiserIndex  -  1
> long accessCounter  -  8
> BlockPriority priority  - Ref  - 4
> volatile boolean markedForEvict  -  1
> AtomicInteger refCount  -  16 + 4
> long cachedTime  -  8
> Total = 12 (Object) + 51 = 63
> ConcurrentHashMap Map.Entry  -  40
> blocksByHFile ConcurrentSkipListSet Entry  -  40
> Total = 29 + 63 + 80 = 172
> For 10 million blocks we will end up having 1.6GB of heap size.  
> This jira aims to reduce this as much as possible



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18430) Typo in "contributing to documentation" page


 [ 
https://issues.apache.org/jira/browse/HBASE-18430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Coral updated HBASE-18430:
--
Status: Patch Available  (was: Open)

> Typo in "contributing to documentation" page
> 
>
> Key: HBASE-18430
> URL: https://issues.apache.org/jira/browse/HBASE-18430
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: Coral
>Assignee: Coral
> Attachments: HBASE-18430.patch
>
>
> Be sure to set the component to Documentation in addition any other involved 
> components.
> Missing "to." Should be: 
> Be sure to set the component to Documentation in addition *to* any other 
> involved components.
> http://hbase.apache.org/book.html#appendix_contributing_to_documentation



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18430) Typo in "contributing to documentation" page


 [ 
https://issues.apache.org/jira/browse/HBASE-18430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Coral updated HBASE-18430:
--
Attachment: HBASE-18430.patch

I created a patch for this. 

> Typo in "contributing to documentation" page
> 
>
> Key: HBASE-18430
> URL: https://issues.apache.org/jira/browse/HBASE-18430
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: Coral
>Assignee: Coral
> Attachments: HBASE-18430.patch
>
>
> Be sure to set the component to Documentation in addition any other involved 
> components.
> Missing "to." Should be: 
> Be sure to set the component to Documentation in addition *to* any other 
> involved components.
> http://hbase.apache.org/book.html#appendix_contributing_to_documentation



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18107) [AMv2] Remove DispatchMergingRegionsRequest & DispatchMergingRegions

2017-07-21 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096803#comment-16096803
 ] 

stack commented on HBASE-18107:
---

This looks excellent to me.

Let me try and apply (seems to have rotted already but let me mess with it 
will be back).

Thanks [~easyliangjob]

> [AMv2] Remove DispatchMergingRegionsRequest & DispatchMergingRegions
> 
>
> Key: HBASE-18107
> URL: https://issues.apache.org/jira/browse/HBASE-18107
> Project: HBase
>  Issue Type: Sub-task
>  Components: Region Assignment
>Affects Versions: 3.0.0
>Reporter: stack
>Assignee: Yi Liang
> Fix For: 3.0.0
>
> Attachments: draft.patch, hbase-18107-master.v1.patch, 
> HBASE-18107-master-v2.patch
>
>
> They don't align with how we have named the Split equivalents; i.e. 
> SplitRegion (so should be MergeRegion...). They probably have these awkward 
> names because the obvious slots are occupied... so this may not be fixable 
> but filing issue anyways.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18024) HRegion#initializeRegionInternals should not re-create .hregioninfo file when the region directory no longer exists


[ 
https://issues.apache.org/jira/browse/HBASE-18024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096799#comment-16096799
 ] 

Esteban Gutierrez commented on HBASE-18024:
---

Both failures are related to the patch, looking into.

> HRegion#initializeRegionInternals should not re-create .hregioninfo file when 
> the region directory no longer exists
> ---
>
> Key: HBASE-18024
> URL: https://issues.apache.org/jira/browse/HBASE-18024
> Project: HBase
>  Issue Type: Bug
>  Components: Region Assignment, regionserver
>Affects Versions: 2.0.0, 1.4.0, 1.3.1, 1.2.5
>Reporter: Esteban Gutierrez
>Assignee: Esteban Gutierrez
> Attachments: HBASE-18024.001.patch
>
>
> When a RegionSever attempts to open a region, during initialization the RS 
> tries to open the {{/data///.hregioninfo}} 
> file, however if the {{.hregioninfofile}} doesn't exist, the RegionServer 
> will create a new one on {{HRegionFileSystem#checkRegionInfoOnFilesystem}}. A 
> side effect of that tools like hbck will incorrectly assume an inconsistency 
> due the presence of this new {{.hregioninfofile}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-7386) Investigate providing some supervisor support for znode deletion

2017-07-21 Thread Samir Ahmic (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096776#comment-16096776
 ] 

Samir Ahmic commented on HBASE-7386:


Thanks [~stack].  Why python supervisor? Well we originally started this story 
around it, and after some time testing it, at least for me,  choosing mature 
and well proven process control system instead of writing custom bash scripts 
has multiple advantages. 
To be honest work here extends original issue of just removing stale znodes to 
creating watchdog over hbase processes and making alternative option for 
managing cluster but when we started tackling supervisor approach why not offer 
folks chance to 
less worry when rs process dies because it will be automatically restarted :) 
Also python supervisor has set of very cool futures like, auto-restart, event 
listeners (that may execute arbitrary code based on process state) an so on, 
and folks may start creating  they own  listeners for different proposes.
Btw i will address shellcheck and pylint issues in next patch. 

> Investigate providing some supervisor support for znode deletion
> 
>
> Key: HBASE-7386
> URL: https://issues.apache.org/jira/browse/HBASE-7386
> Project: HBase
>  Issue Type: Task
>  Components: master, regionserver, scripts
>Reporter: Gregory Chanan
>Assignee: stack
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: HBASE-7386-bin.patch, HBASE-7386-bin-v2.patch, 
> HBASE-7386-bin-v3.patch, HBASE-7386-conf.patch, HBASE-7386-conf-v2.patch, 
> HBASE-7386-conf-v3.patch, HBASE-7386-master-00.patch, HBASE-7386-src.patch, 
> HBASE-7386-v0.patch, supervisordconfigs-v0.patch
>
>
> There a couple of JIRAs for deleting the znode on a process failure:
> HBASE-5844 (RS)
> HBASE-5926 (Master)
> which are pretty neat; on process failure, they delete the znode of the 
> underlying process so HBase can recover faster.
> These JIRAs were implemented via the startup scripts; i.e. the script hangs 
> around and waits for the process to exit, then deletes the znode.
> There are a few problems associated with this approach, as listed in the 
> below JIRAs:
> 1) Hides startup output in script
> https://issues.apache.org/jira/browse/HBASE-5844?focusedCommentId=13463401=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13463401
> 2) two hbase processes listed per launched daemon
> https://issues.apache.org/jira/browse/HBASE-5844?focusedCommentId=13463409=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13463409
> 3) Not run by a real supervisor
> https://issues.apache.org/jira/browse/HBASE-5844?focusedCommentId=13463409=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13463409
> 4) Weird output after kill -9 actual process in standalone mode
> https://issues.apache.org/jira/browse/HBASE-5926?focusedCommentId=13506801=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13506801
> 5) Can kill existing RS if called again
> https://issues.apache.org/jira/browse/HBASE-5844?focusedCommentId=13463401=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13463401
> 6) Hides stdout/stderr[6]
> https://issues.apache.org/jira/browse/HBASE-5844?focusedCommentId=13506832=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13506832
> I suspect running in via something like supervisor.d can solve these issues 
> if we provide the right support.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18024) HRegion#initializeRegionInternals should not re-create .hregioninfo file when the region directory no longer exists


[ 
https://issues.apache.org/jira/browse/HBASE-18024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096766#comment-16096766
 ] 

Hadoop QA commented on HBASE-18024:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
31m 26s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
29s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 22m 59s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
 9s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 70m 41s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | 
hadoop.hbase.regionserver.TestWALMonotonicallyIncreasingSeqId |
|   | hadoop.hbase.regionserver.TestStoreFileRefresherChore |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18024 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12878391/HBASE-18024.001.patch 
|
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux 6a7c46fbc9f9 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 31c3eda |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| unit | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7752/artifact/patchprocess/patch-unit-hbase-server.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7752/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7752/console |
| Powered by | Apache Yetus 0.4.0   http://yetus.apache.org |


This message was automatically generated.



> HRegion#initializeRegionInternals should not re-create .hregioninfo file when 
> the region directory no longer exists
>

[jira] [Commented] (HBASE-17908) Upgrade guava


[ 
https://issues.apache.org/jira/browse/HBASE-17908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096757#comment-16096757
 ] 

Hudson commented on HBASE-17908:


SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #3413 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/3413/])
HBASE-17908 Upgrade guava (stack: rev 890d92a90cb5ef8d17e6c3e4036c8077dea4dc86)
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/security/access/Permission.java
* (edit) 
hbase-client/src/test/java/org/apache/hadoop/hbase/security/TestHBaseSaslRpcClient.java
* (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/security/User.java
* (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/net/Address.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationStateZKBase.java
* (edit) 
hbase-prefix-tree/src/test/java/org/apache/hadoop/hbase/codec/prefixtree/row/data/TestRowDataTrivialWithTags.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/namespace/NamespaceTableAndRegionInfo.java
* (edit) 
hbase-prefix-tree/src/test/java/org/apache/hadoop/hbase/codec/prefixtree/builder/TestTreeDepth.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatImpl.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSizeFailures.java
* (edit) 
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestRegionReplicaReplication.java
* (edit) 
hbase-examples/src/main/java/org/apache/hadoop/hbase/client/example/HttpProxyExample.java
* (edit) 
hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java
* (edit) 
hbase-common/src/main/java/org/apache/hadoop/hbase/util/CoprocessorClassLoader.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ReplicationEndpoint.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RegionServerRpcQuotaManager.java
* (edit) 
hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/SimpleProcedureScheduler.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/util/DirectMemoryUtils.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallableWithReplicas.java
* (edit) 
hbase-prefix-tree/src/test/java/org/apache/hadoop/hbase/codec/prefixtree/row/TestRowEncoder.java
* (edit) 
hbase-prefix-tree/src/test/java/org/apache/hadoop/hbase/codec/prefixtree/timestamp/TestTimestampData.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestBackupStatusProgress.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHashTable.java
* (edit) 
hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/procedure/flush/MasterFlushTableProcedureManager.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapred/TestGroupingTableMap.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/filter/PageFilter.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ColumnPaginationFilter.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExponentialCompactionWindowFactory.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsvParser.java
* (edit) 
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/SimpleLoadBalancer.java
* (edit) 
hbase-common/src/main/java/org/apache/hadoop/hbase/util/ClassLoaderBase.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/AbstractTestDateTieredCompactionPolicy.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/tool/TestCanaryTool.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestSyncTable.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
* (edit) 
hbase-prefix-tree/src/test/java/org/apache/hadoop/hbase/codec/prefixtree/row/data/TestRowDataSearcherRowMiss.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/favored/FavoredNodeAssignmentHelper.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/NamespaceTableCfWALEntryFilter.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestRpcServerSlowConnectionSetup.java
* (edit) 
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsOfflineMode.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionReplayEvents.java
* (edit)

[jira] [Updated] (HBASE-18025) CatalogJanitor should collect outdated RegionStates from the AM


 [ 
https://issues.apache.org/jira/browse/HBASE-18025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Esteban Gutierrez updated HBASE-18025:
--
Attachment: HBASE-18025.002.patch

> CatalogJanitor should collect outdated RegionStates from the AM
> ---
>
> Key: HBASE-18025
> URL: https://issues.apache.org/jira/browse/HBASE-18025
> Project: HBase
>  Issue Type: Bug
>Reporter: Esteban Gutierrez
>Assignee: Esteban Gutierrez
> Attachments: HBASE-18025.001.patch, HBASE-18025.002.patch
>
>
> I don't think this will matter on the long run for HBase 2, but at least in 
> branch-1 and the current master we keep in multiple places copies of the 
> region states in the master and this copies include information like the HRI. 
> A problem that we have observed is when region replicas are being used and 
> there is a split, the region replica from parent doesn't get collected from 
> the region states and when the balancer tries to assign the old parent region 
> replica, this will cause the RegionServer to create a new HRI with the 
> details of the parent causing an inconstancy (see HBASE-18024).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18025) CatalogJanitor should collect outdated RegionStates from the AM


 [ 
https://issues.apache.org/jira/browse/HBASE-18025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Esteban Gutierrez updated HBASE-18025:
--
Attachment: HBASE-18025.001.patch

> CatalogJanitor should collect outdated RegionStates from the AM
> ---
>
> Key: HBASE-18025
> URL: https://issues.apache.org/jira/browse/HBASE-18025
> Project: HBase
>  Issue Type: Bug
>Reporter: Esteban Gutierrez
>Assignee: Esteban Gutierrez
> Attachments: HBASE-18025.001.patch
>
>
> I don't think this will matter on the long run for HBase 2, but at least in 
> branch-1 and the current master we keep in multiple places copies of the 
> region states in the master and this copies include information like the HRI. 
> A problem that we have observed is when region replicas are being used and 
> there is a split, the region replica from parent doesn't get collected from 
> the region states and when the balancer tries to assign the old parent region 
> replica, this will cause the RegionServer to create a new HRI with the 
> details of the parent causing an inconstancy (see HBASE-18024).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18025) CatalogJanitor should collect outdated RegionStates from the AM


 [ 
https://issues.apache.org/jira/browse/HBASE-18025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Esteban Gutierrez updated HBASE-18025:
--
Status: Patch Available  (was: In Progress)

> CatalogJanitor should collect outdated RegionStates from the AM
> ---
>
> Key: HBASE-18025
> URL: https://issues.apache.org/jira/browse/HBASE-18025
> Project: HBase
>  Issue Type: Bug
>Reporter: Esteban Gutierrez
>Assignee: Esteban Gutierrez
> Attachments: HBASE-18025.001.patch
>
>
> I don't think this will matter on the long run for HBase 2, but at least in 
> branch-1 and the current master we keep in multiple places copies of the 
> region states in the master and this copies include information like the HRI. 
> A problem that we have observed is when region replicas are being used and 
> there is a split, the region replica from parent doesn't get collected from 
> the region states and when the balancer tries to assign the old parent region 
> replica, this will cause the RegionServer to create a new HRI with the 
> details of the parent causing an inconstancy (see HBASE-18024).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-15816) Provide client with ability to set priority on Operations


 [ 
https://issues.apache.org/jira/browse/HBASE-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

churro morales updated HBASE-15816:
---
Status: Patch Available  (was: Open)

> Provide client with ability to set priority on Operations 
> --
>
> Key: HBASE-15816
> URL: https://issues.apache.org/jira/browse/HBASE-15816
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: churro morales
>Assignee: churro morales
> Attachments: HBASE-15816.patch, HBASE-15816.v1.branch-1.patch, 
> HBASE-15816-v1.patch, HBASE-15816.v2.patch
>
>
> First round will just be to expose the ability to set priorities for client 
> operations.  For more background: 
> http://mail-archives.apache.org/mod_mbox/hbase-dev/201604.mbox/%3CCA+RK=_BG_o=q8HMptcP2WauAinmEsL+15f3YEJuz=qbpcya...@mail.gmail.com%3E
> Next step would be to remove AnnotationReadingPriorityFunction and have the 
> client send priorities explicitly.  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-15816) Provide client with ability to set priority on Operations


 [ 
https://issues.apache.org/jira/browse/HBASE-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

churro morales updated HBASE-15816:
---
Status: Open  (was: Patch Available)

> Provide client with ability to set priority on Operations 
> --
>
> Key: HBASE-15816
> URL: https://issues.apache.org/jira/browse/HBASE-15816
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: churro morales
>Assignee: churro morales
> Attachments: HBASE-15816.patch, HBASE-15816.v1.branch-1.patch, 
> HBASE-15816-v1.patch, HBASE-15816.v2.patch
>
>
> First round will just be to expose the ability to set priorities for client 
> operations.  For more background: 
> http://mail-archives.apache.org/mod_mbox/hbase-dev/201604.mbox/%3CCA+RK=_BG_o=q8HMptcP2WauAinmEsL+15f3YEJuz=qbpcya...@mail.gmail.com%3E
> Next step would be to remove AnnotationReadingPriorityFunction and have the 
> client send priorities explicitly.  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-15816) Provide client with ability to set priority on Operations


 [ 
https://issues.apache.org/jira/browse/HBASE-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

churro morales updated HBASE-15816:
---
Attachment: HBASE-15816.v1.branch-1.patch

HBASE-15816.v2 looks like it applies cleanly to master and branch-2.  Attached 
is the branch-1 patch.  I think that should cover it.  Branch-1 patch is 
slightly different than the other patch as the controller logic is still inside 
HTable. 

> Provide client with ability to set priority on Operations 
> --
>
> Key: HBASE-15816
> URL: https://issues.apache.org/jira/browse/HBASE-15816
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: churro morales
>Assignee: churro morales
> Attachments: HBASE-15816.patch, HBASE-15816.v1.branch-1.patch, 
> HBASE-15816-v1.patch, HBASE-15816.v2.patch
>
>
> First round will just be to expose the ability to set priorities for client 
> operations.  For more background: 
> http://mail-archives.apache.org/mod_mbox/hbase-dev/201604.mbox/%3CCA+RK=_BG_o=q8HMptcP2WauAinmEsL+15f3YEJuz=qbpcya...@mail.gmail.com%3E
> Next step would be to remove AnnotationReadingPriorityFunction and have the 
> client send priorities explicitly.  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18430) Typo in "contributing to documentation" page


 [ 
https://issues.apache.org/jira/browse/HBASE-18430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Coral updated HBASE-18430:
--
Summary: Typo in "contributing to documentation" page  (was: Typo in 
"contributing to documentation)

> Typo in "contributing to documentation" page
> 
>
> Key: HBASE-18430
> URL: https://issues.apache.org/jira/browse/HBASE-18430
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: Coral
>Assignee: Coral
>
> Be sure to set the component to Documentation in addition any other involved 
> components.
> Missing "to." Should be: 
> Be sure to set the component to Documentation in addition *to* any other 
> involved components.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18430) Typo in "contributing to documentation" page


 [ 
https://issues.apache.org/jira/browse/HBASE-18430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Coral updated HBASE-18430:
--
Description: 
Be sure to set the component to Documentation in addition any other involved 
components.

Missing "to." Should be: 

Be sure to set the component to Documentation in addition *to* any other 
involved components.

http://hbase.apache.org/book.html#appendix_contributing_to_documentation

  was:
Be sure to set the component to Documentation in addition any other involved 
components.

Missing "to." Should be: 

Be sure to set the component to Documentation in addition *to* any other 
involved components.


> Typo in "contributing to documentation" page
> 
>
> Key: HBASE-18430
> URL: https://issues.apache.org/jira/browse/HBASE-18430
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: Coral
>Assignee: Coral
>
> Be sure to set the component to Documentation in addition any other involved 
> components.
> Missing "to." Should be: 
> Be sure to set the component to Documentation in addition *to* any other 
> involved components.
> http://hbase.apache.org/book.html#appendix_contributing_to_documentation



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18430) Typo in "contributing to documentation" page


[ 
https://issues.apache.org/jira/browse/HBASE-18430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096722#comment-16096722
 ] 

Coral commented on HBASE-18430:
---

Hi [~misty]! I'm taking over the HBase doc at Cloudera. I've heard many good 
things about you and your doc is great, so thank you.  :)

I'm just fixing this little one while I figure out the process of contributing 
upstream. I'll definitely take you up on your offer for review in the future. I 
appreciate it! 

> Typo in "contributing to documentation" page
> 
>
> Key: HBASE-18430
> URL: https://issues.apache.org/jira/browse/HBASE-18430
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: Coral
>Assignee: Coral
>
> Be sure to set the component to Documentation in addition any other involved 
> components.
> Missing "to." Should be: 
> Be sure to set the component to Documentation in addition *to* any other 
> involved components.
> http://hbase.apache.org/book.html#appendix_contributing_to_documentation



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18430) Typo in "contributing to documentation


[ 
https://issues.apache.org/jira/browse/HBASE-18430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096697#comment-16096697
 ] 

Misty Stanley-Jones commented on HBASE-18430:
-

Hi [~coral.w]! Please shout out if you need any help on this or when you'd like 
a review. Thanks!

> Typo in "contributing to documentation
> --
>
> Key: HBASE-18430
> URL: https://issues.apache.org/jira/browse/HBASE-18430
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: Coral
>Assignee: Coral
>
> Be sure to set the component to Documentation in addition any other involved 
> components.
> Missing "to." Should be: 
> Be sure to set the component to Documentation in addition *to* any other 
> involved components.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17908) Upgrade guava


[ 
https://issues.apache.org/jira/browse/HBASE-17908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096695#comment-16096695
 ] 

Hudson commented on HBASE-17908:


FAILURE: Integrated in Jenkins build HBase-2.0 #212 (See 
[https://builds.apache.org/job/HBase-2.0/212/])
HBASE-17908 Upgrade guava (stack: rev d5c6e1101614b806eafd32da07eeeb16c299c1a1)
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java
* (edit) 
hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java
* (edit) 
hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEdit.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/BaseReplicationEndpoint.java
* (edit) 
hbase-common/src/main/java/org/apache/hadoop/hbase/io/crypto/Context.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WAL.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFilter.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestRpcServerSlowConnectionSetup.java
* (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java
* (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/util/Threads.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat2.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/QuotaCache.java
* (edit) 
hbase-common/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKConfig.java
* (edit) hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/SyncTable.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionConfiguration.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestWithDisabledAuthorization.java
* (edit) 
hbase-prefix-tree/src/test/java/org/apache/hadoop/hbase/codec/prefixtree/column/TestColumnBuilder.java
* (edit) 
hbase-metrics/src/test/java/org/apache/hadoop/hbase/metrics/impl/TestRefCountingMap.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/monitoring/TaskMonitor.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestBackupDelete.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/util/JvmPauseMonitor.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ServerNonceManager.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupAdminImpl.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RegionServerRpcQuotaManager.java
* (edit) 
hbase-common/src/test/java/org/apache/hadoop/hbase/codec/TestCellCodec.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* (edit) 
hbase-prefix-tree/src/test/java/org/apache/hadoop/hbase/codec/prefixtree/column/TestColumnData.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/RegionLocationFinder.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestLogsCleaner.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotDescriptionUtils.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestBackupShowHistory.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestRpcHandlerException.java
* (edit) 
hbase-common/src/main/java/org/apache/hadoop/hbase/io/crypto/aes/CommonsCryptoAES.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CombinedBlockCache.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RowProcessor.java
* (edit) 
hbase-client/src/main/java/org/apache/hadoop/hbase/filter/TimestampsFilter.java
* (edit) 
hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/util/MetricSampleQuantiles.java
* (edit) 
hbase-common/src/main/java/org/apache/hadoop/hbase/io/hadoopbackport/ThrottledInputStream.java
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/io/asyncfs/AsyncFSOutputHelper.java
* (edit)

[jira] [Updated] (HBASE-18024) HRegion#initializeRegionInternals should not re-create .hregioninfo file when the region directory no longer exists


 [ 
https://issues.apache.org/jira/browse/HBASE-18024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Esteban Gutierrez updated HBASE-18024:
--
Attachment: HBASE-18024.001.patch

> HRegion#initializeRegionInternals should not re-create .hregioninfo file when 
> the region directory no longer exists
> ---
>
> Key: HBASE-18024
> URL: https://issues.apache.org/jira/browse/HBASE-18024
> Project: HBase
>  Issue Type: Bug
>  Components: Region Assignment, regionserver
>Affects Versions: 2.0.0, 1.4.0, 1.3.1, 1.2.5
>Reporter: Esteban Gutierrez
>Assignee: Esteban Gutierrez
> Attachments: HBASE-18024.001.patch
>
>
> When a RegionSever attempts to open a region, during initialization the RS 
> tries to open the {{/data///.hregioninfo}} 
> file, however if the {{.hregioninfofile}} doesn't exist, the RegionServer 
> will create a new one on {{HRegionFileSystem#checkRegionInfoOnFilesystem}}. A 
> side effect of that tools like hbck will incorrectly assume an inconsistency 
> due the presence of this new {{.hregioninfofile}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18024) HRegion#initializeRegionInternals should not re-create .hregioninfo file when the region directory no longer exists


 [ 
https://issues.apache.org/jira/browse/HBASE-18024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Esteban Gutierrez updated HBASE-18024:
--
Affects Version/s: 2.0.0
   Status: Patch Available  (was: In Progress)

> HRegion#initializeRegionInternals should not re-create .hregioninfo file when 
> the region directory no longer exists
> ---
>
> Key: HBASE-18024
> URL: https://issues.apache.org/jira/browse/HBASE-18024
> Project: HBase
>  Issue Type: Bug
>  Components: Region Assignment, regionserver
>Affects Versions: 1.2.5, 1.3.1, 2.0.0, 1.4.0
>Reporter: Esteban Gutierrez
>Assignee: Esteban Gutierrez
> Attachments: HBASE-18024.001.patch
>
>
> When a RegionSever attempts to open a region, during initialization the RS 
> tries to open the {{/data///.hregioninfo}} 
> file, however if the {{.hregioninfofile}} doesn't exist, the RegionServer 
> will create a new one on {{HRegionFileSystem#checkRegionInfoOnFilesystem}}. A 
> side effect of that tools like hbck will incorrectly assume an inconsistency 
> due the presence of this new {{.hregioninfofile}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18430) Typo in "contributing to documentation

Coral created HBASE-18430:
-

 Summary: Typo in "contributing to documentation
 Key: HBASE-18430
 URL: https://issues.apache.org/jira/browse/HBASE-18430
 Project: HBase
  Issue Type: Bug
  Components: documentation
Reporter: Coral
Assignee: Coral


Be sure to set the component to Documentation in addition any other involved 
components.

Missing "to." Should be: 

Be sure to set the component to Documentation in addition *to* any other 
involved components.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18389) Remove byte[] from formal parameter of sizeOf() of ClassSize, ClassSize.MemoryLayout and ClassSize.UnsafeLayout


 [ 
https://issues.apache.org/jira/browse/HBASE-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-18389:
---
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

[~water] Thanks for the patch.

> Remove byte[] from formal parameter of sizeOf() of ClassSize, 
> ClassSize.MemoryLayout and ClassSize.UnsafeLayout
> ---
>
> Key: HBASE-18389
> URL: https://issues.apache.org/jira/browse/HBASE-18389
> Project: HBase
>  Issue Type: Improvement
>  Components: util
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18389.master.000.patch, 
> HBASE-18389.master.001.patch, HBASE-18389.master.002.patch, 
> HBASE-18389.master.003.patch
>
>
> In ClassSize class and its internal static class, sizeOf() function has 2 
> formal parameters: byte[] b and int len. But the function's internal logic 
> does not use or refer to byte[] b. Could be removed.
> {code:title=hbase-common/src/main/java/org/apache/hadoop/hbase/util/ClassSize.java|borderStyle=solid}
> // Class of ClassSize
> public static long sizeOf(byte[] b, int len) {
>   return memoryLayout.sizeOf(b, len);
> }
> // Class of ClassSize.MemoryLayout
> long sizeOf(byte[] b, int len) {
>   return align(arrayHeaderSize() + len);
> }
> // Class of ClassSize.UnsafeLayout
> long sizeOf(byte[] b, int len) {
>   return align(arrayHeaderSize() + len * 
> UnsafeAccess.theUnsafe.ARRAY_BYTE_INDEX_SCALE);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18389) Remove byte[] from formal parameter of sizeOf() of ClassSize, ClassSize.MemoryLayout and ClassSize.UnsafeLayout


 [ 
https://issues.apache.org/jira/browse/HBASE-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-18389:
---
Fix Version/s: 2.0.0-alpha-2
   3.0.0

> Remove byte[] from formal parameter of sizeOf() of ClassSize, 
> ClassSize.MemoryLayout and ClassSize.UnsafeLayout
> ---
>
> Key: HBASE-18389
> URL: https://issues.apache.org/jira/browse/HBASE-18389
> Project: HBase
>  Issue Type: Bug
>  Components: util
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18389.master.000.patch, 
> HBASE-18389.master.001.patch, HBASE-18389.master.002.patch, 
> HBASE-18389.master.003.patch
>
>
> In ClassSize class and its internal static class, sizeOf() function has 2 
> formal parameters: byte[] b and int len. But the function's internal logic 
> does not use or refer to byte[] b. Could be removed.
> {code:title=hbase-common/src/main/java/org/apache/hadoop/hbase/util/ClassSize.java|borderStyle=solid}
> // Class of ClassSize
> public static long sizeOf(byte[] b, int len) {
>   return memoryLayout.sizeOf(b, len);
> }
> // Class of ClassSize.MemoryLayout
> long sizeOf(byte[] b, int len) {
>   return align(arrayHeaderSize() + len);
> }
> // Class of ClassSize.UnsafeLayout
> long sizeOf(byte[] b, int len) {
>   return align(arrayHeaderSize() + len * 
> UnsafeAccess.theUnsafe.ARRAY_BYTE_INDEX_SCALE);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18389) Remove byte[] from formal parameter of sizeOf() of ClassSize, ClassSize.MemoryLayout and ClassSize.UnsafeLayout


 [ 
https://issues.apache.org/jira/browse/HBASE-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-18389:
---
Issue Type: Improvement  (was: Bug)

> Remove byte[] from formal parameter of sizeOf() of ClassSize, 
> ClassSize.MemoryLayout and ClassSize.UnsafeLayout
> ---
>
> Key: HBASE-18389
> URL: https://issues.apache.org/jira/browse/HBASE-18389
> Project: HBase
>  Issue Type: Improvement
>  Components: util
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18389.master.000.patch, 
> HBASE-18389.master.001.patch, HBASE-18389.master.002.patch, 
> HBASE-18389.master.003.patch
>
>
> In ClassSize class and its internal static class, sizeOf() function has 2 
> formal parameters: byte[] b and int len. But the function's internal logic 
> does not use or refer to byte[] b. Could be removed.
> {code:title=hbase-common/src/main/java/org/apache/hadoop/hbase/util/ClassSize.java|borderStyle=solid}
> // Class of ClassSize
> public static long sizeOf(byte[] b, int len) {
>   return memoryLayout.sizeOf(b, len);
> }
> // Class of ClassSize.MemoryLayout
> long sizeOf(byte[] b, int len) {
>   return align(arrayHeaderSize() + len);
> }
> // Class of ClassSize.UnsafeLayout
> long sizeOf(byte[] b, int len) {
>   return align(arrayHeaderSize() + len * 
> UnsafeAccess.theUnsafe.ARRAY_BYTE_INDEX_SCALE);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18404) Small typo on ACID documentation page


[ 
https://issues.apache.org/jira/browse/HBASE-18404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096657#comment-16096657
 ] 

Sean Busbey commented on HBASE-18404:
-

Not your fault at all. While adding the comment about how the -1 is a false 
negative is helpful, I think this should have been handled properly without 
YETUS-527 (since the change is in site files).

> Small typo on ACID documentation page
> -
>
> Key: HBASE-18404
> URL: https://issues.apache.org/jira/browse/HBASE-18404
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Affects Versions: 1.3.1
>Reporter: Michael Crutcher
>Assignee: Coral
>Priority: Trivial
> Fix For: 3.0.0
>
> Attachments: HBASE-18404.patch
>
>
> I noticed a couple of occurrences of the "word" wholely on the ACID semantics 
> doc page (https://hbase.apache.org/acid-semantics.html)
> This should be "wholly".



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18389) Remove byte[] from formal parameter of sizeOf() of ClassSize, ClassSize.MemoryLayout and ClassSize.UnsafeLayout


[ 
https://issues.apache.org/jira/browse/HBASE-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096654#comment-16096654
 ] 

Chia-Ping Tsai commented on HBASE-18389:


Will commit it shortly.

> Remove byte[] from formal parameter of sizeOf() of ClassSize, 
> ClassSize.MemoryLayout and ClassSize.UnsafeLayout
> ---
>
> Key: HBASE-18389
> URL: https://issues.apache.org/jira/browse/HBASE-18389
> Project: HBase
>  Issue Type: Bug
>  Components: util
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-18389.master.000.patch, 
> HBASE-18389.master.001.patch, HBASE-18389.master.002.patch, 
> HBASE-18389.master.003.patch
>
>
> In ClassSize class and its internal static class, sizeOf() function has 2 
> formal parameters: byte[] b and int len. But the function's internal logic 
> does not use or refer to byte[] b. Could be removed.
> {code:title=hbase-common/src/main/java/org/apache/hadoop/hbase/util/ClassSize.java|borderStyle=solid}
> // Class of ClassSize
> public static long sizeOf(byte[] b, int len) {
>   return memoryLayout.sizeOf(b, len);
> }
> // Class of ClassSize.MemoryLayout
> long sizeOf(byte[] b, int len) {
>   return align(arrayHeaderSize() + len);
> }
> // Class of ClassSize.UnsafeLayout
> long sizeOf(byte[] b, int len) {
>   return align(arrayHeaderSize() + len * 
> UnsafeAccess.theUnsafe.ARRAY_BYTE_INDEX_SCALE);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18415) Fix client.TestClientScannerRPCTimeout


 [ 
https://issues.apache.org/jira/browse/HBASE-18415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-18415:
---
Priority: Major  (was: Minor)

> Fix client.TestClientScannerRPCTimeout
> --
>
> Key: HBASE-18415
> URL: https://issues.apache.org/jira/browse/HBASE-18415
> Project: HBase
>  Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>
> {quote}
> Running org.apache.hadoop.hbase.client.TestClientScannerRPCTimeout
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 10.046 sec 
> <<< FAILURE! - in org.apache.hadoop.hbase.client.TestClientScannerRPCTimeout
> testScannerNextRPCTimesout(org.apache.hadoop.hbase.client.TestClientScannerRPCTimeout)
>   Time elapsed: 4.186 sec  <<< ERROR!
> org.apache.hadoop.hbase.TableExistsException: testScannerNextRPCTimesout
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
> Method)
> at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at 
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
> at 
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
> at 
> org.apache.hadoop.hbase.util.ForeignExceptionUtil.toIOException(ForeignExceptionUtil.java:45)
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin$ProcedureFuture.convertResult(HBaseAdmin.java:4773)
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin$ProcedureFuture.waitProcedureResult(HBaseAdmin.java:4731)
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin$ProcedureFuture.get(HBaseAdmin.java:4664)
> at 
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:678)
> at 
> org.apache.hadoop.hbase.HBaseTestingUtility.createTable(HBaseTestingUtility.java:1500)
> at 
> org.apache.hadoop.hbase.HBaseTestingUtility.createTable(HBaseTestingUtility.java:1547)
> at 
> org.apache.hadoop.hbase.HBaseTestingUtility.createTable(HBaseTestingUtility.java:1438)
> at 
> org.apache.hadoop.hbase.HBaseTestingUtility.createTable(HBaseTestingUtility.java:1414)
> at 
> org.apache.hadoop.hbase.HBaseTestingUtility.createTable(HBaseTestingUtility.java:1370)
> at 
> org.apache.hadoop.hbase.client.TestClientScannerRPCTimeout.testScannerNextRPCTimesout(TestClientScannerRPCTimeout.java:92)
> Caused by: org.apache.hadoop.ipc.RemoteException: testScannerNextRPCTimesout
> at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.prepareCreate(CreateTableProcedure.java:286)
> at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:107)
> at 
> org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:59)
> at 
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:139)
> at 
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:506)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1152)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:940)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:893)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:76)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java:478)
> {quote}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18415) Fix client.TestClientScannerRPCTimeout