[jira] [Commented] (HBASE-16074) ITBLL fails, reports lost big or tiny families

[Anoop Sam John (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367260#comment-15367260
 ] 

Anoop Sam John commented on HBASE-16074:


Excellent detective work boss..  Oh combination of clean looking 2 patches made 
a big issue..  So we should be fixing it in such a way to correctly exclude the 
fake keys?

> ITBLL fails, reports lost big or tiny families
> --
>
> Key: HBASE-16074
> URL: https://issues.apache.org/jira/browse/HBASE-16074
> Project: HBase
>  Issue Type: Bug
>  Components: integration tests
>Affects Versions: 1.3.0, 0.98.20
>Reporter: Mikhail Antonov
>Assignee: Mikhail Antonov
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21
>
> Attachments: 16074.test.branch-1.3.patch, 16074.test.patch, 
> HBASE-16074.branch-1.3.001.patch, HBASE-16074.branch-1.3.002.patch, 
> HBASE-16074.branch-1.3.003.patch, HBASE-16074.branch-1.3.003.patch, 
> changes_to_stress_ITBLL.patch, changes_to_stress_ITBLL__a_bit_relaxed_.patch, 
> itbll log with failure, itbll log with success
>
>
> Underlying MR jobs succeed but I'm seeing the following in the logs (mid-size 
> distributed test cluster):
> ERROR test.IntegrationTestBigLinkedList$Verify: Found nodes which lost big or 
> tiny families, count=164
> I do not know exactly yet whether it's a bug, a test issue or env setup 
> issue, but need figure it out. Opening this to raise awareness and see if 
> someone saw that recently.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16176) Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of contention when many threads reading a StoreFile

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367259#comment-15367259
 ] 

stack commented on HBASE-16176:
---

Let me run this patch on cluster overnight in loop to see if fixes scan issue.

> Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of 
> contention when many threads reading a StoreFile
> --
>
> Key: HBASE-16176
> URL: https://issues.apache.org/jira/browse/HBASE-16176
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21
>
> Attachments: HBASE-16176.branch-1.3.001.patch, 
> HBASE-16176.branch-1.3.002.patch, HBASE-16176.branch-1.3.002.patch, 
> HBASE-16176.branch-1.3.003.patch
>
>
> Debugging the parent issue, came up with some improvements on old HBASE-15650 
> "Remove TimeRangeTracker as point of contention when many threads reading a 
> StoreFile". Lets get them in. Here are the changes:
> {code}
>   6  Change HFile Writer constructor so we pass in the TimeRangeTracker, 
> if one,
>   7  on construction rather than set later (the flag and reference were 
> not
>   8  volatile so could have made for issues in concurrent case) 2. Make 
> sure the
>   9  construction of a TimeRange from a TimeRangeTracer on open of an 
> HFile Reader
>  10  never makes a bad minimum value, one that would preclude us reading 
> any
>  11  values from a file (add a log and set min to 0)
>  12 M hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
>  13  Call through to next constructor (if minStamp was 0, we'd skip 
> setting allTime=true)
>  14 M 
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
>  15  Add constructor override that takes a TimeRangeTracker (set when 
> flushing but
>  16  not when compacting)
>  17 M 
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
>  18  Add override creating an HFile in tmp that takes a TimeRangeTracker
>  19 M 
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
>  20  Add override for HFile Writer that takes a TimeRangeTracker
>  21  Take it on construction instead of having it passed by a setter 
> later (flags
>  22  and reference set by the setter were not volatile... could have been 
> prob
>  23  in concurrent case)
>  24 M 
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
>  25  Log WARN if bad initial TimeRange value (and then 'fix' it)
>  26 M 
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java
>  27  A few tests to prove serialization works as expected and that we'll 
> get a bad min if
>  28  not constructed properly.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16176) Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of contention when many threads reading a StoreFile

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367257#comment-15367257
 ] 

stack commented on HBASE-16176:
---

This last patch, 003, includes fix for the parent scan problem. Here is the 
commit message:

{code}
 TimeRangeTracker as point of contention when many threads reading a StoreFile
 Fixes HBASE-16074 ITBLL fails, reports lost big or tiny families broken
 scanning because of a side effect of a clean up in  HBASE-15650 to make
 TimeRange construction consistent exposed a latent issue in
 TimeRange#compare. See HBASE-16074 for more detail.
Also change HFile Writer constructor so we pass in the TimeRangeTracker, if one,
on construction rather than set later (the flag and reference were not volatile
so could have made for issues in concurrent case). And make sure the 
construction
of a TimeRange from a TimeRangeTracer on open of an HFile Reader never makes a
bad minimum value, one that would preclude us reading any values from a file
(set min to 0)
M hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
 Call through to next constructor (if minStamp was 0, we'd skip setting
 allTime=true). Add asserts that timestamps are not < 0 cos it messes
 us up if they are (we already were checking for < 0 on construction but
 assert passed in timestamps are not < 0).
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
 Add constructor override that takes a TimeRangeTracker (set when flushing
 but not when compacting)
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
 Add override creating an HFile in tmp that takes a TimeRangeTracker
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
 Add override for HFile Writer that takes a TimeRangeTracker Take it on
 construction instead of having it passed by a setter later (flags and
 reference set by the setter were not volatile... could have been prob
 in concurrent case)
M 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
 Log WARN if bad initial TimeRange value (and then 'fix' it)
M 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java
 A few tests to prove serialization works as expected and that we'll get a bad 
min if not constructed properly.
M 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
 Handle OLDEST_TIMESTAMP explictly. Don't expect TimeRange to do it.
M 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestQueryMatcher.java
 Refactor from junit3 to junit4 and add test for this weird case.
{code}

> Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of 
> contention when many threads reading a StoreFile
> --
>
> Key: HBASE-16176
> URL: https://issues.apache.org/jira/browse/HBASE-16176
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21
>
> Attachments: HBASE-16176.branch-1.3.001.patch, 
> HBASE-16176.branch-1.3.002.patch, HBASE-16176.branch-1.3.002.patch, 
> HBASE-16176.branch-1.3.003.patch
>
>
> Debugging the parent issue, came up with some improvements on old HBASE-15650 
> "Remove TimeRangeTracker as point of contention when many threads reading a 
> StoreFile". Lets get them in. Here are the changes:
> {code}
>   6  Change HFile Writer constructor so we pass in the TimeRangeTracker, 
> if one,
>   7  on construction rather than set later (the flag and reference were 
> not
>   8  volatile so could have made for issues in concurrent case) 2. Make 
> sure the
>   9  construction of a TimeRange from a TimeRangeTracer on open of an 
> HFile Reader
>  10  never makes a bad minimum value, one that would preclude us reading 
> any
>  11  values from a file (add a log and set min to 0)
>  12 M hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
>  13  Call through to next constructor (if minStamp was 0, we'd skip 
> setting allTime=true)
>  14 M 
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
>  15  Add constructor override that takes a TimeRangeTracker (set when 
> flushing but
>  16  not when compacting)
>  17 M 
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
>  18  Add override creating an HFile in tmp that takes a TimeRangeTracker
>  19 M 
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
>  20  Add override for HFile Writer that takes a TimeRangeTracker
>  21  Take it on construction instead of having it passed by a setter 
> 

[jira] [Updated] (HBASE-16176) Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of contention when many threads reading a StoreFile

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-16176:
--
Attachment: HBASE-16176.branch-1.3.003.patch

> Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of 
> contention when many threads reading a StoreFile
> --
>
> Key: HBASE-16176
> URL: https://issues.apache.org/jira/browse/HBASE-16176
> Project: HBase
>  Issue Type: Sub-task
>  Components: Performance
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21
>
> Attachments: HBASE-16176.branch-1.3.001.patch, 
> HBASE-16176.branch-1.3.002.patch, HBASE-16176.branch-1.3.002.patch, 
> HBASE-16176.branch-1.3.003.patch
>
>
> Debugging the parent issue, came up with some improvements on old HBASE-15650 
> "Remove TimeRangeTracker as point of contention when many threads reading a 
> StoreFile". Lets get them in. Here are the changes:
> {code}
>   6  Change HFile Writer constructor so we pass in the TimeRangeTracker, 
> if one,
>   7  on construction rather than set later (the flag and reference were 
> not
>   8  volatile so could have made for issues in concurrent case) 2. Make 
> sure the
>   9  construction of a TimeRange from a TimeRangeTracer on open of an 
> HFile Reader
>  10  never makes a bad minimum value, one that would preclude us reading 
> any
>  11  values from a file (add a log and set min to 0)
>  12 M hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
>  13  Call through to next constructor (if minStamp was 0, we'd skip 
> setting allTime=true)
>  14 M 
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
>  15  Add constructor override that takes a TimeRangeTracker (set when 
> flushing but
>  16  not when compacting)
>  17 M 
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
>  18  Add override creating an HFile in tmp that takes a TimeRangeTracker
>  19 M 
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
>  20  Add override for HFile Writer that takes a TimeRangeTracker
>  21  Take it on construction instead of having it passed by a setter 
> later (flags
>  22  and reference set by the setter were not volatile... could have been 
> prob
>  23  in concurrent case)
>  24 M 
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
>  25  Log WARN if bad initial TimeRange value (and then 'fix' it)
>  26 M 
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java
>  27  A few tests to prove serialization works as expected and that we'll 
> get a bad min if
>  28  not constructed properly.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16055) PutSortReducer loses any Visibility/acl attribute set on the Puts

[Andrew Purtell (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367252#comment-15367252
 ] 

Andrew Purtell commented on HBASE-16055:


Sure [~ram_krish] but please check that all is well and relevant unit tests 
pass first. 

> PutSortReducer loses any Visibility/acl attribute set on the Puts 
> --
>
> Key: HBASE-16055
> URL: https://issues.apache.org/jira/browse/HBASE-16055
> Project: HBase
>  Issue Type: Bug
>  Components: security
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 2.0.0, 1.0.4, 1.4.0
>
> Attachments: HBASE-16055_1.patch, HBASE-16055_2.patch
>
>
> Based on a user discussion, I think as the user pointed out rightly, when a 
> PutSortReducer is used any visibility attribute or external attribute set on 
> the Put will be lost as we create KVs out of the cells in the puts whereas 
> the ACL and visibility are all set as Attributes. 
> In TextSortReducer we tend to read the information we tend to read the 
> information from the parsed line but here in PutSortReducer we don't do it. I 
> think this problem should be in all the existing versions where we support 
> Tags. Correct me if am wrong here. 
> [~anoop.hbase], [~andrew.purt...@gmail.com]?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16055) PutSortReducer loses any Visibility/acl attribute set on the Puts

[ramkrishna.s.vasudevan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367251#comment-15367251
 ] 

ramkrishna.s.vasudevan commented on HBASE-16055:


[~apurtell]
Sorry about the commit. Let me take create a new patch. I think the problem was 
that the patch did not resolve the conflicts in the test file. I actually 
resolved it but did not update in the patch. I have corrected that. Can i 
recommit that patch?

> PutSortReducer loses any Visibility/acl attribute set on the Puts 
> --
>
> Key: HBASE-16055
> URL: https://issues.apache.org/jira/browse/HBASE-16055
> Project: HBase
>  Issue Type: Bug
>  Components: security
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 2.0.0, 1.0.4, 1.4.0
>
> Attachments: HBASE-16055_1.patch, HBASE-16055_2.patch
>
>
> Based on a user discussion, I think as the user pointed out rightly, when a 
> PutSortReducer is used any visibility attribute or external attribute set on 
> the Put will be lost as we create KVs out of the cells in the puts whereas 
> the ACL and visibility are all set as Attributes. 
> In TextSortReducer we tend to read the information we tend to read the 
> information from the parsed line but here in PutSortReducer we don't do it. I 
> think this problem should be in all the existing versions where we support 
> Tags. Correct me if am wrong here. 
> [~anoop.hbase], [~andrew.purt...@gmail.com]?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16074) ITBLL fails, reports lost big or tiny families

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367240#comment-15367240
 ] 

stack commented on HBASE-16074:
---

What is going on is that 'fake keys' -- 'last on row' keys made to prime 
Scanners -- are getting included in the response.

A combination of two seemingly unrelated commits introduced this rare issue 
found by @mikhails' 1.3 ITBLL runs.

The first commit was a small innocent looking optimization where if the 
TimeRange was essentially all time, then we'd just save on doing any compares 
at all and just return a 0 to indicate include:

Author: anoopsjohn 
Date:   Tue Apr 14 11:39:06 2015 +0530

HBASE-13447 Bypass logic in TimeRange.compare.

The next commit was HBASE-15650 whose thrust was converting the readpath to use 
an immutable TimeRange rather than the expensive mutable TimeRangeTracker. This 
conversion was NOT responsible for the bug found here. This was distracting. 
Rather it was a side effect of some cleanup in TimeRange done as part of 
HBASE-15650. TimeRange had a bunch of constructors and each was doing 
initialization in a slightly different way. Most were setting a flag 'allTime' 
to indicate a TimeRange that was inclusive of all times. The cleanup made 
Constructors call through and be consistent setting allTime. This made allTime 
true when it was supposed to be, everywhere, but where it had not been set on 
certain code paths before.

Scanning, at certain points, we'll shove fake keys into the heaps to start the 
Scan going or to move to next columns/rows. One such fake key is the last on 
row set in StoreScanner when we want to go to the next row. This fake key has a 
timestamp of HConstants.OLDEST_TIMESTAMP; i.e. Long.MIN_TIMESTAMP, i.e. < 0. 
TimeRange does 0 => Long.MAX_LONG. HBASE-15650 making allTime set where it 
should be made it so the optimization done in HBASE-13447 triggered where it 
didn't previous return a 0 -- because when allTime, 0 indicates inclusion -- 
rather than return the expected -1 that the ScanQueryMatcher#match was relying 
upon as means of excluding fake keys.

Let me make the fix as part of the work over in HBASE-16176. This should take 
care of the TRT issue.


> ITBLL fails, reports lost big or tiny families
> --
>
> Key: HBASE-16074
> URL: https://issues.apache.org/jira/browse/HBASE-16074
> Project: HBase
>  Issue Type: Bug
>  Components: integration tests
>Affects Versions: 1.3.0, 0.98.20
>Reporter: Mikhail Antonov
>Assignee: Mikhail Antonov
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21
>
> Attachments: 16074.test.branch-1.3.patch, 16074.test.patch, 
> HBASE-16074.branch-1.3.001.patch, HBASE-16074.branch-1.3.002.patch, 
> HBASE-16074.branch-1.3.003.patch, HBASE-16074.branch-1.3.003.patch, 
> changes_to_stress_ITBLL.patch, changes_to_stress_ITBLL__a_bit_relaxed_.patch, 
> itbll log with failure, itbll log with success
>
>
> Underlying MR jobs succeed but I'm seeing the following in the logs (mid-size 
> distributed test cluster):
> ERROR test.IntegrationTestBigLinkedList$Verify: Found nodes which lost big or 
> tiny families, count=164
> I do not know exactly yet whether it's a bug, a test issue or env setup 
> issue, but need figure it out. Opening this to raise awareness and see if 
> someone saw that recently.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16160) Get the UnsupportedOperationException when using delegation token with encryption

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367213#comment-15367213
 ] 

Hadoop QA commented on HBASE-16160:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
1s {color} | {color:green} The patch appears to include 3 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 
1s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s 
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s 
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
56s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s 
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s 
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 45s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 33s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
53s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
26m 13s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 91m 26s {color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
19s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 133m 9s {color} 
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.replication.TestReplicationSmallTests |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12816743/HBASE-16160.002.patch 
|
| JIRA Issue | HBASE-16160 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 17edca6 |
| Default Java | 1.7.0_80 |
| Multi-JDK versions |  /home/jenkins/tools/java/jdk1.8.0:1.8.0 
/home/jenkins/jenkins-slave/tools/hudson.model.JDK/JDK_1.7_latest_:1.7.0_80 |
| findbugs | 

[jira] [Commented] (HBASE-16195) Should not add chunk into chunkQueue if not using chunk pool in HeapMemStoreLAB

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367208#comment-15367208
 ] 

Ted Yu commented on HBASE-16195:


{code}
+  int queueLength = conf.getInt(CHUNK_QUEUE_LENGTH, (int) 
(memstoreFlushSize / chunkSize));
{code}
What if the value for CHUNK_QUEUE_LENGTH deviates from (int) (memstoreFlushSize 
/ chunkSize) by a wide margin ?

> Should not add chunk into chunkQueue if not using chunk pool in 
> HeapMemStoreLAB
> ---
>
> Key: HBASE-16195
> URL: https://issues.apache.org/jira/browse/HBASE-16195
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-16195.patch
>
>
> Problem description and analysis please refer to HBASE-16193



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16195) Should not add chunk into chunkQueue if not using chunk pool in HeapMemStoreLAB

[Yu Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yu Li updated HBASE-16195:
--
Attachment: HBASE-16195.patch

The initial patch:
1. Change to only initialize and use chunkQueue when chunkPool is in use.
2. Limit chunkQueue length, max allowed memory taken by chunk queue is memstore 
flush size by default. Introduced property 
{{hbase.hregion.memstore.mslab.chunkqueue.length}} to set it.
3. Remove the chunk from chunkQueue after retiring it. Notice that chunkPool 
might be full in which case the {{putbackChunks}} call won't be able to clear 
chunkQueue, so we should remove the chunk explicitly.

> Should not add chunk into chunkQueue if not using chunk pool in 
> HeapMemStoreLAB
> ---
>
> Key: HBASE-16195
> URL: https://issues.apache.org/jira/browse/HBASE-16195
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-16195.patch
>
>
> Problem description and analysis please refer to HBASE-16193



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16190) IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must be positive

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367171#comment-15367171
 ] 

Hudson commented on HBASE-16190:


SUCCESS: Integrated in HBase-1.2-IT #547 (See 
[https://builds.apache.org/job/HBase-1.2-IT/547/])
HBASE-16190 IntegrationTestDDLMasterFailover failed with (enis: rev 
075f33a06ab27f35fc9d4e46b7bf2220d47992d1)
* 
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestDDLMasterFailover.java


> IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must 
> be positive
> -
>
> Key: HBASE-16190
> URL: https://issues.apache.org/jira/browse/HBASE-16190
> Project: HBase
>  Issue Type: Bug
>Reporter: Romil Choksi
>Assignee: Romil Choksi
>Priority: Minor
>  Labels: integration-test
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 1.2.3
>
> Attachments: 16190.v1.txt, HBASE-16190.patch
>
>
> IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must 
> be positive
> {code}
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:21,661 INFO  [main] hbase.IntegrationTestDDLMasterFailover: Runtime is 
> up
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:22,026 ERROR [main] hbase.IntegrationTestDDLMasterFailover: Found 
> exception in thread: Thread-11
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:21,384 INFO  [Thread-16] hbase.IntegrationTestDDLMasterFailover: 
> Performing Action: CREATE_TABLE
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:22,027 INFO  [Thread-16] hbase.IntegrationTestDDLMasterFailover: 
> Thread-16 stopped
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:20,506 INFO  [Thread-30] hbase.IntegrationTestDDLMasterFailover: 
> Performing Action: ADD_COLUMNFAMILY
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|java.lang.IllegalArgumentException:
>  n must be positive
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|at 
> java.util.Random.nextInt(Random.java:300)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.JVMRandom.nextInt(JVMRandom.java:118)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.RandomUtils.nextInt(RandomUtils.java:88)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.RandomUtils.nextInt(RandomUtils.java:74)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$TableAction.selectTable(IntegrationTestDDLMasterFailover.java:212)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$AddColumnFamilyAction.perform(IntegrationTestDDLMasterFailover.java:421)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$Worker.run(IntegrationTestDDLMasterFailover.java:695)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16190) IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must be positive

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367155#comment-15367155
 ] 

Hudson commented on HBASE-16190:


FAILURE: Integrated in HBase-Trunk_matrix #1188 (See 
[https://builds.apache.org/job/HBase-Trunk_matrix/1188/])
HBASE-16190 IntegrationTestDDLMasterFailover failed with (enis: rev 
17edca6346722072a72ec7942e719214083e63a2)
* 
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestDDLMasterFailover.java


> IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must 
> be positive
> -
>
> Key: HBASE-16190
> URL: https://issues.apache.org/jira/browse/HBASE-16190
> Project: HBase
>  Issue Type: Bug
>Reporter: Romil Choksi
>Assignee: Romil Choksi
>Priority: Minor
>  Labels: integration-test
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 1.2.3
>
> Attachments: 16190.v1.txt, HBASE-16190.patch
>
>
> IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must 
> be positive
> {code}
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:21,661 INFO  [main] hbase.IntegrationTestDDLMasterFailover: Runtime is 
> up
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:22,026 ERROR [main] hbase.IntegrationTestDDLMasterFailover: Found 
> exception in thread: Thread-11
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:21,384 INFO  [Thread-16] hbase.IntegrationTestDDLMasterFailover: 
> Performing Action: CREATE_TABLE
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:22,027 INFO  [Thread-16] hbase.IntegrationTestDDLMasterFailover: 
> Thread-16 stopped
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:20,506 INFO  [Thread-30] hbase.IntegrationTestDDLMasterFailover: 
> Performing Action: ADD_COLUMNFAMILY
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|java.lang.IllegalArgumentException:
>  n must be positive
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|at 
> java.util.Random.nextInt(Random.java:300)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.JVMRandom.nextInt(JVMRandom.java:118)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.RandomUtils.nextInt(RandomUtils.java:88)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.RandomUtils.nextInt(RandomUtils.java:74)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$TableAction.selectTable(IntegrationTestDDLMasterFailover.java:212)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$AddColumnFamilyAction.perform(IntegrationTestDDLMasterFailover.java:421)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$Worker.run(IntegrationTestDDLMasterFailover.java:695)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random read

[Hiroshi Ikeda (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367154#comment-15367154
 ] 

Hiroshi Ikeda commented on HBASE-15716:
---

I realized I should have named the test class using the plural form instead of 
the singular one.

> HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random 
> read
> --
>
> Key: HBASE-15716
> URL: https://issues.apache.org/jira/browse/HBASE-15716
> Project: HBase
>  Issue Type: Bug
>  Components: Performance
>Reporter: stack
>Assignee: stack
> Attachments: 
> 15716.implementation.using.ScannerReadPoints.branch-1.patch, 
> 15716.prune.synchronizations.patch, 15716.prune.synchronizations.v3.patch, 
> 15716.prune.synchronizations.v4.patch, 15716.prune.synchronizations.v4.patch, 
> 15716.wip.more_to_be_done.patch, HBASE-15716.branch-1.001.patch, 
> HBASE-15716.branch-1.002.patch, HBASE-15716.branch-1.003.patch, 
> HBASE-15716.branch-1.004.patch, HBASE-15716.branch-1.005.patch, 
> ScannerReadPoints.java, ScannerReadPoints.v2.java, Screen Shot 2016-04-26 at 
> 2.05.45 PM.png, Screen Shot 2016-04-26 at 2.06.14 PM.png, Screen Shot 
> 2016-04-26 at 2.07.06 PM.png, Screen Shot 2016-04-26 at 2.25.26 PM.png, 
> Screen Shot 2016-04-26 at 6.02.29 PM.png, Screen Shot 2016-04-27 at 9.49.35 
> AM.png, Screen Shot 2016-06-30 at 9.52.52 PM.png, Screen Shot 2016-06-30 at 
> 9.54.08 PM.png, TestScannerReadPointDoesntGoBack.java, 
> TestScannerReadPoints.java, before_after.png, 
> current-branch-1.vs.NoSynchronization.vs.Patch.png, hits.png, 
> remove.locks.patch, remove_cslm.patch
>
>
> Here is a [~lhofhansl] special.
> When we construct the region scanner, we get our read point and then store it 
> with the scanner instance in a Region scoped CSLM. This is done under a 
> synchronize on the CSLM.
> This synchronize on a region-scoped Map creating region scanners is the 
> outstanding point of lock contention according to flight recorder (My work 
> load is workload c, random reads).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16194) Should count in MSLAB chunk allocation into heap size change when adding duplicated cells

[Yu Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yu Li updated HBASE-16194:
--
Attachment: HBASE-16194_v2.patch

Refine using of VisibleForTesting tag, following review comments.

> Should count in MSLAB chunk allocation into heap size change when adding 
> duplicated cells
> -
>
> Key: HBASE-16194
> URL: https://issues.apache.org/jira/browse/HBASE-16194
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 1.2.1, 1.1.5, 0.98.20
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-16194.patch, HBASE-16194_v2.patch
>
>
> See more details about problem description and analysis in HBASE-16193



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random read

[Hiroshi Ikeda (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hiroshi Ikeda updated HBASE-15716:
--
Assignee: stack  (was: Hiroshi Ikeda)

> HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random 
> read
> --
>
> Key: HBASE-15716
> URL: https://issues.apache.org/jira/browse/HBASE-15716
> Project: HBase
>  Issue Type: Bug
>  Components: Performance
>Reporter: stack
>Assignee: stack
> Attachments: 
> 15716.implementation.using.ScannerReadPoints.branch-1.patch, 
> 15716.prune.synchronizations.patch, 15716.prune.synchronizations.v3.patch, 
> 15716.prune.synchronizations.v4.patch, 15716.prune.synchronizations.v4.patch, 
> 15716.wip.more_to_be_done.patch, HBASE-15716.branch-1.001.patch, 
> HBASE-15716.branch-1.002.patch, HBASE-15716.branch-1.003.patch, 
> HBASE-15716.branch-1.004.patch, HBASE-15716.branch-1.005.patch, 
> ScannerReadPoints.java, ScannerReadPoints.v2.java, Screen Shot 2016-04-26 at 
> 2.05.45 PM.png, Screen Shot 2016-04-26 at 2.06.14 PM.png, Screen Shot 
> 2016-04-26 at 2.07.06 PM.png, Screen Shot 2016-04-26 at 2.25.26 PM.png, 
> Screen Shot 2016-04-26 at 6.02.29 PM.png, Screen Shot 2016-04-27 at 9.49.35 
> AM.png, Screen Shot 2016-06-30 at 9.52.52 PM.png, Screen Shot 2016-06-30 at 
> 9.54.08 PM.png, TestScannerReadPointDoesntGoBack.java, 
> TestScannerReadPoints.java, before_after.png, 
> current-branch-1.vs.NoSynchronization.vs.Patch.png, hits.png, 
> remove.locks.patch, remove_cslm.patch
>
>
> Here is a [~lhofhansl] special.
> When we construct the region scanner, we get our read point and then store it 
> with the scanner instance in a Region scoped CSLM. This is done under a 
> synchronize on the CSLM.
> This synchronize on a region-scoped Map creating region scanners is the 
> outstanding point of lock contention according to flight recorder (My work 
> load is workload c, random reads).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random read

[Hiroshi Ikeda (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hiroshi Ikeda updated HBASE-15716:
--
Attachment: TestScannerReadPointDoesntGoBack.java

Added another test to check some correctness.

BTW I'm taking a few days off next week and I'll delay responses.

> HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random 
> read
> --
>
> Key: HBASE-15716
> URL: https://issues.apache.org/jira/browse/HBASE-15716
> Project: HBase
>  Issue Type: Bug
>  Components: Performance
>Reporter: stack
>Assignee: Hiroshi Ikeda
> Attachments: 
> 15716.implementation.using.ScannerReadPoints.branch-1.patch, 
> 15716.prune.synchronizations.patch, 15716.prune.synchronizations.v3.patch, 
> 15716.prune.synchronizations.v4.patch, 15716.prune.synchronizations.v4.patch, 
> 15716.wip.more_to_be_done.patch, HBASE-15716.branch-1.001.patch, 
> HBASE-15716.branch-1.002.patch, HBASE-15716.branch-1.003.patch, 
> HBASE-15716.branch-1.004.patch, HBASE-15716.branch-1.005.patch, 
> ScannerReadPoints.java, ScannerReadPoints.v2.java, Screen Shot 2016-04-26 at 
> 2.05.45 PM.png, Screen Shot 2016-04-26 at 2.06.14 PM.png, Screen Shot 
> 2016-04-26 at 2.07.06 PM.png, Screen Shot 2016-04-26 at 2.25.26 PM.png, 
> Screen Shot 2016-04-26 at 6.02.29 PM.png, Screen Shot 2016-04-27 at 9.49.35 
> AM.png, Screen Shot 2016-06-30 at 9.52.52 PM.png, Screen Shot 2016-06-30 at 
> 9.54.08 PM.png, TestScannerReadPointDoesntGoBack.java, 
> TestScannerReadPoints.java, before_after.png, 
> current-branch-1.vs.NoSynchronization.vs.Patch.png, hits.png, 
> remove.locks.patch, remove_cslm.patch
>
>
> Here is a [~lhofhansl] special.
> When we construct the region scanner, we get our read point and then store it 
> with the scanner instance in a Region scoped CSLM. This is done under a 
> synchronize on the CSLM.
> This synchronize on a region-scoped Map creating region scanners is the 
> outstanding point of lock contention according to flight recorder (My work 
> load is workload c, random reads).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random read

[Hiroshi Ikeda (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hiroshi Ikeda reassigned HBASE-15716:
-

Assignee: Hiroshi Ikeda  (was: stack)

> HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random 
> read
> --
>
> Key: HBASE-15716
> URL: https://issues.apache.org/jira/browse/HBASE-15716
> Project: HBase
>  Issue Type: Bug
>  Components: Performance
>Reporter: stack
>Assignee: Hiroshi Ikeda
> Attachments: 
> 15716.implementation.using.ScannerReadPoints.branch-1.patch, 
> 15716.prune.synchronizations.patch, 15716.prune.synchronizations.v3.patch, 
> 15716.prune.synchronizations.v4.patch, 15716.prune.synchronizations.v4.patch, 
> 15716.wip.more_to_be_done.patch, HBASE-15716.branch-1.001.patch, 
> HBASE-15716.branch-1.002.patch, HBASE-15716.branch-1.003.patch, 
> HBASE-15716.branch-1.004.patch, HBASE-15716.branch-1.005.patch, 
> ScannerReadPoints.java, ScannerReadPoints.v2.java, Screen Shot 2016-04-26 at 
> 2.05.45 PM.png, Screen Shot 2016-04-26 at 2.06.14 PM.png, Screen Shot 
> 2016-04-26 at 2.07.06 PM.png, Screen Shot 2016-04-26 at 2.25.26 PM.png, 
> Screen Shot 2016-04-26 at 6.02.29 PM.png, Screen Shot 2016-04-27 at 9.49.35 
> AM.png, Screen Shot 2016-06-30 at 9.52.52 PM.png, Screen Shot 2016-06-30 at 
> 9.54.08 PM.png, TestScannerReadPoints.java, before_after.png, 
> current-branch-1.vs.NoSynchronization.vs.Patch.png, hits.png, 
> remove.locks.patch, remove_cslm.patch
>
>
> Here is a [~lhofhansl] special.
> When we construct the region scanner, we get our read point and then store it 
> with the scanner instance in a Region scoped CSLM. This is done under a 
> synchronize on the CSLM.
> This synchronize on a region-scoped Map creating region scanners is the 
> outstanding point of lock contention according to flight recorder (My work 
> load is workload c, random reads).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16190) IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must be positive

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367144#comment-15367144
 ] 

Hudson commented on HBASE-16190:


SUCCESS: Integrated in HBase-1.2 #666 (See 
[https://builds.apache.org/job/HBase-1.2/666/])
HBASE-16190 IntegrationTestDDLMasterFailover failed with (enis: rev 
075f33a06ab27f35fc9d4e46b7bf2220d47992d1)
* 
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestDDLMasterFailover.java


> IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must 
> be positive
> -
>
> Key: HBASE-16190
> URL: https://issues.apache.org/jira/browse/HBASE-16190
> Project: HBase
>  Issue Type: Bug
>Reporter: Romil Choksi
>Assignee: Romil Choksi
>Priority: Minor
>  Labels: integration-test
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 1.2.3
>
> Attachments: 16190.v1.txt, HBASE-16190.patch
>
>
> IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must 
> be positive
> {code}
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:21,661 INFO  [main] hbase.IntegrationTestDDLMasterFailover: Runtime is 
> up
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:22,026 ERROR [main] hbase.IntegrationTestDDLMasterFailover: Found 
> exception in thread: Thread-11
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:21,384 INFO  [Thread-16] hbase.IntegrationTestDDLMasterFailover: 
> Performing Action: CREATE_TABLE
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:22,027 INFO  [Thread-16] hbase.IntegrationTestDDLMasterFailover: 
> Thread-16 stopped
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:20,506 INFO  [Thread-30] hbase.IntegrationTestDDLMasterFailover: 
> Performing Action: ADD_COLUMNFAMILY
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|java.lang.IllegalArgumentException:
>  n must be positive
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|at 
> java.util.Random.nextInt(Random.java:300)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.JVMRandom.nextInt(JVMRandom.java:118)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.RandomUtils.nextInt(RandomUtils.java:88)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.RandomUtils.nextInt(RandomUtils.java:74)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$TableAction.selectTable(IntegrationTestDDLMasterFailover.java:212)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$AddColumnFamilyAction.perform(IntegrationTestDDLMasterFailover.java:421)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$Worker.run(IntegrationTestDDLMasterFailover.java:695)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16190) IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must be positive

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367140#comment-15367140
 ] 

Hudson commented on HBASE-16190:


SUCCESS: Integrated in HBase-1.3-IT #745 (See 
[https://builds.apache.org/job/HBase-1.3-IT/745/])
HBASE-16190 IntegrationTestDDLMasterFailover failed with (enis: rev 
ee63706eed5fb3b72056c0228aae12104366076a)
* 
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestDDLMasterFailover.java


> IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must 
> be positive
> -
>
> Key: HBASE-16190
> URL: https://issues.apache.org/jira/browse/HBASE-16190
> Project: HBase
>  Issue Type: Bug
>Reporter: Romil Choksi
>Assignee: Romil Choksi
>Priority: Minor
>  Labels: integration-test
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 1.2.3
>
> Attachments: 16190.v1.txt, HBASE-16190.patch
>
>
> IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must 
> be positive
> {code}
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:21,661 INFO  [main] hbase.IntegrationTestDDLMasterFailover: Runtime is 
> up
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:22,026 ERROR [main] hbase.IntegrationTestDDLMasterFailover: Found 
> exception in thread: Thread-11
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:21,384 INFO  [Thread-16] hbase.IntegrationTestDDLMasterFailover: 
> Performing Action: CREATE_TABLE
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:22,027 INFO  [Thread-16] hbase.IntegrationTestDDLMasterFailover: 
> Thread-16 stopped
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:20,506 INFO  [Thread-30] hbase.IntegrationTestDDLMasterFailover: 
> Performing Action: ADD_COLUMNFAMILY
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|java.lang.IllegalArgumentException:
>  n must be positive
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|at 
> java.util.Random.nextInt(Random.java:300)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.JVMRandom.nextInt(JVMRandom.java:118)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.RandomUtils.nextInt(RandomUtils.java:88)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.RandomUtils.nextInt(RandomUtils.java:74)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$TableAction.selectTable(IntegrationTestDDLMasterFailover.java:212)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$AddColumnFamilyAction.perform(IntegrationTestDDLMasterFailover.java:421)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$Worker.run(IntegrationTestDDLMasterFailover.java:695)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16095) Add priority to TableDescriptor and priority region open thread pool

[Enis Soztutar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367135#comment-15367135
 ] 

Enis Soztutar commented on HBASE-16095:
---

I have run the failed tests locally. it passes except for HBASE-16185.

Any interest in reviewing this? 

> Add priority to TableDescriptor and priority region open thread pool
> 
>
> Key: HBASE-16095
> URL: https://issues.apache.org/jira/browse/HBASE-16095
> Project: HBase
>  Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.4.0
>
> Attachments: hbase-16095_v0.patch, hbase-16095_v1.patch, 
> hbase-16095_v2.patch
>
>
> This is in the similar area with HBASE-15816, and also required with the 
> current secondary indexing for Phoenix. 
> The problem with P secondary indexes is that data table regions depend on 
> index regions to be able to make progress. Possible distributed deadlocks can 
> be prevented via custom RpcScheduler + RpcController configuration via 
> HBASE-11048 and PHOENIX-938. However, region opening also has the same 
> deadlock situation, because data region open has to replay the WAL edits to 
> the index regions. There is only 1 thread pool to open regions with 3 workers 
> by default. So if the cluster is recovering / restarting from scratch, the 
> deadlock happens because some index regions cannot be opened due to them 
> being in the same queue waiting for data regions to open (which waits for  
> RPC'ing to index regions which is not open). This is reproduced in almost all 
> Phoenix secondary index clusters (mutable table w/o transactions) that we 
> see. 
> The proposal is to have a "high priority" region opening thread pool, and 
> have the HTD carry the relative priority of a table. This maybe useful for 
> other "framework" level tables from Phoenix, Tephra, Trafodian, etc if they 
> want some specific tables to become online faster. 
> As a follow up patch, we can also take a look at how this priority 
> information can be used by the rpc scheduler on the server side or rpc 
> controller on the client side, so that we do not have to set priorities 
> manually per-operation. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16190) IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must be positive

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367127#comment-15367127
 ] 

Hudson commented on HBASE-16190:


SUCCESS: Integrated in HBase-1.3 #772 (See 
[https://builds.apache.org/job/HBase-1.3/772/])
HBASE-16190 IntegrationTestDDLMasterFailover failed with (enis: rev 
ee63706eed5fb3b72056c0228aae12104366076a)
* 
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestDDLMasterFailover.java


> IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must 
> be positive
> -
>
> Key: HBASE-16190
> URL: https://issues.apache.org/jira/browse/HBASE-16190
> Project: HBase
>  Issue Type: Bug
>Reporter: Romil Choksi
>Assignee: Romil Choksi
>Priority: Minor
>  Labels: integration-test
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 1.2.3
>
> Attachments: 16190.v1.txt, HBASE-16190.patch
>
>
> IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must 
> be positive
> {code}
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:21,661 INFO  [main] hbase.IntegrationTestDDLMasterFailover: Runtime is 
> up
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:22,026 ERROR [main] hbase.IntegrationTestDDLMasterFailover: Found 
> exception in thread: Thread-11
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:21,384 INFO  [Thread-16] hbase.IntegrationTestDDLMasterFailover: 
> Performing Action: CREATE_TABLE
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:22,027 INFO  [Thread-16] hbase.IntegrationTestDDLMasterFailover: 
> Thread-16 stopped
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:20,506 INFO  [Thread-30] hbase.IntegrationTestDDLMasterFailover: 
> Performing Action: ADD_COLUMNFAMILY
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|java.lang.IllegalArgumentException:
>  n must be positive
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|at 
> java.util.Random.nextInt(Random.java:300)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.JVMRandom.nextInt(JVMRandom.java:118)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.RandomUtils.nextInt(RandomUtils.java:88)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.RandomUtils.nextInt(RandomUtils.java:74)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$TableAction.selectTable(IntegrationTestDDLMasterFailover.java:212)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$AddColumnFamilyAction.perform(IntegrationTestDDLMasterFailover.java:421)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$Worker.run(IntegrationTestDDLMasterFailover.java:695)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-16200) TestReplicationSmallTests is failing consistently

[Enis Soztutar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Enis Soztutar resolved HBASE-16200.
---
Resolution: Duplicate

Dup of HBASE-16185. 

> TestReplicationSmallTests is failing consistently
> -
>
> Key: HBASE-16200
> URL: https://issues.apache.org/jira/browse/HBASE-16200
> Project: HBase
>  Issue Type: Bug
>Reporter: Enis Soztutar
>
> Could not find it in jenkins run, but this test fails consistently in local 
> run. 
> https://builds.apache.org/view/All/job/HBase-Trunk_matrix/1187/jdk=latest1.7,label=yahoo-not-h2/testReport/org.apache.hadoop.hbase.replication/
> {code}
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; 
> support was removed in 8.0
> Running org.apache.hadoop.hbase.replication.TestReplicationSmallTests
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 24.815 sec 
> <<< FAILURE! - in 
> org.apache.hadoop.hbase.replication.TestReplicationSmallTests
> testReplicationStatus(org.apache.hadoop.hbase.replication.TestReplicationSmallTests)
>   Time elapsed: 1.65 sec  <<< FAILURE!
> java.lang.AssertionError: failed to get ReplicationLoadSourceList
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at 
> org.apache.hadoop.hbase.replication.TestReplicationSmallTests.testReplicationStatus(TestReplicationSmallTests.java:741)
> Results :
> Failed tests: 
>   TestReplicationSmallTests.testReplicationStatus:741 failed to get 
> ReplicationLoadSourceList
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16200) TestReplicationSmallTests is failing consistently

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367111#comment-15367111
 ] 

Ted Yu commented on HBASE-16200:


I reported HBASE-16185 already.

Mark this as a dup ?

> TestReplicationSmallTests is failing consistently
> -
>
> Key: HBASE-16200
> URL: https://issues.apache.org/jira/browse/HBASE-16200
> Project: HBase
>  Issue Type: Bug
>Reporter: Enis Soztutar
>
> Could not find it in jenkins run, but this test fails consistently in local 
> run. 
> https://builds.apache.org/view/All/job/HBase-Trunk_matrix/1187/jdk=latest1.7,label=yahoo-not-h2/testReport/org.apache.hadoop.hbase.replication/
> {code}
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; 
> support was removed in 8.0
> Running org.apache.hadoop.hbase.replication.TestReplicationSmallTests
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 24.815 sec 
> <<< FAILURE! - in 
> org.apache.hadoop.hbase.replication.TestReplicationSmallTests
> testReplicationStatus(org.apache.hadoop.hbase.replication.TestReplicationSmallTests)
>   Time elapsed: 1.65 sec  <<< FAILURE!
> java.lang.AssertionError: failed to get ReplicationLoadSourceList
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at 
> org.apache.hadoop.hbase.replication.TestReplicationSmallTests.testReplicationStatus(TestReplicationSmallTests.java:741)
> Results :
> Failed tests: 
>   TestReplicationSmallTests.testReplicationStatus:741 failed to get 
> ReplicationLoadSourceList
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-16200) TestReplicationSmallTests is failing consistently

[Enis Soztutar (JIRA)]

Enis Soztutar created HBASE-16200:
-

 Summary: TestReplicationSmallTests is failing consistently
 Key: HBASE-16200
 URL: https://issues.apache.org/jira/browse/HBASE-16200
 Project: HBase
  Issue Type: Bug
Reporter: Enis Soztutar


Could not find it in jenkins run, but this test fails consistently in local 
run. 

https://builds.apache.org/view/All/job/HBase-Trunk_matrix/1187/jdk=latest1.7,label=yahoo-not-h2/testReport/org.apache.hadoop.hbase.replication/

{code}
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; 
support was removed in 8.0
Running org.apache.hadoop.hbase.replication.TestReplicationSmallTests
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 24.815 sec <<< 
FAILURE! - in org.apache.hadoop.hbase.replication.TestReplicationSmallTests
testReplicationStatus(org.apache.hadoop.hbase.replication.TestReplicationSmallTests)
  Time elapsed: 1.65 sec  <<< FAILURE!
java.lang.AssertionError: failed to get ReplicationLoadSourceList
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at 
org.apache.hadoop.hbase.replication.TestReplicationSmallTests.testReplicationStatus(TestReplicationSmallTests.java:741)


Results :

Failed tests: 
  TestReplicationSmallTests.testReplicationStatus:741 failed to get 
ReplicationLoadSourceList
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16160) Get the UnsupportedOperationException when using delegation token with encryption

[Colin Ma (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367107#comment-15367107
 ] 

Colin Ma commented on HBASE-16160:
--

[~ghelmling], thanks for review, the test case is updated as your comments. You 
can review them in Review Board.

> Get the UnsupportedOperationException when using delegation token with 
> encryption
> -
>
> Key: HBASE-16160
> URL: https://issues.apache.org/jira/browse/HBASE-16160
> Project: HBase
>  Issue Type: Bug
>Reporter: Colin Ma
>Assignee: Colin Ma
> Attachments: HBASE-16160.001.patch, HBASE-16160.002.patch
>
>
> Using delegation token with encryption, when do the Put operation, will get 
> the following exception:
> [RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345] 
> ipc.CallRunner(161): 
> RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345: caught: 
> java.lang.UnsupportedOperationException
> at java.nio.ByteBuffer.array(ByteBuffer.java:959)
> at 
> org.apache.hadoop.hbase.ipc.BufferChain.getBytes(BufferChain.java:66)
> at 
> org.apache.hadoop.hbase.ipc.RpcServer$Call.wrapWithSasl(RpcServer.java:547)
> at 
> org.apache.hadoop.hbase.ipc.RpcServer$Call.setResponse(RpcServer.java:467)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:140)
> at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189)
> at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16160) Get the UnsupportedOperationException when using delegation token with encryption

[Colin Ma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Colin Ma updated HBASE-16160:
-
Attachment: HBASE-16160.002.patch

> Get the UnsupportedOperationException when using delegation token with 
> encryption
> -
>
> Key: HBASE-16160
> URL: https://issues.apache.org/jira/browse/HBASE-16160
> Project: HBase
>  Issue Type: Bug
>Reporter: Colin Ma
>Assignee: Colin Ma
> Attachments: HBASE-16160.001.patch, HBASE-16160.002.patch
>
>
> Using delegation token with encryption, when do the Put operation, will get 
> the following exception:
> [RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345] 
> ipc.CallRunner(161): 
> RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345: caught: 
> java.lang.UnsupportedOperationException
> at java.nio.ByteBuffer.array(ByteBuffer.java:959)
> at 
> org.apache.hadoop.hbase.ipc.BufferChain.getBytes(BufferChain.java:66)
> at 
> org.apache.hadoop.hbase.ipc.RpcServer$Call.wrapWithSasl(RpcServer.java:547)
> at 
> org.apache.hadoop.hbase.ipc.RpcServer$Call.setResponse(RpcServer.java:467)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:140)
> at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189)
> at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16160) Get the UnsupportedOperationException when using delegation token with encryption

[Colin Ma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Colin Ma updated HBASE-16160:
-
Attachment: (was: HBASE-16160.002.patch)

> Get the UnsupportedOperationException when using delegation token with 
> encryption
> -
>
> Key: HBASE-16160
> URL: https://issues.apache.org/jira/browse/HBASE-16160
> Project: HBase
>  Issue Type: Bug
>Reporter: Colin Ma
>Assignee: Colin Ma
> Attachments: HBASE-16160.001.patch
>
>
> Using delegation token with encryption, when do the Put operation, will get 
> the following exception:
> [RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345] 
> ipc.CallRunner(161): 
> RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345: caught: 
> java.lang.UnsupportedOperationException
> at java.nio.ByteBuffer.array(ByteBuffer.java:959)
> at 
> org.apache.hadoop.hbase.ipc.BufferChain.getBytes(BufferChain.java:66)
> at 
> org.apache.hadoop.hbase.ipc.RpcServer$Call.wrapWithSasl(RpcServer.java:547)
> at 
> org.apache.hadoop.hbase.ipc.RpcServer$Call.setResponse(RpcServer.java:467)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:140)
> at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189)
> at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16160) Get the UnsupportedOperationException when using delegation token with encryption

[Colin Ma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Colin Ma updated HBASE-16160:
-
Attachment: HBASE-16160.002.patch

> Get the UnsupportedOperationException when using delegation token with 
> encryption
> -
>
> Key: HBASE-16160
> URL: https://issues.apache.org/jira/browse/HBASE-16160
> Project: HBase
>  Issue Type: Bug
>Reporter: Colin Ma
>Assignee: Colin Ma
> Attachments: HBASE-16160.001.patch, HBASE-16160.002.patch
>
>
> Using delegation token with encryption, when do the Put operation, will get 
> the following exception:
> [RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345] 
> ipc.CallRunner(161): 
> RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345: caught: 
> java.lang.UnsupportedOperationException
> at java.nio.ByteBuffer.array(ByteBuffer.java:959)
> at 
> org.apache.hadoop.hbase.ipc.BufferChain.getBytes(BufferChain.java:66)
> at 
> org.apache.hadoop.hbase.ipc.RpcServer$Call.wrapWithSasl(RpcServer.java:547)
> at 
> org.apache.hadoop.hbase.ipc.RpcServer$Call.setResponse(RpcServer.java:467)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:140)
> at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189)
> at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16190) IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must be positive

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367097#comment-15367097
 ] 

Hudson commented on HBASE-16190:


SUCCESS: Integrated in HBase-1.4 #275 (See 
[https://builds.apache.org/job/HBase-1.4/275/])
HBASE-16190 IntegrationTestDDLMasterFailover failed with (enis: rev 
ed07a38d594ac5871e53e20f5060d489ef440696)
* 
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestDDLMasterFailover.java


> IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must 
> be positive
> -
>
> Key: HBASE-16190
> URL: https://issues.apache.org/jira/browse/HBASE-16190
> Project: HBase
>  Issue Type: Bug
>Reporter: Romil Choksi
>Assignee: Romil Choksi
>Priority: Minor
>  Labels: integration-test
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 1.2.3
>
> Attachments: 16190.v1.txt, HBASE-16190.patch
>
>
> IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must 
> be positive
> {code}
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:21,661 INFO  [main] hbase.IntegrationTestDDLMasterFailover: Runtime is 
> up
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:22,026 ERROR [main] hbase.IntegrationTestDDLMasterFailover: Found 
> exception in thread: Thread-11
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:21,384 INFO  [Thread-16] hbase.IntegrationTestDDLMasterFailover: 
> Performing Action: CREATE_TABLE
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:22,027 INFO  [Thread-16] hbase.IntegrationTestDDLMasterFailover: 
> Thread-16 stopped
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:20,506 INFO  [Thread-30] hbase.IntegrationTestDDLMasterFailover: 
> Performing Action: ADD_COLUMNFAMILY
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|java.lang.IllegalArgumentException:
>  n must be positive
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|at 
> java.util.Random.nextInt(Random.java:300)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.JVMRandom.nextInt(JVMRandom.java:118)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.RandomUtils.nextInt(RandomUtils.java:88)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.RandomUtils.nextInt(RandomUtils.java:74)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$TableAction.selectTable(IntegrationTestDDLMasterFailover.java:212)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$AddColumnFamilyAction.perform(IntegrationTestDDLMasterFailover.java:421)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$Worker.run(IntegrationTestDDLMasterFailover.java:695)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16172) Unify the retry logic in ScannerCallableWithReplicas and RpcRetryingCallerWithReadReplicas

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367088#comment-15367088
 ] 

Hadoop QA commented on HBASE-16172:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 1s 
{color} | {color:blue} The patch file was not named according to hbase's naming 
conventions. Please see 
https://yetus.apache.org/documentation/0.2.1/precommit-patchnames for 
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 19s 
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 
6s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s 
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s 
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
26s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 
55s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s 
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s 
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s 
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 
3s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 13s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 13s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 51s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
26m 58s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 46s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 57s 
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 83m 31s {color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
26s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 133m 25s {color} 
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests | 
org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2 |
|   | 

[jira] [Commented] (HBASE-15925) compat-module maven variable not evaluated

[Nick Dimiduk (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367074#comment-15367074
 ] 

Nick Dimiduk commented on HBASE-15925:
--

I'm no longer convinced this can be confirmed with visual inspection. 
[~dportabella] can you plug this repository and version into your project and 
verify? Maybe you can contribute a patch to test this to our 
[hbase-downstreamer|https://github.com/saintstack/hbase-downstreamer] project.

> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
>  Issue Type: Bug
>  Components: build
>Affects Versions: 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.0.3, 1.1.5
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15925.1.patch
>
>
> Looks like we've regressed on HBASE-8488. Have a look at the dependency 
> artifacts list on 
> http://mvnrepository.com/artifact/org.apache.hbase/hbase-testing-util/1.2.1. 
> Notice the direct dependency's artifactId is {{$\{compat.module\}}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15925) compat-module maven variable not evaluated

[Nick Dimiduk (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367070#comment-15367070
 ] 

Nick Dimiduk commented on HBASE-15925:
--

Comparing the pom in the repo on r2 vs that in central for 1.2.1, I don't 
understand why the slf4j.version property a couple lines down is apparently 
parsed correctly while the compat module variable is not.

> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
>  Issue Type: Bug
>  Components: build
>Affects Versions: 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.0.3, 1.1.5
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15925.1.patch
>
>
> Looks like we've regressed on HBASE-8488. Have a look at the dependency 
> artifacts list on 
> http://mvnrepository.com/artifact/org.apache.hbase/hbase-testing-util/1.2.1. 
> Notice the direct dependency's artifactId is {{$\{compat.module\}}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16199) Add daughter ReginInfo parameter to preSplit method.

[Jun Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jun Tsai updated HBASE-16199:
-
Description: 
Using hbase coprocessor interface I want to get daughters info in preSplit 
method.such as:
parent.getCoprocessorHost().preSplit(hri_a,hri_b);

  was:
Using hbase coprocess interface I want to get daughters info in preSplit 
method.such as:
parent.getCoprocessorHost().preSplit(hri_a,hri_b);


> Add daughter ReginInfo parameter to preSplit method.
> 
>
> Key: HBASE-16199
> URL: https://issues.apache.org/jira/browse/HBASE-16199
> Project: HBase
>  Issue Type: New Feature
>  Components: Coprocessors
>Reporter: Jun Tsai
>Priority: Minor
>
> Using hbase coprocessor interface I want to get daughters info in preSplit 
> method.such as:
> parent.getCoprocessorHost().preSplit(hri_a,hri_b);



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-16199) Add daughter ReginInfo parameter to preSplit method.

[Jun Tsai (JIRA)]

Jun Tsai created HBASE-16199:


 Summary: Add daughter ReginInfo parameter to preSplit method.
 Key: HBASE-16199
 URL: https://issues.apache.org/jira/browse/HBASE-16199
 Project: HBase
  Issue Type: New Feature
  Components: Coprocessors
Reporter: Jun Tsai
Priority: Minor


Using hbase coprocess interface I want to get daughters info in preSplit 
method.such as:
parent.getCoprocessorHost().preSplit(hri_a,hri_b);



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (HBASE-15925) compat-module maven variable not evaluated

[Nick Dimiduk (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk reopened HBASE-15925:
--

Reopening issue. Sorry fellas.

> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
>  Issue Type: Bug
>  Components: build
>Affects Versions: 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.0.3, 1.1.5
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15925.1.patch
>
>
> Looks like we've regressed on HBASE-8488. Have a look at the dependency 
> artifacts list on 
> http://mvnrepository.com/artifact/org.apache.hbase/hbase-testing-util/1.2.1. 
> Notice the direct dependency's artifactId is {{$\{compat.module\}}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15925) compat-module maven variable not evaluated

[Nick Dimiduk (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367059#comment-15367059
 ] 

Nick Dimiduk commented on HBASE-15925:
--

Nice timing [~dportabella]. The latest RC for 1.2.2 has been posted, and 
includes a staged maven repository -- this is what will be promoted to maven 
central assuming this release passes. Take a look: 
https://repository.apache.org/content/repositories/orgapachehbase-1142/

>From what I can tell, this is not fixed in the RC. 
>https://repository.apache.org/content/repositories/orgapachehbase-1142/org/apache/hbase/hbase-testing-util/1.2.2/hbase-testing-util-1.2.2.pom

FYI [~busbey]

> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
>  Issue Type: Bug
>  Components: build
>Affects Versions: 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.0.3, 1.1.5
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15925.1.patch
>
>
> Looks like we've regressed on HBASE-8488. Have a look at the dependency 
> artifacts list on 
> http://mvnrepository.com/artifact/org.apache.hbase/hbase-testing-util/1.2.1. 
> Notice the direct dependency's artifactId is {{$\{compat.module\}}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16132) Scan does not return all the result when regionserver is busy

[Nick Dimiduk (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367056#comment-15367056
 ] 

Nick Dimiduk commented on HBASE-16132:
--

Thanks [~carp84].

> Scan does not return all the result when regionserver is busy
> -
>
> Key: HBASE-16132
> URL: https://issues.apache.org/jira/browse/HBASE-16132
> Project: HBase
>  Issue Type: Bug
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-16132.patch, HBASE-16132_v2.patch, 
> HBASE-16132_v3.patch, HBASE-16132_v3.patch, TestScanMissingData.java
>
>
> We have find some corner case, when regionserver is busy and last a long 
> time. Some scanner may return null even if they do not scan all data.
> We find in ScannerCallableWithReplicas there is a case do not handler 
> correct, when cs.poll timeout and do not return any result , it is will 
> return a null result, so scan get null result, and end the scan. 
>  {code}
> try {
>   Future> f = cs.poll(timeout, 
> TimeUnit.MILLISECONDS);
>   if (f != null) {
> Pair r = f.get(timeout, 
> TimeUnit.MILLISECONDS);
> if (r != null && r.getSecond() != null) {
>   updateCurrentlyServingReplica(r.getSecond(), r.getFirst(), done, 
> pool);
> }
> return r == null ? null : r.getFirst(); // great we got an answer
>   }
> } catch (ExecutionException e) {
>   RpcRetryingCallerWithReadReplicas.throwEnrichedException(e, retries);
> } catch (CancellationException e) {
>   throw new InterruptedIOException(e.getMessage());
> } catch (InterruptedException e) {
>   throw new InterruptedIOException(e.getMessage());
> } catch (TimeoutException e) {
>   throw new InterruptedIOException(e.getMessage());
> } finally {
>   // We get there because we were interrupted or because one or more of 
> the
>   // calls succeeded or failed. In all case, we stop all our tasks.
>   cs.cancelAll();
> }
> return null; // unreachable
>  {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16093) Splits failed before creating daughter regions leave meta inconsistent

[Nick Dimiduk (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367049#comment-15367049
 ] 

Nick Dimiduk commented on HBASE-16093:
--

[~enis] disabled by default is orthogonal to my concern re: tests.

> Splits failed before creating daughter regions leave meta inconsistent
> --
>
> Key: HBASE-16093
> URL: https://issues.apache.org/jira/browse/HBASE-16093
> Project: HBase
>  Issue Type: Bug
>  Components: master, Region Assignment
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Critical
> Fix For: 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-16093.branch-1.patch
>
>
> This is on branch-1 based code only.
> Here's the sequence of events.
> # A regionserver opens a new region. That regions looks like it should split.
> # So the regionserver starts a split transaction.
> # Split transaction starts execute
> # Split transaction encounters an error in stepsBeforePONR
> # Split transaction starts rollback
> # Split transaction notifies master that it's rolling back using 
> HMasterRpcServices#reportRegionStateTransition
> # AssignmentManager#onRegionTransition is called with SPLIT_REVERTED
> # AssignmentManager#onRegionSplitReverted is called.
> # That onlines the parent region and offlines the daughter regions.
> However the daughter regions were never created in meta so all that gets done 
> is that state for those rows gets OFFLINE. Now all clients trying to get the 
> parent instead get the offline daughter.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15935) Have a separate Walker task running concurrently with Generator

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367050#comment-15367050
 ] 

stack commented on HBASE-15935:
---

Yes.

I like the idea of concurrent read/write.

You know me. Just trying to figure tooling that'll help us debug stuff when it 
goes awry.

> Have a separate Walker task running concurrently with Generator   
> --
>
> Key: HBASE-15935
> URL: https://issues.apache.org/jira/browse/HBASE-15935
> Project: HBase
>  Issue Type: Sub-task
>  Components: integration tests
>Reporter: Joseph
>Assignee: Joseph
>Priority: Minor
> Attachments: HBASE-15935.patch
>
>
> Keep track of which linked lists have been flushed in an HBase table, so that 
> we can concurrently Walk these lists during the Generation phase. This will 
> allow us to test:
> 1. HBase under concurrent read/writes
> 2. Availability of data immediately after flushes (as opposed to waiting till 
> the Verification phase)
> The review diff can be found at:
> https://reviews.apache.org/r/48294/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14933) check_compatibility.sh does not work with jdk8

[Nick Dimiduk (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-14933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367048#comment-15367048
 ] 

Nick Dimiduk commented on HBASE-14933:
--

Resolved as fixed with no commits? Should be a different resolution I would 
think.

> check_compatibility.sh does not work with jdk8
> --
>
> Key: HBASE-14933
> URL: https://issues.apache.org/jira/browse/HBASE-14933
> Project: HBase
>  Issue Type: Task
>  Components: scripts
>Reporter: Nick Dimiduk
>Assignee: Dima Spivak
>Priority: Minor
> Fix For: 2.0.0
>
>
> Specifically, Oracle jdk1.8.0_65 on OSX and OpenJDk 1.8.0_45-internal-b14 on 
> ubuntu.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15800) check_compatibility.sh broken as of upstream b8f125f

[Nick Dimiduk (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367047#comment-15367047
 ] 

Nick Dimiduk commented on HBASE-15800:
--

Sorry I didn't get around to trying you patch [~dimaspivak]. Thanks for picking 
this one up!

> check_compatibility.sh broken as of upstream b8f125f
> 
>
> Key: HBASE-15800
> URL: https://issues.apache.org/jira/browse/HBASE-15800
> Project: HBase
>  Issue Type: Bug
>  Components: build
>Affects Versions: 2.0.0
>Reporter: Nick Dimiduk
>Assignee: Dima Spivak
>Priority: Minor
>
> Previously (recently as 2 weeks ago) it was possible to use the 
> {{dev-support/check_compatibility.sh}} script against the tags under {{rel}}. 
> Now ({{5e0}}) this is no longer the case. On a mac,
> {noformat}
> $ echo $JAVA_HOME
> /Library/Java/JavaVirtualMachines/jdk1.7.0_80.jdk/Contents/Home
> $ GETOPT=/usr/local/Cellar/gnu-getopt/1.1.6/bin/getopt 
> ./dev-support/check_compatibility.sh -r 
> https://git-wip-us.apache.org/repos/asf/hbase.git rel/1.1.4 branch-1.1
> ...
> Running the Java API Compliance Checker...
> ERROR: can't access 
> './dev-support/target/compatibility/1/hbase-annotations/target/hbase-annotations-1.1.4.jar,./dev-support/target/compatibility/1/hbase-checkstyle/target/hbase-checkstyle-1.1.
> 4.jar,./dev-support/target/compatibility/1/hbase-client/target/hbase-client-1.1.4.jar,./dev-support/target/compatibility/1/hbase-common/target/hbase-common-1.1.4.jar,./dev-support/target/compat
> ibility/1/hbase-examples/target/hbase-examples-1.1.4.jar,./dev-support/target/compatibility/1/hbase-hadoop-compat/target/hbase-hadoop-compat-1.1.4.jar,./dev-support/target/compatibility/1/hbase
> -hadoop2-compat/target/hbase-hadoop2-compat-1.1.4.jar,./dev-support/target/compatibility/1/hbase-it/target/hbase-it-1.1.4.jar,./dev-support/target/compatibility/1/hbase-prefix-tree/target/hbase
> -prefix-tree-1.1.4.jar,./dev-support/target/compatibility/1/hbase-procedure/target/hbase-procedure-1.1.4.jar,./dev-support/target/compatibility/1/hbase-protocol/target/hbase-protocol-1.1.4.jar,
> ./dev-support/target/compatibility/1/hbase-resource-bundle/target/hbase-resource-bundle-1.1.4.jar,./dev-support/target/compatibility/1/hbase-rest/target/hbase-rest-1.1.4.jar,./dev-support/targe
> t/compatibility/1/hbase-server/target/hbase-server-1.1.4.jar,./dev-support/target/compatibility/1/hbase-shell/target/hbase-shell-1.1.4.jar,./dev-support/target/compatibility/1/hbase-testing-uti
> l/target/hbase-testing-util-1.1.4.jar,./dev-support/target/compatibility/1/hbase-thrift/target/hbase-thrift-1.1.4.jar'
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16129) check_compatibility.sh is broken when using Java API Compliance Checker v1.7

[Nick Dimiduk (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367045#comment-15367045
 ] 

Nick Dimiduk commented on HBASE-16129:
--

bq. Then can we please pick back the fix to all branches so maintainers there 
will get accurate results?

Late to the party, but I usually run the script from master branch, since I 
assume it's not a high priority item for backports.

> check_compatibility.sh is broken when using Java API Compliance Checker v1.7
> 
>
> Key: HBASE-16129
> URL: https://issues.apache.org/jira/browse/HBASE-16129
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: Dima Spivak
>Assignee: Dima Spivak
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6, 0.98.21
>
> Attachments: HBASE-16129_v1.patch, HBASE-16129_v2.patch, 
> HBASE-16129_v3.patch
>
>
> As part of HBASE-16073, we hardcoded check_compatiblity.sh to check out the 
> v1.7 tag of Java ACC. Unfortunately, just running it between two branches 
> that I know have incompatibilities, I get 0 incompatibilities (and 0 classes 
> read). Looks like this version doesn't properly traverse through JARs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16158) Automate runs of check_compatibility.sh on upstream infra

[Nick Dimiduk (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367043#comment-15367043
 ] 

Nick Dimiduk commented on HBASE-16158:
--

Automation here would be great!

> Automate runs of check_compatibility.sh on upstream infra
> -
>
> Key: HBASE-16158
> URL: https://issues.apache.org/jira/browse/HBASE-16158
> Project: HBase
>  Issue Type: Task
>  Components: test
>Reporter: Dima Spivak
>Assignee: Dima Spivak
>
> Now that we got {{check_compatiblity.sh}} working again, perhaps we should 
> think about having it run regularly upstream? One possibility would be to tie 
> it into Yetus runs so that the tool gets run on every commit between the 
> branch in question and a designated Git reference (e.g. branch-1.2 could run 
> against the earlier release of the 1.2 line) and simply grepping the output 
> to make sure that the number of problems and warnings doesn't exceed a 
> designated number. The other would be to run every branch in its own Jenkins 
> job following the same workflow, but on a nightly basis. What do you guys and 
> gals think would be best?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16132) Scan does not return all the result when regionserver is busy

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367020#comment-15367020
 ] 

Hudson commented on HBASE-16132:


SUCCESS: Integrated in HBase-1.1-JDK7 #1741 (See 
[https://builds.apache.org/job/HBase-1.1-JDK7/1741/])
HBASE-16132 Scan does not return all the result when regionserver is (liyu: rev 
dcc1243f61332b696e1df193d553249b469c0e63)
* 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallableWithReplicas.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java


> Scan does not return all the result when regionserver is busy
> -
>
> Key: HBASE-16132
> URL: https://issues.apache.org/jira/browse/HBASE-16132
> Project: HBase
>  Issue Type: Bug
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-16132.patch, HBASE-16132_v2.patch, 
> HBASE-16132_v3.patch, HBASE-16132_v3.patch, TestScanMissingData.java
>
>
> We have find some corner case, when regionserver is busy and last a long 
> time. Some scanner may return null even if they do not scan all data.
> We find in ScannerCallableWithReplicas there is a case do not handler 
> correct, when cs.poll timeout and do not return any result , it is will 
> return a null result, so scan get null result, and end the scan. 
>  {code}
> try {
>   Future> f = cs.poll(timeout, 
> TimeUnit.MILLISECONDS);
>   if (f != null) {
> Pair r = f.get(timeout, 
> TimeUnit.MILLISECONDS);
> if (r != null && r.getSecond() != null) {
>   updateCurrentlyServingReplica(r.getSecond(), r.getFirst(), done, 
> pool);
> }
> return r == null ? null : r.getFirst(); // great we got an answer
>   }
> } catch (ExecutionException e) {
>   RpcRetryingCallerWithReadReplicas.throwEnrichedException(e, retries);
> } catch (CancellationException e) {
>   throw new InterruptedIOException(e.getMessage());
> } catch (InterruptedException e) {
>   throw new InterruptedIOException(e.getMessage());
> } catch (TimeoutException e) {
>   throw new InterruptedIOException(e.getMessage());
> } finally {
>   // We get there because we were interrupted or because one or more of 
> the
>   // calls succeeded or failed. In all case, we stop all our tasks.
>   cs.cancelAll();
> }
> return null; // unreachable
>  {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16190) IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must be positive

[Enis Soztutar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Enis Soztutar updated HBASE-16190:
--
   Resolution: Fixed
Fix Version/s: 1.2.3
   1.1.6
   1.4.0
   1.3.0
   2.0.0
   Status: Resolved  (was: Patch Available)

I've committed this. Thanks [~romil.choksi] and [~chenheng]. 

> IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must 
> be positive
> -
>
> Key: HBASE-16190
> URL: https://issues.apache.org/jira/browse/HBASE-16190
> Project: HBase
>  Issue Type: Bug
>Reporter: Romil Choksi
>Assignee: Romil Choksi
>Priority: Minor
>  Labels: integration-test
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.6, 1.2.3
>
> Attachments: 16190.v1.txt, HBASE-16190.patch
>
>
> IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must 
> be positive
> {code}
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:21,661 INFO  [main] hbase.IntegrationTestDDLMasterFailover: Runtime is 
> up
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:22,026 ERROR [main] hbase.IntegrationTestDDLMasterFailover: Found 
> exception in thread: Thread-11
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:21,384 INFO  [Thread-16] hbase.IntegrationTestDDLMasterFailover: 
> Performing Action: CREATE_TABLE
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:22,027 INFO  [Thread-16] hbase.IntegrationTestDDLMasterFailover: 
> Thread-16 stopped
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:20,506 INFO  [Thread-30] hbase.IntegrationTestDDLMasterFailover: 
> Performing Action: ADD_COLUMNFAMILY
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|java.lang.IllegalArgumentException:
>  n must be positive
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|at 
> java.util.Random.nextInt(Random.java:300)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.JVMRandom.nextInt(JVMRandom.java:118)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.RandomUtils.nextInt(RandomUtils.java:88)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.RandomUtils.nextInt(RandomUtils.java:74)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$TableAction.selectTable(IntegrationTestDDLMasterFailover.java:212)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$AddColumnFamilyAction.perform(IntegrationTestDDLMasterFailover.java:421)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$Worker.run(IntegrationTestDDLMasterFailover.java:695)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15682) HBase Backup Phase 3: Possible data loss during incremental WAL files copy

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366974#comment-15366974
 ] 

Ted Yu commented on HBASE-15682:


{code}
+StringUtils.join(incrBackupFileList, ",") +" to "
{code}
  public static String join(CharSequence separator, Iterable strings) {

Separator goes first.
{code}
+List converted = convertFilesFromWALtoOldWAL(missingFiles);
{code}
The above is called twice: once inside updateIncrBackupFileList() and once 
after calling updateIncrBackupFileList().
See if the return value can be reused.
{code}
+for(String path: missingFiles){
+  if(path.indexOf(Path.SEPARATOR + HConstants.HREGION_LOGDIR_NAME) < 0) {
{code}
nit: insert space between for and (.
{code}
+while(!success){
{code}
Should there be limit on the number of retries ?

> HBase Backup Phase 3: Possible data loss during incremental WAL files copy
> --
>
> Key: HBASE-15682
> URL: https://issues.apache.org/jira/browse/HBASE-15682
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: HBASE-15682-v2.patch
>
>
> We collect list of files in WAL and oldWALs directory and launch DistCp job. 
> Some files can be moved from WALs to oldWALs  directory by RS during job's 
> run, what can result in potential data loss.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16172) Unify the retry logic in ScannerCallableWithReplicas and RpcRetryingCallerWithReadReplicas

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-16172:
---
Attachment: 16172.v4.txt

> Unify the retry logic in ScannerCallableWithReplicas and 
> RpcRetryingCallerWithReadReplicas
> --
>
> Key: HBASE-16172
> URL: https://issues.apache.org/jira/browse/HBASE-16172
> Project: HBase
>  Issue Type: Bug
>Reporter: Yu Li
>Assignee: Ted Yu
> Attachments: 16172.branch-1.v4.txt, 16172.v1.txt, 16172.v2.txt, 
> 16172.v2.txt, 16172.v3.txt, 16172.v4.txt, 16172.v4.txt
>
>
> The issue is pointed out by [~devaraj] in HBASE-16132 (Thanks D.D.), that in 
> {{RpcRetryingCallerWithReadReplicas#call}} we will call 
> {{ResultBoundedCompletionService#take}} instead of {{poll}} to dead-wait on 
> the second one if the first replica timed out, while in 
> {{ScannerCallableWithReplicas#call}} we still use 
> {{ResultBoundedCompletionService#poll}} with some timeout for the 2nd replica.
> This JIRA aims at discussing whether to unify the logic in these two kinds of 
> caller with region replica and taking action if necessary.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16190) IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must be positive

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366962#comment-15366962
 ] 

Hadoop QA commented on HBASE-16190:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 1s 
{color} | {color:blue} The patch file was not named according to hbase's naming 
conventions. Please see 
https://yetus.apache.org/documentation/0.2.1/precommit-patchnames for 
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m 
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s 
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 28s 
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 0s 
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s 
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 10s 
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 31s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 21s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 21s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
36m 6s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 0s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 9s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 15s 
{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
8s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 45m 47s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12816713/16190.v1.txt |
| JIRA Issue | HBASE-16190 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / e9f5db7 |
| Default Java | 

[jira] [Commented] (HBASE-3727) MultiHFileOutputFormat

[yi liang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366963#comment-15366963
 ] 

yi liang commented on HBASE-3727:
-

i have testing HFileOutputFormat2 using Export, and importtsv. All of 
them work fine with current HFileOutputFormat2. So i think  there is no 
regression to HFileOutputFormat2.

> MultiHFileOutputFormat
> --
>
> Key: HBASE-3727
> URL: https://issues.apache.org/jira/browse/HBASE-3727
> Project: HBase
>  Issue Type: New Feature
>Affects Versions: 2.0.0
>Reporter: Andrew Purtell
>Assignee: yi liang
>Priority: Minor
> Attachments: HBASE-3727-V3.patch, MH2.patch, 
> MultiHFileOutputFormat.java, MultiHFileOutputFormat.java, 
> MultiHFileOutputFormat.java, TestMultiHFileOutputFormat.java
>
>
> Like MultiTableOutputFormat, but outputting HFiles. Key is tablename as an 
> IBW. Creates sub-writers (code cut and pasted from HFileOutputFormat) on 
> demand that produce HFiles in per-table subdirectories of the configured 
> output path. Does not currently support partitioning for existing tables / 
> incremental update.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-15988) User needs to initiate full backup for new table(s) being added for incremental backup

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu resolved HBASE-15988.

  Resolution: Fixed
Assignee: Ted Yu
Hadoop Flags: Reviewed

Thanks for the review.

Ran tests which passed.

> User needs to initiate full backup for new table(s) being added for 
> incremental backup
> --
>
> Key: HBASE-15988
> URL: https://issues.apache.org/jira/browse/HBASE-15988
> Project: HBase
>  Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Ted Yu
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: 15988.v1.txt, 15988.v2.txt, 15988.v3.txt, 15988.v4.txt
>
>
> When a new table is added to backup table set, the incremental backup 
> involving the new table should be full backup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15988) User needs to initiate full backup for new table(s) being added for incremental backup

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-15988:
---
Summary: User needs to initiate full backup for new table(s) being added 
for incremental backup  (was: Backup set add command MUST initiate full backup 
for a table(s) being added)

> User needs to initiate full backup for new table(s) being added for 
> incremental backup
> --
>
> Key: HBASE-15988
> URL: https://issues.apache.org/jira/browse/HBASE-15988
> Project: HBase
>  Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: 15988.v1.txt, 15988.v2.txt, 15988.v3.txt, 15988.v4.txt
>
>
> When a new table is added to backup table set, the incremental backup 
> involving the new table should be full backup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16132) Scan does not return all the result when regionserver is busy

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366956#comment-15366956
 ] 

Hudson commented on HBASE-16132:


FAILURE: Integrated in HBase-1.1-JDK8 #1828 (See 
[https://builds.apache.org/job/HBase-1.1-JDK8/1828/])
HBASE-16132 Scan does not return all the result when regionserver is (liyu: rev 
dcc1243f61332b696e1df193d553249b469c0e63)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java
* 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallableWithReplicas.java


> Scan does not return all the result when regionserver is busy
> -
>
> Key: HBASE-16132
> URL: https://issues.apache.org/jira/browse/HBASE-16132
> Project: HBase
>  Issue Type: Bug
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-16132.patch, HBASE-16132_v2.patch, 
> HBASE-16132_v3.patch, HBASE-16132_v3.patch, TestScanMissingData.java
>
>
> We have find some corner case, when regionserver is busy and last a long 
> time. Some scanner may return null even if they do not scan all data.
> We find in ScannerCallableWithReplicas there is a case do not handler 
> correct, when cs.poll timeout and do not return any result , it is will 
> return a null result, so scan get null result, and end the scan. 
>  {code}
> try {
>   Future> f = cs.poll(timeout, 
> TimeUnit.MILLISECONDS);
>   if (f != null) {
> Pair r = f.get(timeout, 
> TimeUnit.MILLISECONDS);
> if (r != null && r.getSecond() != null) {
>   updateCurrentlyServingReplica(r.getSecond(), r.getFirst(), done, 
> pool);
> }
> return r == null ? null : r.getFirst(); // great we got an answer
>   }
> } catch (ExecutionException e) {
>   RpcRetryingCallerWithReadReplicas.throwEnrichedException(e, retries);
> } catch (CancellationException e) {
>   throw new InterruptedIOException(e.getMessage());
> } catch (InterruptedException e) {
>   throw new InterruptedIOException(e.getMessage());
> } catch (TimeoutException e) {
>   throw new InterruptedIOException(e.getMessage());
> } finally {
>   // We get there because we were interrupted or because one or more of 
> the
>   // calls succeeded or failed. In all case, we stop all our tasks.
>   cs.cancelAll();
> }
> return null; // unreachable
>  {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15988) Backup set add command MUST initiate full backup for a table(s) being added

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366955#comment-15366955
 ] 

Vladimir Rodionov commented on HBASE-15988:
---

+1, if tests OK.

> Backup set add command MUST initiate full backup for a table(s) being added
> ---
>
> Key: HBASE-15988
> URL: https://issues.apache.org/jira/browse/HBASE-15988
> Project: HBase
>  Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: 15988.v1.txt, 15988.v2.txt, 15988.v3.txt, 15988.v4.txt
>
>
> When a new table is added to backup table set, the incremental backup 
> involving the new table should be full backup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16196) Update jruby to a newer version.

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366912#comment-15366912
 ] 

Hadoop QA commented on HBASE-16196:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 0s 
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 0s 
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s 
{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s 
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 
0s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 1s 
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 33s 
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
47s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 1s 
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 32s 
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 31s 
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 11s 
{color} | {color:red} hbase-shell in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 1m 41s 
{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 21s 
{color} | {color:red} hbase-shell in the patch failed with JDK v1.8.0. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 1m 21s 
{color} | {color:red} root in the patch failed with JDK v1.8.0. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 21s {color} 
| {color:red} hbase-shell in the patch failed with JDK v1.8.0. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 21s {color} 
| {color:red} root in the patch failed with JDK v1.8.0. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 13s 
{color} | {color:red} hbase-shell in the patch failed with JDK v1.7.0_80. 
{color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 57s 
{color} | {color:red} root in the patch failed with JDK v1.7.0_80. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 13s {color} 
| {color:red} hbase-shell in the patch failed with JDK v1.7.0_80. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 57s {color} 
| {color:red} root in the patch failed with JDK v1.7.0_80. {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s 
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m 41s 
{color} | {color:red} Patch causes 14 errors with Hadoop v2.4.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m 0s 
{color} | {color:red} Patch causes 14 errors with Hadoop v2.4.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m 21s 
{color} | {color:red} Patch causes 14 errors with Hadoop v2.5.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m 38s 
{color} | {color:red} Patch causes 14 errors with Hadoop v2.5.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m 59s 
{color} | {color:red} Patch causes 14 errors with Hadoop v2.5.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m 21s 

[jira] [Updated] (HBASE-16190) IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must be positive

[Enis Soztutar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Enis Soztutar updated HBASE-16190:
--
Assignee: Romil Choksi
  Status: Patch Available  (was: Open)

> IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must 
> be positive
> -
>
> Key: HBASE-16190
> URL: https://issues.apache.org/jira/browse/HBASE-16190
> Project: HBase
>  Issue Type: Bug
>Reporter: Romil Choksi
>Assignee: Romil Choksi
>Priority: Minor
>  Labels: integration-test
> Attachments: 16190.v1.txt, HBASE-16190.patch
>
>
> IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must 
> be positive
> {code}
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:21,661 INFO  [main] hbase.IntegrationTestDDLMasterFailover: Runtime is 
> up
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:22,026 ERROR [main] hbase.IntegrationTestDDLMasterFailover: Found 
> exception in thread: Thread-11
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:21,384 INFO  [Thread-16] hbase.IntegrationTestDDLMasterFailover: 
> Performing Action: CREATE_TABLE
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:22,027 INFO  [Thread-16] hbase.IntegrationTestDDLMasterFailover: 
> Thread-16 stopped
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:20,506 INFO  [Thread-30] hbase.IntegrationTestDDLMasterFailover: 
> Performing Action: ADD_COLUMNFAMILY
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|java.lang.IllegalArgumentException:
>  n must be positive
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|at 
> java.util.Random.nextInt(Random.java:300)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.JVMRandom.nextInt(JVMRandom.java:118)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.RandomUtils.nextInt(RandomUtils.java:88)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.RandomUtils.nextInt(RandomUtils.java:74)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$TableAction.selectTable(IntegrationTestDDLMasterFailover.java:212)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$AddColumnFamilyAction.perform(IntegrationTestDDLMasterFailover.java:421)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$Worker.run(IntegrationTestDDLMasterFailover.java:695)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16190) IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must be positive

[Romil Choksi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Romil Choksi updated HBASE-16190:
-
Attachment: 16190.v1.txt

Attaching a patch here. Verified it by re-running the test and it went fine

> IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must 
> be positive
> -
>
> Key: HBASE-16190
> URL: https://issues.apache.org/jira/browse/HBASE-16190
> Project: HBase
>  Issue Type: Bug
>Reporter: Romil Choksi
>Priority: Minor
>  Labels: integration-test
> Attachments: 16190.v1.txt, HBASE-16190.patch
>
>
> IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must 
> be positive
> {code}
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:21,661 INFO  [main] hbase.IntegrationTestDDLMasterFailover: Runtime is 
> up
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:22,026 ERROR [main] hbase.IntegrationTestDDLMasterFailover: Found 
> exception in thread: Thread-11
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:21,384 INFO  [Thread-16] hbase.IntegrationTestDDLMasterFailover: 
> Performing Action: CREATE_TABLE
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:22,027 INFO  [Thread-16] hbase.IntegrationTestDDLMasterFailover: 
> Thread-16 stopped
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:20,506 INFO  [Thread-30] hbase.IntegrationTestDDLMasterFailover: 
> Performing Action: ADD_COLUMNFAMILY
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|java.lang.IllegalArgumentException:
>  n must be positive
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|at 
> java.util.Random.nextInt(Random.java:300)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.JVMRandom.nextInt(JVMRandom.java:118)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.RandomUtils.nextInt(RandomUtils.java:88)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.RandomUtils.nextInt(RandomUtils.java:74)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$TableAction.selectTable(IntegrationTestDDLMasterFailover.java:212)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$AddColumnFamilyAction.perform(IntegrationTestDDLMasterFailover.java:421)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$Worker.run(IntegrationTestDDLMasterFailover.java:695)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15988) Backup set add command MUST initiate full backup for a table(s) being added

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-15988:
---
Attachment: 15988.v4.txt

Patch v4 addresses Vlad's latest comment

> Backup set add command MUST initiate full backup for a table(s) being added
> ---
>
> Key: HBASE-15988
> URL: https://issues.apache.org/jira/browse/HBASE-15988
> Project: HBase
>  Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: 15988.v1.txt, 15988.v2.txt, 15988.v3.txt, 15988.v4.txt
>
>
> When a new table is added to backup table set, the incremental backup 
> involving the new table should be full backup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15682) HBase Backup Phase 3: Possible data loss during incremental WAL files copy

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366840#comment-15366840
 ] 

Hadoop QA commented on HBASE-15682:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 3s {color} 
| {color:red} HBASE-15682 does not apply to master. Rebase required? Wrong 
Branch? See https://yetus.apache.org/documentation/0.2.1/precommit-patchnames 
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12816524/HBASE-15682-v2.patch |
| JIRA Issue | HBASE-15682 |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/2560/console |
| Powered by | Apache Yetus 0.2.1   http://yetus.apache.org |


This message was automatically generated.



> HBase Backup Phase 3: Possible data loss during incremental WAL files copy
> --
>
> Key: HBASE-15682
> URL: https://issues.apache.org/jira/browse/HBASE-15682
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: HBASE-15682-v2.patch
>
>
> We collect list of files in WAL and oldWALs directory and launch DistCp job. 
> Some files can be moved from WALs to oldWALs  directory by RS during job's 
> run, what can result in potential data loss.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Work stopped] (HBASE-14135) HBase Backup/Restore Phase 3: Merge backup images

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-14135 stopped by Vladimir Rodionov.
-
> HBase Backup/Restore Phase 3: Merge backup images
> -
>
> Key: HBASE-14135
> URL: https://issues.apache.org/jira/browse/HBASE-14135
> Project: HBase
>  Issue Type: New Feature
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>  Labels: backup
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Work stopped] (HBASE-15227) HBase Backup Phase 3: Fault tolerance (client/server) support

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-15227 stopped by Vladimir Rodionov.
-
> HBase Backup Phase 3: Fault tolerance (client/server) support
> -
>
> Key: HBASE-15227
> URL: https://issues.apache.org/jira/browse/HBASE-15227
> Project: HBase
>  Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>  Labels: backup
> Fix For: 2.0.0
>
>
> System must be tolerant to faults. Backup operations MUST be atomic (no 
> partial completion state in system table)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15682) HBase Backup Phase 3: Possible data loss during incremental WAL files copy

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366836#comment-15366836
 ] 

Vladimir Rodionov commented on HBASE-15682:
---

Ping [~tedyu], [~enis]

> HBase Backup Phase 3: Possible data loss during incremental WAL files copy
> --
>
> Key: HBASE-15682
> URL: https://issues.apache.org/jira/browse/HBASE-15682
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: HBASE-15682-v2.patch
>
>
> We collect list of files in WAL and oldWALs directory and launch DistCp job. 
> Some files can be moved from WALs to oldWALs  directory by RS during job's 
> run, what can result in potential data loss.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-15682) HBase Backup Phase 3: Possible data loss during incremental WAL files copy

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov updated HBASE-15682:
--
Status: Patch Available  (was: In Progress)

> HBase Backup Phase 3: Possible data loss during incremental WAL files copy
> --
>
> Key: HBASE-15682
> URL: https://issues.apache.org/jira/browse/HBASE-15682
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: HBASE-15682-v2.patch
>
>
> We collect list of files in WAL and oldWALs directory and launch DistCp job. 
> Some files can be moved from WALs to oldWALs  directory by RS during job's 
> run, what can result in potential data loss.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-16198) Enhance backup history command

[Vladimir Rodionov (JIRA)]

Vladimir Rodionov created HBASE-16198:
-

 Summary: Enhance backup history command
 Key: HBASE-16198
 URL: https://issues.apache.org/jira/browse/HBASE-16198
 Project: HBase
  Issue Type: New Feature
Affects Versions: 2.0.0
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov
 Fix For: 2.0.0


We need history per table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Work started] (HBASE-16197) Enchance backup delete command

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-16197 started by Vladimir Rodionov.
-
> Enchance backup delete command
> --
>
> Key: HBASE-16197
> URL: https://issues.apache.org/jira/browse/HBASE-16197
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
>
> Currently, backup delete command deletes only physical files/directories in 
> backup destination. It does not update backup system table (incremental 
> backup table set) and it does not delete related backup images (previous 
> ones), thus creating a hole in dependency chain. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-16197) Enchance backup delete command

[Vladimir Rodionov (JIRA)]

Vladimir Rodionov created HBASE-16197:
-

 Summary: Enchance backup delete command
 Key: HBASE-16197
 URL: https://issues.apache.org/jira/browse/HBASE-16197
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov
 Fix For: 2.0.0


Currently, backup delete command deletes only physical files/directories in 
backup destination. It does not update backup system table (incremental backup 
table set) and it does not delete related backup images (previous ones), thus 
creating a hole in dependency chain. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15988) Backup set add command MUST initiate full backup for a table(s) being added

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366798#comment-15366798
 ] 

Vladimir Rodionov commented on HBASE-15988:
---

# Please replace concat with StringUtils.join
# Fix wording in exception message: "Perform full backup on " + extraTables + " 
first, then retry the command.")


> Backup set add command MUST initiate full backup for a table(s) being added
> ---
>
> Key: HBASE-15988
> URL: https://issues.apache.org/jira/browse/HBASE-15988
> Project: HBase
>  Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: 15988.v1.txt, 15988.v2.txt, 15988.v3.txt
>
>
> When a new table is added to backup table set, the incremental backup 
> involving the new table should be full backup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15988) Backup set add command MUST initiate full backup for a table(s) being added

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366742#comment-15366742
 ] 

Ted Yu commented on HBASE-15988:


In the exception message, it is very obvious which tables need to go through 
full backup.
There is no need to dig through log file.

> Backup set add command MUST initiate full backup for a table(s) being added
> ---
>
> Key: HBASE-15988
> URL: https://issues.apache.org/jira/browse/HBASE-15988
> Project: HBase
>  Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: 15988.v1.txt, 15988.v2.txt, 15988.v3.txt
>
>
> When a new table is added to backup table set, the incremental backup 
> involving the new table should be full backup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16190) IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must be positive

[Enis Soztutar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366726#comment-15366726
 ] 

Enis Soztutar commented on HBASE-16190:
---

Patch seems right. Can we also remove the {{tableMap.isEmpty()}} check outside 
the sync block with this patch. It looks like double-checked locking. Although 
it won't cause any issues I think, we should still be safe. 

> IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must 
> be positive
> -
>
> Key: HBASE-16190
> URL: https://issues.apache.org/jira/browse/HBASE-16190
> Project: HBase
>  Issue Type: Bug
>Reporter: Romil Choksi
>Priority: Minor
>  Labels: integration-test
> Attachments: HBASE-16190.patch
>
>
> IntegrationTestDDLMasterFailover failed with IllegalArgumentException: n must 
> be positive
> {code}
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:21,661 INFO  [main] hbase.IntegrationTestDDLMasterFailover: Runtime is 
> up
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:22,026 ERROR [main] hbase.IntegrationTestDDLMasterFailover: Found 
> exception in thread: Thread-11
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:21,384 INFO  [Thread-16] hbase.IntegrationTestDDLMasterFailover: 
> Performing Action: CREATE_TABLE
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:22,027 INFO  [Thread-16] hbase.IntegrationTestDDLMasterFailover: 
> Thread-16 stopped
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|2016-07-05 
> 12:19:20,506 INFO  [Thread-30] hbase.IntegrationTestDDLMasterFailover: 
> Performing Action: ADD_COLUMNFAMILY
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|java.lang.IllegalArgumentException:
>  n must be positive
> 2016-07-05 
> 12:19:22,154|beaver.machine|INFO|4569|14008027684|MainThread|at 
> java.util.Random.nextInt(Random.java:300)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.JVMRandom.nextInt(JVMRandom.java:118)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.RandomUtils.nextInt(RandomUtils.java:88)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.commons.lang.math.RandomUtils.nextInt(RandomUtils.java:74)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$TableAction.selectTable(IntegrationTestDDLMasterFailover.java:212)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$AddColumnFamilyAction.perform(IntegrationTestDDLMasterFailover.java:421)
> 2016-07-05 
> 12:19:22,155|beaver.machine|INFO|4569|14008027684|MainThread|at 
> org.apache.hadoop.hbase.IntegrationTestDDLMasterFailover$Worker.run(IntegrationTestDDLMasterFailover.java:695)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15988) Backup set add command MUST initiate full backup for a table(s) being added

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366716#comment-15366716
 ] 

Vladimir Rodionov commented on HBASE-15988:
---

-1 on patch v3. User will have no idea what to do in this case. Delete tables 
from the set? Which one? They will need to analyze log file to understand what 
to do next? How are you going to provide full instructions in a log file, 
[~tedyu]? You will need to explain user, that 
he(she) needs to run some tables full backup first, then repeat incremental for 
backup set. You do not have this information (which backup set is used) in 
HMaster. cc: [~enis] 


> Backup set add command MUST initiate full backup for a table(s) being added
> ---
>
> Key: HBASE-15988
> URL: https://issues.apache.org/jira/browse/HBASE-15988
> Project: HBase
>  Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: 15988.v1.txt, 15988.v2.txt, 15988.v3.txt
>
>
> When a new table is added to backup table set, the incremental backup 
> involving the new table should be full backup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15935) Have a separate Walker task running concurrently with Generator

[Elliott Clark (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366715#comment-15366715
 ] 

Elliott Clark commented on HBASE-15935:
---

[~stack] You ok with a commit after the comment above ?

> Have a separate Walker task running concurrently with Generator   
> --
>
> Key: HBASE-15935
> URL: https://issues.apache.org/jira/browse/HBASE-15935
> Project: HBase
>  Issue Type: Sub-task
>  Components: integration tests
>Reporter: Joseph
>Assignee: Joseph
>Priority: Minor
> Attachments: HBASE-15935.patch
>
>
> Keep track of which linked lists have been flushed in an HBase table, so that 
> we can concurrently Walk these lists during the Generation phase. This will 
> allow us to test:
> 1. HBase under concurrent read/writes
> 2. Availability of data immediately after flushes (as opposed to waiting till 
> the Verification phase)
> The review diff can be found at:
> https://reviews.apache.org/r/48294/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16196) Update jruby to a newer version.

[Matt Mullins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Mullins updated HBASE-16196:
-
Attachment: hbase-16196.branch-1.patch

Rebase to branch-1.

> Update jruby to a newer version.
> 
>
> Key: HBASE-16196
> URL: https://issues.apache.org/jira/browse/HBASE-16196
> Project: HBase
>  Issue Type: Bug
>Reporter: Elliott Clark
>Assignee: Matt Mullins
> Attachments: 0001-Update-to-JRuby-9.1.2.0-and-JLine-2.12.patch, 
> hbase-16196.branch-1.patch
>
>
> Ruby 1.8.7 is no longer maintained.
> The TTY library in the old jruby is bad. The newer one is less bad.
> Since this is only a dependency on the hbase-shell module and not on 
> hbase-client or hbase-server this should be a pretty simple thing that 
> doesn't have any backwards compat issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16096) Replication keeps accumulating znodes

[Joseph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph updated HBASE-16096:
---
Status: Patch Available  (was: Open)

I am not super sure why affecting the ReplicationPeerZkImpl would cause the 
tests to fail, so I'm just trying a version without this change. This is not 
for committing, I am just trying to debug.

> Replication keeps accumulating znodes
> -
>
> Key: HBASE-16096
> URL: https://issues.apache.org/jira/browse/HBASE-16096
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Affects Versions: 1.2.0, 2.0.0, 1.3.0
>Reporter: Ashu Pachauri
>Assignee: Joseph
> Attachments: HBASE-16096-v1.patch, HBASE-16096-v2.patch, 
> HBASE-16096.patch
>
>
> If there is an error while creating the replication source on adding the 
> peer, the source if not added to the in memory list of sources but the 
> replication peer is. 
> However, in such a scenario, when you remove the peer, it is deleted from 
> zookeeper successfully but for removing the in memory list of peers, we wait 
> for the corresponding sources to get deleted (which as we said don't exist 
> because of error creating the source). 
> The problem here is the ordering of operations for adding/removing source and 
> peer. 
> Modifying the code to always remove queues from the underlying storage, even 
> if there exists no sources also requires a small refactoring of 
> TableBasedReplicationQueuesImpl to not abort on removeQueues() of an empty 
> queue



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16096) Replication keeps accumulating znodes

[Joseph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph updated HBASE-16096:
---
Attachment: HBASE-16096-v2.patch

> Replication keeps accumulating znodes
> -
>
> Key: HBASE-16096
> URL: https://issues.apache.org/jira/browse/HBASE-16096
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Affects Versions: 2.0.0, 1.2.0, 1.3.0
>Reporter: Ashu Pachauri
>Assignee: Joseph
> Attachments: HBASE-16096-v1.patch, HBASE-16096-v2.patch, 
> HBASE-16096.patch
>
>
> If there is an error while creating the replication source on adding the 
> peer, the source if not added to the in memory list of sources but the 
> replication peer is. 
> However, in such a scenario, when you remove the peer, it is deleted from 
> zookeeper successfully but for removing the in memory list of peers, we wait 
> for the corresponding sources to get deleted (which as we said don't exist 
> because of error creating the source). 
> The problem here is the ordering of operations for adding/removing source and 
> peer. 
> Modifying the code to always remove queues from the underlying storage, even 
> if there exists no sources also requires a small refactoring of 
> TableBasedReplicationQueuesImpl to not abort on removeQueues() of an empty 
> queue



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16096) Replication keeps accumulating znodes

[Joseph (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph updated HBASE-16096:
---
Status: Open  (was: Patch Available)

> Replication keeps accumulating znodes
> -
>
> Key: HBASE-16096
> URL: https://issues.apache.org/jira/browse/HBASE-16096
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Affects Versions: 1.2.0, 2.0.0, 1.3.0
>Reporter: Ashu Pachauri
>Assignee: Joseph
> Attachments: HBASE-16096-v1.patch, HBASE-16096-v2.patch, 
> HBASE-16096.patch
>
>
> If there is an error while creating the replication source on adding the 
> peer, the source if not added to the in memory list of sources but the 
> replication peer is. 
> However, in such a scenario, when you remove the peer, it is deleted from 
> zookeeper successfully but for removing the in memory list of peers, we wait 
> for the corresponding sources to get deleted (which as we said don't exist 
> because of error creating the source). 
> The problem here is the ordering of operations for adding/removing source and 
> peer. 
> Modifying the code to always remove queues from the underlying storage, even 
> if there exists no sources also requires a small refactoring of 
> TableBasedReplicationQueuesImpl to not abort on removeQueues() of an empty 
> queue



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16160) Get the UnsupportedOperationException when using delegation token with encryption

[Gary Helmling (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366671#comment-15366671
 ] 

Gary Helmling commented on HBASE-16160:
---

The fix looks reasonable to me, but the added test is largely a copy-n-paste of 
TestGenerateDelegationToken, with a modification to the config and an added 
test. 

Can you please refactor the test case?  Either find a way to add it in to the 
existing TestGenerateDelegationToken class or else refactor this and 
TestGenerateDelegationToken to share a common base for the configuration and 
cluster setup/teardown.

> Get the UnsupportedOperationException when using delegation token with 
> encryption
> -
>
> Key: HBASE-16160
> URL: https://issues.apache.org/jira/browse/HBASE-16160
> Project: HBase
>  Issue Type: Bug
>Reporter: Colin Ma
>Assignee: Colin Ma
> Attachments: HBASE-16160.001.patch
>
>
> Using delegation token with encryption, when do the Put operation, will get 
> the following exception:
> [RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345] 
> ipc.CallRunner(161): 
> RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345: caught: 
> java.lang.UnsupportedOperationException
> at java.nio.ByteBuffer.array(ByteBuffer.java:959)
> at 
> org.apache.hadoop.hbase.ipc.BufferChain.getBytes(BufferChain.java:66)
> at 
> org.apache.hadoop.hbase.ipc.RpcServer$Call.wrapWithSasl(RpcServer.java:547)
> at 
> org.apache.hadoop.hbase.ipc.RpcServer$Call.setResponse(RpcServer.java:467)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:140)
> at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189)
> at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11240) Print hdfs pipeline when hlog's sync is slow

[Enis Soztutar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366670#comment-15366670
 ] 

Enis Soztutar commented on HBASE-11240:
---

No worries. Thanks [~chia7712].

> Print hdfs pipeline when hlog's sync is slow
> 
>
> Key: HBASE-11240
> URL: https://issues.apache.org/jira/browse/HBASE-11240
> Project: HBase
>  Issue Type: Improvement
>  Components: Operability, wal
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
> Fix For: 0.99.0, 2.0.0
>
> Attachments: HBASE-11240-addendum.diff, HBASE-11240-trunk-v1.diff, 
> HBASE-11240-trunk-v2.diff
>
>
> Sometimes the slow sync of hlog writer is because there is an abnormal 
> datanode in the pipeline. So it will be helpful to print the pipeline of slow 
> sync to diagnose those problems.
> The ultimate solution is to join the trace of HBase and HDFS.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16174) Hook cell test up, and fix broken cell test.

[Elliott Clark (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-16174:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Hook cell test up, and fix broken cell test.
> 
>
> Key: HBASE-16174
> URL: https://issues.apache.org/jira/browse/HBASE-16174
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Attachments: HBASE-16174.HBASE-14850.patch
>
>
> Make sure that cell test is working properly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16196) Update jruby to a newer version.

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366654#comment-15366654
 ] 

Hadoop QA commented on HBASE-16196:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s {color} 
| {color:red} HBASE-16196 does not apply to master. Rebase required? Wrong 
Branch? See https://yetus.apache.org/documentation/0.2.1/precommit-patchnames 
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12816683/0001-Update-to-JRuby-9.1.2.0-and-JLine-2.12.patch
 |
| JIRA Issue | HBASE-16196 |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/2557/console |
| Powered by | Apache Yetus 0.2.1   http://yetus.apache.org |


This message was automatically generated.



> Update jruby to a newer version.
> 
>
> Key: HBASE-16196
> URL: https://issues.apache.org/jira/browse/HBASE-16196
> Project: HBase
>  Issue Type: Bug
>Reporter: Elliott Clark
>Assignee: Matt Mullins
> Attachments: 0001-Update-to-JRuby-9.1.2.0-and-JLine-2.12.patch
>
>
> Ruby 1.8.7 is no longer maintained.
> The TTY library in the old jruby is bad. The newer one is less bad.
> Since this is only a dependency on the hbase-shell module and not on 
> hbase-client or hbase-server this should be a pretty simple thing that 
> doesn't have any backwards compat issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16196) Update jruby to a newer version.

[Elliott Clark (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-16196:
--
Status: Patch Available  (was: Open)

> Update jruby to a newer version.
> 
>
> Key: HBASE-16196
> URL: https://issues.apache.org/jira/browse/HBASE-16196
> Project: HBase
>  Issue Type: Bug
>Reporter: Elliott Clark
>Assignee: Matt Mullins
> Attachments: 0001-Update-to-JRuby-9.1.2.0-and-JLine-2.12.patch
>
>
> Ruby 1.8.7 is no longer maintained.
> The TTY library in the old jruby is bad. The newer one is less bad.
> Since this is only a dependency on the hbase-shell module and not on 
> hbase-client or hbase-server this should be a pretty simple thing that 
> doesn't have any backwards compat issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16196) Update jruby to a newer version.

[Matt Mullins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Mullins updated HBASE-16196:
-
Attachment: 0001-Update-to-JRuby-9.1.2.0-and-JLine-2.12.patch

Attached initial stab at upgrading JRuby and JLine.

> Update jruby to a newer version.
> 
>
> Key: HBASE-16196
> URL: https://issues.apache.org/jira/browse/HBASE-16196
> Project: HBase
>  Issue Type: Bug
>Reporter: Elliott Clark
>Assignee: Matt Mullins
> Attachments: 0001-Update-to-JRuby-9.1.2.0-and-JLine-2.12.patch
>
>
> Ruby 1.8.7 is no longer maintained.
> The TTY library in the old jruby is bad. The newer one is less bad.
> Since this is only a dependency on the hbase-shell module and not on 
> hbase-client or hbase-server this should be a pretty simple thing that 
> doesn't have any backwards compat issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11240) Print hdfs pipeline when hlog's sync is slow

[ChiaPing Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ChiaPing Tsai updated HBASE-11240:
--
Assignee: Liu Shaohui  (was: ChiaPing Tsai)

> Print hdfs pipeline when hlog's sync is slow
> 
>
> Key: HBASE-11240
> URL: https://issues.apache.org/jira/browse/HBASE-11240
> Project: HBase
>  Issue Type: Improvement
>  Components: Operability, wal
>Reporter: Liu Shaohui
>Assignee: Liu Shaohui
> Fix For: 0.99.0, 2.0.0
>
> Attachments: HBASE-11240-addendum.diff, HBASE-11240-trunk-v1.diff, 
> HBASE-11240-trunk-v2.diff
>
>
> Sometimes the slow sync of hlog writer is because there is an abnormal 
> datanode in the pipeline. So it will be helpful to print the pipeline of slow 
> sync to diagnose those problems.
> The ultimate solution is to join the trace of HBase and HDFS.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16196) Update jruby to a newer version.

[Elliott Clark (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-16196:
--
Assignee: Matt Mullins

> Update jruby to a newer version.
> 
>
> Key: HBASE-16196
> URL: https://issues.apache.org/jira/browse/HBASE-16196
> Project: HBase
>  Issue Type: Bug
>Reporter: Elliott Clark
>Assignee: Matt Mullins
>
> Ruby 1.8.7 is no longer maintained.
> The TTY library in the old jruby is bad. The newer one is less bad.
> Since this is only a dependency on the hbase-shell module and not on 
> hbase-client or hbase-server this should be a pretty simple thing that 
> doesn't have any backwards compat issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11240) Print hdfs pipeline when hlog's sync is slow

[ChiaPing Tsai (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366635#comment-15366635
 ] 

ChiaPing Tsai commented on HBASE-11240:
---

hi [~enis]

This is my fault and it is accidentally pressed.
How can I do to restore the Assignee ?

Thanks

> Print hdfs pipeline when hlog's sync is slow
> 
>
> Key: HBASE-11240
> URL: https://issues.apache.org/jira/browse/HBASE-11240
> Project: HBase
>  Issue Type: Improvement
>  Components: Operability, wal
>Reporter: Liu Shaohui
>Assignee: ChiaPing Tsai
> Fix For: 0.99.0, 2.0.0
>
> Attachments: HBASE-11240-addendum.diff, HBASE-11240-trunk-v1.diff, 
> HBASE-11240-trunk-v2.diff
>
>
> Sometimes the slow sync of hlog writer is because there is an abnormal 
> datanode in the pipeline. So it will be helpful to print the pipeline of slow 
> sync to diagnose those problems.
> The ultimate solution is to join the trace of HBase and HDFS.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16172) Unify the retry logic in ScannerCallableWithReplicas and RpcRetryingCallerWithReadReplicas

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366628#comment-15366628
 ] 

Hadoop QA commented on HBASE-16172:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 1s 
{color} | {color:blue} The patch file was not named according to hbase's naming 
conventions. Please see 
https://yetus.apache.org/documentation/0.2.1/precommit-patchnames for 
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 14s 
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 
56s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 0s 
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s 
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
56s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
25s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 
50s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s 
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s 
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s 
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 
2s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 1s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 49s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
56s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
15m 16s {color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s 
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s 
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 34s 
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 85m 23s 
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
27s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 122m 34s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/1281/16172.branch-1.v4.txt 
|
| JIRA Issue | 

[jira] [Commented] (HBASE-15935) Have a separate Walker task running concurrently with Generator

[Joseph (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366607#comment-15366607
 ] 

Joseph commented on HBASE-15935:


I guess coming back to this, we also send a non-zero return code on Generation 
jobs that do fail this check along with the Counter's of the failed types. So 
the Loop will fail the instant the first failed Generation phase returns. Also 
it might be difficult to kill the Generation mapper job, because I'm not sure 
if the error will be reproducible, so if we throw an error the task might 
succeed on another retry? So I think it might not be necessary to fail the task 
that soon, what are your thoughts?

> Have a separate Walker task running concurrently with Generator   
> --
>
> Key: HBASE-15935
> URL: https://issues.apache.org/jira/browse/HBASE-15935
> Project: HBase
>  Issue Type: Sub-task
>  Components: integration tests
>Reporter: Joseph
>Assignee: Joseph
>Priority: Minor
> Attachments: HBASE-15935.patch
>
>
> Keep track of which linked lists have been flushed in an HBase table, so that 
> we can concurrently Walk these lists during the Generation phase. This will 
> allow us to test:
> 1. HBase under concurrent read/writes
> 2. Availability of data immediately after flushes (as opposed to waiting till 
> the Verification phase)
> The review diff can be found at:
> https://reviews.apache.org/r/48294/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16185) TestReplicationSmallTests fails in master branch

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366597#comment-15366597
 ] 

Ted Yu commented on HBASE-16185:


[~eclark]:
One more ping.

> TestReplicationSmallTests fails in master branch
> 
>
> Key: HBASE-16185
> URL: https://issues.apache.org/jira/browse/HBASE-16185
> Project: HBase
>  Issue Type: Test
>Reporter: Ted Yu
>
> As commit 581d2b7de517ee29b81b62c521ef5ca27c41f38d, the following test 
> failure can be reproduced:
> {code}
> testReplicationStatus(org.apache.hadoop.hbase.replication.TestReplicationSmallTests)
>   Time elapsed: 2.691 sec  <<< FAILURE!
> java.lang.AssertionError: failed to get ReplicationLoadSourceList
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at 
> org.apache.hadoop.hbase.replication.TestReplicationSmallTests.testReplicationStatus(TestReplicationSmallTests.java:741)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15988) Backup set add command MUST initiate full backup for a table(s) being added

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366594#comment-15366594
 ] 

Ted Yu commented on HBASE-15988:


As I mentioned previously, running incremental backup would be longer than 
expected if we trigger full backup first for the new table(s).
RPC timeout would result.

I suggest using patch v3.


> Backup set add command MUST initiate full backup for a table(s) being added
> ---
>
> Key: HBASE-15988
> URL: https://issues.apache.org/jira/browse/HBASE-15988
> Project: HBase
>  Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: 15988.v1.txt, 15988.v2.txt, 15988.v3.txt
>
>
> When a new table is added to backup table set, the incremental backup 
> involving the new table should be full backup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15988) Backup set add command MUST initiate full backup for a table(s) being added

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366570#comment-15366570
 ] 

Vladimir Rodionov commented on HBASE-15988:
---

Yeah, you right. And it should not be. OK, in this case, incremental backup for 
table set should be split into two parts:

# full backup for new tables
# followed by incremental backup for all tables in a backup set

What do you think, [~tedyu]?

> Backup set add command MUST initiate full backup for a table(s) being added
> ---
>
> Key: HBASE-15988
> URL: https://issues.apache.org/jira/browse/HBASE-15988
> Project: HBase
>  Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: 15988.v1.txt, 15988.v2.txt, 15988.v3.txt
>
>
> When a new table is added to backup table set, the incremental backup 
> involving the new table should be full backup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-16196) Update jruby to a newer version.

[Elliott Clark (JIRA)]

Elliott Clark created HBASE-16196:
-

 Summary: Update jruby to a newer version.
 Key: HBASE-16196
 URL: https://issues.apache.org/jira/browse/HBASE-16196
 Project: HBase
  Issue Type: Bug
Reporter: Elliott Clark


Ruby 1.8.7 is no longer maintained.
The TTY library in the old jruby is bad. The newer one is less bad.

Since this is only a dependency on the hbase-shell module and not on 
hbase-client or hbase-server this should be a pretty simple thing that doesn't 
have any backwards compat issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11240) Print hdfs pipeline when hlog's sync is slow

[Enis Soztutar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366529#comment-15366529
 ] 

Enis Soztutar commented on HBASE-11240:
---

[~chia7712] why did you assign this to yourself?

> Print hdfs pipeline when hlog's sync is slow
> 
>
> Key: HBASE-11240
> URL: https://issues.apache.org/jira/browse/HBASE-11240
> Project: HBase
>  Issue Type: Improvement
>  Components: Operability, wal
>Reporter: Liu Shaohui
>Assignee: ChiaPing Tsai
> Fix For: 0.99.0, 2.0.0
>
> Attachments: HBASE-11240-addendum.diff, HBASE-11240-trunk-v1.diff, 
> HBASE-11240-trunk-v2.diff
>
>
> Sometimes the slow sync of hlog writer is because there is an abnormal 
> datanode in the pipeline. So it will be helpful to print the pipeline of slow 
> sync to diagnose those problems.
> The ultimate solution is to join the trace of HBase and HDFS.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16169) RegionSizeCalculator should not depend on master

[Thiruvel Thirumoolan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366514#comment-15366514
 ] 

Thiruvel Thirumoolan commented on HBASE-16169:
--

[~eclark], This aligns with the general principle that master should be out of 
the read/write path. In this case, we use the Master API as part of split 
calculation.

We also believe this is the right direction for us internally since this 
approach scales well. ClusterStatus gets bulkier and bulkier as the cluster and 
tables grow. It also doesn't help that most of the information from 
ClusterStatus is thrown away, since it doesn't even belong to the table we are 
interested in.

> RegionSizeCalculator should not depend on master
> 
>
> Key: HBASE-16169
> URL: https://issues.apache.org/jira/browse/HBASE-16169
> Project: HBase
>  Issue Type: Sub-task
>  Components: mapreduce, scaling
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-16169.master.000.patch
>
>
> RegionSizeCalculator is needed for better split generation of MR jobs. This 
> requires RegionLoad which can be obtained via ClusterStatus, i.e. accessing 
> Master. We don't want master to be in this path.
> The proposal is to add an API to the RegionServer that gets RegionLoad of all 
> regions hosted on it or those of a table if specified. RegionSizeCalculator 
> can use the latter.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15988) Backup set add command MUST initiate full backup for a table(s) being added

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366500#comment-15366500
 ] 

Ted Yu commented on HBASE-15988:


bq. You get incremental backup set (using backup root)

Backup root is not part of backup set addition command.

> Backup set add command MUST initiate full backup for a table(s) being added
> ---
>
> Key: HBASE-15988
> URL: https://issues.apache.org/jira/browse/HBASE-15988
> Project: HBase
>  Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: 15988.v1.txt, 15988.v2.txt, 15988.v3.txt
>
>
> When a new table is added to backup table set, the incremental backup 
> involving the new table should be full backup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15988) Backup set add command MUST initiate full backup for a table(s) being added

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366497#comment-15366497
 ] 

Vladimir Rodionov commented on HBASE-15988:
---

{quote}
Without backup root dir, how would the check be done ?
{quote}

I do not think that we need backup set per backup destination. You get 
incremental backup set (using backup root), then retrieve backup set (no backup 
root), compare them and make decision. 

> Backup set add command MUST initiate full backup for a table(s) being added
> ---
>
> Key: HBASE-15988
> URL: https://issues.apache.org/jira/browse/HBASE-15988
> Project: HBase
>  Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: 15988.v1.txt, 15988.v2.txt, 15988.v3.txt
>
>
> When a new table is added to backup table set, the incremental backup 
> involving the new table should be full backup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16055) PutSortReducer loses any Visibility/acl attribute set on the Puts

[Andrew Purtell (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-16055:
---
Fix Version/s: (was: 0.98.21)

> PutSortReducer loses any Visibility/acl attribute set on the Puts 
> --
>
> Key: HBASE-16055
> URL: https://issues.apache.org/jira/browse/HBASE-16055
> Project: HBase
>  Issue Type: Bug
>  Components: security
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 2.0.0, 1.0.4, 1.4.0
>
> Attachments: HBASE-16055_1.patch, HBASE-16055_2.patch
>
>
> Based on a user discussion, I think as the user pointed out rightly, when a 
> PutSortReducer is used any visibility attribute or external attribute set on 
> the Put will be lost as we create KVs out of the cells in the puts whereas 
> the ACL and visibility are all set as Attributes. 
> In TextSortReducer we tend to read the information we tend to read the 
> information from the parsed line but here in PutSortReducer we don't do it. I 
> think this problem should be in all the existing versions where we support 
> Tags. Correct me if am wrong here. 
> [~anoop.hbase], [~andrew.purt...@gmail.com]?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16055) PutSortReducer loses any Visibility/acl attribute set on the Puts

[Andrew Purtell (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366493#comment-15366493
 ] 

Andrew Purtell commented on HBASE-16055:


The 0.98 commit ec506bb breaks compilation. I have reverted it with 60ff612

{noformat}
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile 
(default-testCompile) on project hbase-server: Compilation failure: Compilation 
failure:
[ERROR] 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java:[225,11]
 error: cannot find symbol
[ERROR] 
[ERROR] location: variable p of type Put
[ERROR] 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java:[505,4]
 error: method doIncrementalLoadTest in class TestHFileOutputFormat2 cannot be 
applied to given types;
[ERROR] required: boolean,boolean,boolean,String
[ERROR] found: boolean,boolean
[ERROR] reason: actual and formal argument lists differ in length
[ERROR] 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java:[511,4]
 error: method doIncrementalLoadTest in class TestHFileOutputFormat2 cannot be 
applied to given types;
[ERROR] required: boolean,boolean,boolean,String
[ERROR] found: boolean,boolean
[ERROR] reason: actual and formal argument lists differ in length
[ERROR] 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java:[667,4]
 error: method setupRandomGeneratorMapper in class TestHFileOutputFormat2 
cannot be applied to given types;
[ERROR] required: Job,boolean
[ERROR] found: Job
[ERROR] reason: actual and formal argument lists differ in length
[ERROR] 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java:[1019,6]
 error: method setupRandomGeneratorMapper in class TestHFileOutputFormat2 
cannot be applied to given types;
{noformat}


> PutSortReducer loses any Visibility/acl attribute set on the Puts 
> --
>
> Key: HBASE-16055
> URL: https://issues.apache.org/jira/browse/HBASE-16055
> Project: HBase
>  Issue Type: Bug
>  Components: security
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 2.0.0, 1.0.4, 1.4.0
>
> Attachments: HBASE-16055_1.patch, HBASE-16055_2.patch
>
>
> Based on a user discussion, I think as the user pointed out rightly, when a 
> PutSortReducer is used any visibility attribute or external attribute set on 
> the Put will be lost as we create KVs out of the cells in the puts whereas 
> the ACL and visibility are all set as Attributes. 
> In TextSortReducer we tend to read the information we tend to read the 
> information from the parsed line but here in PutSortReducer we don't do it. I 
> think this problem should be in all the existing versions where we support 
> Tags. Correct me if am wrong here. 
> [~anoop.hbase], [~andrew.purt...@gmail.com]?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16055) PutSortReducer loses any Visibility/acl attribute set on the Puts

[Andrew Purtell (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366481#comment-15366481
 ] 

Andrew Purtell commented on HBASE-16055:


Thanks [~ram_krish] I will check 0.98 head locally too

> PutSortReducer loses any Visibility/acl attribute set on the Puts 
> --
>
> Key: HBASE-16055
> URL: https://issues.apache.org/jira/browse/HBASE-16055
> Project: HBase
>  Issue Type: Bug
>  Components: security
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 2.0.0, 1.0.4, 1.4.0, 0.98.21
>
> Attachments: HBASE-16055_1.patch, HBASE-16055_2.patch
>
>
> Based on a user discussion, I think as the user pointed out rightly, when a 
> PutSortReducer is used any visibility attribute or external attribute set on 
> the Put will be lost as we create KVs out of the cells in the puts whereas 
> the ACL and visibility are all set as Attributes. 
> In TextSortReducer we tend to read the information we tend to read the 
> information from the parsed line but here in PutSortReducer we don't do it. I 
> think this problem should be in all the existing versions where we support 
> Tags. Correct me if am wrong here. 
> [~anoop.hbase], [~andrew.purt...@gmail.com]?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15988) Backup set add command MUST initiate full backup for a table(s) being added

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366480#comment-15366480
 ] 

Ted Yu commented on HBASE-15988:


Here is how incremental table set is retrieved:
{code}
incrTableSet = table.getIncrementalBackupTableSet(targetRootDir);
{code}
Here is API for adding table(s) to backup set:
{code}
  public void addToBackupSet(String name, TableName[] tables) throws 
IOException {
{code}
Without backup root dir, how would the check be done ?

> Backup set add command MUST initiate full backup for a table(s) being added
> ---
>
> Key: HBASE-15988
> URL: https://issues.apache.org/jira/browse/HBASE-15988
> Project: HBase
>  Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: 15988.v1.txt, 15988.v2.txt, 15988.v3.txt
>
>
> When a new table is added to backup table set, the incremental backup 
> involving the new table should be full backup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-15988) Backup set add command MUST initiate full backup for a table(s) being added

[Vladimir Rodionov (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366435#comment-15366435
 ] 

Vladimir Rodionov commented on HBASE-15988:
---

{quote}
What if user intends to add a table to existing backup set and perform full 
backup on the updated backup set ?
{quote}

[~tedyu], they can take a full backup on a set first, then add new tables with 
*-force* key.  Command must provide immediate feedback (failed, succeeded) - 
not postponed one.

> Backup set add command MUST initiate full backup for a table(s) being added
> ---
>
> Key: HBASE-15988
> URL: https://issues.apache.org/jira/browse/HBASE-15988
> Project: HBase
>  Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>  Labels: backup
> Fix For: 2.0.0
>
> Attachments: 15988.v1.txt, 15988.v2.txt, 15988.v3.txt
>
>
> When a new table is added to backup table set, the incremental backup 
> involving the new table should be full backup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-16194) Should count in MSLAB chunk allocation into heap size change when adding duplicated cells

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366429#comment-15366429
 ] 

Ted Yu commented on HBASE-16194:


{code}
158   int getCellLength(Cell cell) {
{code}
Add @VisibleForTesting to the above method.

The test is in same package as the classes. So methods can be package private 
instead of public, right ?

> Should count in MSLAB chunk allocation into heap size change when adding 
> duplicated cells
> -
>
> Key: HBASE-16194
> URL: https://issues.apache.org/jira/browse/HBASE-16194
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 1.2.1, 1.1.5, 0.98.20
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-16194.patch
>
>
> See more details about problem description and analysis in HBASE-16193



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-16172) Unify the retry logic in ScannerCallableWithReplicas and RpcRetryingCallerWithReadReplicas

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-16172:
---
Attachment: 16172.branch-1.v4.txt

> Unify the retry logic in ScannerCallableWithReplicas and 
> RpcRetryingCallerWithReadReplicas
> --
>
> Key: HBASE-16172
> URL: https://issues.apache.org/jira/browse/HBASE-16172
> Project: HBase
>  Issue Type: Bug
>Reporter: Yu Li
>Assignee: Ted Yu
> Attachments: 16172.branch-1.v4.txt, 16172.v1.txt, 16172.v2.txt, 
> 16172.v2.txt, 16172.v3.txt, 16172.v4.txt
>
>
> The issue is pointed out by [~devaraj] in HBASE-16132 (Thanks D.D.), that in 
> {{RpcRetryingCallerWithReadReplicas#call}} we will call 
> {{ResultBoundedCompletionService#take}} instead of {{poll}} to dead-wait on 
> the second one if the first replica timed out, while in 
> {{ScannerCallableWithReplicas#call}} we still use 
> {{ResultBoundedCompletionService#poll}} with some timeout for the 2nd replica.
> This JIRA aims at discussing whether to unify the logic in these two kinds of 
> caller with region replica and taking action if necessary.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)