[jira] [Updated] (HBASE-20374) End point Coprocessors do not integrate with AccessController/ VisibilityController

[Anoop Sam John (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anoop Sam John updated HBASE-20374:
---
Summary: End point Coprocessors do not integrate with AccessController/ 
VisibilityController  (was: Coprocessors do not integrate with 
VisibilityController)

> End point Coprocessors do not integrate with AccessController/ 
> VisibilityController
> ---
>
> Key: HBASE-20374
> URL: https://issues.apache.org/jira/browse/HBASE-20374
> Project: HBase
>  Issue Type: Bug
>Reporter: Jim Hughes
>Priority: Major
>
> One cannot easily combine an aggregating coprocessor with the cell-level 
> visibility implemented by the VisibilityController.  As a concrete example, 
> suppose one wanted to use the ColumnAggregation Processor with cells with 
> different authorizations.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20128) Add new UTs which extends the old replication UTs but set replication scope to SERIAL

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431749#comment-16431749
 ] 

Hadoop QA commented on HBASE-20128:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 10 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
 5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
 4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
13s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  1m  
0s{color} | {color:red} hbase-server: The patch generated 10 new + 26 unchanged 
- 0 fixed = 36 total (was 26) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
13s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
12m 51s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}148m 16s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
16s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}188m 20s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20128 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12918295/HBASE-20128.v3.patch |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux b00a2d5bcb2f 4.4.0-104-generic #127-Ubuntu SMP Mon Dec 11 
12:16:42 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 93498ddc59 |
| maven | version: Apache Maven 3.5.3 
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_162 |
| findbugs | v3.1.0-RC3 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-HBASE-Build/12365/artifact/patchprocess/diff-checkstyle-hbase-server.txt
 |
| unit | 
https://builds.apache.org/job/PreCommit-HBASE-Build/12365/artifact/patchprocess/patch-unit-hbase-server.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/12365/testReport/ |
| Max. process+thread count | 

[jira] [Commented] (HBASE-20352) [Chore] Backport HBASE-18309 to branch-1

[Yu Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431747#comment-16431747
 ] 

Yu Li commented on HBASE-20352:
---

+1, patch LGTM.

I could help commit but we need one more +1 before that :-)

> [Chore] Backport HBASE-18309 to branch-1
> 
>
> Key: HBASE-20352
> URL: https://issues.apache.org/jira/browse/HBASE-20352
> Project: HBase
>  Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
> Attachments: HBASE-20352.branch-1.001.patch
>
>
> Using multiple threads to scan directory and to clean old WALs



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20142) Copy master doc into branch-2 and edit to make it suit 2.0.0

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431673#comment-16431673
 ] 

stack commented on HBASE-20142:
---

Pushed to branch-2.0 a copy from master branch of content of src/main/*

commit 9ef75b96d196ed9169189626808e191187487f37 (HEAD -> 2.0, origin/branch-2.0)
Author: Michael Stack 
Date:   Mon Apr 9 20:30:18 2018 -0700

HBASE-20142 Copy master doc into branch-2 and edit to make it suit 2.0.0

Copy from master of the content at src/main back to branch-2.0.

I then removed backup and spark files from _chapter subdir and
applied HBASE-17918, doc on serial replication, in reverse.


Leaving this issue because will do this a few more times before we release.

> Copy master doc into branch-2 and edit to make it suit 2.0.0
> 
>
> Key: HBASE-20142
> URL: https://issues.apache.org/jira/browse/HBASE-20142
> Project: HBase
>  Issue Type: Sub-task
>Reporter: stack
>Priority: Major
>
> Place-holder for task to be done before we RC... copy the master refguide 
> into branch-2 and do a pass to make sure it hbase-2 appropriate.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19079) Support setting up two clusters with A and S state

[Duo Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-19079:
--
Attachment: HBASE-19079-HBASE-19064-v1.patch

> Support setting up two clusters with A and S state
> --
>
> Key: HBASE-19079
> URL: https://issues.apache.org/jira/browse/HBASE-19079
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>  Labels: HBASE-19064
> Fix For: HBASE-19064
>
> Attachments: HBASE-19079-HBASE-19064-v1.patch, 
> HBASE-19079-HBASE-19064.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19343) Restore snapshot makes split parent region online

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431653#comment-16431653
 ] 

Hudson commented on HBASE-19343:


SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1099 (See 
[https://builds.apache.org/job/HBase-1.2-IT/1099/])
HBASE-19343 Restore snapshot makes split parent region online (tedyu: rev 
c202144591a6399ab27e248f2ad5881f93df71df)
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/RestoreSnapshotHelper.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestRestoreSnapshotFromClient.java


> Restore snapshot makes split parent region online 
> --
>
> Key: HBASE-19343
> URL: https://issues.apache.org/jira/browse/HBASE-19343
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Major
> Fix For: 1.5.0, 1.2.7, 1.3.3, 1.4.4
>
> Attachments: 19343.tst, HBASE-19343-branch-1-v2.patch, 
> HBASE-19343-branch-1.2.patch, HBASE-19343-branch-1.3.patch, 
> HBASE-19343-branch-1.patch, Snapshot.jpg
>
>
> Restore snapshot makes parent split region online as shown in the attached 
> snapshot.
> Steps to reproduce
> =
> 1. Create table
> 2. Insert few records into the table
> 3. flush the table
> 4. Split the table
> 5. Create snapshot before catalog janitor clears the parent region entry from 
> meta.
> 6. Restore snapshot
> We can see the problem in meta entries,
> Meta content before restore snapshot:
> {noformat}
> t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:regioninfo, timestamp=1511537565964, value={ENCODED => 
> 077a12b0b3c91b053fa95223635f9543, NAME => 
> 't1,,1511537529449.077a12b0b3c91b053fa95223635f9543.', STARTKEY =>
>   '', ENDKEY => 
> '', OFFLINE => true, SPLIT => true}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:seqnumDuringOpen, timestamp=1511537530107, 
> value=\x00\x00\x00\x00\x00\x00\x00\x02
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:server, timestamp=1511537530107, value=host-xx:16020
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:serverstartcode, timestamp=1511537530107, value=1511537511523
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:splitA, timestamp=1511537565964, value={ENCODED => 
> 3c7c866d4df370c586131a4cbe0ef6a8, NAME => 
> 't1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8.', STARTKEY => '',
>   ENDKEY => 'm'}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:splitB, timestamp=1511537565964, value={ENCODED => 
> dc7facd824c85b94e5bf6a2e6b5f5efc, NAME => 
> 't1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.', STARTKEY => 'm
>  ', ENDKEY => ''}
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:regioninfo, timestamp=1511537566075, value={ENCODED => 
> 3c7c866d4df370c586131a4cbe0ef6a8, NAME => 
> 't1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8.', STARTKEY =>
>   '', ENDKEY => 
> 'm'}
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:seqnumDuringOpen, timestamp=1511537566075, 
> value=\x00\x00\x00\x00\x00\x00\x00\x02
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:server, timestamp=1511537566075, value=host-xx:16020
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:serverstartcode, timestamp=1511537566075, value=1511537511523
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:regioninfo, timestamp=1511537566069, value={ENCODED => 
> dc7facd824c85b94e5bf6a2e6b5f5efc, NAME => 
> 't1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.', STARTKEY =
>  > 'm', ENDKEY => 
> ''}
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:seqnumDuringOpen, timestamp=1511537566069, 
> value=\x00\x00\x00\x00\x00\x00\x00\x08
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:server, timestamp=1511537566069, value=host-xx:16020
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:serverstartcode, timestamp=1511537566069, value=1511537511523
> {noformat}
> Meta content after restore snapshot:
> {noformat}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:regioninfo, timestamp=1511537667635, value={ENCODED => 
> 077a12b0b3c91b053fa95223635f9543, NAME => 
> 

[jira] [Commented] (HBASE-20248) [ITBLL] UNREFERENCED rows

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431650#comment-16431650
 ] 

stack commented on HBASE-20248:
---

Ok. Tried 25B twice. First time failed with HBASE-20366. Second time ran longer 
but failed because I filled all the disks (I am keeping all WALs, recovered 
edits, and hfiles, so I can trace dataloss should it arise). Let me go back 
to 5 or 10B runs.

> [ITBLL] UNREFERENCED rows
> -
>
> Key: HBASE-20248
> URL: https://issues.apache.org/jira/browse/HBASE-20248
> Project: HBase
>  Issue Type: Sub-task
>  Components: dataloss
>Affects Versions: 2.0.0
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0
>
>
> From parent, saw unreferenced rows in a run yesterday against tip of 
> branch-2. Saw similar in a run from a week or so ago.
> Enabling DEBUG and rerunning to see if I can get to root of dataloss. See 
> https://docs.google.com/document/d/14Tvu5yWYNBDFkh8xCqLkU9tlyNWhJv3GjDGOkqZU1eE/edit#
>  for old debugging trickery.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19343) Restore snapshot makes split parent region online

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-19343:
---
Fix Version/s: 1.2.7

> Restore snapshot makes split parent region online 
> --
>
> Key: HBASE-19343
> URL: https://issues.apache.org/jira/browse/HBASE-19343
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Major
> Fix For: 1.5.0, 1.2.7, 1.3.3, 1.4.4
>
> Attachments: 19343.tst, HBASE-19343-branch-1-v2.patch, 
> HBASE-19343-branch-1.2.patch, HBASE-19343-branch-1.3.patch, 
> HBASE-19343-branch-1.patch, Snapshot.jpg
>
>
> Restore snapshot makes parent split region online as shown in the attached 
> snapshot.
> Steps to reproduce
> =
> 1. Create table
> 2. Insert few records into the table
> 3. flush the table
> 4. Split the table
> 5. Create snapshot before catalog janitor clears the parent region entry from 
> meta.
> 6. Restore snapshot
> We can see the problem in meta entries,
> Meta content before restore snapshot:
> {noformat}
> t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:regioninfo, timestamp=1511537565964, value={ENCODED => 
> 077a12b0b3c91b053fa95223635f9543, NAME => 
> 't1,,1511537529449.077a12b0b3c91b053fa95223635f9543.', STARTKEY =>
>   '', ENDKEY => 
> '', OFFLINE => true, SPLIT => true}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:seqnumDuringOpen, timestamp=1511537530107, 
> value=\x00\x00\x00\x00\x00\x00\x00\x02
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:server, timestamp=1511537530107, value=host-xx:16020
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:serverstartcode, timestamp=1511537530107, value=1511537511523
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:splitA, timestamp=1511537565964, value={ENCODED => 
> 3c7c866d4df370c586131a4cbe0ef6a8, NAME => 
> 't1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8.', STARTKEY => '',
>   ENDKEY => 'm'}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:splitB, timestamp=1511537565964, value={ENCODED => 
> dc7facd824c85b94e5bf6a2e6b5f5efc, NAME => 
> 't1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.', STARTKEY => 'm
>  ', ENDKEY => ''}
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:regioninfo, timestamp=1511537566075, value={ENCODED => 
> 3c7c866d4df370c586131a4cbe0ef6a8, NAME => 
> 't1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8.', STARTKEY =>
>   '', ENDKEY => 
> 'm'}
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:seqnumDuringOpen, timestamp=1511537566075, 
> value=\x00\x00\x00\x00\x00\x00\x00\x02
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:server, timestamp=1511537566075, value=host-xx:16020
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:serverstartcode, timestamp=1511537566075, value=1511537511523
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:regioninfo, timestamp=1511537566069, value={ENCODED => 
> dc7facd824c85b94e5bf6a2e6b5f5efc, NAME => 
> 't1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.', STARTKEY =
>  > 'm', ENDKEY => 
> ''}
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:seqnumDuringOpen, timestamp=1511537566069, 
> value=\x00\x00\x00\x00\x00\x00\x00\x08
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:server, timestamp=1511537566069, value=host-xx:16020
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:serverstartcode, timestamp=1511537566069, value=1511537511523
> {noformat}
> Meta content after restore snapshot:
> {noformat}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:regioninfo, timestamp=1511537667635, value={ENCODED => 
> 077a12b0b3c91b053fa95223635f9543, NAME => 
> 't1,,1511537529449.077a12b0b3c91b053fa95223635f9543.', STARTKEY =>
>   '', ENDKEY => 
> ''}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:seqnumDuringOpen, timestamp=1511537667635, 
> value=\x00\x00\x00\x00\x00\x00\x00\x0A
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:server, timestamp=1511537667635, value=host-xx:16020
>  

[jira] [Commented] (HBASE-19343) Restore snapshot makes split parent region online

[Pankaj Kumar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431637#comment-16431637
 ] 

Pankaj Kumar commented on HBASE-19343:
--

Hi [~yuzhih...@gmail.com], please commit this fix to branch-1.2 also, have 
already attached the patch for that.

> Restore snapshot makes split parent region online 
> --
>
> Key: HBASE-19343
> URL: https://issues.apache.org/jira/browse/HBASE-19343
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Major
> Fix For: 1.5.0, 1.3.3, 1.4.4
>
> Attachments: 19343.tst, HBASE-19343-branch-1-v2.patch, 
> HBASE-19343-branch-1.2.patch, HBASE-19343-branch-1.3.patch, 
> HBASE-19343-branch-1.patch, Snapshot.jpg
>
>
> Restore snapshot makes parent split region online as shown in the attached 
> snapshot.
> Steps to reproduce
> =
> 1. Create table
> 2. Insert few records into the table
> 3. flush the table
> 4. Split the table
> 5. Create snapshot before catalog janitor clears the parent region entry from 
> meta.
> 6. Restore snapshot
> We can see the problem in meta entries,
> Meta content before restore snapshot:
> {noformat}
> t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:regioninfo, timestamp=1511537565964, value={ENCODED => 
> 077a12b0b3c91b053fa95223635f9543, NAME => 
> 't1,,1511537529449.077a12b0b3c91b053fa95223635f9543.', STARTKEY =>
>   '', ENDKEY => 
> '', OFFLINE => true, SPLIT => true}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:seqnumDuringOpen, timestamp=1511537530107, 
> value=\x00\x00\x00\x00\x00\x00\x00\x02
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:server, timestamp=1511537530107, value=host-xx:16020
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:serverstartcode, timestamp=1511537530107, value=1511537511523
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:splitA, timestamp=1511537565964, value={ENCODED => 
> 3c7c866d4df370c586131a4cbe0ef6a8, NAME => 
> 't1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8.', STARTKEY => '',
>   ENDKEY => 'm'}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:splitB, timestamp=1511537565964, value={ENCODED => 
> dc7facd824c85b94e5bf6a2e6b5f5efc, NAME => 
> 't1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.', STARTKEY => 'm
>  ', ENDKEY => ''}
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:regioninfo, timestamp=1511537566075, value={ENCODED => 
> 3c7c866d4df370c586131a4cbe0ef6a8, NAME => 
> 't1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8.', STARTKEY =>
>   '', ENDKEY => 
> 'm'}
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:seqnumDuringOpen, timestamp=1511537566075, 
> value=\x00\x00\x00\x00\x00\x00\x00\x02
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:server, timestamp=1511537566075, value=host-xx:16020
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:serverstartcode, timestamp=1511537566075, value=1511537511523
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:regioninfo, timestamp=1511537566069, value={ENCODED => 
> dc7facd824c85b94e5bf6a2e6b5f5efc, NAME => 
> 't1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.', STARTKEY =
>  > 'm', ENDKEY => 
> ''}
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:seqnumDuringOpen, timestamp=1511537566069, 
> value=\x00\x00\x00\x00\x00\x00\x00\x08
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:server, timestamp=1511537566069, value=host-xx:16020
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:serverstartcode, timestamp=1511537566069, value=1511537511523
> {noformat}
> Meta content after restore snapshot:
> {noformat}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:regioninfo, timestamp=1511537667635, value={ENCODED => 
> 077a12b0b3c91b053fa95223635f9543, NAME => 
> 't1,,1511537529449.077a12b0b3c91b053fa95223635f9543.', STARTKEY =>
>   '', ENDKEY => 
> ''}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:seqnumDuringOpen, timestamp=1511537667635, 
> value=\x00\x00\x00\x00\x00\x00\x00\x0A
>  

[jira] [Commented] (HBASE-19343) Restore snapshot makes split parent region online

[Pankaj Kumar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431634#comment-16431634
 ] 

Pankaj Kumar commented on HBASE-19343:
--

Hi [~mdrob], this bug is not applicable for master branch, test case was 
successful in branch-2/master. 

> Restore snapshot makes split parent region online 
> --
>
> Key: HBASE-19343
> URL: https://issues.apache.org/jira/browse/HBASE-19343
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Major
> Fix For: 1.5.0, 1.3.3, 1.4.4
>
> Attachments: 19343.tst, HBASE-19343-branch-1-v2.patch, 
> HBASE-19343-branch-1.2.patch, HBASE-19343-branch-1.3.patch, 
> HBASE-19343-branch-1.patch, Snapshot.jpg
>
>
> Restore snapshot makes parent split region online as shown in the attached 
> snapshot.
> Steps to reproduce
> =
> 1. Create table
> 2. Insert few records into the table
> 3. flush the table
> 4. Split the table
> 5. Create snapshot before catalog janitor clears the parent region entry from 
> meta.
> 6. Restore snapshot
> We can see the problem in meta entries,
> Meta content before restore snapshot:
> {noformat}
> t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:regioninfo, timestamp=1511537565964, value={ENCODED => 
> 077a12b0b3c91b053fa95223635f9543, NAME => 
> 't1,,1511537529449.077a12b0b3c91b053fa95223635f9543.', STARTKEY =>
>   '', ENDKEY => 
> '', OFFLINE => true, SPLIT => true}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:seqnumDuringOpen, timestamp=1511537530107, 
> value=\x00\x00\x00\x00\x00\x00\x00\x02
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:server, timestamp=1511537530107, value=host-xx:16020
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:serverstartcode, timestamp=1511537530107, value=1511537511523
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:splitA, timestamp=1511537565964, value={ENCODED => 
> 3c7c866d4df370c586131a4cbe0ef6a8, NAME => 
> 't1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8.', STARTKEY => '',
>   ENDKEY => 'm'}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:splitB, timestamp=1511537565964, value={ENCODED => 
> dc7facd824c85b94e5bf6a2e6b5f5efc, NAME => 
> 't1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.', STARTKEY => 'm
>  ', ENDKEY => ''}
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:regioninfo, timestamp=1511537566075, value={ENCODED => 
> 3c7c866d4df370c586131a4cbe0ef6a8, NAME => 
> 't1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8.', STARTKEY =>
>   '', ENDKEY => 
> 'm'}
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:seqnumDuringOpen, timestamp=1511537566075, 
> value=\x00\x00\x00\x00\x00\x00\x00\x02
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:server, timestamp=1511537566075, value=host-xx:16020
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:serverstartcode, timestamp=1511537566075, value=1511537511523
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:regioninfo, timestamp=1511537566069, value={ENCODED => 
> dc7facd824c85b94e5bf6a2e6b5f5efc, NAME => 
> 't1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.', STARTKEY =
>  > 'm', ENDKEY => 
> ''}
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:seqnumDuringOpen, timestamp=1511537566069, 
> value=\x00\x00\x00\x00\x00\x00\x00\x08
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:server, timestamp=1511537566069, value=host-xx:16020
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:serverstartcode, timestamp=1511537566069, value=1511537511523
> {noformat}
> Meta content after restore snapshot:
> {noformat}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:regioninfo, timestamp=1511537667635, value={ENCODED => 
> 077a12b0b3c91b053fa95223635f9543, NAME => 
> 't1,,1511537529449.077a12b0b3c91b053fa95223635f9543.', STARTKEY =>
>   '', ENDKEY => 
> ''}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:seqnumDuringOpen, timestamp=1511537667635, 
> value=\x00\x00\x00\x00\x00\x00\x00\x0A
>  

[jira] [Commented] (HBASE-20243) [Shell] Add shell command to create a new table by cloning the existent table

[Guangxu Cheng (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431629#comment-16431629
 ] 

Guangxu Cheng commented on HBASE-20243:
---

rubocop warnings are unrelated.

ruby-lint does not seem to understand the code that well.

I ran the test locally and both passed for me too.

Retry again :)

> [Shell] Add shell command to create a new table by cloning the existent table
> -
>
> Key: HBASE-20243
> URL: https://issues.apache.org/jira/browse/HBASE-20243
> Project: HBase
>  Issue Type: Improvement
>  Components: shell
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Minor
> Fix For: 2.1.0
>
> Attachments: HBASE-20243.master.001.patch, 
> HBASE-20243.master.002.patch, HBASE-20243.master.003.patch, 
> HBASE-20243.master.004.patch, HBASE-20243.master.005.patch, 
> HBASE-20243.master.006.patch, HBASE-20243.master.007.patch, 
> HBASE-20243.master.008.patch, HBASE-20243.master.008.patch
>
>
> In the production environment, we need to create a new table every day. The 
> schema and the split keys of the table are the same as that of yesterday's 
> table, only the name of the table is different. For example, 
> x_20180321,x_20180322 etc.But now there is no convenient command to 
> do this. So we may need such a command(clone_table) to create a new table by 
> cloning the existent table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20243) [Shell] Add shell command to create a new table by cloning the existent table

[Guangxu Cheng (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guangxu Cheng updated HBASE-20243:
--
Attachment: HBASE-20243.master.008.patch

> [Shell] Add shell command to create a new table by cloning the existent table
> -
>
> Key: HBASE-20243
> URL: https://issues.apache.org/jira/browse/HBASE-20243
> Project: HBase
>  Issue Type: Improvement
>  Components: shell
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Minor
> Fix For: 2.1.0
>
> Attachments: HBASE-20243.master.001.patch, 
> HBASE-20243.master.002.patch, HBASE-20243.master.003.patch, 
> HBASE-20243.master.004.patch, HBASE-20243.master.005.patch, 
> HBASE-20243.master.006.patch, HBASE-20243.master.007.patch, 
> HBASE-20243.master.008.patch, HBASE-20243.master.008.patch
>
>
> In the production environment, we need to create a new table every day. The 
> schema and the split keys of the table are the same as that of yesterday's 
> table, only the name of the table is different. For example, 
> x_20180321,x_20180322 etc.But now there is no convenient command to 
> do this. So we may need such a command(clone_table) to create a new table by 
> cloning the existent table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20128) Add new UTs which extends the old replication UTs but set replication scope to SERIAL

[Zheng Hu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431610#comment-16431610
 ] 

Zheng Hu commented on HBASE-20128:
--

Hadoop QA again.

> Add new UTs which extends the old replication UTs but set replication scope 
> to SERIAL
> -
>
> Key: HBASE-20128
> URL: https://issues.apache.org/jira/browse/HBASE-20128
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-20128.v1.patch, HBASE-20128.v2.patch, 
> HBASE-20128.v3.patch, HBASE-20128.v3.patch
>
>
> Make sure that the basic function for replicationstill works. The serial 
> replication UTs are focused on order.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20128) Add new UTs which extends the old replication UTs but set replication scope to SERIAL

[Zheng Hu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Hu updated HBASE-20128:
-
Attachment: HBASE-20128.v3.patch

> Add new UTs which extends the old replication UTs but set replication scope 
> to SERIAL
> -
>
> Key: HBASE-20128
> URL: https://issues.apache.org/jira/browse/HBASE-20128
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-20128.v1.patch, HBASE-20128.v2.patch, 
> HBASE-20128.v3.patch, HBASE-20128.v3.patch
>
>
> Make sure that the basic function for replicationstill works. The serial 
> replication UTs are focused on order.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-17918) document serial replication

[Duo Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-17918:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Pushed to master and branch-2.

> document serial replication
> ---
>
> Key: HBASE-17918
> URL: https://issues.apache.org/jira/browse/HBASE-17918
> Project: HBase
>  Issue Type: Task
>  Components: documentation, Replication
>Reporter: Sean Busbey
>Assignee: Yi Mei
>Priority: Critical
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-17918-v4.patch, HBASE-17918.v1.patch, 
> HBASE-17918.v2.patch, HBASE-17918.v3.patch
>
>
> It looks like HBASE-9465 addresses one of the major flaws in our existing 
> replication (namely that order of delivery is not assured). All I see in the 
> reference guide is a note on {{hbase.serial.replication.waitingMs}}. Instead 
> we should cover this in the replication section, especially given that we 
> call out the order of delivery limitation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20375) Remove use of getCurrentUserCredentials in hbase-spark module

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431536#comment-16431536
 ] 

Hadoop QA commented on HBASE-20375:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue}  0m  
1s{color} | {color:blue} The patch file was not named according to hbase's 
naming conventions. Please see 
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for 
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
 5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green}  0m 
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green}  0m 
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green}  0m 
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  4m 
31s{color} | {color:green} hbase-spark in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
 7s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m  6s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20375 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12918289/20375.v2.txt |
| Optional Tests |  asflicense  scalac  scaladoc  unit  compile  |
| uname | Linux 78e557f2cd55 4.4.0-104-generic #127-Ubuntu SMP Mon Dec 11 
12:16:42 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 17f930c4d6 |
| maven | version: Apache Maven 3.5.3 
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/12364/testReport/ |
| Max. process+thread count | 934 (vs. ulimit of 1) |
| modules | C: hbase-spark U: hbase-spark |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/12364/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> Remove use of getCurrentUserCredentials in hbase-spark module
> -
>
> Key: HBASE-20375
> URL: https://issues.apache.org/jira/browse/HBASE-20375
> Project: HBase
>  Issue Type: Bug
>  Components: spark
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20375.v1.txt, 20375.v2.txt
>
>
> When compiling hbase-spark module against Spark 2.3.0 release, we would get:
> {code}
> [ERROR] 
> /a/hbase/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/HBaseContext.scala:68:
>  error: value getCurrentUserCredentials is not a member of 
> org.apache.spark.deploy.SparkHadoopUtil
> [ERROR]   @transient var credentials = 
> SparkHadoopUtil.get.getCurrentUserCredentials()
> [ERROR]^
> [ERROR] 
> /a/hbase/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/HBaseContext.scala:236:
>  error: value getCurrentUserCredentials is not a member of 
> org.apache.spark.deploy.   

[jira] [Commented] (HBASE-20375) Remove use of getCurrentUserCredentials in hbase-spark module

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431516#comment-16431516
 ] 

Ted Yu commented on HBASE-20375:


Patch v2 encloses the log in an if statement.

> Remove use of getCurrentUserCredentials in hbase-spark module
> -
>
> Key: HBASE-20375
> URL: https://issues.apache.org/jira/browse/HBASE-20375
> Project: HBase
>  Issue Type: Bug
>  Components: spark
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20375.v1.txt, 20375.v2.txt
>
>
> When compiling hbase-spark module against Spark 2.3.0 release, we would get:
> {code}
> [ERROR] 
> /a/hbase/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/HBaseContext.scala:68:
>  error: value getCurrentUserCredentials is not a member of 
> org.apache.spark.deploy.SparkHadoopUtil
> [ERROR]   @transient var credentials = 
> SparkHadoopUtil.get.getCurrentUserCredentials()
> [ERROR]^
> [ERROR] 
> /a/hbase/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/HBaseContext.scala:236:
>  error: value getCurrentUserCredentials is not a member of 
> org.apache.spark.deploy.   SparkHadoopUtil
> [ERROR] credentials = SparkHadoopUtil.get.getCurrentUserCredentials()
> [ERROR]   ^
> [ERROR] two errors found
> {code}
> {{getCurrentUserCredentials}} was removed by SPARK-22372.
> This issue is to replace the call to {{getCurrentUserCredentials}} with call 
> to {{UserGroupInformation.getCurrentUser().getCredentials()}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20375) Remove use of getCurrentUserCredentials in hbase-spark module

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-20375:
---
Attachment: 20375.v2.txt

> Remove use of getCurrentUserCredentials in hbase-spark module
> -
>
> Key: HBASE-20375
> URL: https://issues.apache.org/jira/browse/HBASE-20375
> Project: HBase
>  Issue Type: Bug
>  Components: spark
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20375.v1.txt, 20375.v2.txt
>
>
> When compiling hbase-spark module against Spark 2.3.0 release, we would get:
> {code}
> [ERROR] 
> /a/hbase/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/HBaseContext.scala:68:
>  error: value getCurrentUserCredentials is not a member of 
> org.apache.spark.deploy.SparkHadoopUtil
> [ERROR]   @transient var credentials = 
> SparkHadoopUtil.get.getCurrentUserCredentials()
> [ERROR]^
> [ERROR] 
> /a/hbase/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/HBaseContext.scala:236:
>  error: value getCurrentUserCredentials is not a member of 
> org.apache.spark.deploy.   SparkHadoopUtil
> [ERROR] credentials = SparkHadoopUtil.get.getCurrentUserCredentials()
> [ERROR]   ^
> [ERROR] two errors found
> {code}
> {{getCurrentUserCredentials}} was removed by SPARK-22372.
> This issue is to replace the call to {{getCurrentUserCredentials}} with call 
> to {{UserGroupInformation.getCurrentUser().getCredentials()}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20375) Remove use of getCurrentUserCredentials in hbase-spark module

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431513#comment-16431513
 ] 

Hadoop QA commented on HBASE-20375:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue}  0m  
1s{color} | {color:blue} The patch file was not named according to hbase's 
naming conventions. Please see 
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for 
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
 8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green}  0m 
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green}  0m 
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green}  0m 
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  4m 
42s{color} | {color:green} hbase-spark in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
 8s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 27s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20375 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12918286/20375.v1.txt |
| Optional Tests |  asflicense  scalac  scaladoc  unit  compile  |
| uname | Linux 93431e218682 4.4.0-104-generic #127-Ubuntu SMP Mon Dec 11 
12:16:42 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 17f930c4d6 |
| maven | version: Apache Maven 3.5.3 
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/12363/testReport/ |
| Max. process+thread count | 936 (vs. ulimit of 1) |
| modules | C: hbase-spark U: hbase-spark |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/12363/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> Remove use of getCurrentUserCredentials in hbase-spark module
> -
>
> Key: HBASE-20375
> URL: https://issues.apache.org/jira/browse/HBASE-20375
> Project: HBase
>  Issue Type: Bug
>  Components: spark
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20375.v1.txt
>
>
> When compiling hbase-spark module against Spark 2.3.0 release, we would get:
> {code}
> [ERROR] 
> /a/hbase/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/HBaseContext.scala:68:
>  error: value getCurrentUserCredentials is not a member of 
> org.apache.spark.deploy.SparkHadoopUtil
> [ERROR]   @transient var credentials = 
> SparkHadoopUtil.get.getCurrentUserCredentials()
> [ERROR]^
> [ERROR] 
> /a/hbase/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/HBaseContext.scala:236:
>  error: value getCurrentUserCredentials is not a member of 
> org.apache.spark.deploy.   SparkHadoopUtil
> [ERROR]

[jira] [Commented] (HBASE-20375) Remove use of getCurrentUserCredentials in hbase-spark module

[Wei-Chiu Chuang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431504#comment-16431504
 ] 

Wei-Chiu Chuang commented on HBASE-20375:
-

The change seems good to me. I'm curious about the next debug message:
{code:java}
logDebug("appliedCredentials:" + appliedCredentials + ",credentials:" + 
credentials)
{code}
IIUC, it would call Credentials.toString() to convert the object into string, 
regardless of log level. That doesn't sound like a great approach from a 
performance perspective.

> Remove use of getCurrentUserCredentials in hbase-spark module
> -
>
> Key: HBASE-20375
> URL: https://issues.apache.org/jira/browse/HBASE-20375
> Project: HBase
>  Issue Type: Bug
>  Components: spark
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20375.v1.txt
>
>
> When compiling hbase-spark module against Spark 2.3.0 release, we would get:
> {code}
> [ERROR] 
> /a/hbase/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/HBaseContext.scala:68:
>  error: value getCurrentUserCredentials is not a member of 
> org.apache.spark.deploy.SparkHadoopUtil
> [ERROR]   @transient var credentials = 
> SparkHadoopUtil.get.getCurrentUserCredentials()
> [ERROR]^
> [ERROR] 
> /a/hbase/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/HBaseContext.scala:236:
>  error: value getCurrentUserCredentials is not a member of 
> org.apache.spark.deploy.   SparkHadoopUtil
> [ERROR] credentials = SparkHadoopUtil.get.getCurrentUserCredentials()
> [ERROR]   ^
> [ERROR] two errors found
> {code}
> {{getCurrentUserCredentials}} was removed by SPARK-22372.
> This issue is to replace the call to {{getCurrentUserCredentials}} with call 
> to {{UserGroupInformation.getCurrentUser().getCredentials()}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19343) Restore snapshot makes split parent region online

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431497#comment-16431497
 ] 

Hudson commented on HBASE-19343:


Results for branch branch-1.3
[build #290 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/290/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/290//General_Nightly_Build_Report/]


(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/290//JDK7_Nightly_Build_Report/]


(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/290//JDK8_Nightly_Build_Report_(Hadoop2)/]




(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> Restore snapshot makes split parent region online 
> --
>
> Key: HBASE-19343
> URL: https://issues.apache.org/jira/browse/HBASE-19343
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Major
> Fix For: 1.5.0, 1.3.3, 1.4.4
>
> Attachments: 19343.tst, HBASE-19343-branch-1-v2.patch, 
> HBASE-19343-branch-1.2.patch, HBASE-19343-branch-1.3.patch, 
> HBASE-19343-branch-1.patch, Snapshot.jpg
>
>
> Restore snapshot makes parent split region online as shown in the attached 
> snapshot.
> Steps to reproduce
> =
> 1. Create table
> 2. Insert few records into the table
> 3. flush the table
> 4. Split the table
> 5. Create snapshot before catalog janitor clears the parent region entry from 
> meta.
> 6. Restore snapshot
> We can see the problem in meta entries,
> Meta content before restore snapshot:
> {noformat}
> t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:regioninfo, timestamp=1511537565964, value={ENCODED => 
> 077a12b0b3c91b053fa95223635f9543, NAME => 
> 't1,,1511537529449.077a12b0b3c91b053fa95223635f9543.', STARTKEY =>
>   '', ENDKEY => 
> '', OFFLINE => true, SPLIT => true}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:seqnumDuringOpen, timestamp=1511537530107, 
> value=\x00\x00\x00\x00\x00\x00\x00\x02
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:server, timestamp=1511537530107, value=host-xx:16020
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:serverstartcode, timestamp=1511537530107, value=1511537511523
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:splitA, timestamp=1511537565964, value={ENCODED => 
> 3c7c866d4df370c586131a4cbe0ef6a8, NAME => 
> 't1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8.', STARTKEY => '',
>   ENDKEY => 'm'}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:splitB, timestamp=1511537565964, value={ENCODED => 
> dc7facd824c85b94e5bf6a2e6b5f5efc, NAME => 
> 't1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.', STARTKEY => 'm
>  ', ENDKEY => ''}
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:regioninfo, timestamp=1511537566075, value={ENCODED => 
> 3c7c866d4df370c586131a4cbe0ef6a8, NAME => 
> 't1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8.', STARTKEY =>
>   '', ENDKEY => 
> 'm'}
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:seqnumDuringOpen, timestamp=1511537566075, 
> value=\x00\x00\x00\x00\x00\x00\x00\x02
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:server, timestamp=1511537566075, value=host-xx:16020
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:serverstartcode, timestamp=1511537566075, value=1511537511523
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:regioninfo, timestamp=1511537566069, value={ENCODED => 
> dc7facd824c85b94e5bf6a2e6b5f5efc, NAME => 
> 't1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.', STARTKEY =
>  > 'm', ENDKEY => 
> ''}
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:seqnumDuringOpen, timestamp=1511537566069, 
> value=\x00\x00\x00\x00\x00\x00\x00\x08
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:server, timestamp=1511537566069, value=host-xx:16020
>  

[jira] [Commented] (HBASE-15466) precommit should not run all java goals when given a docs-only patch

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431496#comment-16431496
 ] 

Hudson commented on HBASE-15466:


Results for branch branch-1.3
[build #290 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/290/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/290//General_Nightly_Build_Report/]


(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/290//JDK7_Nightly_Build_Report/]


(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/290//JDK8_Nightly_Build_Report_(Hadoop2)/]




(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> precommit should not run all java goals when given a docs-only patch
> 
>
> Key: HBASE-15466
> URL: https://issues.apache.org/jira/browse/HBASE-15466
> Project: HBase
>  Issue Type: Improvement
>  Components: build
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.4, 2.0.0
>
> Attachments: HBASE-15466.0.patch, HBASE-15466.1.patch, 
> HBASE-15466.2.patch, HBASE-15466.3.patch
>
>
> Right now docs-only patches (those that only impact the top level 
> src/main/site, src/main/asciidoc, or src/main/xslt) run through all of the 
> java related precommit checks, including test4tests and the full unit test 
> suite.
> Since we know these paths don't require those checks, we should update our 
> personality to skip them. (or fix our project structure to match "the maven 
> way".)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20375) Remove use of getCurrentUserCredentials in hbase-spark module

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-20375:
---
Attachment: 20375.v1.txt

> Remove use of getCurrentUserCredentials in hbase-spark module
> -
>
> Key: HBASE-20375
> URL: https://issues.apache.org/jira/browse/HBASE-20375
> Project: HBase
>  Issue Type: Bug
>  Components: spark
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20375.v1.txt
>
>
> When compiling hbase-spark module against Spark 2.3.0 release, we would get:
> {code}
> [ERROR] 
> /a/hbase/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/HBaseContext.scala:68:
>  error: value getCurrentUserCredentials is not a member of 
> org.apache.spark.deploy.SparkHadoopUtil
> [ERROR]   @transient var credentials = 
> SparkHadoopUtil.get.getCurrentUserCredentials()
> [ERROR]^
> [ERROR] 
> /a/hbase/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/HBaseContext.scala:236:
>  error: value getCurrentUserCredentials is not a member of 
> org.apache.spark.deploy.   SparkHadoopUtil
> [ERROR] credentials = SparkHadoopUtil.get.getCurrentUserCredentials()
> [ERROR]   ^
> [ERROR] two errors found
> {code}
> {{getCurrentUserCredentials}} was removed by SPARK-22372.
> This issue is to replace the call to {{getCurrentUserCredentials}} with call 
> to {{UserGroupInformation.getCurrentUser().getCredentials()}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20375) Remove use of getCurrentUserCredentials in hbase-spark module

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-20375:
---
Component/s: spark

> Remove use of getCurrentUserCredentials in hbase-spark module
> -
>
> Key: HBASE-20375
> URL: https://issues.apache.org/jira/browse/HBASE-20375
> Project: HBase
>  Issue Type: Bug
>  Components: spark
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20375.v1.txt
>
>
> When compiling hbase-spark module against Spark 2.3.0 release, we would get:
> {code}
> [ERROR] 
> /a/hbase/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/HBaseContext.scala:68:
>  error: value getCurrentUserCredentials is not a member of 
> org.apache.spark.deploy.SparkHadoopUtil
> [ERROR]   @transient var credentials = 
> SparkHadoopUtil.get.getCurrentUserCredentials()
> [ERROR]^
> [ERROR] 
> /a/hbase/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/HBaseContext.scala:236:
>  error: value getCurrentUserCredentials is not a member of 
> org.apache.spark.deploy.   SparkHadoopUtil
> [ERROR] credentials = SparkHadoopUtil.get.getCurrentUserCredentials()
> [ERROR]   ^
> [ERROR] two errors found
> {code}
> {{getCurrentUserCredentials}} was removed by SPARK-22372.
> This issue is to replace the call to {{getCurrentUserCredentials}} with call 
> to {{UserGroupInformation.getCurrentUser().getCredentials()}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20375) Remove use of getCurrentUserCredentials in hbase-spark module

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-20375:
---
Status: Patch Available  (was: Open)

> Remove use of getCurrentUserCredentials in hbase-spark module
> -
>
> Key: HBASE-20375
> URL: https://issues.apache.org/jira/browse/HBASE-20375
> Project: HBase
>  Issue Type: Bug
>  Components: spark
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20375.v1.txt
>
>
> When compiling hbase-spark module against Spark 2.3.0 release, we would get:
> {code}
> [ERROR] 
> /a/hbase/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/HBaseContext.scala:68:
>  error: value getCurrentUserCredentials is not a member of 
> org.apache.spark.deploy.SparkHadoopUtil
> [ERROR]   @transient var credentials = 
> SparkHadoopUtil.get.getCurrentUserCredentials()
> [ERROR]^
> [ERROR] 
> /a/hbase/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/HBaseContext.scala:236:
>  error: value getCurrentUserCredentials is not a member of 
> org.apache.spark.deploy.   SparkHadoopUtil
> [ERROR] credentials = SparkHadoopUtil.get.getCurrentUserCredentials()
> [ERROR]   ^
> [ERROR] two errors found
> {code}
> {{getCurrentUserCredentials}} was removed by SPARK-22372.
> This issue is to replace the call to {{getCurrentUserCredentials}} with call 
> to {{UserGroupInformation.getCurrentUser().getCredentials()}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HBASE-20375) Remove use of getCurrentUserCredentials in hbase-spark module

[Ted Yu (JIRA)]

Ted Yu created HBASE-20375:
--

 Summary: Remove use of getCurrentUserCredentials in hbase-spark 
module
 Key: HBASE-20375
 URL: https://issues.apache.org/jira/browse/HBASE-20375
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu


When compiling hbase-spark module against Spark 2.3.0 release, we would get:
{code}
[ERROR] 
/a/hbase/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/HBaseContext.scala:68:
 error: value getCurrentUserCredentials is not a member of 
org.apache.spark.deploy.SparkHadoopUtil
[ERROR]   @transient var credentials = 
SparkHadoopUtil.get.getCurrentUserCredentials()
[ERROR]^
[ERROR] 
/a/hbase/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/HBaseContext.scala:236:
 error: value getCurrentUserCredentials is not a member of 
org.apache.spark.deploy.   SparkHadoopUtil
[ERROR] credentials = SparkHadoopUtil.get.getCurrentUserCredentials()
[ERROR]   ^
[ERROR] two errors found
{code}
{{getCurrentUserCredentials}} was removed by SPARK-22372.

This issue is to replace the call to {{getCurrentUserCredentials}} with call to 
{{UserGroupInformation.getCurrentUser().getCredentials()}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-15466) precommit should not run all java goals when given a docs-only patch

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431468#comment-16431468
 ] 

Hudson commented on HBASE-15466:


Results for branch branch-2
[build #591 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/591/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/591//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/591//JDK8_Nightly_Build_Report_(Hadoop2)/]


(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/591//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> precommit should not run all java goals when given a docs-only patch
> 
>
> Key: HBASE-15466
> URL: https://issues.apache.org/jira/browse/HBASE-15466
> Project: HBase
>  Issue Type: Improvement
>  Components: build
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.4, 2.0.0
>
> Attachments: HBASE-15466.0.patch, HBASE-15466.1.patch, 
> HBASE-15466.2.patch, HBASE-15466.3.patch
>
>
> Right now docs-only patches (those that only impact the top level 
> src/main/site, src/main/asciidoc, or src/main/xslt) run through all of the 
> java related precommit checks, including test4tests and the full unit test 
> suite.
> Since we know these paths don't require those checks, we should update our 
> personality to skip them. (or fix our project structure to match "the maven 
> way".)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20363) TestNamespaceAuditor.testRegionMerge is flaky

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431467#comment-16431467
 ] 

Hudson commented on HBASE-20363:


Results for branch branch-2
[build #591 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/591/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/591//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/591//JDK8_Nightly_Build_Report_(Hadoop2)/]


(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/591//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> TestNamespaceAuditor.testRegionMerge is flaky
> -
>
> Key: HBASE-20363
> URL: https://issues.apache.org/jira/browse/HBASE-20363
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-20363-addendum.patch, HBASE-20363.patch
>
>
> I think it is easy to find out the problem. We haven't done a compaction 
> after merging and then try to split the region. The split will fail because 
> of there are still reference files.
> https://builds.apache.org/job/HBASE-Flaky-Tests/28972/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.namespace.TestNamespaceAuditor-output.txt
> {noformat}
> 2018-04-08 05:29:49,742 INFO  
> [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=43056] 
> master.HMaster$2(1644): Client=jenkins//67.195.81.155 split 
> TestNamespaceAuditor_regiontest:table2,,1523165387443.c6dd29ca77051607ab50a1edfa5f076f.
> 2018-04-08 05:29:49,745 DEBUG 
> [RpcServer.priority.FPBQ.Fifo.handler=1,queue=0,port=38141] 
> regionserver.HRegion(1360): Region 
> TestNamespaceAuditor_regiontest:table2,,1523165387443.c6dd29ca77051607ab50a1edfa5f076f.
>  is not mergeable because it has references
> 2018-04-08 05:29:49,746 DEBUG 
> [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=43056] 
> assignment.SplitTableRegionProcedure(174): Splittable=false rit=OPEN, 
> location=asf911.gq1.ygridcore.net,38141,1523165245520
> 2018-04-08 05:29:49,747 DEBUG 
> [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=43056] 
> ipc.CallRunner(142): callId: 1092 service: MasterService methodName: 
> SplitRegion size: 113 connection: 67.195.81.155:58584 deadline: 
> 1523165449742, exception=org.apache.hadoop.hbase.DoNotRetryIOException: 
> c6dd29ca77051607ab50a1edfa5f076f NOT splittable
> 2018-04-08 05:29:49,752 INFO  [Time-limited test] client.HBaseAdmin$15(907): 
> Started disable of TestNames
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20363) TestNamespaceAuditor.testRegionMerge is flaky

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431448#comment-16431448
 ] 

Hudson commented on HBASE-20363:


Results for branch branch-2.0
[build #151 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/151/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/151//General_Nightly_Build_Report/]




(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/151//JDK8_Nightly_Build_Report_(Hadoop2)/]


(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/151//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> TestNamespaceAuditor.testRegionMerge is flaky
> -
>
> Key: HBASE-20363
> URL: https://issues.apache.org/jira/browse/HBASE-20363
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-20363-addendum.patch, HBASE-20363.patch
>
>
> I think it is easy to find out the problem. We haven't done a compaction 
> after merging and then try to split the region. The split will fail because 
> of there are still reference files.
> https://builds.apache.org/job/HBASE-Flaky-Tests/28972/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.namespace.TestNamespaceAuditor-output.txt
> {noformat}
> 2018-04-08 05:29:49,742 INFO  
> [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=43056] 
> master.HMaster$2(1644): Client=jenkins//67.195.81.155 split 
> TestNamespaceAuditor_regiontest:table2,,1523165387443.c6dd29ca77051607ab50a1edfa5f076f.
> 2018-04-08 05:29:49,745 DEBUG 
> [RpcServer.priority.FPBQ.Fifo.handler=1,queue=0,port=38141] 
> regionserver.HRegion(1360): Region 
> TestNamespaceAuditor_regiontest:table2,,1523165387443.c6dd29ca77051607ab50a1edfa5f076f.
>  is not mergeable because it has references
> 2018-04-08 05:29:49,746 DEBUG 
> [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=43056] 
> assignment.SplitTableRegionProcedure(174): Splittable=false rit=OPEN, 
> location=asf911.gq1.ygridcore.net,38141,1523165245520
> 2018-04-08 05:29:49,747 DEBUG 
> [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=43056] 
> ipc.CallRunner(142): callId: 1092 service: MasterService methodName: 
> SplitRegion size: 113 connection: 67.195.81.155:58584 deadline: 
> 1523165449742, exception=org.apache.hadoop.hbase.DoNotRetryIOException: 
> c6dd29ca77051607ab50a1edfa5f076f NOT splittable
> 2018-04-08 05:29:49,752 INFO  [Time-limited test] client.HBaseAdmin$15(907): 
> Started disable of TestNames
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-15466) precommit should not run all java goals when given a docs-only patch

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431449#comment-16431449
 ] 

Hudson commented on HBASE-15466:


Results for branch branch-2.0
[build #151 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/151/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/151//General_Nightly_Build_Report/]




(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/151//JDK8_Nightly_Build_Report_(Hadoop2)/]


(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/151//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> precommit should not run all java goals when given a docs-only patch
> 
>
> Key: HBASE-15466
> URL: https://issues.apache.org/jira/browse/HBASE-15466
> Project: HBase
>  Issue Type: Improvement
>  Components: build
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.4, 2.0.0
>
> Attachments: HBASE-15466.0.patch, HBASE-15466.1.patch, 
> HBASE-15466.2.patch, HBASE-15466.3.patch
>
>
> Right now docs-only patches (those that only impact the top level 
> src/main/site, src/main/asciidoc, or src/main/xslt) run through all of the 
> java related precommit checks, including test4tests and the full unit test 
> suite.
> Since we know these paths don't require those checks, we should update our 
> personality to skip them. (or fix our project structure to match "the maven 
> way".)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19663) site build fails complaining "javadoc: error - class file for javax.annotation.meta.TypeQualifierNickname not found"

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431367#comment-16431367
 ] 

stack commented on HBASE-19663:
---

Need to figure this one out. Its in the way of my building site. I'd forgotten 
about it. Need to figure how to do w/o the workaround that drops user... doc.

> site build fails complaining "javadoc: error - class file for 
> javax.annotation.meta.TypeQualifierNickname not found"
> 
>
> Key: HBASE-19663
> URL: https://issues.apache.org/jira/browse/HBASE-19663
> Project: HBase
>  Issue Type: Bug
>  Components: website
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: script.sh
>
>
> Cryptic failure trying to build beta-1 RC. Fails like this:
> {code}
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 03:54 min
> [INFO] Finished at: 2017-12-29T01:13:15-08:00
> [INFO] Final Memory: 381M/9165M
> [INFO] 
> 
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on project 
> hbase: Error generating maven-javadoc-plugin:2.10.3:aggregate:
> [ERROR] Exit code: 1 - warning: unknown enum constant When.ALWAYS
> [ERROR] reason: class file for javax.annotation.meta.When not found
> [ERROR] warning: unknown enum constant When.UNKNOWN
> [ERROR] warning: unknown enum constant When.MAYBE
> [ERROR] 
> /home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java:762:
>  warning - Tag @link: malformed: "#matchingRows(Cell, byte[]))"
> [ERROR] 
> /home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java:762:
>  warning - Tag @link: reference not found: #matchingRows(Cell, byte[]))
> [ERROR] 
> /home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java:762:
>  warning - Tag @link: reference not found: #matchingRows(Cell, byte[]))
> [ERROR] javadoc: warning - Class javax.annotation.Nonnull not found.
> [ERROR] javadoc: error - class file for 
> javax.annotation.meta.TypeQualifierNickname not found
> [ERROR]
> [ERROR] Command line was: /home/stack/bin/jdk1.8.0_151/jre/../bin/javadoc 
> -J-Xmx2G @options @packages
> [ERROR]
> [ERROR] Refer to the generated Javadoc files in 
> '/home/stack/hbase.git/target/site/apidocs' dir.
> [ERROR] -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please 
> read the following articles:
> [ERROR] [Help 1] 
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> {code}
> javax.annotation.meta.TypeQualifierNickname is out of jsr305 but we don't 
> include this anywhere according to mvn dependency.
> Happens building the User API both test and main.
> Excluding these lines gets us passing again:
> {code}
>   3511   
>   3512 
> org.apache.yetus.audience.tools.IncludePublicAnnotationsStandardDoclet
>   3513   
>   3514   
>   3515 org.apache.yetus
>   3516 audience-annotations
>   3517 ${audience-annotations.version}
>   3518   
> + 3519   true
> {code}
> Tried upgrading to newer mvn site (ours is three years old) but that a 
> different set of problems.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19663) site build fails complaining "javadoc: error - class file for javax.annotation.meta.TypeQualifierNickname not found"

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-19663:
--
Fix Version/s: 2.0.0

> site build fails complaining "javadoc: error - class file for 
> javax.annotation.meta.TypeQualifierNickname not found"
> 
>
> Key: HBASE-19663
> URL: https://issues.apache.org/jira/browse/HBASE-19663
> Project: HBase
>  Issue Type: Bug
>  Components: website
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: script.sh
>
>
> Cryptic failure trying to build beta-1 RC. Fails like this:
> {code}
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 03:54 min
> [INFO] Finished at: 2017-12-29T01:13:15-08:00
> [INFO] Final Memory: 381M/9165M
> [INFO] 
> 
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on project 
> hbase: Error generating maven-javadoc-plugin:2.10.3:aggregate:
> [ERROR] Exit code: 1 - warning: unknown enum constant When.ALWAYS
> [ERROR] reason: class file for javax.annotation.meta.When not found
> [ERROR] warning: unknown enum constant When.UNKNOWN
> [ERROR] warning: unknown enum constant When.MAYBE
> [ERROR] 
> /home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java:762:
>  warning - Tag @link: malformed: "#matchingRows(Cell, byte[]))"
> [ERROR] 
> /home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java:762:
>  warning - Tag @link: reference not found: #matchingRows(Cell, byte[]))
> [ERROR] 
> /home/stack/hbase.git/hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java:762:
>  warning - Tag @link: reference not found: #matchingRows(Cell, byte[]))
> [ERROR] javadoc: warning - Class javax.annotation.Nonnull not found.
> [ERROR] javadoc: error - class file for 
> javax.annotation.meta.TypeQualifierNickname not found
> [ERROR]
> [ERROR] Command line was: /home/stack/bin/jdk1.8.0_151/jre/../bin/javadoc 
> -J-Xmx2G @options @packages
> [ERROR]
> [ERROR] Refer to the generated Javadoc files in 
> '/home/stack/hbase.git/target/site/apidocs' dir.
> [ERROR] -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please 
> read the following articles:
> [ERROR] [Help 1] 
> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
> {code}
> javax.annotation.meta.TypeQualifierNickname is out of jsr305 but we don't 
> include this anywhere according to mvn dependency.
> Happens building the User API both test and main.
> Excluding these lines gets us passing again:
> {code}
>   3511   
>   3512 
> org.apache.yetus.audience.tools.IncludePublicAnnotationsStandardDoclet
>   3513   
>   3514   
>   3515 org.apache.yetus
>   3516 audience-annotations
>   3517 ${audience-annotations.version}
>   3518   
> + 3519   true
> {code}
> Tried upgrading to newer mvn site (ours is three years old) but that a 
> different set of problems.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-15466) precommit should not run all java goals when given a docs-only patch

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431357#comment-16431357
 ] 

Hudson commented on HBASE-15466:


Results for branch branch-1.2
[build #294 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/294/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/294//General_Nightly_Build_Report/]


(/) {color:green}+1 jdk7 checks{color}
-- For more information [see jdk7 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/294//JDK7_Nightly_Build_Report/]


(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/294//JDK8_Nightly_Build_Report_(Hadoop2)/]




(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> precommit should not run all java goals when given a docs-only patch
> 
>
> Key: HBASE-15466
> URL: https://issues.apache.org/jira/browse/HBASE-15466
> Project: HBase
>  Issue Type: Improvement
>  Components: build
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.4, 2.0.0
>
> Attachments: HBASE-15466.0.patch, HBASE-15466.1.patch, 
> HBASE-15466.2.patch, HBASE-15466.3.patch
>
>
> Right now docs-only patches (those that only impact the top level 
> src/main/site, src/main/asciidoc, or src/main/xslt) run through all of the 
> java related precommit checks, including test4tests and the full unit test 
> suite.
> Since we know these paths don't require those checks, we should update our 
> personality to skip them. (or fix our project structure to match "the maven 
> way".)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20338) WALProcedureStore#recoverLease() should have fixed sleeps for retrying rollWriter()

[Umesh Agashe (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431343#comment-16431343
 ] 

Umesh Agashe commented on HBASE-20338:
--

[~mdrob], can you review and commit this patch? Thanks!

> WALProcedureStore#recoverLease() should have fixed sleeps for retrying 
> rollWriter()
> ---
>
> Key: HBASE-20338
> URL: https://issues.apache.org/jira/browse/HBASE-20338
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0-beta-2
>Reporter: Umesh Agashe
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HBASE-20338.master.001.patch, 
> HBASE-20338.master.002.patch, HBASE-20338.master.003.patch, 
> HBASE-20338.master.004.patch, HBASE-20338.master.005.patch
>
>
> In our internal testing we observed that logs are getting flooded due to 
> continuous loop in WALProcedureStore#recoverLease():
> {code}
>   while (isRunning()) {
> // Get Log-MaxID and recover lease on old logs
> try {
>   flushLogId = initOldLogs(oldLogs);
> } catch (FileNotFoundException e) {
>   LOG.warn("Someone else is active and deleted logs. retrying.", e);
>   oldLogs = getLogFiles();
>   continue;
> }
> // Create new state-log
> if (!rollWriter(flushLogId + 1)) {
>   // someone else has already created this log
>   LOG.debug("Someone else has already created log " + flushLogId);
>   continue;
> }
> {code}
> rollWriter() fails to create a new file. Error messages in HDFS namenode logs 
> around same time:
> {code}
> INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 8020, call 
> org.apache.hadoop.hdfs.protocol.ClientProtocol.create from 
> 172.31.121.196:38508 Call#3141 Retry#0
> java.io.IOException: Exeption while contacting value generator
> at 
> org.apache.hadoop.crypto.key.kms.ValueQueue.getAtMost(ValueQueue.java:389)
> at 
> org.apache.hadoop.crypto.key.kms.ValueQueue.getNext(ValueQueue.java:291)
> at 
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.generateEncryptedKey(KMSClientProvider.java:724)
> at 
> org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.generateEncryptedKey(KeyProviderCryptoExtension.java:511)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem$2.run(FSNamesystem.java:2680)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem$2.run(FSNamesystem.java:2676)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
> at 
> org.apache.hadoop.security.SecurityUtil.doAsUser(SecurityUtil.java:477)
> at 
> org.apache.hadoop.security.SecurityUtil.doAsLoginUser(SecurityUtil.java:458)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.generateEncryptedDataEncryptionKey(FSNamesystem.java:2675)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2815)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2712)
> at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:604)
> at 
> org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.create(AuthorizationProviderProxyClientProtocol.java:115)
> at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:412)
> at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2226)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2220)
> Caused by: java.net.ConnectException: Connection refused (Connection refused)
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at 
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> at 
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at 

[jira] [Commented] (HBASE-20338) WALProcedureStore#recoverLease() should have fixed sleeps for retrying rollWriter()

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431321#comment-16431321
 ] 

Hadoop QA commented on HBASE-20338:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
53s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
57s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
14m 52s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
35s{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
10s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 11s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20338 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12918259/HBASE-20338.master.005.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 8eb49cf37150 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 17f930c4d6 |
| maven | version: Apache Maven 3.5.3 
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_162 |
| findbugs | v3.1.0-RC3 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/12362/testReport/ |
| Max. process+thread count | 279 (vs. ulimit of 1) |
| modules | C: hbase-procedure U: hbase-procedure |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/12362/console |
| 

[jira] [Commented] (HBASE-20338) WALProcedureStore#recoverLease() should have fixed sleeps for retrying rollWriter()

[Umesh Agashe (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431270#comment-16431270
 ] 

Umesh Agashe commented on HBASE-20338:
--

+1 for 005 patch. Thanks [~jojochuang]!

> WALProcedureStore#recoverLease() should have fixed sleeps for retrying 
> rollWriter()
> ---
>
> Key: HBASE-20338
> URL: https://issues.apache.org/jira/browse/HBASE-20338
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0-beta-2
>Reporter: Umesh Agashe
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HBASE-20338.master.001.patch, 
> HBASE-20338.master.002.patch, HBASE-20338.master.003.patch, 
> HBASE-20338.master.004.patch, HBASE-20338.master.005.patch
>
>
> In our internal testing we observed that logs are getting flooded due to 
> continuous loop in WALProcedureStore#recoverLease():
> {code}
>   while (isRunning()) {
> // Get Log-MaxID and recover lease on old logs
> try {
>   flushLogId = initOldLogs(oldLogs);
> } catch (FileNotFoundException e) {
>   LOG.warn("Someone else is active and deleted logs. retrying.", e);
>   oldLogs = getLogFiles();
>   continue;
> }
> // Create new state-log
> if (!rollWriter(flushLogId + 1)) {
>   // someone else has already created this log
>   LOG.debug("Someone else has already created log " + flushLogId);
>   continue;
> }
> {code}
> rollWriter() fails to create a new file. Error messages in HDFS namenode logs 
> around same time:
> {code}
> INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 8020, call 
> org.apache.hadoop.hdfs.protocol.ClientProtocol.create from 
> 172.31.121.196:38508 Call#3141 Retry#0
> java.io.IOException: Exeption while contacting value generator
> at 
> org.apache.hadoop.crypto.key.kms.ValueQueue.getAtMost(ValueQueue.java:389)
> at 
> org.apache.hadoop.crypto.key.kms.ValueQueue.getNext(ValueQueue.java:291)
> at 
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.generateEncryptedKey(KMSClientProvider.java:724)
> at 
> org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.generateEncryptedKey(KeyProviderCryptoExtension.java:511)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem$2.run(FSNamesystem.java:2680)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem$2.run(FSNamesystem.java:2676)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
> at 
> org.apache.hadoop.security.SecurityUtil.doAsUser(SecurityUtil.java:477)
> at 
> org.apache.hadoop.security.SecurityUtil.doAsLoginUser(SecurityUtil.java:458)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.generateEncryptedDataEncryptionKey(FSNamesystem.java:2675)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2815)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2712)
> at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:604)
> at 
> org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.create(AuthorizationProviderProxyClientProtocol.java:115)
> at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:412)
> at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2226)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2220)
> Caused by: java.net.ConnectException: Connection refused (Connection refused)
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at 
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> at 
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at 
> 

[jira] [Commented] (HBASE-20149) Purge dev javadoc from bin tarball (or make a separate tarball of javadoc)

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431212#comment-16431212
 ] 

Sean Busbey commented on HBASE-20149:
-

okay addendum pushed. +1 on v3.

> Purge dev javadoc from bin tarball (or make a separate tarball of javadoc)
> --
>
> Key: HBASE-20149
> URL: https://issues.apache.org/jira/browse/HBASE-20149
> Project: HBase
>  Issue Type: Sub-task
>  Components: build, community, documentation
>Reporter: stack
>Assignee: Artem Ervits
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20149.branch-2.0.001.patch, 
> HBASE-20149.branch-2.0.002.patch, HBASE-20149.branch-2.0.003.patch
>
>
> The bin tarball is too fat (Chia-Ping and Josh noticed it on the beta-2 
> vote). A note to the dev list subsequently resulted in suggestion that we 
> just purge dev javadoc (or even all javadoc) from bin tarball (Andrew). Sean 
> was good w/ it and suggested perhaps we could do a javadoc only tgz. Let me 
> look into this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20338) WALProcedureStore#recoverLease() should have fixed sleeps for retrying rollWriter()

[Wei-Chiu Chuang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431210#comment-16431210
 ] 

Wei-Chiu Chuang commented on HBASE-20338:
-

... apparently I can't write code. Uploade rev 005.

> WALProcedureStore#recoverLease() should have fixed sleeps for retrying 
> rollWriter()
> ---
>
> Key: HBASE-20338
> URL: https://issues.apache.org/jira/browse/HBASE-20338
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0-beta-2
>Reporter: Umesh Agashe
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HBASE-20338.master.001.patch, 
> HBASE-20338.master.002.patch, HBASE-20338.master.003.patch, 
> HBASE-20338.master.004.patch, HBASE-20338.master.005.patch
>
>
> In our internal testing we observed that logs are getting flooded due to 
> continuous loop in WALProcedureStore#recoverLease():
> {code}
>   while (isRunning()) {
> // Get Log-MaxID and recover lease on old logs
> try {
>   flushLogId = initOldLogs(oldLogs);
> } catch (FileNotFoundException e) {
>   LOG.warn("Someone else is active and deleted logs. retrying.", e);
>   oldLogs = getLogFiles();
>   continue;
> }
> // Create new state-log
> if (!rollWriter(flushLogId + 1)) {
>   // someone else has already created this log
>   LOG.debug("Someone else has already created log " + flushLogId);
>   continue;
> }
> {code}
> rollWriter() fails to create a new file. Error messages in HDFS namenode logs 
> around same time:
> {code}
> INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 8020, call 
> org.apache.hadoop.hdfs.protocol.ClientProtocol.create from 
> 172.31.121.196:38508 Call#3141 Retry#0
> java.io.IOException: Exeption while contacting value generator
> at 
> org.apache.hadoop.crypto.key.kms.ValueQueue.getAtMost(ValueQueue.java:389)
> at 
> org.apache.hadoop.crypto.key.kms.ValueQueue.getNext(ValueQueue.java:291)
> at 
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.generateEncryptedKey(KMSClientProvider.java:724)
> at 
> org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.generateEncryptedKey(KeyProviderCryptoExtension.java:511)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem$2.run(FSNamesystem.java:2680)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem$2.run(FSNamesystem.java:2676)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
> at 
> org.apache.hadoop.security.SecurityUtil.doAsUser(SecurityUtil.java:477)
> at 
> org.apache.hadoop.security.SecurityUtil.doAsLoginUser(SecurityUtil.java:458)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.generateEncryptedDataEncryptionKey(FSNamesystem.java:2675)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2815)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2712)
> at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:604)
> at 
> org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.create(AuthorizationProviderProxyClientProtocol.java:115)
> at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:412)
> at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2226)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2220)
> Caused by: java.net.ConnectException: Connection refused (Connection refused)
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at 
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> at 
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at 

[jira] [Updated] (HBASE-20338) WALProcedureStore#recoverLease() should have fixed sleeps for retrying rollWriter()

[Wei-Chiu Chuang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wei-Chiu Chuang updated HBASE-20338:

Attachment: HBASE-20338.master.005.patch

> WALProcedureStore#recoverLease() should have fixed sleeps for retrying 
> rollWriter()
> ---
>
> Key: HBASE-20338
> URL: https://issues.apache.org/jira/browse/HBASE-20338
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0-beta-2
>Reporter: Umesh Agashe
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HBASE-20338.master.001.patch, 
> HBASE-20338.master.002.patch, HBASE-20338.master.003.patch, 
> HBASE-20338.master.004.patch, HBASE-20338.master.005.patch
>
>
> In our internal testing we observed that logs are getting flooded due to 
> continuous loop in WALProcedureStore#recoverLease():
> {code}
>   while (isRunning()) {
> // Get Log-MaxID and recover lease on old logs
> try {
>   flushLogId = initOldLogs(oldLogs);
> } catch (FileNotFoundException e) {
>   LOG.warn("Someone else is active and deleted logs. retrying.", e);
>   oldLogs = getLogFiles();
>   continue;
> }
> // Create new state-log
> if (!rollWriter(flushLogId + 1)) {
>   // someone else has already created this log
>   LOG.debug("Someone else has already created log " + flushLogId);
>   continue;
> }
> {code}
> rollWriter() fails to create a new file. Error messages in HDFS namenode logs 
> around same time:
> {code}
> INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 8020, call 
> org.apache.hadoop.hdfs.protocol.ClientProtocol.create from 
> 172.31.121.196:38508 Call#3141 Retry#0
> java.io.IOException: Exeption while contacting value generator
> at 
> org.apache.hadoop.crypto.key.kms.ValueQueue.getAtMost(ValueQueue.java:389)
> at 
> org.apache.hadoop.crypto.key.kms.ValueQueue.getNext(ValueQueue.java:291)
> at 
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.generateEncryptedKey(KMSClientProvider.java:724)
> at 
> org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.generateEncryptedKey(KeyProviderCryptoExtension.java:511)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem$2.run(FSNamesystem.java:2680)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem$2.run(FSNamesystem.java:2676)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
> at 
> org.apache.hadoop.security.SecurityUtil.doAsUser(SecurityUtil.java:477)
> at 
> org.apache.hadoop.security.SecurityUtil.doAsLoginUser(SecurityUtil.java:458)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.generateEncryptedDataEncryptionKey(FSNamesystem.java:2675)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2815)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2712)
> at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:604)
> at 
> org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.create(AuthorizationProviderProxyClientProtocol.java:115)
> at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:412)
> at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2226)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2220)
> Caused by: java.net.ConnectException: Connection refused (Connection refused)
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at 
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> at 
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at 
> 

[jira] [Created] (HBASE-20374) Coprocessors do not integrate with VisibilityController

[Jim Hughes (JIRA)]

Jim Hughes created HBASE-20374:
--

 Summary: Coprocessors do not integrate with VisibilityController
 Key: HBASE-20374
 URL: https://issues.apache.org/jira/browse/HBASE-20374
 Project: HBase
  Issue Type: Bug
Reporter: Jim Hughes


One cannot easily combine an aggregating coprocessor with the cell-level 
visibility implemented by the VisibilityController.  As a concrete example, 
suppose one wanted to use the ColumnAggregation Processor with cells with 
different authorizations.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20149) Purge dev javadoc from bin tarball (or make a separate tarball of javadoc)

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431186#comment-16431186
 ] 

Sean Busbey commented on HBASE-20149:
-

i thought of a clearer way to say this. We make 4 javadoc reports:

* apidocs - user facing main api javadocs. currently for a release line, 
published on website and linked from menu. included in the bin tarball as of 
now and with this patch
* devapidocs - hbase internal javadocs. currently for a release line, published 
on the website but not linked from the menu. included in the bin tarball now, 
but not after this patch.
* testapidocs - user facing test scope api javadocs. currently for a release 
line, not published. included in the bin tarball now and with this patch.
* testdevapidocs - hbase internal test scope javadocs. currently for a release 
line, not published. included in the bin tarball now but not after this patch.

does that make sense for the state of play?

300 MiB isn't that much in the scheme of things. I'll go push an addendum.

> Purge dev javadoc from bin tarball (or make a separate tarball of javadoc)
> --
>
> Key: HBASE-20149
> URL: https://issues.apache.org/jira/browse/HBASE-20149
> Project: HBase
>  Issue Type: Sub-task
>  Components: build, community, documentation
>Reporter: stack
>Assignee: Artem Ervits
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20149.branch-2.0.001.patch, 
> HBASE-20149.branch-2.0.002.patch, HBASE-20149.branch-2.0.003.patch
>
>
> The bin tarball is too fat (Chia-Ping and Josh noticed it on the beta-2 
> vote). A note to the dev list subsequently resulted in suggestion that we 
> just purge dev javadoc (or even all javadoc) from bin tarball (Andrew). Sean 
> was good w/ it and suggested perhaps we could do a javadoc only tgz. Let me 
> look into this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20149) Purge dev javadoc from bin tarball (or make a separate tarball of javadoc)

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431181#comment-16431181
 ] 

Sean Busbey commented on HBASE-20149:
-

 dev api is there. the 1.2 / 1.1 release specific docs published apidocs and 
devapidocs but only link to the user facing apidocs from the website menu.  So 
I did the same for 2.0. That means the link to the devapidocs works fine even 
though it's not in the landing page menu. But the testdevapidoc don't.

> Purge dev javadoc from bin tarball (or make a separate tarball of javadoc)
> --
>
> Key: HBASE-20149
> URL: https://issues.apache.org/jira/browse/HBASE-20149
> Project: HBase
>  Issue Type: Sub-task
>  Components: build, community, documentation
>Reporter: stack
>Assignee: Artem Ervits
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20149.branch-2.0.001.patch, 
> HBASE-20149.branch-2.0.002.patch, HBASE-20149.branch-2.0.003.patch
>
>
> The bin tarball is too fat (Chia-Ping and Josh noticed it on the beta-2 
> vote). A note to the dev list subsequently resulted in suggestion that we 
> just purge dev javadoc (or even all javadoc) from bin tarball (Andrew). Sean 
> was good w/ it and suggested perhaps we could do a javadoc only tgz. Let me 
> look into this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20338) WALProcedureStore#recoverLease() should have fixed sleeps for retrying rollWriter()

[Umesh Agashe (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431174#comment-16431174
 ] 

Umesh Agashe commented on HBASE-20338:
--

[~jojochuang], with patch 004, looks like we sleep first time only?

> WALProcedureStore#recoverLease() should have fixed sleeps for retrying 
> rollWriter()
> ---
>
> Key: HBASE-20338
> URL: https://issues.apache.org/jira/browse/HBASE-20338
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0-beta-2
>Reporter: Umesh Agashe
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HBASE-20338.master.001.patch, 
> HBASE-20338.master.002.patch, HBASE-20338.master.003.patch, 
> HBASE-20338.master.004.patch
>
>
> In our internal testing we observed that logs are getting flooded due to 
> continuous loop in WALProcedureStore#recoverLease():
> {code}
>   while (isRunning()) {
> // Get Log-MaxID and recover lease on old logs
> try {
>   flushLogId = initOldLogs(oldLogs);
> } catch (FileNotFoundException e) {
>   LOG.warn("Someone else is active and deleted logs. retrying.", e);
>   oldLogs = getLogFiles();
>   continue;
> }
> // Create new state-log
> if (!rollWriter(flushLogId + 1)) {
>   // someone else has already created this log
>   LOG.debug("Someone else has already created log " + flushLogId);
>   continue;
> }
> {code}
> rollWriter() fails to create a new file. Error messages in HDFS namenode logs 
> around same time:
> {code}
> INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 8020, call 
> org.apache.hadoop.hdfs.protocol.ClientProtocol.create from 
> 172.31.121.196:38508 Call#3141 Retry#0
> java.io.IOException: Exeption while contacting value generator
> at 
> org.apache.hadoop.crypto.key.kms.ValueQueue.getAtMost(ValueQueue.java:389)
> at 
> org.apache.hadoop.crypto.key.kms.ValueQueue.getNext(ValueQueue.java:291)
> at 
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.generateEncryptedKey(KMSClientProvider.java:724)
> at 
> org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.generateEncryptedKey(KeyProviderCryptoExtension.java:511)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem$2.run(FSNamesystem.java:2680)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem$2.run(FSNamesystem.java:2676)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
> at 
> org.apache.hadoop.security.SecurityUtil.doAsUser(SecurityUtil.java:477)
> at 
> org.apache.hadoop.security.SecurityUtil.doAsLoginUser(SecurityUtil.java:458)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.generateEncryptedDataEncryptionKey(FSNamesystem.java:2675)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2815)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2712)
> at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:604)
> at 
> org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.create(AuthorizationProviderProxyClientProtocol.java:115)
> at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:412)
> at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2226)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2220)
> Caused by: java.net.ConnectException: Connection refused (Connection refused)
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at 
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> at 
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at 
> 

[jira] [Commented] (HBASE-19343) Restore snapshot makes split parent region online

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431163#comment-16431163
 ] 

Hudson commented on HBASE-19343:


Results for branch branch-1.4
[build #282 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/282/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/282//General_Nightly_Build_Report/]


(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/282//JDK7_Nightly_Build_Report/]


(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/282//JDK8_Nightly_Build_Report_(Hadoop2)/]




(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> Restore snapshot makes split parent region online 
> --
>
> Key: HBASE-19343
> URL: https://issues.apache.org/jira/browse/HBASE-19343
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Major
> Fix For: 1.5.0, 1.3.3, 1.4.4
>
> Attachments: 19343.tst, HBASE-19343-branch-1-v2.patch, 
> HBASE-19343-branch-1.2.patch, HBASE-19343-branch-1.3.patch, 
> HBASE-19343-branch-1.patch, Snapshot.jpg
>
>
> Restore snapshot makes parent split region online as shown in the attached 
> snapshot.
> Steps to reproduce
> =
> 1. Create table
> 2. Insert few records into the table
> 3. flush the table
> 4. Split the table
> 5. Create snapshot before catalog janitor clears the parent region entry from 
> meta.
> 6. Restore snapshot
> We can see the problem in meta entries,
> Meta content before restore snapshot:
> {noformat}
> t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:regioninfo, timestamp=1511537565964, value={ENCODED => 
> 077a12b0b3c91b053fa95223635f9543, NAME => 
> 't1,,1511537529449.077a12b0b3c91b053fa95223635f9543.', STARTKEY =>
>   '', ENDKEY => 
> '', OFFLINE => true, SPLIT => true}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:seqnumDuringOpen, timestamp=1511537530107, 
> value=\x00\x00\x00\x00\x00\x00\x00\x02
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:server, timestamp=1511537530107, value=host-xx:16020
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:serverstartcode, timestamp=1511537530107, value=1511537511523
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:splitA, timestamp=1511537565964, value={ENCODED => 
> 3c7c866d4df370c586131a4cbe0ef6a8, NAME => 
> 't1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8.', STARTKEY => '',
>   ENDKEY => 'm'}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:splitB, timestamp=1511537565964, value={ENCODED => 
> dc7facd824c85b94e5bf6a2e6b5f5efc, NAME => 
> 't1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.', STARTKEY => 'm
>  ', ENDKEY => ''}
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:regioninfo, timestamp=1511537566075, value={ENCODED => 
> 3c7c866d4df370c586131a4cbe0ef6a8, NAME => 
> 't1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8.', STARTKEY =>
>   '', ENDKEY => 
> 'm'}
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:seqnumDuringOpen, timestamp=1511537566075, 
> value=\x00\x00\x00\x00\x00\x00\x00\x02
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:server, timestamp=1511537566075, value=host-xx:16020
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:serverstartcode, timestamp=1511537566075, value=1511537511523
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:regioninfo, timestamp=1511537566069, value={ENCODED => 
> dc7facd824c85b94e5bf6a2e6b5f5efc, NAME => 
> 't1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.', STARTKEY =
>  > 'm', ENDKEY => 
> ''}
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:seqnumDuringOpen, timestamp=1511537566069, 
> value=\x00\x00\x00\x00\x00\x00\x00\x08
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:server, timestamp=1511537566069, value=host-xx:16020
>  

[jira] [Commented] (HBASE-15466) precommit should not run all java goals when given a docs-only patch

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431162#comment-16431162
 ] 

Hudson commented on HBASE-15466:


Results for branch branch-1.4
[build #282 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/282/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/282//General_Nightly_Build_Report/]


(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/282//JDK7_Nightly_Build_Report/]


(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/282//JDK8_Nightly_Build_Report_(Hadoop2)/]




(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> precommit should not run all java goals when given a docs-only patch
> 
>
> Key: HBASE-15466
> URL: https://issues.apache.org/jira/browse/HBASE-15466
> Project: HBase
>  Issue Type: Improvement
>  Components: build
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.4, 2.0.0
>
> Attachments: HBASE-15466.0.patch, HBASE-15466.1.patch, 
> HBASE-15466.2.patch, HBASE-15466.3.patch
>
>
> Right now docs-only patches (those that only impact the top level 
> src/main/site, src/main/asciidoc, or src/main/xslt) run through all of the 
> java related precommit checks, including test4tests and the full unit test 
> suite.
> Since we know these paths don't require those checks, we should update our 
> personality to skip them. (or fix our project structure to match "the maven 
> way".)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20188) [TESTING] Performance

[Eshcar Hillel (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431148#comment-16431148
 ] 

Eshcar Hillel commented on HBASE-20188:
---

I attached the results of some additional experiments we ran with 8GB heap.
We created a new workload  [^workloadx]. It is a write-only skewed 
distribution, with client side batching.
Results show that IMC with 2% active and 2 pipeline segments has an advantage 
over none in write-only workloads.
Script to run the experiments is here  [^HBASE-20188-xac.sh], and relevant 
hbase-site configuration here  [^hbase-site.xml] 
we would still like to investigate the read latency with IMC we have some 
direction we plan to explore; will come back with results.

> [TESTING] Performance
> -
>
> Key: HBASE-20188
> URL: https://issues.apache.org/jira/browse/HBASE-20188
> Project: HBase
>  Issue Type: Umbrella
>  Components: Performance
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, 
> HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB.pdf, HBase 2.0 
> performance evaluation - Basic vs None_ system settings.pdf, 
> ITBLL2.5B_1.2.7vs2.0.0_cpu.png, ITBLL2.5B_1.2.7vs2.0.0_gctime.png, 
> ITBLL2.5B_1.2.7vs2.0.0_iops.png, ITBLL2.5B_1.2.7vs2.0.0_load.png, 
> ITBLL2.5B_1.2.7vs2.0.0_memheap.png, ITBLL2.5B_1.2.7vs2.0.0_memstore.png, 
> ITBLL2.5B_1.2.7vs2.0.0_ops.png, 
> ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, 
> YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, 
> YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, 
> flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, 
> hbase-site.xml, lock.127.workloadc.20180402T200918Z.svg, 
> lock.2.memsize2.c.20180403T160257Z.svg, run_ycsb.sh, tree.txt, workloadx
>
>
> How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor 
> that it is much slower, that the problem is the asyncwal writing. Does 
> in-memory compaction slow us down or speed us up? What happens when you 
> enable offheaping?
> Keep notes here in this umbrella issue. Need to be able to say something 
> about perf when 2.0.0 ships.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20188) [TESTING] Performance

[Eshcar Hillel (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eshcar Hillel updated HBASE-20188:
--
Attachment: hbase-site.xml
workloadx
HBASE-20188-xac.sh

> [TESTING] Performance
> -
>
> Key: HBASE-20188
> URL: https://issues.apache.org/jira/browse/HBASE-20188
> Project: HBase
>  Issue Type: Umbrella
>  Components: Performance
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, 
> HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB.pdf, HBase 2.0 
> performance evaluation - Basic vs None_ system settings.pdf, 
> ITBLL2.5B_1.2.7vs2.0.0_cpu.png, ITBLL2.5B_1.2.7vs2.0.0_gctime.png, 
> ITBLL2.5B_1.2.7vs2.0.0_iops.png, ITBLL2.5B_1.2.7vs2.0.0_load.png, 
> ITBLL2.5B_1.2.7vs2.0.0_memheap.png, ITBLL2.5B_1.2.7vs2.0.0_memstore.png, 
> ITBLL2.5B_1.2.7vs2.0.0_ops.png, 
> ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, 
> YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, 
> YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, 
> flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, 
> hbase-site.xml, lock.127.workloadc.20180402T200918Z.svg, 
> lock.2.memsize2.c.20180403T160257Z.svg, run_ycsb.sh, tree.txt, workloadx
>
>
> How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor 
> that it is much slower, that the problem is the asyncwal writing. Does 
> in-memory compaction slow us down or speed us up? What happens when you 
> enable offheaping?
> Keep notes here in this umbrella issue. Need to be able to say something 
> about perf when 2.0.0 ships.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20188) [TESTING] Performance

[Eshcar Hillel (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eshcar Hillel updated HBASE-20188:
--
Attachment: HBase 2.0 performance evaluation - 8GB.pdf

> [TESTING] Performance
> -
>
> Key: HBASE-20188
> URL: https://issues.apache.org/jira/browse/HBASE-20188
> Project: HBase
>  Issue Type: Umbrella
>  Components: Performance
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: CAM-CONFIG-V01.patch, HBASE-20188.sh, HBase 2.0 
> performance evaluation - 8GB.pdf, HBase 2.0 performance evaluation - Basic vs 
> None_ system settings.pdf, ITBLL2.5B_1.2.7vs2.0.0_cpu.png, 
> ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, 
> ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, 
> ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, 
> ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, 
> YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, 
> YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, 
> flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, 
> lock.127.workloadc.20180402T200918Z.svg, 
> lock.2.memsize2.c.20180403T160257Z.svg, run_ycsb.sh, tree.txt
>
>
> How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor 
> that it is much slower, that the problem is the asyncwal writing. Does 
> in-memory compaction slow us down or speed us up? What happens when you 
> enable offheaping?
> Keep notes here in this umbrella issue. Need to be able to say something 
> about perf when 2.0.0 ships.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19761) Fix Checkstyle errors in hbase-zookeeper

[Jan Hentschel (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431115#comment-16431115
 ] 

Jan Hentschel commented on HBASE-19761:
---

I like the idea from [~mdrob].

> Fix Checkstyle errors in hbase-zookeeper
> 
>
> Key: HBASE-19761
> URL: https://issues.apache.org/jira/browse/HBASE-19761
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Jan Hentschel
>Assignee: maoling
>Priority: Minor
> Attachments: HBASE-19761-master-v0.patch
>
>
> Fix the remaining Checkstyle errors in the *hbase-zookeeper* module and 
> enable Checkstyle to fail on violations.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20149) Purge dev javadoc from bin tarball (or make a separate tarball of javadoc)

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1643#comment-1643
 ] 

stack commented on HBASE-20149:
---

Removing dev api from hbase.apache.org menu?

Or, just not building dev api?

Either are good by me including pushing the dev api... I'm easy. Might be ok 
having devapi up on website... 

> Purge dev javadoc from bin tarball (or make a separate tarball of javadoc)
> --
>
> Key: HBASE-20149
> URL: https://issues.apache.org/jira/browse/HBASE-20149
> Project: HBase
>  Issue Type: Sub-task
>  Components: build, community, documentation
>Reporter: stack
>Assignee: Artem Ervits
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20149.branch-2.0.001.patch, 
> HBASE-20149.branch-2.0.002.patch, HBASE-20149.branch-2.0.003.patch
>
>
> The bin tarball is too fat (Chia-Ping and Josh noticed it on the beta-2 
> vote). A note to the dev list subsequently resulted in suggestion that we 
> just purge dev javadoc (or even all javadoc) from bin tarball (Andrew). Sean 
> was good w/ it and suggested perhaps we could do a javadoc only tgz. Let me 
> look into this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20149) Purge dev javadoc from bin tarball (or make a separate tarball of javadoc)

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431102#comment-16431102
 ] 

Sean Busbey commented on HBASE-20149:
-

shoot. the {{testdevapidocs}} link doesn't work because in HBASE-20365 I left 
out the {{test*docs}}.

What do you think about just leaving it out? I could push an addendum that 
includes {{test*docs}} now (~300MiB more on top of the ~500MiB already in the 
v2 docs) , or we could wait until someone asks about it.

> Purge dev javadoc from bin tarball (or make a separate tarball of javadoc)
> --
>
> Key: HBASE-20149
> URL: https://issues.apache.org/jira/browse/HBASE-20149
> Project: HBase
>  Issue Type: Sub-task
>  Components: build, community, documentation
>Reporter: stack
>Assignee: Artem Ervits
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20149.branch-2.0.001.patch, 
> HBASE-20149.branch-2.0.002.patch, HBASE-20149.branch-2.0.003.patch
>
>
> The bin tarball is too fat (Chia-Ping and Josh noticed it on the beta-2 
> vote). A note to the dev list subsequently resulted in suggestion that we 
> just purge dev javadoc (or even all javadoc) from bin tarball (Andrew). Sean 
> was good w/ it and suggested perhaps we could do a javadoc only tgz. Let me 
> look into this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-15466) precommit should not run all java goals when given a docs-only patch

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431084#comment-16431084
 ] 

Hudson commented on HBASE-15466:


Results for branch master
[build #290 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/290/]: (x) 
*{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/290//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/290//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/290//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> precommit should not run all java goals when given a docs-only patch
> 
>
> Key: HBASE-15466
> URL: https://issues.apache.org/jira/browse/HBASE-15466
> Project: HBase
>  Issue Type: Improvement
>  Components: build
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.5.0, 1.2.7, 1.3.3, 1.4.4, 2.0.0
>
> Attachments: HBASE-15466.0.patch, HBASE-15466.1.patch, 
> HBASE-15466.2.patch, HBASE-15466.3.patch
>
>
> Right now docs-only patches (those that only impact the top level 
> src/main/site, src/main/asciidoc, or src/main/xslt) run through all of the 
> java related precommit checks, including test4tests and the full unit test 
> suite.
> Since we know these paths don't require those checks, we should update our 
> personality to skip them. (or fix our project structure to match "the maven 
> way".)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20322) CME in StoreScanner causes region server crash

[Mike Drob (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431054#comment-16431054
 ] 

Mike Drob commented on HBASE-20322:
---

Thanks!

> CME in StoreScanner causes region server crash
> --
>
> Key: HBASE-20322
> URL: https://issues.apache.org/jira/browse/HBASE-20322
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.2
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
>Priority: Major
> Fix For: 1.5.0, 1.3.3, 1.4.4
>
> Attachments: HBASE-20322.branch-1.3.001.patch, 
> HBASE-20322.branch-1.3.002-addendum.patch, HBASE-20322.branch-1.3.002.patch, 
> HBASE-20322.branch-1.4.001.patch
>
>
> RS crashed with ConcurrentModificationException on our 1.3 cluster, stack 
> trace below. [~toffer] and I checked and there is a race condition between 
> flush and scanner close. When StoreScanner.updateReaders() is updating the 
> scanners after a newly flushed file (in this trace below a region close 
> during a split), the client's scanner could be closing thus causing CME.
> Its rare, but since it crashes the region server, needs to be fixed.
> FATAL regionserver.HRegionServer [regionserver/] : ABORTING region server 
> : Replay of WAL required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: 
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2579)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2255)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2217)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2207)
> at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1501)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1420)
> at 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:398)
> at 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:278)
> at 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:566)
> at 
> org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
> at 
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:154)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.ConcurrentModificationException
> at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
> at java.util.ArrayList$Itr.next(ArrayList.java:851)
> at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.clearAndClose(StoreScanner.java:797)
> at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.updateReaders(StoreScanner.java:825)
> at 
> org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:1155)
> PS: ignore the line no in the above stack trace, method calls should help 
> understand whats happening.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20322) CME in StoreScanner causes region server crash

[Thiruvel Thirumoolan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431034#comment-16431034
 ] 

Thiruvel Thirumoolan commented on HBASE-20322:
--

[~mdrob] - I raised HBASE-20373 to confirm that and fix it. Will get to it 
sometime this week or next.

> CME in StoreScanner causes region server crash
> --
>
> Key: HBASE-20322
> URL: https://issues.apache.org/jira/browse/HBASE-20322
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 1.3.2
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
>Priority: Major
> Fix For: 1.5.0, 1.3.3, 1.4.4
>
> Attachments: HBASE-20322.branch-1.3.001.patch, 
> HBASE-20322.branch-1.3.002-addendum.patch, HBASE-20322.branch-1.3.002.patch, 
> HBASE-20322.branch-1.4.001.patch
>
>
> RS crashed with ConcurrentModificationException on our 1.3 cluster, stack 
> trace below. [~toffer] and I checked and there is a race condition between 
> flush and scanner close. When StoreScanner.updateReaders() is updating the 
> scanners after a newly flushed file (in this trace below a region close 
> during a split), the client's scanner could be closing thus causing CME.
> Its rare, but since it crashes the region server, needs to be fixed.
> FATAL regionserver.HRegionServer [regionserver/] : ABORTING region server 
> : Replay of WAL required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region: 
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2579)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2255)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2217)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2207)
> at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1501)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1420)
> at 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:398)
> at 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:278)
> at 
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:566)
> at 
> org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
> at 
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:154)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.ConcurrentModificationException
> at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
> at java.util.ArrayList$Itr.next(ArrayList.java:851)
> at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.clearAndClose(StoreScanner.java:797)
> at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.updateReaders(StoreScanner.java:825)
> at 
> org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:1155)
> PS: ignore the line no in the above stack trace, method calls should help 
> understand whats happening.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HBASE-20373) Check and forward port HBASE-20322 (RS crash due to CME in StoreScanner)

[Thiruvel Thirumoolan (JIRA)]

Thiruvel Thirumoolan created HBASE-20373:


 Summary: Check and forward port HBASE-20322 (RS crash due to CME 
in StoreScanner)
 Key: HBASE-20373
 URL: https://issues.apache.org/jira/browse/HBASE-20373
 Project: HBase
  Issue Type: Bug
Reporter: Thiruvel Thirumoolan
Assignee: Thiruvel Thirumoolan


I think the same problem that causes HBASE-20322 in 1.x exists in branch-2 
also. This jira is to confirm that and fix it if required.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20338) WALProcedureStore#recoverLease() should have fixed sleeps for retrying rollWriter()

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430986#comment-16430986
 ] 

Hadoop QA commented on HBASE-20338:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
11s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
15s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red}  2m  
1s{color} | {color:red} The patch causes 10 errors with Hadoop v2.6.5. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red}  4m  
5s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red}  6m 
22s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color} 
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  4m  
5s{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
 8s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m  8s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20338 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12918203/HBASE-20338.master.004.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 30e49733dd17 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 17f930c4d6 |
| maven | version: Apache Maven 3.5.3 
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_162 |
| findbugs | v3.1.0-RC3 |
| 

[jira] [Commented] (HBASE-20244) NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430984#comment-16430984
 ] 

Ted Yu commented on HBASE-20244:


Test failure was not related to patch.

> NoSuchMethodException when retrieving private method 
> decryptEncryptedDataEncryptionKey from DFSClient
> -
>
> Key: HBASE-20244
> URL: https://issues.apache.org/jira/browse/HBASE-20244
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 20244.v1.txt, 20244.v1.txt, 20244.v1.txt
>
>
> I was running unit test against hadoop 3.0.1 RC and saw the following in test 
> output:
> {code}
> ERROR [RS-EventLoopGroup-3-3] 
> asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(267): Couldn't properly 
> initialize access to HDFS internals. Please update  your WAL Provider to not 
> make use of the 'asyncfs' provider. See HBASE-16110 for more information.
> java.lang.NoSuchMethodException: 
> org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo)
>   at java.lang.Class.getDeclaredMethod(Class.java:2130)
>   at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:232)
>   at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:262)
>   at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:661)
>   at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:118)
>   at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:720)
>   at 
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:715)
>   at 
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
>   at 
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500)
>   at 
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479)
>   at 
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
>   at 
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104)
>   at 
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
>   at 
> org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:306)
>   at 
> org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:341)
>   at 
> org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633)
>   at 
> org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
> {code}
> The private method was moved by HDFS-12574 to HdfsKMSUtil with different 
> signature.
> To accommodate the above method movement, it seems we need to call the 
> following method of DFSClient :
> {code}
>   public KeyProvider getKeyProvider() throws IOException {
> {code}
> Since the new decryptEncryptedDataEncryptionKey method has this signature:
> {code}
>   static KeyVersion decryptEncryptedDataEncryptionKey(FileEncryptionInfo
> feInfo, KeyProvider keyProvider) throws IOException {
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-15814) Miss important information in Document of HBase Security

[Josh Elser (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430968#comment-16430968
 ] 

Josh Elser commented on HBASE-15814:


I think the biggest decision to make would be whether we should have one 
"Security" chapter which encompasses all relevant security banter (e.g. "core", 
thrift1, thrift2, REST) or if we should leave the Security to 
fundamental/"core" components, and leave security for each of the related 
components.

The CDH docs do the former while the HBase Book does the latter which is why 
the CDH docs look so much more expansive (I think). Maybe a good minimal change 
would be to proactively call out the other sections in our Book's Security 
chapter? (e.g. "By the way, if you care about $service, read this chapter too")

> Miss important information in Document of HBase Security
> 
>
> Key: HBASE-15814
> URL: https://issues.apache.org/jira/browse/HBASE-15814
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Reporter: Heng Chen
>Priority: Major
>
> I have deployed secure cluster recently, and found we miss important 
> information in http://hbase.apache.org/book.html#security
> Some configurations like 
> {code}
> 
>   hbase.regionserver.kerberos.principal 
>   hbase/_h...@your-realm.com 
>  
>  
>   hbase.regionserver.keytab.file 
>   /etc/hbase/conf/hbase.keytab 
> 
>  
>   hbase.master.kerberos.principal 
>   hbase/_h...@your-realm.com 
>  
>  
> hbase.master.keytab.file 
> /etc/hbase/conf/hbase.keytab 
> 
> {code}
> And i found more detailed document in 
> http://www.cloudera.com/documentation/enterprise/5-5-x/topics/cdh_sg_hbase_authentication.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20149) Purge dev javadoc from bin tarball (or make a separate tarball of javadoc)

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430964#comment-16430964
 ] 

stack commented on HBASE-20149:
---

.003 points local install to new /2.0/ version-specific doc when you click on 
links for dev apis. Link was added by below

commit 116a8085178d9d5bc55ab48a7e48a3969a0fd787
Author: Sean Busbey 
Date:   Mon Apr 9 01:07:02 2018 -0500

HBASE-20365 add 2.0 docs to menu.

> Purge dev javadoc from bin tarball (or make a separate tarball of javadoc)
> --
>
> Key: HBASE-20149
> URL: https://issues.apache.org/jira/browse/HBASE-20149
> Project: HBase
>  Issue Type: Sub-task
>  Components: build, community, documentation
>Reporter: stack
>Assignee: Artem Ervits
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20149.branch-2.0.001.patch, 
> HBASE-20149.branch-2.0.002.patch, HBASE-20149.branch-2.0.003.patch
>
>
> The bin tarball is too fat (Chia-Ping and Josh noticed it on the beta-2 
> vote). A note to the dev list subsequently resulted in suggestion that we 
> just purge dev javadoc (or even all javadoc) from bin tarball (Andrew). Sean 
> was good w/ it and suggested perhaps we could do a javadoc only tgz. Let me 
> look into this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20149) Purge dev javadoc from bin tarball (or make a separate tarball of javadoc)

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-20149:
--
Attachment: HBASE-20149.branch-2.0.003.patch

> Purge dev javadoc from bin tarball (or make a separate tarball of javadoc)
> --
>
> Key: HBASE-20149
> URL: https://issues.apache.org/jira/browse/HBASE-20149
> Project: HBase
>  Issue Type: Sub-task
>  Components: build, community, documentation
>Reporter: stack
>Assignee: Artem Ervits
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20149.branch-2.0.001.patch, 
> HBASE-20149.branch-2.0.002.patch, HBASE-20149.branch-2.0.003.patch
>
>
> The bin tarball is too fat (Chia-Ping and Josh noticed it on the beta-2 
> vote). A note to the dev list subsequently resulted in suggestion that we 
> just purge dev javadoc (or even all javadoc) from bin tarball (Andrew). Sean 
> was good w/ it and suggested perhaps we could do a javadoc only tgz. Let me 
> look into this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20372) [website] move stuff from more than 2 years ago to old news

[Josh Elser (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430957#comment-16430957
 ] 

Josh Elser commented on HBASE-20372:


+1

> [website] move stuff from more than 2 years ago to old news
> ---
>
> Key: HBASE-20372
> URL: https://issues.apache.org/jira/browse/HBASE-20372
> Project: HBase
>  Issue Type: Task
>  Components: website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Attachments: HBASE-20372.0.patch
>
>
> "current news" currently covers back to march 2014, which seems too long.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19343) Restore snapshot makes split parent region online

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430947#comment-16430947
 ] 

Ted Yu commented on HBASE-19343:


See 19343.tst which applies on master.

The new test passes on master.

> Restore snapshot makes split parent region online 
> --
>
> Key: HBASE-19343
> URL: https://issues.apache.org/jira/browse/HBASE-19343
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Major
> Fix For: 1.5.0, 1.3.3, 1.4.4
>
> Attachments: 19343.tst, HBASE-19343-branch-1-v2.patch, 
> HBASE-19343-branch-1.2.patch, HBASE-19343-branch-1.3.patch, 
> HBASE-19343-branch-1.patch, Snapshot.jpg
>
>
> Restore snapshot makes parent split region online as shown in the attached 
> snapshot.
> Steps to reproduce
> =
> 1. Create table
> 2. Insert few records into the table
> 3. flush the table
> 4. Split the table
> 5. Create snapshot before catalog janitor clears the parent region entry from 
> meta.
> 6. Restore snapshot
> We can see the problem in meta entries,
> Meta content before restore snapshot:
> {noformat}
> t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:regioninfo, timestamp=1511537565964, value={ENCODED => 
> 077a12b0b3c91b053fa95223635f9543, NAME => 
> 't1,,1511537529449.077a12b0b3c91b053fa95223635f9543.', STARTKEY =>
>   '', ENDKEY => 
> '', OFFLINE => true, SPLIT => true}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:seqnumDuringOpen, timestamp=1511537530107, 
> value=\x00\x00\x00\x00\x00\x00\x00\x02
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:server, timestamp=1511537530107, value=host-xx:16020
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:serverstartcode, timestamp=1511537530107, value=1511537511523
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:splitA, timestamp=1511537565964, value={ENCODED => 
> 3c7c866d4df370c586131a4cbe0ef6a8, NAME => 
> 't1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8.', STARTKEY => '',
>   ENDKEY => 'm'}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:splitB, timestamp=1511537565964, value={ENCODED => 
> dc7facd824c85b94e5bf6a2e6b5f5efc, NAME => 
> 't1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.', STARTKEY => 'm
>  ', ENDKEY => ''}
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:regioninfo, timestamp=1511537566075, value={ENCODED => 
> 3c7c866d4df370c586131a4cbe0ef6a8, NAME => 
> 't1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8.', STARTKEY =>
>   '', ENDKEY => 
> 'm'}
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:seqnumDuringOpen, timestamp=1511537566075, 
> value=\x00\x00\x00\x00\x00\x00\x00\x02
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:server, timestamp=1511537566075, value=host-xx:16020
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:serverstartcode, timestamp=1511537566075, value=1511537511523
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:regioninfo, timestamp=1511537566069, value={ENCODED => 
> dc7facd824c85b94e5bf6a2e6b5f5efc, NAME => 
> 't1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.', STARTKEY =
>  > 'm', ENDKEY => 
> ''}
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:seqnumDuringOpen, timestamp=1511537566069, 
> value=\x00\x00\x00\x00\x00\x00\x00\x08
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:server, timestamp=1511537566069, value=host-xx:16020
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:serverstartcode, timestamp=1511537566069, value=1511537511523
> {noformat}
> Meta content after restore snapshot:
> {noformat}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:regioninfo, timestamp=1511537667635, value={ENCODED => 
> 077a12b0b3c91b053fa95223635f9543, NAME => 
> 't1,,1511537529449.077a12b0b3c91b053fa95223635f9543.', STARTKEY =>
>   '', ENDKEY => 
> ''}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:seqnumDuringOpen, timestamp=1511537667635, 
> value=\x00\x00\x00\x00\x00\x00\x00\x0A
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> 

[jira] [Updated] (HBASE-19343) Restore snapshot makes split parent region online

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-19343:
---
Attachment: 19343.tst

> Restore snapshot makes split parent region online 
> --
>
> Key: HBASE-19343
> URL: https://issues.apache.org/jira/browse/HBASE-19343
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Major
> Fix For: 1.5.0, 1.3.3, 1.4.4
>
> Attachments: 19343.tst, HBASE-19343-branch-1-v2.patch, 
> HBASE-19343-branch-1.2.patch, HBASE-19343-branch-1.3.patch, 
> HBASE-19343-branch-1.patch, Snapshot.jpg
>
>
> Restore snapshot makes parent split region online as shown in the attached 
> snapshot.
> Steps to reproduce
> =
> 1. Create table
> 2. Insert few records into the table
> 3. flush the table
> 4. Split the table
> 5. Create snapshot before catalog janitor clears the parent region entry from 
> meta.
> 6. Restore snapshot
> We can see the problem in meta entries,
> Meta content before restore snapshot:
> {noformat}
> t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:regioninfo, timestamp=1511537565964, value={ENCODED => 
> 077a12b0b3c91b053fa95223635f9543, NAME => 
> 't1,,1511537529449.077a12b0b3c91b053fa95223635f9543.', STARTKEY =>
>   '', ENDKEY => 
> '', OFFLINE => true, SPLIT => true}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:seqnumDuringOpen, timestamp=1511537530107, 
> value=\x00\x00\x00\x00\x00\x00\x00\x02
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:server, timestamp=1511537530107, value=host-xx:16020
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:serverstartcode, timestamp=1511537530107, value=1511537511523
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:splitA, timestamp=1511537565964, value={ENCODED => 
> 3c7c866d4df370c586131a4cbe0ef6a8, NAME => 
> 't1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8.', STARTKEY => '',
>   ENDKEY => 'm'}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:splitB, timestamp=1511537565964, value={ENCODED => 
> dc7facd824c85b94e5bf6a2e6b5f5efc, NAME => 
> 't1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.', STARTKEY => 'm
>  ', ENDKEY => ''}
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:regioninfo, timestamp=1511537566075, value={ENCODED => 
> 3c7c866d4df370c586131a4cbe0ef6a8, NAME => 
> 't1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8.', STARTKEY =>
>   '', ENDKEY => 
> 'm'}
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:seqnumDuringOpen, timestamp=1511537566075, 
> value=\x00\x00\x00\x00\x00\x00\x00\x02
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:server, timestamp=1511537566075, value=host-xx:16020
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:serverstartcode, timestamp=1511537566075, value=1511537511523
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:regioninfo, timestamp=1511537566069, value={ENCODED => 
> dc7facd824c85b94e5bf6a2e6b5f5efc, NAME => 
> 't1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.', STARTKEY =
>  > 'm', ENDKEY => 
> ''}
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:seqnumDuringOpen, timestamp=1511537566069, 
> value=\x00\x00\x00\x00\x00\x00\x00\x08
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:server, timestamp=1511537566069, value=host-xx:16020
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:serverstartcode, timestamp=1511537566069, value=1511537511523
> {noformat}
> Meta content after restore snapshot:
> {noformat}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:regioninfo, timestamp=1511537667635, value={ENCODED => 
> 077a12b0b3c91b053fa95223635f9543, NAME => 
> 't1,,1511537529449.077a12b0b3c91b053fa95223635f9543.', STARTKEY =>
>   '', ENDKEY => 
> ''}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:seqnumDuringOpen, timestamp=1511537667635, 
> value=\x00\x00\x00\x00\x00\x00\x00\x0A
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:server, timestamp=1511537667635, value=host-xx:16020
>  

[jira] [Commented] (HBASE-20366) Procedure State != ProcedureState.RUNNABLE; IllegalArgumentException

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430938#comment-16430938
 ] 

stack commented on HBASE-20366:
---

After more study, the move procedure's unassign has not finished. It has set 
CLOSED in hbase:meta but the procedure has not yet completed. The move 
procedure is suspended waiting on the unassign to complete so it can move to 
the assign step. It wants the unassign to let go of the region lock before it 
can more on.

This check for RUNNABLE state seems too constrained. We should let through the 
suspended procedures.

Let me keep an eye out for this failure type.

> Procedure State != ProcedureState.RUNNABLE; IllegalArgumentException
> 
>
> Key: HBASE-20366
> URL: https://issues.apache.org/jira/browse/HBASE-20366
> Project: HBase
>  Issue Type: Bug
>  Components: amv2
>Reporter: stack
>Priority: Critical
>
> PE Worker dies and Region offlined because Procedure not runable when 
> procedure goes to run it. It looks like this:
> {code}
> 2018-04-07 19:58:50,589 INFO  [PEWorker-5] 
> procedure.MasterProcedureScheduler: pid=8304, 
> state=WAITING:MOVE_REGION_ASSIGN; MoveRegionProcedure 
> hri=IntegrationTestBigLinkedList,p\xC3\x11\xB2,1523155040553.187ee18fb3dd1a7ac1f9f2b667160729.,
>  source=ve0534.halxg.cloudera.com,16020,1523153184521, 
> destination=ve0542.halxg.cloudera.com,16020,1523155964184 checking lock on 
> 187ee18fb3dd1a7ac1f9f2b667160729
> 2018-04-07 19:58:50,589 INFO  [PEWorker-14] 
> procedure.MasterProcedureScheduler: pid=8302, 
> state=RUNNABLE:MOVE_REGION_ASSIGN; MoveRegionProcedure 
> hri=IntegrationTestBigLinkedList,\xEC0\x83\x96*\x86Qsh\xD82\x1E\xAB\x06$\x89,1523151456082.84e97ce42aeb78a2abaf8f17a278b735.,
>  source=ve0534.halxg.cloudera.com,16020,1523153184521, 
> destination=ve0542.halxg.cloudera.com,16020,1523155964184 checking lock on 
> 84e97ce42aeb78a2abaf8f17a278b735  
>   
> 2018-04-07 
> 19:58:50,591 WARN  [PEWorker-5] procedure2.ProcedureExecutor: Worker 
> terminating UNNATURALLY null
> java.lang.IllegalArgumentException: pid=8304, 
> state=WAITING:MOVE_REGION_ASSIGN; MoveRegionProcedure 
> hri=IntegrationTestBigLinkedList,p\xC3\x11\xB2,1523155040553.187ee18fb3dd1a7ac1f9f2b667160729.,
>  source=ve0534.halxg.cloudera.com,16020,1523153184521, 
> destination=ve0542.halxg.cloudera.com,16020,1523155964184
>   at 
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkArgument(Preconditions.java:134)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1430)
>   
>   
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1221)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:75)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1741)
> {code}
> This killed my job because it offlined a region.
> Narrative:
>  * Balancer moves this region
>  * Move procedure does dispatch to unassign...
>  * Suspiciously, the close comes in unannounced.. .its as though it a close 
> from another procedure...
>  2018-04-07 19:58:24,296 INFO  [PEWorker-9] assignment.RegionStateStore: 
> pid=8305 updating hbase:meta 
> row=IntegrationTestBigLinkedList,p\xC3\x11\xB2,1523155040553.187ee18fb3dd1a7ac1f9f2b667160729.,
>  regionState=CLOSED
>  * Master is killed by monkey.
>  * Recovery. Region is in CLOSED state.
>  * We go to schedule the move region procedure again... Its state must have 
> not been updated on master crash.
>  2018-04-07 19:58:50,589 INFO  [PEWorker-5] 
> procedure.MasterProcedureScheduler: pid=8304, 
> state=WAITING:MOVE_REGION_ASSIGN; MoveRegionProcedure 
> hri=IntegrationTestBigLinkedList,p\xC3\x11\xB2,1523155040553.187ee18fb3dd1a7ac1f9f2b667160729.,
>  source=ve0534.halxg.cloudera.com,16020,1523153184521, 
> destination=ve0542.halxg.cloudera.com,16020,1523155964184 checking lock on 
> 187ee18fb3dd1a7ac1f9f2b667160729
>  * And then we get
>  2018-04-07 19:58:50,591 WARN  [PEWorker-5] procedure2.ProcedureExecutor: 
> Worker terminating UNNATURALLY null   
>   
>   
>  java.lang.IllegalArgumentException: pid=8304, 
> 

[jira] [Updated] (HBASE-20338) WALProcedureStore#recoverLease() should have fixed sleeps for retrying rollWriter()

[Wei-Chiu Chuang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wei-Chiu Chuang updated HBASE-20338:

Attachment: HBASE-20338.master.004.patch

> WALProcedureStore#recoverLease() should have fixed sleeps for retrying 
> rollWriter()
> ---
>
> Key: HBASE-20338
> URL: https://issues.apache.org/jira/browse/HBASE-20338
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0-beta-2
>Reporter: Umesh Agashe
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HBASE-20338.master.001.patch, 
> HBASE-20338.master.002.patch, HBASE-20338.master.003.patch, 
> HBASE-20338.master.004.patch
>
>
> In our internal testing we observed that logs are getting flooded due to 
> continuous loop in WALProcedureStore#recoverLease():
> {code}
>   while (isRunning()) {
> // Get Log-MaxID and recover lease on old logs
> try {
>   flushLogId = initOldLogs(oldLogs);
> } catch (FileNotFoundException e) {
>   LOG.warn("Someone else is active and deleted logs. retrying.", e);
>   oldLogs = getLogFiles();
>   continue;
> }
> // Create new state-log
> if (!rollWriter(flushLogId + 1)) {
>   // someone else has already created this log
>   LOG.debug("Someone else has already created log " + flushLogId);
>   continue;
> }
> {code}
> rollWriter() fails to create a new file. Error messages in HDFS namenode logs 
> around same time:
> {code}
> INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 8020, call 
> org.apache.hadoop.hdfs.protocol.ClientProtocol.create from 
> 172.31.121.196:38508 Call#3141 Retry#0
> java.io.IOException: Exeption while contacting value generator
> at 
> org.apache.hadoop.crypto.key.kms.ValueQueue.getAtMost(ValueQueue.java:389)
> at 
> org.apache.hadoop.crypto.key.kms.ValueQueue.getNext(ValueQueue.java:291)
> at 
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.generateEncryptedKey(KMSClientProvider.java:724)
> at 
> org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.generateEncryptedKey(KeyProviderCryptoExtension.java:511)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem$2.run(FSNamesystem.java:2680)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem$2.run(FSNamesystem.java:2676)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
> at 
> org.apache.hadoop.security.SecurityUtil.doAsUser(SecurityUtil.java:477)
> at 
> org.apache.hadoop.security.SecurityUtil.doAsLoginUser(SecurityUtil.java:458)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.generateEncryptedDataEncryptionKey(FSNamesystem.java:2675)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2815)
> at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2712)
> at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:604)
> at 
> org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.create(AuthorizationProviderProxyClientProtocol.java:115)
> at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:412)
> at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2226)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2220)
> Caused by: java.net.ConnectException: Connection refused (Connection refused)
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at 
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> at 
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at 
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)

[jira] [Commented] (HBASE-20372) [website] move stuff from more than 2 years ago to old news

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430930#comment-16430930
 ] 

Hadoop QA commented on HBASE-20372:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 12m 
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 13m 
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
20s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m  7s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20372 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12918190/HBASE-20372.0.patch |
| Optional Tests |  asflicense  mvnsite  xml  |
| uname | Linux bfbc6c677013 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / ee87de9bfd |
| maven | version: Apache Maven 3.5.3 
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Max. process+thread count | 94 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/12359/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> [website] move stuff from more than 2 years ago to old news
> ---
>
> Key: HBASE-20372
> URL: https://issues.apache.org/jira/browse/HBASE-20372
> Project: HBase
>  Issue Type: Task
>  Components: website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Attachments: HBASE-20372.0.patch
>
>
> "current news" currently covers back to march 2014, which seems too long.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20068) Hadoopcheck project health check uses default maven repo instead of yetus managed ones

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430928#comment-16430928
 ] 

Hadoop QA commented on HBASE-20068:
---

(!) A patch to the testing environment has been detected. 
Re-executing against the patched versions to perform further tests. 
The console is at 
https://builds.apache.org/job/PreCommit-HBASE-Build/12360/console in case of 
problems.


> Hadoopcheck project health check uses default maven repo instead of yetus 
> managed ones
> --
>
> Key: HBASE-20068
> URL: https://issues.apache.org/jira/browse/HBASE-20068
> Project: HBase
>  Issue Type: Bug
>  Components: community, test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-20068.0.patch, HBASE-20068.1.patch
>
>
> Recently had a precommit run fail hadoop check for all 3 versions with 
> {code}
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-install-plugin:2.5.2:install (default-install) 
> on project hbase-thrift: Failed to install metadata 
> org.apache.hbase:hbase-thrift:3.0.0-SNAPSHOT/maven-metadata.xml: Could not 
> parse metadata 
> /home/jenkins/.m2/repository/org/apache/hbase/hbase-thrift/3.0.0-SNAPSHOT/maven-metadata-local.xml:
>  in epilog non whitespace content is not allowed but got / (position: END_TAG 
> seen ...\n/... @25:2)  -> [Help 1]
> {code}
> Looks like maven repo corruption.
> Also the path {{/home/jenkins/.m2/repository}} means that those invocations 
> are using the jenkins user repo, which isn't safe since there are multiple 
> executors. either the plugin isn't using the yetus provided maven repo path 
> or our yetus invocation isn't telling yetus to provide its own maven repo 
> path.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20068) Hadoopcheck project health check uses default maven repo instead of yetus managed ones

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430929#comment-16430929
 ] 

Hadoop QA commented on HBASE-20068:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue}  0m  
3s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green}  0m 
 1s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
 8s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}  0m 31s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20068 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12918201/HBASE-20068.1.patch |
| Optional Tests |  asflicense  shellcheck  shelldocs  |
| uname | Linux d985183566aa 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 17f930c4d6 |
| maven | version: Apache Maven 3.5.3 
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| shellcheck | v0.4.4 |
| Max. process+thread count | 47 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/12360/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> Hadoopcheck project health check uses default maven repo instead of yetus 
> managed ones
> --
>
> Key: HBASE-20068
> URL: https://issues.apache.org/jira/browse/HBASE-20068
> Project: HBase
>  Issue Type: Bug
>  Components: community, test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-20068.0.patch, HBASE-20068.1.patch
>
>
> Recently had a precommit run fail hadoop check for all 3 versions with 
> {code}
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-install-plugin:2.5.2:install (default-install) 
> on project hbase-thrift: Failed to install metadata 
> org.apache.hbase:hbase-thrift:3.0.0-SNAPSHOT/maven-metadata.xml: Could not 
> parse metadata 
> /home/jenkins/.m2/repository/org/apache/hbase/hbase-thrift/3.0.0-SNAPSHOT/maven-metadata-local.xml:
>  in epilog non whitespace content is not allowed but got / (position: END_TAG 
> seen ...\n/... @25:2)  -> [Help 1]
> {code}
> Looks like maven repo corruption.
> Also the path {{/home/jenkins/.m2/repository}} means that those invocations 
> are using the jenkins user repo, which isn't safe since there are multiple 
> executors. either the plugin isn't using the yetus provided maven repo path 
> or our yetus invocation isn't telling yetus to provide its own maven repo 
> path.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19343) Restore snapshot makes split parent region online

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430927#comment-16430927
 ] 

Hudson commented on HBASE-19343:


SUCCESS: Integrated in Jenkins build HBase-1.3-IT #391 (See 
[https://builds.apache.org/job/HBase-1.3-IT/391/])
HBASE-19343 Restore snapshot makes split parent region online (tedyu: rev 
1197ecaa3a809a96dc55b3d3543f4d0eb5c04e24)
* (edit) 
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/RestoreSnapshotHelper.java
* (edit) 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestRestoreSnapshotFromClient.java


> Restore snapshot makes split parent region online 
> --
>
> Key: HBASE-19343
> URL: https://issues.apache.org/jira/browse/HBASE-19343
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Major
> Fix For: 1.5.0, 1.3.3, 1.4.4
>
> Attachments: HBASE-19343-branch-1-v2.patch, 
> HBASE-19343-branch-1.2.patch, HBASE-19343-branch-1.3.patch, 
> HBASE-19343-branch-1.patch, Snapshot.jpg
>
>
> Restore snapshot makes parent split region online as shown in the attached 
> snapshot.
> Steps to reproduce
> =
> 1. Create table
> 2. Insert few records into the table
> 3. flush the table
> 4. Split the table
> 5. Create snapshot before catalog janitor clears the parent region entry from 
> meta.
> 6. Restore snapshot
> We can see the problem in meta entries,
> Meta content before restore snapshot:
> {noformat}
> t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:regioninfo, timestamp=1511537565964, value={ENCODED => 
> 077a12b0b3c91b053fa95223635f9543, NAME => 
> 't1,,1511537529449.077a12b0b3c91b053fa95223635f9543.', STARTKEY =>
>   '', ENDKEY => 
> '', OFFLINE => true, SPLIT => true}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:seqnumDuringOpen, timestamp=1511537530107, 
> value=\x00\x00\x00\x00\x00\x00\x00\x02
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:server, timestamp=1511537530107, value=host-xx:16020
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:serverstartcode, timestamp=1511537530107, value=1511537511523
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:splitA, timestamp=1511537565964, value={ENCODED => 
> 3c7c866d4df370c586131a4cbe0ef6a8, NAME => 
> 't1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8.', STARTKEY => '',
>   ENDKEY => 'm'}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:splitB, timestamp=1511537565964, value={ENCODED => 
> dc7facd824c85b94e5bf6a2e6b5f5efc, NAME => 
> 't1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.', STARTKEY => 'm
>  ', ENDKEY => ''}
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:regioninfo, timestamp=1511537566075, value={ENCODED => 
> 3c7c866d4df370c586131a4cbe0ef6a8, NAME => 
> 't1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8.', STARTKEY =>
>   '', ENDKEY => 
> 'm'}
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:seqnumDuringOpen, timestamp=1511537566075, 
> value=\x00\x00\x00\x00\x00\x00\x00\x02
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:server, timestamp=1511537566075, value=host-xx:16020
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:serverstartcode, timestamp=1511537566075, value=1511537511523
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:regioninfo, timestamp=1511537566069, value={ENCODED => 
> dc7facd824c85b94e5bf6a2e6b5f5efc, NAME => 
> 't1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.', STARTKEY =
>  > 'm', ENDKEY => 
> ''}
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:seqnumDuringOpen, timestamp=1511537566069, 
> value=\x00\x00\x00\x00\x00\x00\x00\x08
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:server, timestamp=1511537566069, value=host-xx:16020
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:serverstartcode, timestamp=1511537566069, value=1511537511523
> {noformat}
> Meta content after restore snapshot:
> {noformat}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:regioninfo, timestamp=1511537667635, value={ENCODED => 
> 077a12b0b3c91b053fa95223635f9543, NAME => 
> 

[jira] [Commented] (HBASE-20296) Remove last pushed sequence ids when removing tables from a peer

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430921#comment-16430921
 ] 

Hudson commented on HBASE-20296:


Results for branch HBASE-20046-branch-2
[build #7 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> Remove last pushed sequence ids when removing tables from a peer
> 
>
> Key: HBASE-20296
> URL: https://issues.apache.org/jira/browse/HBASE-20296
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20296-v1.patch, HBASE-20296.patch, 
> HBASE-20296.patch, HBASE-20296.patch, HBASE-20296.patch
>
>
> Discussed with [~zghaobac] and [~openinx] offline, this is the only safe 
> thing to do for now. It is not safe to remove barriers and last pushed 
> sequence ids when deleting a table since we may still have edits which should 
> be replicated.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20371) website landing page should draw attention to CFP

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-20371:

   Resolution: Fixed
Fix Version/s: 3.0.0
   Status: Resolved  (was: Patch Available)

pushed to master. thanks for the review!

> website landing page should draw attention to CFP
> -
>
> Key: HBASE-20371
> URL: https://issues.apache.org/jira/browse/HBASE-20371
> Project: HBase
>  Issue Type: Task
>  Components: website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-20371.0.patch
>
>
> I missed that the CFP for HBaseCon NA West 2018 is still open because I 
> didn't click the link to the conf page. The "News" section should mention 
> that CFP is open until it is not.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20285) Delete all last pushed sequence ids when removing a peer or removing the serial flag for a peer

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430920#comment-16430920
 ] 

Hudson commented on HBASE-20285:


Results for branch HBASE-20046-branch-2
[build #7 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> Delete all last pushed sequence ids when removing a peer or removing the 
> serial flag for a peer
> ---
>
> Key: HBASE-20285
> URL: https://issues.apache.org/jira/browse/HBASE-20285
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20285-v1.patch, HBASE-20285-v2.patch, 
> HBASE-20285.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20138) Find a way to deal with the conflicts when updating replication position

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430919#comment-16430919
 ] 

Hudson commented on HBASE-20138:


Results for branch HBASE-20046-branch-2
[build #7 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> Find a way to deal with the conflicts when updating replication position
> 
>
> Key: HBASE-20138
> URL: https://issues.apache.org/jira/browse/HBASE-20138
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Duo Zhang
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20138.v1.patch, HBASE-20138.v2.patch, 
> HBASE-20138.v3.patch, HBASE-20138.v4.patch, HBASE-20138.v5.patch, 
> HBASE-20138.v6.patch, HBASE-20138.v7.patch
>
>
> For now if a table is not created with SERIAL_REPLICATION_SCOPE and later 
> converted to SERIAL_REPLICATION_SCOPE , then we may have multiple replication 
> sources which replicate the different ranges for the same region and update 
> the queue storage concurrently. This will cause problem if the lower range 
> finish at last since the replication position will be wrong...
> Need to find a way to deal with this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20361) Non-successive TableInputSplits may wrongly be merged by auto balancing feature

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430902#comment-16430902
 ] 

Hudson commented on HBASE-20361:


Results for branch HBASE-20046-branch-2
[build #7 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> Non-successive TableInputSplits may wrongly be merged by auto balancing 
> feature
> ---
>
> Key: HBASE-20361
> URL: https://issues.apache.org/jira/browse/HBASE-20361
> Project: HBase
>  Issue Type: Bug
>  Components: mapreduce
>Reporter: Yuki Tawara
>Assignee: Yuki Tawara
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: HBASE-20361.master.001.patch, 
> HBASE-20361.master.002.patch
>
>
> TableInputFormatBase class offers users a mechanism to exclude specific 
> splits from returned list of TableInputFormatBase#getSplits through 
> TableInputFormatBase#includeRegionInSplit.
> It also offers users a feature called "auto balancing" to mitigate data skew 
> by splitting large splits and merging small splits.
> If a user overrides TableInputFormatBase#includeRegionInSplit, i th split and 
> i+1 th split may not be successive(i th split's end key is smaller than i+1 
> th split's start key).
> If he or she further enable auto balancing feature, non-successive splits can 
> be merged, which means excluded splits between merged non-successive splits 
> "revive".
> To avoid such cases, we should not merge non-successive splits.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20147) Serial replication will be stuck if we create a table with serial replication but add it to a peer after there are region moves

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430915#comment-16430915
 ] 

Hudson commented on HBASE-20147:


Results for branch HBASE-20046-branch-2
[build #7 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> Serial replication will be stuck if we create a table with serial replication 
> but add it to a peer after there are region moves
> ---
>
> Key: HBASE-20147
> URL: https://issues.apache.org/jira/browse/HBASE-20147
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20147-v1.patch, HBASE-20147-v2.patch, 
> HBASE-20147-v3.patch, HBASE-20147-v4.patch, HBASE-20147-v5.patch, 
> HBASE-20147-v6.patch, HBASE-20147-v7.patch, HBASE-20147-v8.patch, 
> HBASE-20147.patch, HBASE-20147.patch
>
>
> The start point for serial replication is that, if we are in the first range 
> then we are safe to push. And we will record replication barrier when the 
> replication scope is set to SERIAL even if the table is not contained in any 
> peers. So it could happen that, we record several barriers in the meta 
> already and then we add a peer which contains this table. So when 
> replicating, we will find that we are not in the first range, and then the 
> replication will be stuck.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20116) Optimize the region last pushed sequence id layout on zk

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430913#comment-16430913
 ] 

Hudson commented on HBASE-20116:


Results for branch HBASE-20046-branch-2
[build #7 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> Optimize the region last pushed sequence id layout on zk
> 
>
> Key: HBASE-20116
> URL: https://issues.apache.org/jira/browse/HBASE-20116
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Duo Zhang
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20116-addendum.patch, HBASE-20116.v1.patch, 
> HBASE-20116.v2.patch, HBASE-20116.v3.patch, HBASE-20116.v4.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20125) Add UT for serial replication after region split and merge

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430907#comment-16430907
 ] 

Hudson commented on HBASE-20125:


Results for branch HBASE-20046-branch-2
[build #7 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> Add UT for serial replication after region split and merge
> --
>
> Key: HBASE-20125
> URL: https://issues.apache.org/jira/browse/HBASE-20125
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20125-v1.patch, HBASE-20125.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20117) Cleanup the unused replication barriers in meta table

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430911#comment-16430911
 ] 

Hudson commented on HBASE-20117:


Results for branch HBASE-20046-branch-2
[build #7 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> Cleanup the unused replication barriers in meta table
> -
>
> Key: HBASE-20117
> URL: https://issues.apache.org/jira/browse/HBASE-20117
> Project: HBase
>  Issue Type: Sub-task
>  Components: master, Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20117-v1.patch, HBASE-20117-v2.patch, 
> HBASE-20117.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20227) Add UT for ReplicationUtils.contains method

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430917#comment-16430917
 ] 

Hudson commented on HBASE-20227:


Results for branch HBASE-20046-branch-2
[build #7 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> Add UT for ReplicationUtils.contains method
> ---
>
> Key: HBASE-20227
> URL: https://issues.apache.org/jira/browse/HBASE-20227
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication, test
>Reporter: Duo Zhang
>Assignee: Jingyun Tian
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20227-v8.patch, HBASE-20227.master.001.patch, 
> HBASE-20227.master.002.patch, HBASE-20227.master.003.patch, 
> HBASE-20227.master.005.patch, HBASE-20227.master.006.patch, 
> HBASE-20227.master.007.patch
>
>
> The method is extracted from NamespaceTableCfWALEntryFilter. But 
> NamespaceTableCfWALEntryFilter is in the hbase-server module so the UTs which 
> tests the function of this method are also in the hbase-server module. If 
> later we modify ReplicationUtils.contains without touching any code in 
> hbase-server, then we will not run the UTs for this method and may introduce 
> bugs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20115) Reimplement serial replication based on the new replication storage layer

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430905#comment-16430905
 ] 

Hudson commented on HBASE-20115:


Results for branch HBASE-20046-branch-2
[build #7 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> Reimplement serial replication based on the new replication storage layer
> -
>
> Key: HBASE-20115
> URL: https://issues.apache.org/jira/browse/HBASE-20115
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20115-v1.patch, HBASE-20115-v2.patch, 
> HBASE-20115-v3.patch, HBASE-20115-v4.patch, HBASE-20115.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20127) Add UT for serial replication after failover

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430918#comment-16430918
 ] 

Hudson commented on HBASE-20127:


Results for branch HBASE-20046-branch-2
[build #7 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> Add UT for serial replication after failover
> 
>
> Key: HBASE-20127
> URL: https://issues.apache.org/jira/browse/HBASE-20127
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication, test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20127-v1.patch, HBASE-20127.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20167) Optimize the implementation of ReplicationSourceWALReader

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430909#comment-16430909
 ] 

Hudson commented on HBASE-20167:


Results for branch HBASE-20046-branch-2
[build #7 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> Optimize the implementation of ReplicationSourceWALReader
> -
>
> Key: HBASE-20167
> URL: https://issues.apache.org/jira/browse/HBASE-20167
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20167-v1.patch, HBASE-20167-v2.patch, 
> HBASE-20167.patch
>
>
> After HBASE-20148, serial replication will be an option for peer. Since an 
> instance of ReplicationSourceWALReader can only belongs to one peer, we do 
> not need to add the so many 'if' in the implementation of readWALEntries to 
> check whether we should consider serial replication. We can just make a sub 
> class or something similiar for serial replication to make the code clean.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20242) The open sequence number will grow if we fail to open a region after writing the max sequence id file

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430914#comment-16430914
 ] 

Hudson commented on HBASE-20242:


Results for branch HBASE-20046-branch-2
[build #7 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> The open sequence number will grow if we fail to open a region after writing 
> the max sequence id file
> -
>
> Key: HBASE-20242
> URL: https://issues.apache.org/jira/browse/HBASE-20242
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20242.patch
>
>
> We write the current max sequence id plus one as the new max sequence id so 
> if we fail before report RIT to master, when assign next time the open 
> sequence number will increase by one.
> But it is possible that we fail before writing the open region marker so 
> there will no actual WAL with the sequence id which is the open sequence 
> number minus one. This will cause the serial replication to be stuck.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20050) Reimplement updateReplicationPositions logic in serial replication based on the newly introduced replication storage layer

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430904#comment-16430904
 ] 

Hudson commented on HBASE-20050:


Results for branch HBASE-20046-branch-2
[build #7 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> Reimplement updateReplicationPositions logic in serial replication based on 
> the newly introduced replication storage layer
> --
>
> Key: HBASE-20050
> URL: https://issues.apache.org/jira/browse/HBASE-20050
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20050.patch, HBASE-20050.v1.patch, 
> HBASE-20050.v2.patch, HBASE-20050.v3.patch
>
>
> Now, in serial replication,  need to update  the replication position both in 
> zookeeper and meta table, which may led to inconsistent problem. I'll 
> re-implement this part based on the uniform replication storage layer. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20362) TestMasterShutdown.testMasterShutdownBeforeStartingAnyRegionServer is flaky

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430903#comment-16430903
 ] 

Hudson commented on HBASE-20362:


Results for branch HBASE-20046-branch-2
[build #7 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> TestMasterShutdown.testMasterShutdownBeforeStartingAnyRegionServer is flaky
> ---
>
> Key: HBASE-20362
> URL: https://issues.apache.org/jira/browse/HBASE-20362
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-20362.patch
>
>
> {code}
> Thread shutdownThread = new Thread("Shutdown-Thread") {
>   @Override
>   public void run() {
> LOG.info("Before call to shutdown master");
> try {
>   try (Connection connection =
>   ConnectionFactory.createConnection(util.getConfiguration())) {
> try (Admin admin = connection.getAdmin()) {
>   admin.shutdown();
> }
>   }
>   LOG.info("After call to shutdown master");
>   cluster.waitOnMaster(MASTER_INDEX);
> } catch (Exception e) {
> }
>   }
> };
> {code}
> https://builds.apache.org/job/HBASE-Flaky-Tests/28970/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.TestMasterShutdown-output.txt
> In the output for a failed running, we only have 'Before call to shutdown 
> master' but no 'After call to shutdown master', so I think there must be 
> something wrong when calling admin.shutdown, but in the catch block below we 
> just ignore the exception.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20271) ReplicationSourceWALReader.switched should use the file name instead of the path object directly

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430916#comment-16430916
 ] 

Hudson commented on HBASE-20271:


Results for branch HBASE-20046-branch-2
[build #7 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> ReplicationSourceWALReader.switched should use the file name instead of the 
> path object directly
> 
>
> Key: HBASE-20271
> URL: https://issues.apache.org/jira/browse/HBASE-20271
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20271.patch
>
>
> {noformat}
> 2018-03-24 08:29:29,965 ERROR 
> [RS_REFRESH_PEER-regionserver/ubuntu:0-0.replicationSource,2.replicationSource.shipperubuntu%2C35197%2C1521851267085,2]
>  helpers.MarkerIgnoringBase(159): * ABORTING region server 
> ubuntu,35197,1521851267085: Failed to operate on replication queue *
> org.apache.hadoop.hbase.replication.ReplicationException: Failed to set log 
> position (serverName=ubuntu,35197,1521851267085, queueId=2, 
> fileName=ubuntu%2C35197%2C1521851267085.1521851344947, position=2533)
>   at 
> org.apache.hadoop.hbase.replication.ZKReplicationQueueStorage.setWALPosition(ZKReplicationQueueStorage.java:237)
>   at 
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.lambda$9(ReplicationSourceManager.java:488)
>   at 
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.abortWhenFail(ReplicationSourceManager.java:455)
>   at 
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.logPositionAndCleanOldLogs(ReplicationSourceManager.java:488)
>   at 
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.updateLogPosition(ReplicationSourceShipper.java:232)
>   at 
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.shipEdits(ReplicationSourceShipper.java:134)
>   at 
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:104)
> Caused by: org.apache.zookeeper.KeeperException$NoNodeException: 
> KeeperErrorCode = NoNode
>   at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
>   at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:1006)
>   at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:910)
>   at 
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.multi(RecoverableZooKeeper.java:663)
>   at 
> org.apache.hadoop.hbase.zookeeper.ZKUtil.multiOrSequential(ZKUtil.java:1670)
>   at 
> org.apache.hadoop.hbase.replication.ZKReplicationQueueStorage.setWALPosition(ZKReplicationQueueStorage.java:235)
>   ... 6 more
> 2018-03-24 08:29:30,025 ERROR 
> [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37509] 
> master.MasterRpcServices(508): Region server ubuntu,35197,1521851267085 
> reported a fatal error:
> * ABORTING region server ubuntu,35197,1521851267085: Failed to operate on 
> replication queue *
> Cause:
> org.apache.hadoop.hbase.replication.ReplicationException: Failed to set log 
> position (serverName=ubuntu,35197,1521851267085, queueId=2, 
> fileName=ubuntu%2C35197%2C1521851267085.1521851344947, position=2533)
>   at 
> org.apache.hadoop.hbase.replication.ZKReplicationQueueStorage.setWALPosition(ZKReplicationQueueStorage.java:237)
>   at 
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.lambda$9(ReplicationSourceManager.java:488)
>   at 
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.abortWhenFail(ReplicationSourceManager.java:455)
>   at 
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.logPositionAndCleanOldLogs(ReplicationSourceManager.java:488)
>   at 
> 

[jira] [Commented] (HBASE-20129) Add UT for serial replication checker

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430906#comment-16430906
 ] 

Hudson commented on HBASE-20129:


Results for branch HBASE-20046-branch-2
[build #7 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> Add UT for serial replication checker
> -
>
> Key: HBASE-20129
> URL: https://issues.apache.org/jira/browse/HBASE-20129
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20129-v1.patch, HBASE-20129.patch
>
>
> Now it is a separated class so it is much easier to write UT to test the 
> corner cases.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20148) Make serial replication as a option for a peer instead of a table

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430908#comment-16430908
 ] 

Hudson commented on HBASE-20148:


Results for branch HBASE-20046-branch-2
[build #7 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> Make serial replication as a option for a peer instead of a table
> -
>
> Key: HBASE-20148
> URL: https://issues.apache.org/jira/browse/HBASE-20148
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20148-v1.patch, HBASE-20148-v2.patch, 
> HBASE-20148-v3.patch, HBASE-20148.patch
>
>
> Discussed with [~zghaobac] and [~openinx] offline, HBASE-20147 can only be 
> solved by adding new stages when creating a new replication peer. It will 
> also effect he processing even if you do not have any table with serial 
> replication. And also, the new logic introduced in ReplicationSource related 
> classes will also be executed even if you do not have serial replication 
> tables.
> So, we think that it maybe a better choice to move the option to 
> ReplicationPeerConfig. Since if there is one table in the peer requires 
> serial replication, then all the tables in this peer will be automatically 
> 'serial' since the wal for these tables are mixed with the one which requires 
> serial replication.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20206) WALEntryStream should not switch WAL file silently

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430912#comment-16430912
 ] 

Hudson commented on HBASE-20206:


Results for branch HBASE-20046-branch-2
[build #7 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> WALEntryStream should not switch WAL file silently
> --
>
> Key: HBASE-20206
> URL: https://issues.apache.org/jira/browse/HBASE-20206
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20206-v1.patch, HBASE-20206-v2.patch, 
> HBASE-20206-v3.patch, HBASE-20206-v4.patch, HBASE-20206-v5.patch, 
> HBASE-20206-v6.patch, HBASE-20206.patch
>
>
> The returned WALEntryBatch contains a file name and position, if we allow 
> switching WAL file silently then the position maybe wrong since it may be for 
> another file...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20165) Shell command to make a normal peer to be a serial replication peer

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430910#comment-16430910
 ] 

Hudson commented on HBASE-20165:


Results for branch HBASE-20046-branch-2
[build #7 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20046-branch-2/7//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> Shell command to make a normal peer to be a serial replication peer
> ---
>
> Key: HBASE-20165
> URL: https://issues.apache.org/jira/browse/HBASE-20165
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: HBASE-20165.v1.patch, HBASE-20165.v2.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19343) Restore snapshot makes split parent region online

[Mike Drob (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430899#comment-16430899
 ] 

Mike Drob commented on HBASE-19343:
---

[~pankaj2461], [~yuzhih...@gmail.com] - 
bq. Have to test this in master and other branches as well.
Did anybody get a chance to look at this? Do we think this is a problem in 
master as well?

> Restore snapshot makes split parent region online 
> --
>
> Key: HBASE-19343
> URL: https://issues.apache.org/jira/browse/HBASE-19343
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Major
> Fix For: 1.5.0, 1.3.3, 1.4.4
>
> Attachments: HBASE-19343-branch-1-v2.patch, 
> HBASE-19343-branch-1.2.patch, HBASE-19343-branch-1.3.patch, 
> HBASE-19343-branch-1.patch, Snapshot.jpg
>
>
> Restore snapshot makes parent split region online as shown in the attached 
> snapshot.
> Steps to reproduce
> =
> 1. Create table
> 2. Insert few records into the table
> 3. flush the table
> 4. Split the table
> 5. Create snapshot before catalog janitor clears the parent region entry from 
> meta.
> 6. Restore snapshot
> We can see the problem in meta entries,
> Meta content before restore snapshot:
> {noformat}
> t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:regioninfo, timestamp=1511537565964, value={ENCODED => 
> 077a12b0b3c91b053fa95223635f9543, NAME => 
> 't1,,1511537529449.077a12b0b3c91b053fa95223635f9543.', STARTKEY =>
>   '', ENDKEY => 
> '', OFFLINE => true, SPLIT => true}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:seqnumDuringOpen, timestamp=1511537530107, 
> value=\x00\x00\x00\x00\x00\x00\x00\x02
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:server, timestamp=1511537530107, value=host-xx:16020
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:serverstartcode, timestamp=1511537530107, value=1511537511523
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:splitA, timestamp=1511537565964, value={ENCODED => 
> 3c7c866d4df370c586131a4cbe0ef6a8, NAME => 
> 't1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8.', STARTKEY => '',
>   ENDKEY => 'm'}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:splitB, timestamp=1511537565964, value={ENCODED => 
> dc7facd824c85b94e5bf6a2e6b5f5efc, NAME => 
> 't1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.', STARTKEY => 'm
>  ', ENDKEY => ''}
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:regioninfo, timestamp=1511537566075, value={ENCODED => 
> 3c7c866d4df370c586131a4cbe0ef6a8, NAME => 
> 't1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8.', STARTKEY =>
>   '', ENDKEY => 
> 'm'}
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:seqnumDuringOpen, timestamp=1511537566075, 
> value=\x00\x00\x00\x00\x00\x00\x00\x02
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:server, timestamp=1511537566075, value=host-xx:16020
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:serverstartcode, timestamp=1511537566075, value=1511537511523
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:regioninfo, timestamp=1511537566069, value={ENCODED => 
> dc7facd824c85b94e5bf6a2e6b5f5efc, NAME => 
> 't1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.', STARTKEY =
>  > 'm', ENDKEY => 
> ''}
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:seqnumDuringOpen, timestamp=1511537566069, 
> value=\x00\x00\x00\x00\x00\x00\x00\x08
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:server, timestamp=1511537566069, value=host-xx:16020
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:serverstartcode, timestamp=1511537566069, value=1511537511523
> {noformat}
> Meta content after restore snapshot:
> {noformat}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:regioninfo, timestamp=1511537667635, value={ENCODED => 
> 077a12b0b3c91b053fa95223635f9543, NAME => 
> 't1,,1511537529449.077a12b0b3c91b053fa95223635f9543.', STARTKEY =>
>   '', ENDKEY => 
> ''}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:seqnumDuringOpen, timestamp=1511537667635, 
> 

[jira] [Commented] (HBASE-20371) website landing page should draw attention to CFP

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430896#comment-16430896
 ] 

Hadoop QA commented on HBASE-20371:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 13m 
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 16m 
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
15s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 45s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20371 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12918187/HBASE-20371.0.patch |
| Optional Tests |  asflicense  mvnsite  xml  |
| uname | Linux 57a0027b63de 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / ee87de9bfd |
| maven | version: Apache Maven 3.5.3 
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Max. process+thread count | 84 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/12358/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> website landing page should draw attention to CFP
> -
>
> Key: HBASE-20371
> URL: https://issues.apache.org/jira/browse/HBASE-20371
> Project: HBase
>  Issue Type: Task
>  Components: website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Attachments: HBASE-20371.0.patch
>
>
> I missed that the CFP for HBaseCon NA West 2018 is still open because I 
> didn't click the link to the conf page. The "News" section should mention 
> that CFP is open until it is not.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20068) Hadoopcheck project health check uses default maven repo instead of yetus managed ones

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-20068:

Status: Patch Available  (was: In Progress)

-v1
  - accounts for HBASE-15466 landing with another direct invocation of {{MAVEN}}
  - disable shellcheck warnings present, since that's what yetus does when it 
uses the executor function internally.

> Hadoopcheck project health check uses default maven repo instead of yetus 
> managed ones
> --
>
> Key: HBASE-20068
> URL: https://issues.apache.org/jira/browse/HBASE-20068
> Project: HBase
>  Issue Type: Bug
>  Components: community, test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-20068.0.patch, HBASE-20068.1.patch
>
>
> Recently had a precommit run fail hadoop check for all 3 versions with 
> {code}
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-install-plugin:2.5.2:install (default-install) 
> on project hbase-thrift: Failed to install metadata 
> org.apache.hbase:hbase-thrift:3.0.0-SNAPSHOT/maven-metadata.xml: Could not 
> parse metadata 
> /home/jenkins/.m2/repository/org/apache/hbase/hbase-thrift/3.0.0-SNAPSHOT/maven-metadata-local.xml:
>  in epilog non whitespace content is not allowed but got / (position: END_TAG 
> seen ...\n/... @25:2)  -> [Help 1]
> {code}
> Looks like maven repo corruption.
> Also the path {{/home/jenkins/.m2/repository}} means that those invocations 
> are using the jenkins user repo, which isn't safe since there are multiple 
> executors. either the plugin isn't using the yetus provided maven repo path 
> or our yetus invocation isn't telling yetus to provide its own maven repo 
> path.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-20068) Hadoopcheck project health check uses default maven repo instead of yetus managed ones

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-20068:

Attachment: HBASE-20068.1.patch

> Hadoopcheck project health check uses default maven repo instead of yetus 
> managed ones
> --
>
> Key: HBASE-20068
> URL: https://issues.apache.org/jira/browse/HBASE-20068
> Project: HBase
>  Issue Type: Bug
>  Components: community, test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-20068.0.patch, HBASE-20068.1.patch
>
>
> Recently had a precommit run fail hadoop check for all 3 versions with 
> {code}
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-install-plugin:2.5.2:install (default-install) 
> on project hbase-thrift: Failed to install metadata 
> org.apache.hbase:hbase-thrift:3.0.0-SNAPSHOT/maven-metadata.xml: Could not 
> parse metadata 
> /home/jenkins/.m2/repository/org/apache/hbase/hbase-thrift/3.0.0-SNAPSHOT/maven-metadata-local.xml:
>  in epilog non whitespace content is not allowed but got / (position: END_TAG 
> seen ...\n/... @25:2)  -> [Help 1]
> {code}
> Looks like maven repo corruption.
> Also the path {{/home/jenkins/.m2/repository}} means that those invocations 
> are using the jenkins user repo, which isn't safe since there are multiple 
> executors. either the plugin isn't using the yetus provided maven repo path 
> or our yetus invocation isn't telling yetus to provide its own maven repo 
> path.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-19079) Support setting up two clusters with A and S state

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-19079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430892#comment-16430892
 ] 

Hadoop QA commented on HBASE-19079:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 4 new or modified test 
files. {color} |
|| || || || {color:brown} HBASE-19064 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  8m 
11s{color} | {color:green} HBASE-19064 passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 
40s{color} | {color:red} hbase-server in HBASE-19064 failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
14s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  6m 
16s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m  
1s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
39s{color} | {color:green} HBASE-19064 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 
34s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  0m 34s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  1m 
12s{color} | {color:red} hbase-server: The patch generated 3 new + 0 unchanged 
- 0 fixed = 3 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
49s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
19m 51s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}115m 28s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
25s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}163m 29s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.wal.TestWALDurability |
|   | hadoop.hbase.client.replication.TestReplicationAdmin |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-19079 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12918162/HBASE-19079-HBASE-19064.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 1d3dda82dda4 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | HBASE-19064 / e7b37cf934 |
| maven | version: Apache Maven 3.5.3 
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_162 |
| compile | 
https://builds.apache.org/job/PreCommit-HBASE-Build/12356/artifact/patchprocess/branch-compile-hbase-server.txt
 |
| findbugs | v3.1.0-RC3 |
| compile | 

[jira] [Updated] (HBASE-20068) Hadoopcheck project health check uses default maven repo instead of yetus managed ones

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-20068:

Status: In Progress  (was: Patch Available)

> Hadoopcheck project health check uses default maven repo instead of yetus 
> managed ones
> --
>
> Key: HBASE-20068
> URL: https://issues.apache.org/jira/browse/HBASE-20068
> Project: HBase
>  Issue Type: Bug
>  Components: community, test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HBASE-20068.0.patch
>
>
> Recently had a precommit run fail hadoop check for all 3 versions with 
> {code}
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-install-plugin:2.5.2:install (default-install) 
> on project hbase-thrift: Failed to install metadata 
> org.apache.hbase:hbase-thrift:3.0.0-SNAPSHOT/maven-metadata.xml: Could not 
> parse metadata 
> /home/jenkins/.m2/repository/org/apache/hbase/hbase-thrift/3.0.0-SNAPSHOT/maven-metadata-local.xml:
>  in epilog non whitespace content is not allowed but got / (position: END_TAG 
> seen ...\n/... @25:2)  -> [Help 1]
> {code}
> Looks like maven repo corruption.
> Also the path {{/home/jenkins/.m2/repository}} means that those invocations 
> are using the jenkins user repo, which isn't safe since there are multiple 
> executors. either the plugin isn't using the yetus provided maven repo path 
> or our yetus invocation isn't telling yetus to provide its own maven repo 
> path.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20364) nightly job gives old results or no results for stages that timeout on SCM

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-20364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430884#comment-16430884
 ] 

Sean Busbey commented on HBASE-20364:
-

different failure mode on HBASE-20362:

{quote}
Results for branch branch-2.0
[build #150 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/150/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/150//General_Nightly_Build_Report/]




(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/150//JDK8_Nightly_Build_Report_(Hadoop2)/]


(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/150//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.
{quote}

build #150 completed successfully entirely.

> nightly job gives old results or no results for stages that timeout on SCM
> --
>
> Key: HBASE-20364
> URL: https://issues.apache.org/jira/browse/HBASE-20364
> Project: HBase
>  Issue Type: Bug
>  Components: test
>Affects Versions: 3.0.0
>Reporter: Sean Busbey
>Priority: Critical
>
> seen in the branch-2.0 nightly report for HBASE-18828:
>  
> {quote}
> Results for branch branch-2.0
>  [build #143 on 
> builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/143/]:
>  (x) *\{color:red}-1 overall\{color}*
> 
> details (if available):
> (/) \{color:green}+1 general checks\{color}
> -- For more information [see general 
> report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/140//General_Nightly_Build_Report/]
>  
> (/) \{color:green}+1 jdk8 hadoop2 checks\{color}
> -- For more information [see jdk8 (hadoop2) 
> report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/143//JDK8_Nightly_Build_Report_(Hadoop2)/]
> (/) \{color:green}+1 jdk8 hadoop3 checks\{color}
> -- For more information [see jdk8 (hadoop3) 
> report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/143//JDK8_Nightly_Build_Report_(Hadoop3)/]
>  
> {quote}
>  
> -1 for the overall build was correct. build #143 failed both the general 
> check and the source tarball check.
>  
> but in the posted comment, we get a false "passing" that links to the general 
> result from build #140. and we get no result for the source tarball at all.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-19343) Restore snapshot makes split parent region online

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-19343:
---
Fix Version/s: 1.4.4
   1.3.3

> Restore snapshot makes split parent region online 
> --
>
> Key: HBASE-19343
> URL: https://issues.apache.org/jira/browse/HBASE-19343
> Project: HBase
>  Issue Type: Bug
>  Components: snapshots
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Major
> Fix For: 1.5.0, 1.3.3, 1.4.4
>
> Attachments: HBASE-19343-branch-1-v2.patch, 
> HBASE-19343-branch-1.2.patch, HBASE-19343-branch-1.3.patch, 
> HBASE-19343-branch-1.patch, Snapshot.jpg
>
>
> Restore snapshot makes parent split region online as shown in the attached 
> snapshot.
> Steps to reproduce
> =
> 1. Create table
> 2. Insert few records into the table
> 3. flush the table
> 4. Split the table
> 5. Create snapshot before catalog janitor clears the parent region entry from 
> meta.
> 6. Restore snapshot
> We can see the problem in meta entries,
> Meta content before restore snapshot:
> {noformat}
> t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:regioninfo, timestamp=1511537565964, value={ENCODED => 
> 077a12b0b3c91b053fa95223635f9543, NAME => 
> 't1,,1511537529449.077a12b0b3c91b053fa95223635f9543.', STARTKEY =>
>   '', ENDKEY => 
> '', OFFLINE => true, SPLIT => true}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:seqnumDuringOpen, timestamp=1511537530107, 
> value=\x00\x00\x00\x00\x00\x00\x00\x02
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:server, timestamp=1511537530107, value=host-xx:16020
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:serverstartcode, timestamp=1511537530107, value=1511537511523
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:splitA, timestamp=1511537565964, value={ENCODED => 
> 3c7c866d4df370c586131a4cbe0ef6a8, NAME => 
> 't1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8.', STARTKEY => '',
>   ENDKEY => 'm'}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:splitB, timestamp=1511537565964, value={ENCODED => 
> dc7facd824c85b94e5bf6a2e6b5f5efc, NAME => 
> 't1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.', STARTKEY => 'm
>  ', ENDKEY => ''}
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:regioninfo, timestamp=1511537566075, value={ENCODED => 
> 3c7c866d4df370c586131a4cbe0ef6a8, NAME => 
> 't1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8.', STARTKEY =>
>   '', ENDKEY => 
> 'm'}
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:seqnumDuringOpen, timestamp=1511537566075, 
> value=\x00\x00\x00\x00\x00\x00\x00\x02
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:server, timestamp=1511537566075, value=host-xx:16020
>  t1,,1511537565718.3c7c866d4df370c586131a4cbe0ef6a8. 
> column=info:serverstartcode, timestamp=1511537566075, value=1511537511523
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:regioninfo, timestamp=1511537566069, value={ENCODED => 
> dc7facd824c85b94e5bf6a2e6b5f5efc, NAME => 
> 't1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.', STARTKEY =
>  > 'm', ENDKEY => 
> ''}
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:seqnumDuringOpen, timestamp=1511537566069, 
> value=\x00\x00\x00\x00\x00\x00\x00\x08
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:server, timestamp=1511537566069, value=host-xx:16020
>  t1,m,1511537565718.dc7facd824c85b94e5bf6a2e6b5f5efc.
> column=info:serverstartcode, timestamp=1511537566069, value=1511537511523
> {noformat}
> Meta content after restore snapshot:
> {noformat}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:regioninfo, timestamp=1511537667635, value={ENCODED => 
> 077a12b0b3c91b053fa95223635f9543, NAME => 
> 't1,,1511537529449.077a12b0b3c91b053fa95223635f9543.', STARTKEY =>
>   '', ENDKEY => 
> ''}
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:seqnumDuringOpen, timestamp=1511537667635, 
> value=\x00\x00\x00\x00\x00\x00\x00\x0A
>  t1,,1511537529449.077a12b0b3c91b053fa95223635f9543. 
> column=info:server, timestamp=1511537667635, value=host-xx:16020
>  

[jira] [Updated] (HBASE-20365) start branch-2.0 specific website section

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-20365:

Resolution: Fixed
Status: Resolved  (was: Patch Available)

thanks [~mdrob]!

> start branch-2.0 specific website section
> -
>
> Key: HBASE-20365
> URL: https://issues.apache.org/jira/browse/HBASE-20365
> Project: HBase
>  Issue Type: Task
>  Components: documentation, website
>Affects Versions: 2.0.0-alpha-1
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20365.0.patch
>
>
> our website should have docs folks can look at related to the 2.0-* line of 
> releases.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work started] (HBASE-20332) shaded mapreduce module shouldn't include hadoop

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-20332 started by Sean Busbey.
---
> shaded mapreduce module shouldn't include hadoop
> 
>
> Key: HBASE-20332
> URL: https://issues.apache.org/jira/browse/HBASE-20332
> Project: HBase
>  Issue Type: Sub-task
>  Components: mapreduce, shading
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 2.0.0
>
>
> AFAICT, we should just entirely skip including hadoop in our shaded mapreduce 
> module
> 1) Folks expect to run yarn / mr apps via {{hadoop jar}} / {{yarn jar}}
> 2) those commands include all the needed Hadoop jars in your classpath by 
> default (both client side and in the containers)
> 3) If you try to use "user classpath first" for your job as a workaround 
> (e.g. for some library your application needs that hadoop provides) then our 
> inclusion of *some but not all* hadoop classes then causes everything to fall 
> over because of mixing rewritten and non-rewritten hadoop classes
> 4) if you don't use "user classpath first" then all of our 
> non-relocated-but-still-shaded hadoop classes are ignored anyways so we're 
> just wasting space



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)