[jira] [Commented] (HBASE-21230) BackupUtils#checkTargetDir doesn't compose error message correctly

2018-10-07 Thread liubangchen (JIRA) 


[ 
https://issues.apache.org/jira/browse/HBASE-21230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16641320#comment-16641320
 ] 

liubangchen commented on HBASE-21230:
-

I think so. Let me fix it.

> BackupUtils#checkTargetDir doesn't compose error message correctly
> --
>
> Key: HBASE-21230
> URL: https://issues.apache.org/jira/browse/HBASE-21230
> Project: HBase
>  Issue Type: Bug
>  Components: backuprestore
>Reporter: Ted Yu
>Assignee: liubangchen
>Priority: Minor
> Attachments: HBASE-21230-1.patch
>
>
> Here is related code from BackupUtils#checkTargetDir:
> {code}
>   String expMsg = e.getMessage();
>   String newMsg = null;
>   if (expMsg.contains("No FileSystem for scheme")) {
> newMsg =
> "Unsupported filesystem scheme found in the backup target url. 
> Error Message: "
> + newMsg;
> {code}
> I think the intention was to concatenate expMsg at the end of newMsg.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-21230) BackupUtils#checkTargetDir doesn't compose error message correctly

2018-10-07 Thread liubangchen (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-21230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liubangchen updated HBASE-21230:

Attachment: HBASE-21230-1.patch
Status: Patch Available  (was: Open)

> BackupUtils#checkTargetDir doesn't compose error message correctly
> --
>
> Key: HBASE-21230
> URL: https://issues.apache.org/jira/browse/HBASE-21230
> Project: HBase
>  Issue Type: Bug
>  Components: backuprestore
>Reporter: Ted Yu
>Assignee: liubangchen
>Priority: Minor
> Attachments: HBASE-21230-1.patch
>
>
> Here is related code from BackupUtils#checkTargetDir:
> {code}
>   String expMsg = e.getMessage();
>   String newMsg = null;
>   if (expMsg.contains("No FileSystem for scheme")) {
> newMsg =
> "Unsupported filesystem scheme found in the backup target url. 
> Error Message: "
> + newMsg;
> {code}
> I think the intention was to concatenate expMsg at the end of newMsg.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HBASE-21269) Forward-port to branch-2 " HBASE-21213 [hbck2] bypass leaves behind state in RegionStates when assign/unassign"

2018-10-07 Thread Jingyun Tian (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-21269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jingyun Tian reassigned HBASE-21269:


Assignee: Jingyun Tian

> Forward-port to branch-2 " HBASE-21213 [hbck2] bypass leaves behind state 
> in RegionStates when assign/unassign"
> ---
>
> Key: HBASE-21269
> URL: https://issues.apache.org/jira/browse/HBASE-21269
> Project: HBase
>  Issue Type: Sub-task
>  Components: amv2
>Reporter: stack
>Assignee: Jingyun Tian
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21269.branch-2.001.patch
>
>
> A bunch of this patch does not apply to branch-2 and master now we don't have 
> AP or UP anymore. Need to figure if we need override in branch-2 and master. 
> Let me upload the forward-port done so far. Can finish this when move to 
> branch-2.2 exercise. FYI [~Apache9]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21269) Forward-port to branch-2 " HBASE-21213 [hbck2] bypass leaves behind state in RegionStates when assign/unassign"

2018-10-07 Thread stack (JIRA) 


[ 
https://issues.apache.org/jira/browse/HBASE-21269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16641298#comment-16641298
 ] 

stack commented on HBASE-21269:
---

@jingyoun tian that would be great. 

> Forward-port to branch-2 " HBASE-21213 [hbck2] bypass leaves behind state 
> in RegionStates when assign/unassign"
> ---
>
> Key: HBASE-21269
> URL: https://issues.apache.org/jira/browse/HBASE-21269
> Project: HBase
>  Issue Type: Sub-task
>  Components: amv2
>Reporter: stack
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21269.branch-2.001.patch
>
>
> A bunch of this patch does not apply to branch-2 and master now we don't have 
> AP or UP anymore. Need to figure if we need override in branch-2 and master. 
> Let me upload the forward-port done so far. Can finish this when move to 
> branch-2.2 exercise. FYI [~Apache9]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21269) Forward-port to branch-2 " HBASE-21213 [hbck2] bypass leaves behind state in RegionStates when assign/unassign"

2018-10-07 Thread Jingyun Tian (JIRA) 


[ 
https://issues.apache.org/jira/browse/HBASE-21269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16641296#comment-16641296
 ] 

Jingyun Tian commented on HBASE-21269:
--

[~stack] Can I take this one? I am currently working on how to make the HBCK 
works for TSRP.

> Forward-port to branch-2 " HBASE-21213 [hbck2] bypass leaves behind state 
> in RegionStates when assign/unassign"
> ---
>
> Key: HBASE-21269
> URL: https://issues.apache.org/jira/browse/HBASE-21269
> Project: HBase
>  Issue Type: Sub-task
>  Components: amv2
>Reporter: stack
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21269.branch-2.001.patch
>
>
> A bunch of this patch does not apply to branch-2 and master now we don't have 
> AP or UP anymore. Need to figure if we need override in branch-2 and master. 
> Let me upload the forward-port done so far. Can finish this when move to 
> branch-2.2 exercise. FYI [~Apache9]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-21216) TestSnapshotFromMaster#testSnapshotHFileArchiving is flaky

2018-10-07 Thread Ted Yu (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-21216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-21216:
---
Description: 
>From 
>https://builds.apache.org/job/HBase-Flaky-Tests/job/branch-2/794/testReport/junit/org.apache.hadoop.hbase.master.cleaner/TestSnapshotFromMaster/testSnapshotHFileArchiving/
> :
{code}
java.lang.AssertionError: Archived hfiles [] and table hfiles 
[9ca09392705f425f9c916beedc10d63c] is missing snapshot 
file:6739a09747e54189a4112a6d8f37e894
at 
org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster.testSnapshotHFileArchiving(TestSnapshotFromMaster.java:370)
{code}
The file appeared in archive dir before hfile cleaners were run:
{code}
2018-09-20 10:38:53,187 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|-archive/
2018-09-20 10:38:53,188 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|data/
2018-09-20 10:38:53,189 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|---default/
2018-09-20 10:38:53,190 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|--test/
2018-09-20 10:38:53,191 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|-1237d57b63a7bdf067a930441a02514a/
2018-09-20 10:38:53,192 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|recovered.edits/
2018-09-20 10:38:53,193 DEBUG [Time-limited test] util.CommonFSUtils(774): 
|---4.seqid
2018-09-20 10:38:53,193 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|-29e1700e09b51223ad2f5811105a4d51/
2018-09-20 10:38:53,194 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|fam/
2018-09-20 10:38:53,195 DEBUG [Time-limited test] util.CommonFSUtils(774): 
|---2c66a18f6c1a4074b84ffbb3245268c4
2018-09-20 10:38:53,196 DEBUG [Time-limited test] util.CommonFSUtils(774): 
|---45bb396c6a5e49629e45a4d56f1e9b14
2018-09-20 10:38:53,196 DEBUG [Time-limited test] util.CommonFSUtils(774): 
|---6739a09747e54189a4112a6d8f37e894
{code}
However, the archive dir became empty after hfile cleaners were run:
{code}
2018-09-20 10:38:53,312 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|-archive/
2018-09-20 10:38:53,313 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|-corrupt/
{code}
Leading to the assertion failure.

This test is one of the top flaky tests.

  was:
>From 
>https://builds.apache.org/job/HBase-Flaky-Tests/job/branch-2/794/testReport/junit/org.apache.hadoop.hbase.master.cleaner/TestSnapshotFromMaster/testSnapshotHFileArchiving/
> :
{code}
java.lang.AssertionError: Archived hfiles [] and table hfiles 
[9ca09392705f425f9c916beedc10d63c] is missing snapshot 
file:6739a09747e54189a4112a6d8f37e894
at 
org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster.testSnapshotHFileArchiving(TestSnapshotFromMaster.java:370)
{code}
The file appeared in archive dir before hfile cleaners were run:
{code}
2018-09-20 10:38:53,187 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|-archive/
2018-09-20 10:38:53,188 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|data/
2018-09-20 10:38:53,189 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|---default/
2018-09-20 10:38:53,190 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|--test/
2018-09-20 10:38:53,191 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|-1237d57b63a7bdf067a930441a02514a/
2018-09-20 10:38:53,192 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|recovered.edits/
2018-09-20 10:38:53,193 DEBUG [Time-limited test] util.CommonFSUtils(774): 
|---4.seqid
2018-09-20 10:38:53,193 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|-29e1700e09b51223ad2f5811105a4d51/
2018-09-20 10:38:53,194 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|fam/
2018-09-20 10:38:53,195 DEBUG [Time-limited test] util.CommonFSUtils(774): 
|---2c66a18f6c1a4074b84ffbb3245268c4
2018-09-20 10:38:53,196 DEBUG [Time-limited test] util.CommonFSUtils(774): 
|---45bb396c6a5e49629e45a4d56f1e9b14
2018-09-20 10:38:53,196 DEBUG [Time-limited test] util.CommonFSUtils(774): 
|---6739a09747e54189a4112a6d8f37e894
{code}
However, the archive dir became empty after hfile cleaners were run:
{code}
2018-09-20 10:38:53,312 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|-archive/
2018-09-20 10:38:53,313 DEBUG [Time-limited test] util.CommonFSUtils(771): 
|-corrupt/
{code}
Leading to the assertion failure.


> TestSnapshotFromMaster#testSnapshotHFileArchiving is flaky
> --
>
> Key: HBASE-21216
> URL: https://issues.apache.org/jira/browse/HBASE-21216
> Project: HBase
>  Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments:

[jira] [Updated] (HBASE-21230) BackupUtils#checkTargetDir doesn't compose error message correctly

2018-10-07 Thread Ted Yu (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-21230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-21230:
---
Description: 
Here is related code from BackupUtils#checkTargetDir:
{code}
  String expMsg = e.getMessage();
  String newMsg = null;
  if (expMsg.contains("No FileSystem for scheme")) {
newMsg =
"Unsupported filesystem scheme found in the backup target url. 
Error Message: "
+ newMsg;
{code}
I think the intention was to concatenate expMsg at the end of newMsg.

  was:
Here is related code:
{code}
  String expMsg = e.getMessage();
  String newMsg = null;
  if (expMsg.contains("No FileSystem for scheme")) {
newMsg =
"Unsupported filesystem scheme found in the backup target url. 
Error Message: "
+ newMsg;
{code}
I think the intention was to concatenate expMsg at the end of newMsg.


> BackupUtils#checkTargetDir doesn't compose error message correctly
> --
>
> Key: HBASE-21230
> URL: https://issues.apache.org/jira/browse/HBASE-21230
> Project: HBase
>  Issue Type: Bug
>  Components: backuprestore
>Reporter: Ted Yu
>Assignee: liubangchen
>Priority: Minor
>
> Here is related code from BackupUtils#checkTargetDir:
> {code}
>   String expMsg = e.getMessage();
>   String newMsg = null;
>   if (expMsg.contains("No FileSystem for scheme")) {
> newMsg =
> "Unsupported filesystem scheme found in the backup target url. 
> Error Message: "
> + newMsg;
> {code}
> I think the intention was to concatenate expMsg at the end of newMsg.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-21238) MapReduceHFileSplitterJob#run shouldn't call System.exit

2018-10-07 Thread Ted Yu (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-21238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-21238:
---
Labels: mapreduce  (was: )

> MapReduceHFileSplitterJob#run shouldn't call System.exit
> 
>
> Key: HBASE-21238
> URL: https://issues.apache.org/jira/browse/HBASE-21238
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>  Labels: mapreduce
>
> {code}
> if (args.length < 2) {
>   usage("Wrong number of arguments: " + args.length);
>   System.exit(-1);
> {code}
> Correct way of handling error condition is through return value of run method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21250) Refactor WALProcedureStore and add more comments for better understanding the implementation

2018-10-07 Thread Hudson (JIRA) 


[ 
https://issues.apache.org/jira/browse/HBASE-21250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16641144#comment-16641144
 ] 

Hudson commented on HBASE-21250:


Results for branch branch-2.1
[build #430 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/430/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/430//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- Something went wrong running this stage, please [check relevant console 
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/430//console].


(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/430//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Refactor WALProcedureStore and add more comments for better understanding the 
> implementation
> 
>
> Key: HBASE-21250
> URL: https://issues.apache.org/jira/browse/HBASE-21250
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21250-v1.patch, HBASE-21250-v2.patch, 
> HBASE-21250-v3.patch, HBASE-21250.patch
>
>
> The implementation is complicated and lack of comments to say how it works.
> {code}
> /**
>  * WAL implementation of the ProcedureStore.
>  * @see ProcedureWALPrettyPrinter for printing content of a single WAL.
>  * @see #main(String[]) to parse a directory of MasterWALProcs.
>  */
> {code}
> I think at least we can move sub classes to separated files to make the class 
> smaller, and add more comments to describe what is going on here.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21250) Refactor WALProcedureStore and add more comments for better understanding the implementation

2018-10-07 Thread Hudson (JIRA) 


[ 
https://issues.apache.org/jira/browse/HBASE-21250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16641118#comment-16641118
 ] 

Hudson commented on HBASE-21250:


Results for branch branch-2
[build #1354 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1354/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- Something went wrong running this stage, please [check relevant console 
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1354//console].




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1354//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1354//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Refactor WALProcedureStore and add more comments for better understanding the 
> implementation
> 
>
> Key: HBASE-21250
> URL: https://issues.apache.org/jira/browse/HBASE-21250
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21250-v1.patch, HBASE-21250-v2.patch, 
> HBASE-21250-v3.patch, HBASE-21250.patch
>
>
> The implementation is complicated and lack of comments to say how it works.
> {code}
> /**
>  * WAL implementation of the ProcedureStore.
>  * @see ProcedureWALPrettyPrinter for printing content of a single WAL.
>  * @see #main(String[]) to parse a directory of MasterWALProcs.
>  */
> {code}
> I think at least we can move sub classes to separated files to make the class 
> smaller, and add more comments to describe what is going on here.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21250) Refactor WALProcedureStore and add more comments for better understanding the implementation

2018-10-07 Thread Hudson (JIRA) 


[ 
https://issues.apache.org/jira/browse/HBASE-21250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16641112#comment-16641112
 ] 

Hudson commented on HBASE-21250:


Results for branch branch-2.0
[build #917 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/917/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/917//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- Something went wrong running this stage, please [check relevant console 
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/917//console].


(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/917//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


> Refactor WALProcedureStore and add more comments for better understanding the 
> implementation
> 
>
> Key: HBASE-21250
> URL: https://issues.apache.org/jira/browse/HBASE-21250
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21250-v1.patch, HBASE-21250-v2.patch, 
> HBASE-21250-v3.patch, HBASE-21250.patch
>
>
> The implementation is complicated and lack of comments to say how it works.
> {code}
> /**
>  * WAL implementation of the ProcedureStore.
>  * @see ProcedureWALPrettyPrinter for printing content of a single WAL.
>  * @see #main(String[]) to parse a directory of MasterWALProcs.
>  */
> {code}
> I think at least we can move sub classes to separated files to make the class 
> smaller, and add more comments to describe what is going on here.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21275) Thrift Server (branch 1 fix) -> Disable TRACE HTTP method for thrift http server (branch 1 only)

2018-10-07 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/HBASE-21275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16641109#comment-16641109
 ] 

Hadoop QA commented on HBASE-21275:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} branch-1.2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m 
25s{color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
45s{color} | {color:green} branch-1.2 passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
43s{color} | {color:green} branch-1.2 passed with JDK v1.7.0_191 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
55s{color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  3m 
33s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
2s{color} | {color:green} branch-1.2 passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
7s{color} | {color:green} branch-1.2 passed with JDK v1.7.0_191 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
47s{color} | {color:green} the patch passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed with JDK v1.7.0_191 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
49s{color} | {color:red} hbase-thrift: The patch generated 2 new + 24 unchanged 
- 0 fixed = 26 total (was 24) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  3m 
22s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
11m 54s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.4.1 2.5.2 2.6.5 2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
54s{color} | {color:green} the patch passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
7s{color} | {color:green} the patch passed with JDK v1.7.0_191 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  6m  
2s{color} | {color:green} hbase-thrift in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
24s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 44s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:34a9b27 |
| JIRA Issue | HBASE-21275 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12942692/HBASE-21275-branch-1.2.001.patch
 |
| Optional Tests |  dupname  asflicense  javac  javadoc  unit  findbugs  
shadedjars  hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux 03501fc574ae 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27

[jira] [Commented] (HBASE-20623) Introduce the helper method "getCellBuilder()" to Mutation

2018-10-07 Thread Chia-Ping Tsai (JIRA) 


[ 
https://issues.apache.org/jira/browse/HBASE-20623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16641078#comment-16641078
 ] 

Chia-Ping Tsai commented on HBASE-20623:


bq. Sorry for my absence for a long time.I have polished up this patch 
again.the QA report is green.can we move on?
Thanks for the patch. Some comments are already in review board.

> Introduce the helper method "getCellBuilder()" to Mutation
> --
>
> Key: HBASE-20623
> URL: https://issues.apache.org/jira/browse/HBASE-20623
> Project: HBase
>  Issue Type: Task
>  Components: API
>Reporter: Chia-Ping Tsai
>Assignee: maoling
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-20623.master.001.patch, 
> HBASE-20623.master.002.patch, HBASE-20623.master.003.patch, 
> HBASE-20623.master.004.patch, HBASE-20623.master.005.patch
>
>
> see 
> [https://lists.apache.org/thread.html/d05bfaa0134502a47f6e1aca56cb0b096d4dd32ddefbbdf28db4952a@%3Cdev.hbase.apache.org%3E]
>  for more details.
> {code:java}
> How about a "getCellBuilder" or "getCellBuilderFactory" method for
> Mutation implementations that gives you a CellBuilder instance that
> already has relevant parts set? Like for a Put instance it should be
> able to already have the Type and Row set.{code}
> mentioned a day or so ago by [~busbey]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-20623) Introduce the helper method "getCellBuilder()" to Mutation

2018-10-07 Thread maoling (JIRA) 


[ 
https://issues.apache.org/jira/browse/HBASE-20623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16641074#comment-16641074
 ] 

maoling commented on HBASE-20623:
-

[~chia7712] 
Sorry for my absence for a long time.I have polished up this patch again.the QA 
report is green.can we move on?

> Introduce the helper method "getCellBuilder()" to Mutation
> --
>
> Key: HBASE-20623
> URL: https://issues.apache.org/jira/browse/HBASE-20623
> Project: HBase
>  Issue Type: Task
>  Components: API
>Reporter: Chia-Ping Tsai
>Assignee: maoling
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-20623.master.001.patch, 
> HBASE-20623.master.002.patch, HBASE-20623.master.003.patch, 
> HBASE-20623.master.004.patch, HBASE-20623.master.005.patch
>
>
> see 
> [https://lists.apache.org/thread.html/d05bfaa0134502a47f6e1aca56cb0b096d4dd32ddefbbdf28db4952a@%3Cdev.hbase.apache.org%3E]
>  for more details.
> {code:java}
> How about a "getCellBuilder" or "getCellBuilderFactory" method for
> Mutation implementations that gives you a CellBuilder instance that
> already has relevant parts set? Like for a Put instance it should be
> able to already have the Type and Row set.{code}
> mentioned a day or so ago by [~busbey]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21219) Hbase incremental backup fails with null pointer exception

2018-10-07 Thread Hudson (JIRA) 


[ 
https://issues.apache.org/jira/browse/HBASE-21219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16641050#comment-16641050
 ] 

Hudson commented on HBASE-21219:


Results for branch master
[build #530 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/530/]: (x) 
*{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- Something went wrong running this stage, please [check relevant console 
output|https://builds.apache.org/job/HBase%20Nightly/job/master/530//console].




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/530//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/530//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Hbase incremental backup fails with null pointer exception
> --
>
> Key: HBASE-21219
> URL: https://issues.apache.org/jira/browse/HBASE-21219
> Project: HBase
>  Issue Type: Bug
>  Components: backuprestore
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-21219-v1.patch
>
>
> hbase backup create incremental hdfs:///bkpHbase_Test/bkpHbase_Test2 -t 
> bkpHbase_Test2 
> 2018-09-21 15:35:31,421 INFO [main] impl.TableBackupClient: Backup 
> backup_1537524313995 started at 1537524331419. 2018-09-21 15:35:31,454 INFO 
> [main] impl.IncrementalBackupManager: Execute roll log procedure for 
> incremental backup ... 2018-09-21 15:35:32,985 ERROR [main] 
> impl.TableBackupClient: Unexpected Exception : java.lang.NullPointerException 
> java.lang.NullPointerException at 
> org.apache.hadoop.hbase.backup.impl.IncrementalBackupManager.getLogFilesForNewBackup(IncrementalBackupManager.java:309)
>  at 
> org.apache.hadoop.hbase.backup.impl.IncrementalBackupManager.getIncrBackupLogFileMap(IncrementalBackupManager.java:103)
>  at 
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.execute(IncrementalTableBackupClient.java:276)
>  at 
> org.apache.hadoop.hbase.backup.impl.BackupAdminImpl.backupTables(BackupAdminImpl.java:601)
>  at 
> org.apache.hadoop.hbase.backup.impl.BackupCommands$CreateCommand.execute(BackupCommands.java:347)
>  at 
> org.apache.hadoop.hbase.backup.BackupDriver.parseAndRun(BackupDriver.java:138)
>  at org.apache.hadoop.hbase.backup.BackupDriver.doWork(BackupDriver.java:171) 
> at org.apache.hadoop.hbase.backup.BackupDriver.run(BackupDriver.java:204) at 
> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at 
> org.apache.hadoop.hbase.backup.BackupDriver.main(BackupDriver.java:179) 
> 2018-09-21 15:35:32,989 ERROR [main] impl.TableBackupClient: 
> BackupId=backup_1537524313995,startts=1537524331419,failedts=1537524332989,failedphase=PREPARE_INCREMENTAL,failedmessage=null
>  2018-09-21 15:35:57,167 ERROR [main] impl.TableBackupClient: Backup 
> backup_1537524313995 failed. 
> Backup session finished. Status: FAILURE 2018-09-21 15:35:57,175 ERROR [main] 
> backup.BackupDriver: Error running 
> command-line tool java.io.IOException: java.lang.NullPointerException at 
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.execute(IncrementalTableBackupClient.java:281)
>  at 
> org.apache.hadoop.hbase.backup.impl.BackupAdminImpl.backupTables(BackupAdminImpl.java:601)
>  at 
> org.apache.hadoop.hbase.backup.impl.BackupCommands$CreateCommand.execute(BackupCommands.java:347)
>  at 
> org.apache.hadoop.hbase.backup.BackupDriver.parseAndRun(BackupDriver.java:138)
>  at org.apache.hadoop.hbase.backup.BackupDriver.doWork(BackupDriver.java:171) 
> at org.apache.hadoop.hbase.backup.BackupDriver.run(BackupDriver.java:204) at 
> org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at 
> org.apache.hadoop.hbase.backup.BackupDriver.main(BackupDriver.java:179) 
> Caused by: java.lang.NullPointerException at 
> org.apache.hadoop.hbase.backup.impl.IncrementalBackupManager.getLogFilesForNewBackup(IncrementalBackupManager.java:309)
>  at 
> org.apache.hadoop.hbase.backup.impl.IncrementalBackupManager.getIncrBackupLogFileMap(IncrementalBackupManager.java:103)
>  at 
> org.apache.hadoop.hbase.backup.impl.IncrementalTableBackupClient.execute(IncrementalTableBackupClient.java:276)
>  ... 7 more



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-21250) Refactor WALProcedureStore and add more comments for better understanding the implementation

2018-10-07 Thread Duo Zhang (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-21250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-21250:
--
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Pushed to branch-2.0+.

Thanks [~stack] and [~allan163] for reviewing.

> Refactor WALProcedureStore and add more comments for better understanding the 
> implementation
> 
>
> Key: HBASE-21250
> URL: https://issues.apache.org/jira/browse/HBASE-21250
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21250-v1.patch, HBASE-21250-v2.patch, 
> HBASE-21250-v3.patch, HBASE-21250.patch
>
>
> The implementation is complicated and lack of comments to say how it works.
> {code}
> /**
>  * WAL implementation of the ProcedureStore.
>  * @see ProcedureWALPrettyPrinter for printing content of a single WAL.
>  * @see #main(String[]) to parse a directory of MasterWALProcs.
>  */
> {code}
> I think at least we can move sub classes to separated files to make the class 
> smaller, and add more comments to describe what is going on here.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HBASE-21254) Need to find a way to limit the number of proc wal files

2018-10-07 Thread Duo Zhang (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-21254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang reassigned HBASE-21254:
-

Assignee: Duo Zhang

> Need to find a way to limit the number of proc wal files
> 
>
> Key: HBASE-21254
> URL: https://issues.apache.org/jira/browse/HBASE-21254
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
>
> For regionserver, we have a max wal file limitation, if we reach the 
> limitation, we will trigger a flush on specific regions so that we can delete 
> old wal files. But for proc wals, we do not have this mechanism, and it will 
> be worse after HBASE-21233, as if there is an old procedure which can not 
> make progress and do not persist its state, we need to keep the old proc wal 
> file for ever...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21250) Refactor WALProcedureStore and add more comments for better understanding the implementation

2018-10-07 Thread Duo Zhang (JIRA) 


[ 
https://issues.apache.org/jira/browse/HBASE-21250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16641004#comment-16641004
 ] 

Duo Zhang commented on HBASE-21250:
---

Let me commit.

> Refactor WALProcedureStore and add more comments for better understanding the 
> implementation
> 
>
> Key: HBASE-21250
> URL: https://issues.apache.org/jira/browse/HBASE-21250
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21250-v1.patch, HBASE-21250-v2.patch, 
> HBASE-21250-v3.patch, HBASE-21250.patch
>
>
> The implementation is complicated and lack of comments to say how it works.
> {code}
> /**
>  * WAL implementation of the ProcedureStore.
>  * @see ProcedureWALPrettyPrinter for printing content of a single WAL.
>  * @see #main(String[]) to parse a directory of MasterWALProcs.
>  */
> {code}
> I think at least we can move sub classes to separated files to make the class 
> smaller, and add more comments to describe what is going on here.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HBASE-21250) Refactor WALProcedureStore and add more comments for better understanding the implementation

2018-10-07 Thread Duo Zhang (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HBASE-21250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang updated HBASE-21250:
--
Fix Version/s: 2.0.3
   2.1.1

> Refactor WALProcedureStore and add more comments for better understanding the 
> implementation
> 
>
> Key: HBASE-21250
> URL: https://issues.apache.org/jira/browse/HBASE-21250
> Project: HBase
>  Issue Type: Sub-task
>  Components: proc-v2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21250-v1.patch, HBASE-21250-v2.patch, 
> HBASE-21250-v3.patch, HBASE-21250.patch
>
>
> The implementation is complicated and lack of comments to say how it works.
> {code}
> /**
>  * WAL implementation of the ProcedureStore.
>  * @see ProcedureWALPrettyPrinter for printing content of a single WAL.
>  * @see #main(String[]) to parse a directory of MasterWALProcs.
>  */
> {code}
> I think at least we can move sub classes to separated files to make the class 
> smaller, and add more comments to describe what is going on here.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)