[jira] [Commented] (HBASE-23838) Adding debug logging to a few ExportSnapshot tests
[
https://issues.apache.org/jira/browse/HBASE-23838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039036#comment-17039036
]
Hudson commented on HBASE-23838:
Results for branch master
[build #1634 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1634/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1634//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1634//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1634//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Adding debug logging to a few ExportSnapshot tests
> --
>
> Key: HBASE-23838
> URL: https://issues.apache.org/jira/browse/HBASE-23838
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-23838-Adding-debug-logging-to-a-few-ExportSnap.patch,
> 0001-HBASE-23838-Adding-debug-logging-to-a-few-ExportSnap.patch
>
>
> Bit of extra logging in tests to help me diagnose where they are going wrong.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23838) Adding debug logging to a few ExportSnapshot tests
[
https://issues.apache.org/jira/browse/HBASE-23838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17038806#comment-17038806
]
Hudson commented on HBASE-23838:
Results for branch branch-2
[build #2480 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2480/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2480//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2480//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2480//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Adding debug logging to a few ExportSnapshot tests
> --
>
> Key: HBASE-23838
> URL: https://issues.apache.org/jira/browse/HBASE-23838
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-23838-Adding-debug-logging-to-a-few-ExportSnap.patch,
> 0001-HBASE-23838-Adding-debug-logging-to-a-few-ExportSnap.patch
>
>
> Bit of extra logging in tests to help me diagnose where they are going wrong.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23838) Adding debug logging to a few ExportSnapshot tests
[
https://issues.apache.org/jira/browse/HBASE-23838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17038716#comment-17038716
]
Michael Stack commented on HBASE-23838:
---
Added another logging addendum to see if can catch where hdfs comes into the
mix (though no hdfs clusters in test). Attached what I pushed as
0001-HBASE-23838-Adding-debug-logging-to-a-few-ExportSnap.patch.
Here are changes applied to master and branch-2:
{code}
>From 0c18fa983be70db49541c27e95896eba874ff0fe Mon Sep 17 00:00:00 2001
From: stack
Date: Mon, 17 Feb 2020 17:17:40 -0800
Subject: [PATCH] HBASE-23838 Adding debug logging to a few ExportSnapshot
tests ADDENDUM 2: More logging to debug.
---
.../org/apache/hadoop/hbase/snapshot/ExportSnapshot.java | 4 ++--
.../apache/hadoop/hbase/snapshot/TestExportSnapshot.java | 9 ++---
.../hbase/snapshot/TestExportSnapshotV1NoCluster.java| 2 +-
3 files changed, 9 insertions(+), 6 deletions(-)
diff --git
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/snapshot/ExportSnapshot.java
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/snapshot/ExportSnapshot.java
index 36b75c43df..198ae97f9b 100644
---
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/snapshot/ExportSnapshot.java
+++
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/snapshot/ExportSnapshot.java
@@ -978,8 +978,8 @@ public class ExportSnapshot extends AbstractHBaseTool
implements Tool {
Path outputSnapshotDir =
SnapshotDescriptionUtils.getCompletedSnapshotDir(targetName, outputRoot);
Path initialOutputSnapshotDir = skipTmp ? outputSnapshotDir :
snapshotTmpDir;
LOG.debug("inputFs={}, inputRoot={}", inputFs.getUri().toString(),
inputRoot);
-LOG.debug("outputFs={}, outputRoot={}, outputFs.getUri().toString(), " +
-"initialOutputSnapshotDir={}", outputRoot.toString(), skipTmp,
initialOutputSnapshotDir);
+LOG.debug("outputFs={}, outputRoot={}, skipTmp={},
initialOutputSnapshotDir={}",
+ outputFs, outputRoot.toString(), skipTmp, initialOutputSnapshotDir);
// Find the necessary directory which need to change owner and group
Path needSetOwnerDir =
SnapshotDescriptionUtils.getSnapshotRootDir(outputRoot);
diff --git
a/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
b/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
index cfdca30f49..973e182133 100644
---
a/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
+++
b/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
@@ -87,13 +87,15 @@ public class TestExportSnapshot {
public static void setUpBaseConf(Configuration conf) {
conf.setBoolean(SnapshotManager.HBASE_SNAPSHOT_ENABLED, true);
conf.setInt("hbase.regionserver.msginterval", 100);
+// If a single node has enough failures (default 3), resource manager will
blacklist it.
+// With only 2 nodes and tests injecting faults, we don't want that.
+conf.setInt("mapreduce.job.maxtaskfailures.per.tracker", 100);
+/*
conf.setInt("hbase.client.pause", 250);
conf.setInt(HConstants.HBASE_CLIENT_RETRIES_NUMBER, 6);
conf.setBoolean("hbase.master.enabletable.roundrobin", true);
conf.setInt("mapreduce.map.maxattempts", 10);
-// If a single node has enough failures (default 3), resource manager will
blacklist it.
-// With only 2 nodes and tests injecting faults, we don't want that.
-conf.setInt("mapreduce.job.maxtaskfailures.per.tracker", 100);
+*/
}
@BeforeClass
@@ -213,6 +215,7 @@ public class TestExportSnapshot {
final RegionPredicate bypassregionPredicate, boolean success) throws
Exception {
URI hdfsUri = FileSystem.get(conf).getUri();
FileSystem fs = FileSystem.get(copyDir.toUri(), conf);
+LOG.info("DEBUG FS {} {} {}, hdfsUri={}", fs, copyDir, copyDir.toUri(),
hdfsUri);
copyDir = copyDir.makeQualified(fs.getUri(), fs.getWorkingDirectory());
List opts = new ArrayList<>();
opts.add("--snapshot");
{code}
> Adding debug logging to a few ExportSnapshot tests
> --
>
> Key: HBASE-23838
> URL: https://issues.apache.org/jira/browse/HBASE-23838
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-23838-Adding-debug-logging-to-a-few-ExportSnap.patch,
> 0001-HBASE-23838-Adding-debug-logging-to-a-few-ExportSnap.patch
>
>
> Bit of extra logging in tests to help me diagnose where they are going wrong.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23838) Adding debug logging to a few ExportSnapshot tests
[
https://issues.apache.org/jira/browse/HBASE-23838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17037422#comment-17037422
]
Hudson commented on HBASE-23838:
Results for branch master
[build #1630 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1630/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1630//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1630//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1630//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Adding debug logging to a few ExportSnapshot tests
> --
>
> Key: HBASE-23838
> URL: https://issues.apache.org/jira/browse/HBASE-23838
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-23838-Adding-debug-logging-to-a-few-ExportSnap.patch
>
>
> Bit of extra logging in tests to help me diagnose where they are going wrong.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23838) Adding debug logging to a few ExportSnapshot tests
[
https://issues.apache.org/jira/browse/HBASE-23838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17037415#comment-17037415
]
Hudson commented on HBASE-23838:
Results for branch branch-2
[build #2474 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2474/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2474//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2474//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2474//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Adding debug logging to a few ExportSnapshot tests
> --
>
> Key: HBASE-23838
> URL: https://issues.apache.org/jira/browse/HBASE-23838
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-23838-Adding-debug-logging-to-a-few-ExportSnap.patch
>
>
> Bit of extra logging in tests to help me diagnose where they are going wrong.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23838) Adding debug logging to a few ExportSnapshot tests
[
https://issues.apache.org/jira/browse/HBASE-23838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17037370#comment-17037370
]
Michael Stack commented on HBASE-23838:
---
Pushed an addendum on branch-2 and master to fix failing TestExportSnapshot:
{code}
diff --git
a/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
b/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
index 4293d461d9..99d9d65d30 100644
---
a/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
+++
b/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
@@ -213,8 +213,7 @@ public class TestExportSnapshot {
final RegionPredicate bypassregionPredicate, boolean success) throws
Exception {
URI hdfsUri = FileSystem.get(conf).getUri();
FileSystem fs = FileSystem.get(copyDir.toUri(), conf);
-copyDir = copyDir.makeQualified(hdfsUri, copyDir);
-
+copyDir = copyDir.makeQualified(fs.getUri(), fs.getWorkingDirectory());
List opts = new ArrayList<>();
opts.add("--snapshot");
opts.add(Bytes.toString(snapshotName));
@@ -227,10 +226,8 @@ public class TestExportSnapshot {
if (overwrite) opts.add("--overwrite");
// Export Snapshot
-ExportSnapshot es = new ExportSnapshot();
-es.setConf(conf);
-int res = run(conf, es, opts.toArray(new String[opts.size()]));
-assertEquals(success ? 0 : 1, res);
+int res = run(conf, new ExportSnapshot(), opts.toArray(new
String[opts.size()]));
+assertEquals("success " + success + ", res=" + res, success ? 0 : 1, res);
if (!success) {
final Path targetDir = new Path(HConstants.SNAPSHOT_DIR_NAME,
Bytes.toString(targetName));
assertFalse(copyDir.toString() + " " + targetDir.toString(),
{code}
> Adding debug logging to a few ExportSnapshot tests
> --
>
> Key: HBASE-23838
> URL: https://issues.apache.org/jira/browse/HBASE-23838
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-23838-Adding-debug-logging-to-a-few-ExportSnap.patch
>
>
> Bit of extra logging in tests to help me diagnose where they are going wrong.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23838) Adding debug logging to a few ExportSnapshot tests
[
https://issues.apache.org/jira/browse/HBASE-23838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036946#comment-17036946
]
Hudson commented on HBASE-23838:
Results for branch master
[build #1629 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1629/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1629//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1629//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1629//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Adding debug logging to a few ExportSnapshot tests
> --
>
> Key: HBASE-23838
> URL: https://issues.apache.org/jira/browse/HBASE-23838
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-23838-Adding-debug-logging-to-a-few-ExportSnap.patch
>
>
> Bit of extra logging in tests to help me diagnose where they are going wrong.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23838) Adding debug logging to a few ExportSnapshot tests
[
https://issues.apache.org/jira/browse/HBASE-23838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036642#comment-17036642
]
Hudson commented on HBASE-23838:
Results for branch branch-2
[build #2472 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2472/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2472//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2472//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2472//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Adding debug logging to a few ExportSnapshot tests
> --
>
> Key: HBASE-23838
> URL: https://issues.apache.org/jira/browse/HBASE-23838
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-23838-Adding-debug-logging-to-a-few-ExportSnap.patch
>
>
> Bit of extra logging in tests to help me diagnose where they are going wrong.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23838) Adding debug logging to a few ExportSnapshot tests
[ https://issues.apache.org/jira/browse/HBASE-23838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036534#comment-17036534 ] Michael Stack commented on HBASE-23838: --- What I pushed on branch-2 and master just now. > Adding debug logging to a few ExportSnapshot tests > -- > > Key: HBASE-23838 > URL: https://issues.apache.org/jira/browse/HBASE-23838 > Project: HBase > Issue Type: Bug >Reporter: Michael Stack >Priority: Trivial > Attachments: > 0001-HBASE-23838-Adding-debug-logging-to-a-few-ExportSnap.patch > > > Bit of extra logging in tests to help me diagnose where they are going wrong. -- This message was sent by Atlassian Jira (v8.3.4#803005)
