[GitHub] [hadoop] steveloughran commented on a change in pull request #1950: HADOOP-16586. ITestS3GuardFsck, others fails when run using a local m…
steveloughran commented on a change in pull request #1950:
URL: https://github.com/apache/hadoop/pull/1950#discussion_r420882639
##
File path:
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/ITestS3GuardToolLocal.java
##
@@ -57,13 +59,18 @@
private static final String[] ABORT_FORCE_OPTIONS = new String[] {"-abort",
"-force", "-verbose"};
+ @Override
+ protected Configuration createConfiguration() {
+Configuration conf = super.createConfiguration();
+conf.set(S3_METADATA_STORE_IMPL, S3GUARD_METASTORE_LOCAL);
Review comment:
thanks. FWIW that removeBaseAndBucket call is varags; you can remove as
many as you need in one go. Code is fine as as is though
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #1950: HADOOP-16586. ITestS3GuardFsck, others fails when run using a local m…
steveloughran commented on a change in pull request #1950:
URL: https://github.com/apache/hadoop/pull/1950#discussion_r418483936
##
File path:
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/ITestS3GuardToolLocal.java
##
@@ -165,18 +165,19 @@ public void testImportNoFilesystem() throws Throwable {
@Test
public void testInfoBucketAndRegionNoFS() throws Throwable {
-intercept(FileNotFoundException.class,
+intercept(UnknownStoreException.class,
Review comment:
we should probably unset bucket probe options too & switch to the
default (2) value; avoid differences in init behaviour
##
File path:
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/ITestS3GuardToolLocal.java
##
@@ -57,13 +59,18 @@
private static final String[] ABORT_FORCE_OPTIONS = new String[] {"-abort",
"-force", "-verbose"};
+ @Override
+ protected Configuration createConfiguration() {
+Configuration conf = super.createConfiguration();
+conf.set(S3_METADATA_STORE_IMPL, S3GUARD_METASTORE_LOCAL);
Review comment:
need to use the `removeBaseAndBucketOverrides()` code in S3ATestUtils to
make sure that any per-bucket setting has turned it on. Look for other uses to
see what to do
##
File path:
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/ITestS3GuardToolLocal.java
##
@@ -97,7 +98,6 @@ public void testImportCommand() throws Exception {
.getListing().size());
assertEquals("Expected 2 items: empty directory and a parent directory", 2,
ms.listChildren(parent).getListing().size());
-assertTrue(children.isAuthoritative());
Review comment:
probably is wrong then :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #1950: HADOOP-16586. ITestS3GuardFsck, others fails when run using a local m…
steveloughran commented on a change in pull request #1950:
URL: https://github.com/apache/hadoop/pull/1950#discussion_r416469326
##
File path:
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/ITestS3GuardToolLocal.java
##
@@ -165,18 +165,19 @@ public void testImportNoFilesystem() throws Throwable {
@Test
public void testInfoBucketAndRegionNoFS() throws Throwable {
-intercept(FileNotFoundException.class,
+intercept(UnknownStoreException.class,
Review comment:
yeah, generally
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #1950: HADOOP-16586. ITestS3GuardFsck, others fails when run using a local m…
steveloughran commented on a change in pull request #1950:
URL: https://github.com/apache/hadoop/pull/1950#discussion_r411349770
##
File path:
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/ITestPartialRenamesDeletes.java
##
@@ -54,7 +54,12 @@
import static org.apache.hadoop.fs.contract.ContractTestUtils.*;
import static org.apache.hadoop.fs.s3a.Constants.*;
import static org.apache.hadoop.fs.s3a.S3ATestUtils.MetricDiff;
-import static org.apache.hadoop.fs.s3a.S3ATestUtils.*;
+import static org.apache.hadoop.fs.s3a.S3ATestUtils.assume;
Review comment:
not sure the expansion here is needed..,you know how imports are such a
backporting troublespot.
in fact, given there's no other diff to this class, I'm not sure this file
needs changing at all
##
File path:
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/ITestS3GuardToolLocal.java
##
@@ -165,18 +165,19 @@ public void testImportNoFilesystem() throws Throwable {
@Test
public void testInfoBucketAndRegionNoFS() throws Throwable {
-intercept(FileNotFoundException.class,
+intercept(UnknownStoreException.class,
Review comment:
if this is triggering failures, I'm not seeing them -but then I do have
fs.s3a.bucket.probe == 0it's probably not triggering any bucket probe at
all. For consistent failures here, the config we run with should have that
option set to 2, "use v2 probe on instantiation".
##
File path:
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/ITestS3GuardToolLocal.java
##
@@ -165,18 +165,19 @@ public void testImportNoFilesystem() throws Throwable {
@Test
public void testInfoBucketAndRegionNoFS() throws Throwable {
-intercept(FileNotFoundException.class,
+intercept(UnknownStoreException.class,
() -> run(BucketInfo.NAME, "-meta",
LOCAL_METADATA, "-region",
"any-region", S3A_THIS_BUCKET_DOES_NOT_EXIST));
}
@Test
public void testInitNegativeRead() throws Throwable {
-runToFailure(INVALID_ARGUMENT,
-Init.NAME, "-meta", LOCAL_METADATA, "-region",
-"eu-west-1",
-READ_FLAG, "-10");
+intercept(CommandFormat.UnknownOptionException.class,
Review comment:
if this is triggering bucket probe failures, it's not doing the -ve arg
check. So here we should either switch to a probe option of 0, or just cut the
test entirely. I'm almost going for the latter given the way people should be
creating tables is with read = write =0 for on-demand use
##
File path:
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/ITestS3GuardToolLocal.java
##
@@ -97,7 +98,6 @@ public void testImportCommand() throws Exception {
.getListing().size());
assertEquals("Expected 2 items: empty directory and a parent directory", 2,
ms.listChildren(parent).getListing().size());
-assertTrue(children.isAuthoritative());
Review comment:
why did you cut this assertion?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
