[
https://issues.apache.org/jira/browse/HADOOP-18168?focusedWorklogId=764077&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764077
]
ASF GitHub Bot logged work on HADOOP-18168:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 29/Apr/22 08:38
Start Date: 29/Apr/22 08:38
Worklog Time Spent: 10m
Work Description: dannycjones commented on code in PR #4140:
URL: https://github.com/apache/hadoop/pull/4140#discussion_r861581105
##########
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ARequesterPays.java:
##########
@@ -102,14 +88,30 @@ public void testRequesterPaysDisabledFails() throws
Throwable {
}
}
- private Path getRequesterPaysPath(Configuration conf) {
- String requesterPaysFile =
- conf.getTrimmed(KEY_REQUESTER_PAYS_FILE, DEFAULT_REQUESTER_PAYS_FILE);
- S3ATestUtils.assume(
- "Empty test property: " + KEY_REQUESTER_PAYS_FILE,
- !requesterPaysFile.isEmpty()
- );
- return new Path(requesterPaysFile);
+ /**
+ * Use this after creating the file system, as this is when bucket-specific
+ * overrides are applied.
+ * @param conf Hadoop configuration from FS to mutate
+ * @param requesterPaysEnabled Indicate if requester pays be on or off
+ */
+ private static void updateConf(
+ Configuration conf,
+ boolean requesterPaysEnabled
Review Comment:
makes sense, will note for future.
i removed this code now since the change was unnecessary and was making the
patch longer. (I had some confusion around the bucket conf propagation when i
wrote it)
Issue Time Tracking
-------------------
Worklog Id: (was: 764077)
Time Spent: 3h 10m (was: 3h)
> ITestMarkerTool.testRunLimitedLandsatAudit failing due to most of bucket
> content purged
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-18168
> URL: https://issues.apache.org/jira/browse/HADOOP-18168
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
> Affects Versions: 3.3.4
> Reporter: Steve Loughran
> Assignee: Daniel Carl Jones
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> {{ITestMarkerTool.testRunLimitedLandsatAudit}} is failing -a scan which was
> meant to stop after the first page of results is finishing because there
> aren't so many objects there.
>
> first visible sign of the landsat-pds cleanup
> now we have requester pays, we could do this against another store with
> stability promises, e.g common crawl.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
