[
https://issues.apache.org/jira/browse/HADOOP-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817658#comment-17817658
]
ASF GitHub Bot commented on HADOOP-14837:
-----------------------------------------
steveloughran commented on code in PR #6407:
URL: https://github.com/apache/hadoop/pull/6407#discussion_r1490936359
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/StoreContext.java:
##########
@@ -37,7 +37,7 @@
import org.apache.hadoop.fs.s3a.S3AFileStatus;
import org.apache.hadoop.fs.s3a.S3AInputPolicy;
import org.apache.hadoop.fs.s3a.S3AStorageStatistics;
-import org.apache.hadoop.fs.s3a.S3ObjectStorageClassFilter;
+import org.apache.hadoop.fs.s3a.api.S3ObjectStorageClassFilter;
Review Comment:
this has to move down to L42 now, as its in a sub package.
##########
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/list/ITestS3AReadRestoredGlacierObjects.java:
##########
@@ -89,7 +92,7 @@ private FileSystem createFiles(String
s3ObjectStorageClassFilter) throws Throwab
FileSystem fs = contract.getTestFileSystem();
Path dir = methodPath();
fs.mkdirs(dir);
Review Comment:
skip this for a marginal increase in performance. create() assumes there's a
dir and just creates the object in the destination path without checks.
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java:
##########
@@ -52,6 +52,7 @@
import java.util.concurrent.atomic.AtomicBoolean;
import javax.annotation.Nullable;
+import org.apache.hadoop.fs.s3a.api.S3ObjectStorageClassFilter;
Review Comment:
same
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Constants.java:
##########
@@ -1525,6 +1526,11 @@ private Constants() {
*/
public static final String READ_RESTORED_GLACIER_OBJECTS =
"fs.s3a.glacier.read.restored.objects";
+ /**
+ * Default value of Read Restored Glacier objects config.
+ */
+ public static final String DEFAULT_READ_RESTORED_GLACIER_OBJECTS =
S3ObjectStorageClassFilter.READ_ALL.toString();
Review Comment:
split onto a new line to keep checkstyle happy
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Listing.java:
##########
@@ -18,6 +18,7 @@
package org.apache.hadoop.fs.s3a;
+import org.apache.hadoop.fs.s3a.api.S3ObjectStorageClassFilter;
Review Comment:
new org.apache imports MUST go into that block for new classes. set your IDE
up for this and life gets simpler for all
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/api/S3ObjectStorageClassFilter.java:
##########
@@ -16,8 +16,9 @@
* limitations under the License.
*/
-package org.apache.hadoop.fs.s3a;
+package org.apache.hadoop.fs.s3a.api;
+import org.apache.hadoop.fs.s3a.S3AFileSystem;
Review Comment:
now, this is a branch new class. in which case the L22 import can go into
the org.apache block and the S3aFilesystem one with it. the only reason it is
in the wrong place in existing code is that the move to repackaged classes was
a big search and replace only: no re-ordering
> Handle S3A "glacier" data
> -------------------------
>
> Key: HADOOP-14837
> URL: https://issues.apache.org/jira/browse/HADOOP-14837
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.0.0-beta1
> Reporter: Steve Loughran
> Assignee: Bhavay Pahuja
> Priority: Minor
> Labels: pull-request-available
>
> SPARK-21797 covers how if you have AWS S3 set to copy some files to glacier,
> they appear in the listing but GETs fail, and so does everything else
> We should think about how best to handle this.
> # report better
> # if listings can identify files which are glaciated then maybe we could have
> an option to filter them out
> # test & see what happens
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
