[
https://issues.apache.org/jira/browse/HADOOP-12020?focusedWorklogId=767833&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-767833
]
ASF GitHub Bot logged work on HADOOP-12020:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 09/May/22 08:54
Start Date: 09/May/22 08:54
Worklog Time Spent: 10m
Work Description: monthonk commented on code in PR #3877:
URL: https://github.com/apache/hadoop/pull/3877#discussion_r867783928
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Constants.java:
##########
@@ -429,6 +429,24 @@ private Constants() {
*/
public static final String CONTENT_ENCODING =
"fs.s3a.object.content.encoding";
+ /**
+ * S3 storage class: standard, reduced_redundancy, intelligent_tiering etc.
+ */
+ public static final String STORAGE_CLASS = "fs.s3a.storage.class";
+
+ /**
+ * All possible values for storage class.
+ */
+ public static final String STORAGE_CLASS_STANDARD = "standard";
+ public static final String STORAGE_CLASS_REDUCED_REDUNDANCY =
"reduced_redundancy";
+ public static final String STORAGE_CLASS_GLACIER = "glacier";
+ public static final String STORAGE_CLASS_STANDARD_INFREQUENT_ACCESS =
"standard_ia";
+ public static final String STORAGE_CLASS_ONEZONE_INFREQUENT_ACCESS =
"onezone_ia";
+ public static final String STORAGE_CLASS_INTELLIGENT_TIERING =
"intelligent_tiering";
+ public static final String STORAGE_CLASS_DEEP_ARCHIVE = "deep_archive";
+ public static final String STORAGE_CLASS_OUTPOSTS = "outposts";
+ public static final String STORAGE_CLASS_GLACIER_INSTANT_RETRIEVAL =
"glacier_ir";
+
Review Comment:
We will have to keep updating these values. However, users will not be
effected by this because these constants are used in our code only, users can
still put any value they like in xml file.
also discussed with Steve, he prefer to have static string in Constants.
Issue Time Tracking
-------------------
Worklog Id: (was: 767833)
Time Spent: 2h 10m (was: 2h)
> Support AWS S3 reduced redundancy storage class
> -----------------------------------------------
>
> Key: HADOOP-12020
> URL: https://issues.apache.org/jira/browse/HADOOP-12020
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 2.7.0
> Environment: Hadoop on AWS
> Reporter: Yann Landrin-Schweitzer
> Assignee: Monthon Klongklaew
> Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Amazon S3 uses, by default, the NORMAL_STORAGE class for s3 objects.
> This offers, according to Amazon's material, 99.99999999% reliability.
> For many applications, however, the 99.99% reliability offered by the
> REDUCED_REDUNDANCY storage class is amply sufficient, and comes with a
> significant cost saving.
> HDFS, when using the legacy s3n protocol, or the new s3a scheme, should
> support overriding the default storage class of created s3 objects so that
> users can take advantage of this cost benefit.
> This would require minor changes of the s3n and s3a drivers, using
> a configuration property fs.s3n.storage.class to override the default storage
> when desirable.
> This override could be implemented in Jets3tNativeFileSystemStore with:
> S3Object object = new S3Object(key);
> ...
> if(storageClass!=null) object.setStorageClass(storageClass);
> It would take a more complex form in s3a, e.g. setting:
> InitiateMultipartUploadRequest initiateMPURequest =
> new InitiateMultipartUploadRequest(bucket, key, om);
> if(storageClass !=null ) {
> initiateMPURequest =
> initiateMPURequest.withStorageClass(storageClass);
> }
> and similar statements in various places.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
