szetszwo commented on code in PR #2858:
URL: https://github.com/apache/ozone/pull/2858#discussion_r910369107
##########
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/ObjectEndpoint.java:
##########
@@ -132,6 +138,8 @@ public class ObjectEndpoint extends EndpointBase {
private List<String> customizableGetHeaders = new ArrayList<>();
private int bufferSize;
+ private int chunkSize;
Review Comment:
Move chunkSize to ObjectEndpointStreaming
##########
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/ObjectEndpoint.java:
##########
@@ -132,6 +138,8 @@ public class ObjectEndpoint extends EndpointBase {
private List<String> customizableGetHeaders = new ArrayList<>();
private int bufferSize;
+ private int chunkSize;
+ private boolean datastreamEnabled;
Review Comment:
Indeed, we should create an `ObjectEndpointStreaming streaming` object and
use streaming != null for checking if streaming is enabled.
##########
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/ObjectEndpoint.java:
##########
@@ -185,11 +200,21 @@ public Response put(
storageType = headers.getHeaderString(STORAGE_CLASS_HEADER);
boolean storageTypeDefault = StringUtils.isEmpty(storageType);
+ if (storageTypeDefault) {
+ storageType = S3StorageType.getDefault(ozoneConfiguration).toString();
+ }
+
// Normal put object
OzoneBucket bucket = getBucket(bucketName);
ReplicationConfig replicationConfig =
getReplicationConfig(bucket, storageType);
+ boolean enableEC = false;
+ if ((replicationConfig != null &&
+ replicationConfig.getReplicationType() == EC) ||
+ bucket.getReplicationConfig() instanceof ECReplicationConfig) {
+ enableEC = true;
+ }
Review Comment:
This is incorrect when replicationConfig != null &&
replicationConfig.getReplicationType() != EC. As an isEc() method as below.
```
static boolean isEc(ReplicationConfig replication, OzoneBucket bucket) {
if (replication != null) {
return replication.getReplicationType() == EC;
} else {
return bucket.getReplicationConfig() instanceof ECReplicationConfig;
}
}
```
##########
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/ObjectEndpoint.java:
##########
@@ -690,18 +720,43 @@ private Response createMultipartKey(String bucket, String
key, long length,
throws IOException, OS3Exception {
try {
OzoneBucket ozoneBucket = getBucket(bucket);
- String copyHeader;
OzoneOutputStream ozoneOutputStream = null;
if ("STREAMING-AWS4-HMAC-SHA256-PAYLOAD"
.equals(headers.getHeaderString("x-amz-content-sha256"))) {
body = new SignedChunksInputStream(body);
}
+ String copyHeader = headers.getHeaderString(COPY_SOURCE_HEADER);
+ String storageType = headers.getHeaderString(STORAGE_CLASS_HEADER);
+ ReplicationConfig replicationConfig =
+ getReplicationConfig(ozoneBucket, storageType);
+
+ boolean enableEC = false;
+ if ((replicationConfig != null &&
+ replicationConfig.getReplicationType() == EC) ||
+ ozoneBucket.getReplicationConfig() instanceof ECReplicationConfig) {
+ enableEC = true;
+ }
+
try {
+ if (datastreamEnabled && !enableEC && copyHeader != null) {
+ Pair<String, String> result = parseSourceHeader(copyHeader);
+ String sourceBucket = result.getLeft();
+ String sourceKey = result.getRight();
+ OzoneBucket sourceOzoneBucket = getBucket(sourceBucket);
+ return ObjectEndpointStreaming
+ .copyMultipartKey(Pair.of(sourceOzoneBucket, sourceKey),
+ Pair.of(ozoneBucket, key), length, partNumber, uploadID,
+ chunkSize, headers);
+ } else if (datastreamEnabled && !enableEC) {
+ return ObjectEndpointStreaming
+ .createMultipartKey(ozoneBucket, key, length, partNumber,
+ uploadID, chunkSize, body);
+ }
+
Review Comment:
Move the logic to ObjectEndpointStreaming.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
