[
https://issues.apache.org/jira/browse/HADOOP-15323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732181#comment-16732181
]
Weiwei Yang commented on HADOOP-15323:
--------------------------------------
Hi [~wujinhu]
Sorry for the late response, I am just back from vacation.
I am confused with following logic
{code}
public boolean copyFile(String srcKey, long srcLen, String dstKey) {
try {
//1, try single copy first
return singleCopy(srcKey, dstKey);
} catch (Exception e) {
//2, if failed(shallow copy not supported), then multi part copy
return multipartCopy(srcKey, srcLen, dstKey);
}
{code}
this assumes any exception thrown by \{{singleCopy}} means shallow copy not
supported, and then it falls back to use \{{multipartCopy}}. What if there were
some other exceptions throws, that means it goes to retry multi part copy and
then fail again? Is there an elegant way to tell if singleCopy is supported or
not?
Another thing, removing a configuration is an incompatible change, it would
affect old clients, is that something to concern about?
Thanks
> AliyunOSS: Improve copy file performance for AliyunOSSFileSystemStore
> ---------------------------------------------------------------------
>
> Key: HADOOP-15323
> URL: https://issues.apache.org/jira/browse/HADOOP-15323
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/oss
> Affects Versions: 2.10.0, 3.2.0, 3.1.1, 2.9.2, 3.0.3, 3.3.0
> Reporter: wujinhu
> Assignee: wujinhu
> Priority: Major
> Attachments: HADOOP-15323.001.patch, HADOOP-15323.002.patch
>
>
> Aliyun OSS will support shallow copy which means server will only copy
> metadata when copy object operation occurs.
> With shallow copy, we can use copyObject api instead of multi-part copy api
> if we do not change object storage type & encryption type & source object is
> uploaded by Put / Multipart upload api.
> We will try to use copyObject api, and check result. If shallow copy disabled
> for this object, then we will use multipart copy. So, I will remove
> fs.oss.multipart.upload.threshold configuration.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
