[
https://issues.apache.org/jira/browse/HADOOP-16306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16837759#comment-16837759
]
wujinhu commented on HADOOP-16306:
----------------------------------
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.aliyun.oss.TestAliyunCredentials
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.322 s
- in org.apache.hadoop.fs.aliyun.oss.TestAliyunCredentials
[INFO] Running org.apache.hadoop.fs.aliyun.oss.TestAliyunOSSInputStream
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.531 s
- in org.apache.hadoop.fs.aliyun.oss.TestAliyunOSSInputStream
[INFO] Running org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContext
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.466 s
- in org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContext
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContextStatistics
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.526 s
- in org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContextStatistics
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContextCreateMkdir
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.833
s - in org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContextCreateMkdir
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContextMainOperations
[WARNING] Tests run: 73, Failures: 0, Errors: 0, Skipped: 4, Time elapsed:
117.442 s - in
org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContextMainOperations
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContextUtil
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.929 s
- in org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContextUtil
[INFO] Running org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContextURI
[WARNING] Tests run: 17, Failures: 0, Errors: 0, Skipped: 1, Time elapsed:
43.218 s - in org.apache.hadoop.fs.aliyun.oss.fileContext.TestOSSFileContextURI
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractSeek
[INFO] Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.009
s - in org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractSeek
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractMkdir
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.501 s
- in org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractMkdir
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractDelete
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.999 s
- in org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractDelete
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractGetFileStatus
[INFO] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.432
s - in
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractGetFileStatus
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractDistCp
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 89.904 s
- in org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractDistCp
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractRename
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.768 s
- in org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractRename
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractRootDir
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.167 s
- in org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractRootDir
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractCreate
[WARNING] Tests run: 11, Failures: 0, Errors: 0, Skipped: 2, Time elapsed:
14.002 s - in
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractCreate
[INFO] Running
org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractOpen
[INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.351
s - in org.apache.hadoop.fs.aliyun.oss.contract.TestAliyunOSSContractOpen
[INFO] Running org.apache.hadoop.fs.aliyun.oss.yarn.TestOSS
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.889 s
- in org.apache.hadoop.fs.aliyun.oss.yarn.TestOSS
[INFO] Running org.apache.hadoop.fs.aliyun.oss.yarn.TestOSSMiniYarnCluster
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.323 s
- in org.apache.hadoop.fs.aliyun.oss.yarn.TestOSSMiniYarnCluster
[INFO] Running org.apache.hadoop.fs.aliyun.oss.TestAliyunOSSFileSystemStore
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 412.957
s - in org.apache.hadoop.fs.aliyun.oss.TestAliyunOSSFileSystemStore
[INFO] Running org.apache.hadoop.fs.aliyun.oss.TestAliyunOSSBlockOutputStream
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 108.327
s - in org.apache.hadoop.fs.aliyun.oss.TestAliyunOSSBlockOutputStream
[INFO] Running org.apache.hadoop.fs.aliyun.oss.TestAliyunOSSFileSystemContract
[INFO] Tests run: 52, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 147.057
s - in org.apache.hadoop.fs.aliyun.oss.TestAliyunOSSFileSystemContract
[INFO]
[INFO] Results:
[INFO]
[WARNING] Tests run: 284, Failures: 0, Errors: 0, Skipped: 7
> AliyunOSS: Remove temporary files when upload small files to OSS
> ----------------------------------------------------------------
>
> Key: HADOOP-16306
> URL: https://issues.apache.org/jira/browse/HADOOP-16306
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/oss
> Affects Versions: 2.10.0, 3.2.0, 2.9.2, 3.0.3, 3.3.0, 3.1.2
> Reporter: wujinhu
> Assignee: wujinhu
> Priority: Major
> Attachments: HADOOP-16306.001.patch, HADOOP-16306.002.patch
>
>
> Currently, AliyunOSSBlockOutputStream will upload files to OSS directly
> instead of multi part upload if their size are less than configured part
> size(fs.oss.multipart.upload.size).
> When caller closes AliyunOSSBlockOutputStream, it will call
> removePartFiles(), however, this method will only remove multi part temporary
> files, so this patch will fix case above.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
