[jira] [Commented] (HDDS-987) MultipartUpload: S3API for list parts of a object
[ https://issues.apache.org/jira/browse/HDDS-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16761013#comment-16761013 ] Bharat Viswanadham commented on HDDS-987: - Thank You [~elek] for review and commit. I missed this API during my initial design document for ozone MPU support. I will look into it. i think this lists all in progress multipart uploads happening in a bucket. But I don't think MPU functionality is incomplete without this API. As the user can complete MPU and also abort MPU. (Without this missing will be, user will not be able to find in progress MPU's happening in a bucket) [~ste...@apache.org] ist-multipart-uploads is used in s3a? > MultipartUpload: S3API for list parts of a object > - > > Key: HDDS-987 > URL: https://issues.apache.org/jira/browse/HDDS-987 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > Fix For: 0.4.0 > > Attachments: HDDS-987.01.patch, HDDS-987.02.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListParts.html -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-987) MultipartUpload: S3API for list parts of a object
[ https://issues.apache.org/jira/browse/HDDS-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16760826#comment-16760826 ] Steve Loughran commented on HDDS-987: - you tried the S3A committer or the new MultipartUploader API & Tests yet? They're the probably places where we use this in our own code Trivia: AWS bill you for MPUs you don't commit. Don't think you can get away without paying for all those incomplete uploads you haven't committed yet. > MultipartUpload: S3API for list parts of a object > - > > Key: HDDS-987 > URL: https://issues.apache.org/jira/browse/HDDS-987 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > Attachments: HDDS-987.01.patch, HDDS-987.02.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListParts.html -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-987) MultipartUpload: S3API for list parts of a object
[ https://issues.apache.org/jira/browse/HDDS-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16760773#comment-16760773 ] Hudson commented on HDDS-987: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15889 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/15889/]) HDDS-987. MultipartUpload: S3API for list parts of a object. Contributed (elek: rev 9ace37b11920989db7c97838b250d43c445267b9) * (add) hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/ListPartsResponse.java * (edit) hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneMultipartUploadPartListParts.java * (edit) hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/ObjectEndpoint.java * (edit) hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestPartUpload.java * (edit) hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/client/OzoneBucketStub.java * (edit) hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestObjectGet.java * (edit) hadoop-ozone/dist/src/main/smoketest/s3/MultipartUpload.robot * (add) hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestListParts.java > MultipartUpload: S3API for list parts of a object > - > > Key: HDDS-987 > URL: https://issues.apache.org/jira/browse/HDDS-987 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > Attachments: HDDS-987.01.patch, HDDS-987.02.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListParts.html -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-987) MultipartUpload: S3API for list parts of a object
[ https://issues.apache.org/jira/browse/HDDS-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16760724#comment-16760724 ] Elek, Marton commented on HDDS-987: --- +1 LGTM Tested all the multipart aws commands manually and everything worked well (except list-multipart-uploads which was not yet implemented). list-parts also worked well, excepted the owner/user field which can be implemented after the s3 block token support. Tests are not related, I can see the same tests in the daily builds... Will commit it soon. > MultipartUpload: S3API for list parts of a object > - > > Key: HDDS-987 > URL: https://issues.apache.org/jira/browse/HDDS-987 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Labels: pull-request-available > Attachments: HDDS-987.01.patch, HDDS-987.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListParts.html -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-987) MultipartUpload: S3API for list parts of a object
[
https://issues.apache.org/jira/browse/HDDS-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757708#comment-16757708
]
Hadoop QA commented on HDDS-987:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 18 line(s) with tabs. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 30m 16s{color}
| {color:red} hadoop-ozone in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 5m
31s{color} | {color:green} hadoop-hdds in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 45m 58s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.ozone.client.rpc.TestOzoneRpcClientWithRatis |
| | hadoop.ozone.client.rpc.TestSecureOzoneRpcClient |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDDS-987 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12957162/HDDS-987.02.patch |
| Optional Tests | asflicense javac javadoc unit findbugs checkstyle |
| uname | Linux 57f524fd4141 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HDDS-Build@2/ozone.sh |
| git revision | trunk / f738b39 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| whitespace |
https://builds.apache.org/job/PreCommit-HDDS-Build/2156/artifact/out/whitespace-tabs.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDDS-Build/2156/artifact/out/patch-unit-hadoop-ozone.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDDS-Build/2156/testReport/ |
| Max. process+thread count | 1139 (vs. ulimit of 1) |
| modules | C: hadoop-ozone/client hadoop-ozone/dist hadoop-ozone/s3gateway U:
hadoop-ozone |
| Console output |
https://builds.apache.org/job/PreCommit-HDDS-Build/2156/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> MultipartUpload: S3API for list parts of a object
> -
>
> Key: HDDS-987
> URL: https://issues.apache.org/jira/browse/HDDS-987
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Labels:
[jira] [Commented] (HDDS-987) MultipartUpload: S3API for list parts of a object
[ https://issues.apache.org/jira/browse/HDDS-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757646#comment-16757646 ] Bharat Viswanadham commented on HDDS-987: - Will upload a new PR, as now HDDS-956 got checked in. > MultipartUpload: S3API for list parts of a object > - > > Key: HDDS-987 > URL: https://issues.apache.org/jira/browse/HDDS-987 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Attachments: HDDS-987.01.patch > > > https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListParts.html -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-987) MultipartUpload: S3API for list parts of a object
[
https://issues.apache.org/jira/browse/HDDS-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750478#comment-16750478
]
Hadoop QA commented on HDDS-987:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} HDDS-987 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HDDS-987 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12956044/HDDS-987.01.patch |
| Console output |
https://builds.apache.org/job/PreCommit-HDDS-Build/2093/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> MultipartUpload: S3API for list parts of a object
> -
>
> Key: HDDS-987
> URL: https://issues.apache.org/jira/browse/HDDS-987
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
> Attachments: HDDS-987.01.patch
>
>
> https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListParts.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-987) MultipartUpload: S3API for list parts of a object
[ https://issues.apache.org/jira/browse/HDDS-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750475#comment-16750475 ] Bharat Viswanadham commented on HDDS-987: - This patch needs to be applied on top of HDDS-948 and HDDS-956. > MultipartUpload: S3API for list parts of a object > - > > Key: HDDS-987 > URL: https://issues.apache.org/jira/browse/HDDS-987 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Attachments: HDDS-987.01.patch > > > https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListParts.html -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
