[jira] [Commented] (HADOOP-6916) Implement append operation for S3FileSystem
[
https://issues.apache.org/jira/browse/HADOOP-6916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16017266#comment-16017266
]
Steve Loughran commented on HADOOP-6916:
I think we can close as a wontfix for S3, but the tests could be lifted out
into the contract tests for those filesystems which do append...we don't have
many (any?) AFAIK
> Implement append operation for S3FileSystem
> ---
>
> Key: HADOOP-6916
> URL: https://issues.apache.org/jira/browse/HADOOP-6916
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs/s3
>Affects Versions: 0.20.2, 0.22.0
>Reporter: Oleg Aleshko
>Priority: Minor
> Attachments: s3_append1.patch
>
>
> Currently org.apache.hadoop.fs.s3.S3FileSystem#append throws an
> IOException("Not supported");
> S3FileSystem should be able to support appending, possibly via common block
> storage logic.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-6916) Implement append operation for S3FileSystem
[ https://issues.apache.org/jira/browse/HADOOP-6916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14080643#comment-14080643 ] Steve Loughran commented on HADOOP-6916: pure s3 hasn't had much maintenance for a while; focus has been on s3n, and soon the s3a successor -which will support big files. But: no append, it's not part of S3 Implement append operation for S3FileSystem --- Key: HADOOP-6916 URL: https://issues.apache.org/jira/browse/HADOOP-6916 Project: Hadoop Common Issue Type: New Feature Components: fs/s3 Affects Versions: 0.20.2, 0.22.0 Reporter: Oleg Aleshko Priority: Minor Attachments: s3_append1.patch Currently org.apache.hadoop.fs.s3.S3FileSystem#append throws an IOException(Not supported); S3FileSystem should be able to support appending, possibly via common block storage logic. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HADOOP-6916) Implement append operation for S3FileSystem
[ https://issues.apache.org/jira/browse/HADOOP-6916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14080191#comment-14080191 ] Allen Wittenauer commented on HADOOP-6916: -- Is this still an issue with all the S3 rework that happened? Implement append operation for S3FileSystem --- Key: HADOOP-6916 URL: https://issues.apache.org/jira/browse/HADOOP-6916 Project: Hadoop Common Issue Type: New Feature Components: fs/s3 Affects Versions: 0.20.2, 0.22.0 Reporter: Oleg Aleshko Priority: Minor Attachments: s3_append1.patch Currently org.apache.hadoop.fs.s3.S3FileSystem#append throws an IOException(Not supported); S3FileSystem should be able to support appending, possibly via common block storage logic. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] Commented: (HADOOP-6916) Implement append operation for S3FileSystem
[ https://issues.apache.org/jira/browse/HADOOP-6916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12918729#action_12918729 ] Eli Collins commented on HADOOP-6916: - Hey Oleg, * Doesn't pos and filePos need to be updated when opening for append? * Doesn't S3OutputStream#setBlocks need to be synchronized? Why does it copy the blocks? * Make sure the unit tests exercise calling close after append. * Have you tested this on S3? eg by setting test.fs.s3.name in src/test/core-site.xml? Hudson currently running, please use ant test-patch from an svn checkout. Thanks, Eli Implement append operation for S3FileSystem --- Key: HADOOP-6916 URL: https://issues.apache.org/jira/browse/HADOOP-6916 Project: Hadoop Common Issue Type: New Feature Components: fs/s3 Affects Versions: 0.20.2, 0.22.0 Reporter: Oleg Aleshko Priority: Minor Fix For: 0.22.0 Attachments: s3_append1.patch Currently org.apache.hadoop.fs.s3.S3FileSystem#append throws an IOException(Not supported); S3FileSystem should be able to support appending, possibly via common block storage logic. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6916) Implement append operation for S3FileSystem
[ https://issues.apache.org/jira/browse/HADOOP-6916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12918731#action_12918731 ] Eli Collins commented on HADOOP-6916: - Forgot to mention this could use some additional tests, eg appending to a file already open for append, appending more than a block, etc. See the hdfs append tests for additional ideas. Implement append operation for S3FileSystem --- Key: HADOOP-6916 URL: https://issues.apache.org/jira/browse/HADOOP-6916 Project: Hadoop Common Issue Type: New Feature Components: fs/s3 Affects Versions: 0.20.2, 0.22.0 Reporter: Oleg Aleshko Priority: Minor Fix For: 0.22.0 Attachments: s3_append1.patch Currently org.apache.hadoop.fs.s3.S3FileSystem#append throws an IOException(Not supported); S3FileSystem should be able to support appending, possibly via common block storage logic. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6916) Implement append operation for S3FileSystem
[ https://issues.apache.org/jira/browse/HADOOP-6916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12912460#action_12912460 ] Oleg Aleshko commented on HADOOP-6916: -- Can this be integrated into the upcoming release? Implement append operation for S3FileSystem --- Key: HADOOP-6916 URL: https://issues.apache.org/jira/browse/HADOOP-6916 Project: Hadoop Common Issue Type: Bug Components: fs/s3 Affects Versions: 0.20.2 Reporter: Oleg Aleshko Priority: Minor Attachments: s3_append1.patch Currently org.apache.hadoop.fs.s3.S3FileSystem#append throws an IOException(Not supported); S3FileSystem should be able to support appending, possibly via common block storage logic. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
