[jira] [Commented] (BEAM-2500) Add support for S3 as a Apache Beam FileSystem

[ https://issues.apache.org/jira/browse/BEAM-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16171891#comment-16171891 ] Steve Loughran commented on BEAM-2500: -- looking at the code, the main thing I'd highli

[jira] [Commented] (BEAM-2500) Add support for S3 as a Apache Beam FileSystem

[ https://issues.apache.org/jira/browse/BEAM-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167614#comment-16167614 ] Steve Loughran commented on BEAM-2500: -- bq. . So we'll have to have a way to stream by

[jira] [Commented] (BEAM-2500) Add support for S3 as a Apache Beam FileSystem

[ https://issues.apache.org/jira/browse/BEAM-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167608#comment-16167608 ] Steve Loughran commented on BEAM-2500: -- This is how Hadoop does its multipart upload

[jira] [Commented] (BEAM-2790) Error while reading from Amazon S3 via Hadoop File System

[ https://issues.apache.org/jira/browse/BEAM-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16152722#comment-16152722 ] Steve Loughran commented on BEAM-2790: -- Looking at Hadoop's FileSystem there is meant

[jira] [Commented] (BEAM-2790) Error while reading from Amazon S3 via Hadoop File System

[ https://issues.apache.org/jira/browse/BEAM-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16140412#comment-16140412 ] Steve Loughran commented on BEAM-2790: -- ...looks like it's only in HDFS. More specific

[jira] [Commented] (BEAM-2790) Error while reading from Amazon S3 via Hadoop File System

[ https://issues.apache.org/jira/browse/BEAM-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16139909#comment-16139909 ] Steve Loughran commented on BEAM-2790: -- OF course, a patch for HADOOP-14603 would make

[jira] [Commented] (BEAM-2500) Add support for S3 as a Apache Beam FileSystem

[ https://issues.apache.org/jira/browse/BEAM-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16123094#comment-16123094 ] Steve Loughran commented on BEAM-2500: -- +also, as well as any fixes to S3A, it'd be sw

[jira] [Commented] (BEAM-2500) Add support for S3 as a Apache Beam FileSystem

[ https://issues.apache.org/jira/browse/BEAM-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16123091#comment-16123091 ] Steve Loughran commented on BEAM-2500: -- * Do a patch for that &the ByteBuffer & I'll l

[jira] [Commented] (BEAM-2500) Add support for S3 as a Apache Beam FileSystem

[ https://issues.apache.org/jira/browse/BEAM-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16120232#comment-16120232 ] Steve Loughran commented on BEAM-2500: -- Thanks for the update ":" is a troublespot ch

[jira] [Commented] (BEAM-2572) Implement an S3 filesystem for Python SDK

[ https://issues.apache.org/jira/browse/BEAM-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16087071#comment-16087071 ] Steve Loughran commented on BEAM-2572: -- Worth mentioning a couple of recent changes in

[jira] [Commented] (BEAM-2572) Implement an S3 filesystem for Python SDK

[ https://issues.apache.org/jira/browse/BEAM-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085648#comment-16085648 ] Steve Loughran commented on BEAM-2572: -- bear in mind that credentials are highly sensi

[jira] [Commented] (BEAM-2500) Add support for S3 as a Apache Beam FileSystem

[ https://issues.apache.org/jira/browse/BEAM-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16073967#comment-16073967 ] Steve Loughran commented on BEAM-2500: -- It's not clear that the S3 clients from EMR or