[
https://issues.apache.org/jira/browse/BEAM-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16140412#comment-16140412
]
Steve Loughran commented on BEAM-2790:
--------------------------------------
...looks like it's only in HDFS. More specifically, it looks like one of those
things the HDFS team added without actually updating the FS spec to say what
it's meant to do, hence there's no cross-fs compliance tests. Pity.
> Error while reading from Amazon S3 via Hadoop File System
> ---------------------------------------------------------
>
> Key: BEAM-2790
> URL: https://issues.apache.org/jira/browse/BEAM-2790
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-extensions
> Affects Versions: 2.0.0, 2.1.0
> Reporter: Ismaël Mejía
> Assignee: Ismaël Mejía
>
> If you try to use hadoop-aws with Beam to read from AWS S3 it breaks because
> S3AInputStream (the implementation of Hadoop's FSDataInputStream) is not
> ByteBufferReadable.
> {code}
> Exception in thread "main" java.lang.UnsupportedOperationException:
> Byte-buffer read unsupported by input stream
> at
> org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:146)
> at
> org.apache.beam.sdk.io.hdfs.HadoopFileSystem$HadoopSeekableByteChannel.read(HadoopFileSystem.java:192)
> at
> org.apache.beam.sdk.io.TextSource$TextBasedReader.tryToEnsureNumberOfBytesInBuffer(TextSource.java:232)
> at
> org.apache.beam.sdk.io.TextSource$TextBasedReader.findSeparatorBounds(TextSource.java:166)
> at
> org.apache.beam.sdk.io.TextSource$TextBasedReader.readNextRecord(TextSource.java:198)
> at
> org.apache.beam.sdk.io.FileBasedSource$FileBasedReader.advanceImpl(FileBasedSource.java:481)
> at
> org.apache.beam.sdk.io.FileBasedSource$FileBasedReader.startImpl(FileBasedSource.java:476)
> at
> org.apache.beam.sdk.io.OffsetBasedSource$OffsetBasedReader.start(OffsetBasedSource.java:261)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
