[
https://issues.apache.org/jira/browse/FLINK-8628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16365216#comment-16365216
]
Aljoscha Krettek commented on FLINK-8628:
-----------------------------------------
Could you please also comment on what is/was in your jar and in the lib folder
as well? If they are setup correctly this should also work without the fixes of
1.4.1.
P.S. 1.4.1 should be out today or tomorrow.
> BucketingSink does not work with S3
> -----------------------------------
>
> Key: FLINK-8628
> URL: https://issues.apache.org/jira/browse/FLINK-8628
> Project: Flink
> Issue Type: Bug
> Components: FileSystem, Streaming
> Affects Versions: 1.4.0
> Reporter: dejan miljkovic
> Priority: Blocker
> Fix For: 1.5.0
>
>
> BucketingSink does not work wit S3. Followed instructions provided on
> [https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/deployment/aws.html]
> but got below exception. Several people are complaining on the same issue.
> [http://mail-archives.apache.org/mod_mbox/flink-user/201801.mbox/%3CCADAFrT9T6WQa25HXR1z1NaL=n8wP9s7aSXxZWxHy=hubo93...@mail.gmail.com%3E]
> [https://lists.apache.org/thread.html/%3CCADAFrT9T6WQa25HXR1z1NaL=n8wP9s7aSXxZWxHy=hubo93...@mail.gmail.com%3E]
> [http://mail-archives.apache.org/mod_mbox/flink-user/201801.mbox/%3CCADAFrT-i+vGe64e__=-dnu4pmpxhvyzvkfqzrhgxbeyhnwa...@mail.gmail.com%3E]
> I don't see any specific bug related to this.
>
> java.lang.RuntimeException: Error while creating FileSystem when initializing
> the state of the BucketingSink.
> at
> org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink.initializeState(BucketingSink.java:358)
> at
> org.apache.flink.streaming.util.functions.StreamingFunctionUtils.tryRestoreFunction(StreamingFunctionUtils.java:178)
> at
> org.apache.flink.streaming.util.functions.StreamingFunctionUtils.restoreFunctionState(StreamingFunctionUtils.java:160)
> at
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.initializeState(AbstractUdfStreamOperator.java:96)
> at
> org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:259)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.initializeOperators(StreamTask.java:694)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.initializeState(StreamTask.java:682)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:253)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Cannot instantiate file system for URI:
> hdfs://localhost:12345/
> at
> org.apache.flink.runtime.fs.hdfs.HadoopFsFactory.create(HadoopFsFactory.java:187)
> at
> org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:401)
> at
> org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink.createHadoopFileSystem(BucketingSink.java:1154)
> at
> org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink.initFileSystem(BucketingSink.java:411)
> at
> org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink.initializeState(BucketingSink.java:355)
> ... 9 more
> Caused by: java.lang.ClassCastException:
> org.apache.hadoop.ipc.ProtobufRpcEngine cannot be cast to
> org.apache.hadoop.ipc.RpcEngine
> at org.apache.hadoop.ipc.RPC.getProtocolEngine(RPC.java:211)
> at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:583)
> at
> org.apache.hadoop.hdfs.NameNodeProxiesClient.createNonHAProxyWithClientProtocol(NameNodeProxiesClient.java:343)
> at
> org.apache.hadoop.hdfs.NameNodeProxiesClient.createProxyWithClientProtocol(NameNodeProxiesClient.java:131)
> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:343)
> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:287)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:156)
> at
> org.apache.flink.runtime.fs.hdfs.HadoopFsFactory.create(HadoopFsFactory.java:159)
> ... 13 more
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
