[
https://issues.apache.org/jira/browse/FLINK-14955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984291#comment-16984291
]
Kostas Kloudas commented on FLINK-14955:
----------------------------------------
This is related to https://issues.apache.org/jira/browse/FLINK-14170
> Not able to write to swift via StreamingFileSink.forBulkFormat
> --------------------------------------------------------------
>
> Key: FLINK-14955
> URL: https://issues.apache.org/jira/browse/FLINK-14955
> Project: Flink
> Issue Type: Bug
> Components: Connectors / FileSystem
> Affects Versions: 1.8.1, 1.9.1
> Reporter: Simya Jose
> Priority: Major
> Attachments: pom.xml
>
>
> not able to use StreamingFileSink to write to swift file storage
>
> *Code*:
> flink version: 1.9.1. ( tried with 1.8.1 as well, same exception)
> scala 2.11
> build tool : maven
> main part of the code:
> val eligibleItems: DataStream[EligibleItem] = env.fromCollection(Seq(
> EligibleItem("pencil"),
> EligibleItem("rubber"),
> EligibleItem("beer")))(TypeInformation.of(classOf[EligibleItem]))
> val factory2: ParquetWriterFactory[EligibleItem] =
> ParquetAvroWriters.forReflectRecord(classOf[EligibleItem])
> val sink: StreamingFileSink[EligibleItem] = StreamingFileSink
> .forBulkFormat(new Path(capHadoopPath),factory2)
> .build()
> eligibleItems.addSink(sink)
> .setParallelism(1)
> .uid("TEST_1")
> .name("TEST")
> *scenario* : when path is set to point to swift ( capHadoopPath =
> "swift://<path>" ) , getting exception -
> _java.lang.UnsupportedOperationException: Recoverable writers on Hadoop are
> only supported for HDFS and for Hadoop version 2.7 or
> newerjava.lang.UnsupportedOperationException: Recoverable writers on Hadoop
> are only supported for HDFS and for Hadoop version 2.7 or newer at
> org.apache.flink.fs.openstackhadoop.shaded.org.apache.flink.runtime.fs.hdfs.HadoopRecoverableWriter.<init>(HadoopRecoverableWriter.java:57)_
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
