Chesnay Schepler created FLINK-20804:
----------------------------------------
Summary: Add SwiftFS end-to-end test
Key: FLINK-20804
URL: https://issues.apache.org/jira/browse/FLINK-20804
Project: Flink
Issue Type: Improvement
Components: FileSystems, Tests
Reporter: Chesnay Schepler
Our swift filesystem is closely aligned with the S3 filesystems in that it
follows the same packaging scheme as flink-fs-hadoop-shaded (used by the S3
filesystems), and thus ideally is kept in sync with these filesystems, for
example in regards to dependencies.
As it stands, this is not the case. While the S3 modules work against Hadoop 3,
swift is still sitting on 2.8.1 . Presumable this difference is due to us
having no infrastructure for actually testing our swift filesystem.
Unfortunately it appears that Switft is rather annoying to setup, but I was
able find a [project|https://github.com/swiftstack/docker-swift] that tries to
provide a docker image containing everything you need. Maybe we could use this
to bootstrap some end-to-end-test, allowing us to finally touch this module
again without fear of completely breaking it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
