aljoscha opened a new pull request #14122:
URL: https://github.com/apache/flink/pull/14122
[FLINK-20098] Don't add flink-connector-files to flink-dist, make
dependencies explicit
We currently add both flink-connector-files and flink-connector-base to
flink-dist.
This implies, that users should use the dependency like this:
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-files</artifactId>
<version>${project.version}</version>
<scope>provided</scope>
</dependency>
which differs from other connectors where users don't need to specify
<scope>provided</scope>.
Also, flink-connector-files had flink-connector-base as a provided
dependency, which means that examples that use this dependency would not
run out-of-box in IntelliJ because transitive provided dependencies will
not be considered.
This removes the dependencies from flink-dist which lets users use the
File Connector like any other connector.
I believe the initial motivation for "providing" the File Connector in
flink-dist was to allow us to use the File Connector under the hood in
methods such as StreamExecutionEnvironment.readFile(...). We could
decide to deprecate and remove those methods or re-add the File
Connector as an explicit (non-provided) dependency again in the future.
## Brief change log
- remove dependencies from `flink-dist`
- change `provided` dependencies to instead be in `compile` scope
- add File Connector modules to Table API uber jars because they are using
them and were relying on them being present in `flink-dist`. This increases the
size of the Table uber jars by a couple of kilo bytes but it should be
negligible.
## Verifying this change
Covered by existing tests.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): maybe
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`may: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: no
- The S3 file system connector: no
## Documentation
Not applicable
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
