Hi Aljoscha, Big +1 for the fat flink distribution, where do you plan to put these connectors ? opt or lib ?
Aljoscha Krettek <aljos...@apache.org> 于2020年4月15日周三 下午3:30写道： > Hi Everyone, > > I'd like to discuss about releasing a more full-featured Flink > distribution. The motivation is that there is friction for SQL/Table API > users that want to use Table connectors which are not there in the > current Flink Distribution. For these users the workflow is currently > roughly: > > - download Flink dist > - configure csv/Kafka/json connectors per configuration > - run SQL client or program > - decrypt error message and research the solution > - download additional connector jars > - program works correctly > > I realize that this can be made to work but if every SQL user has this > as their first experience that doesn't seem good to me. > > My proposal is to provide two versions of the Flink Distribution in the > future: "fat" and "slim" (names to be discussed): > > - slim would be even trimmer than todays distribution > - fat would contain a lot of convenience connectors (yet to be > determined which one) > > And yes, I realize that there are already more dimensions of Flink > releases (Scala version and Java version). > > For background, our current Flink dist has these in the opt directory: > > - flink-azure-fs-hadoop-1.10.0.jar > - flink-cep-scala_2.12-1.10.0.jar > - flink-cep_2.12-1.10.0.jar > - flink-gelly-scala_2.12-1.10.0.jar > - flink-gelly_2.12-1.10.0.jar > - flink-metrics-datadog-1.10.0.jar > - flink-metrics-graphite-1.10.0.jar > - flink-metrics-influxdb-1.10.0.jar > - flink-metrics-prometheus-1.10.0.jar > - flink-metrics-slf4j-1.10.0.jar > - flink-metrics-statsd-1.10.0.jar > - flink-oss-fs-hadoop-1.10.0.jar > - flink-python_2.12-1.10.0.jar > - flink-queryable-state-runtime_2.12-1.10.0.jar > - flink-s3-fs-hadoop-1.10.0.jar > - flink-s3-fs-presto-1.10.0.jar > - flink-shaded-netty-tcnative-dynamic-2.0.25.Final-9.0.jar > - flink-sql-client_2.12-1.10.0.jar > - flink-state-processor-api_2.12-1.10.0.jar > - flink-swift-fs-hadoop-1.10.0.jar > > Current Flink dist is 267M. If we removed everything from opt we would > go down to 126M. I would reccomend this, because the large majority of > the files in opt are probably unused. > > What do you think? > > Best, > Aljoscha > > -- Best Regards Jeff Zhang
