Hi Kyuubi community, I raise this thread to discuss the support versions for Spark and Flink engines.
Currently, Kyuubi supports - Spark 3.2 (deprecated since 1.10, only keep runtime support of engine) - Spark 3.3 (fully supported) - Spark 3.4 (fully supported) - Spark 3.5 (fully supported, default version) - Spark 4.0 (fully supported except for authZ and lineage plugin) - Flink 1.17 (fully supported) - Flink 1.18 (fully supported) - Flink 1.19 (fully supported) - Flink 1.20 (fully supported, default version) - Flink 2.0 (unsupported yet) - Flink 2.1 (unsupported yet) I propose to remove support for Spark 3.2, and deprecate support for Flink 1.17 and 1.18. What does deprecation mean? - Stop supporting for building but still keep engine runtime support by cross version verification. - For Spark engine, stop supporting for connectors, sql extensions, authZ and lineage plugins. WDYT? Thanks, Cheng Pan
