Hi wombatu-kun, Thanks for raising this question. The EPIC of enhancing Spark integration in Hudi 1.0, HUDI-7917, contains the work to remove Hudi support of all Spark versions below Spark 3.3. To remove the support Spark 2.x support, we need first to migrate the integration tests to run on Spark 3.5 instead of Spark 2.4 (see HUDI-7919). The integration tests cover other engine integrations including Hive, Presto, Trino and we need to fix the setup too. So HUDI-7919 is a blocker.
Thanks, - Ethan On Fri, Sep 13, 2024 at 6:15 AM wombatu-kun o_0 <wombatu...@gmail.com> wrote: > Hi all, > > Support for Spark 3.0, 3.1, 3.1 , 3.2 is already removed in Hudi master. > Dropping support for Java 8 - is planned. > Why do Hudi still support Spark 2.4? What do you think about dropping this > support too in the nearest future? >
