Overall I'm +1, but could be convinced otherwise. Spark 2.4 is old and doesn't really function properly because the Spark Catalog API was missing at the time. And people can still use older versions of Iceberg that support Spark 2.4 if they need it because the Iceberg spec guarantees forward compatibility.
That said, I'd love to hear from more people on this. I think it would be great to drop support, but I don't know how many people still use it. Is upgrading Hadoop a good reason to drop support for an engine? Hadoop seems like a minor concern to me unless it is blocking something. Ryan On Thu, Apr 13, 2023 at 12:54 PM Jack Ye <yezhao...@gmail.com> wrote: > +1 for dropping 2.4 support > > Best, > Jack Ye > > On Thu, Apr 13, 2023 at 10:59 AM Fokko Driesprong <fo...@apache.org> > wrote: > >> Hi all, >> >> I'm working on moving to Hadoop 3.x >> <https://github.com/apache/iceberg/pull/7114>, and one thing is that it >> seems to be incompatible with Spark 2.4. I wanted to ask if people are >> still on Spark 2.4 and what we think of dropping the support. The last >> release of Spark 2.4.8 was on 2021-05-17 and it also looks like the 2.4 >> branch on the Spark Github repository is stale, so I don't expect any >> further releases. >> >> Before creating a PR I would like to check on the mail-list if anyone has >> any objections. If so, please let us know. >> >> Thanks, >> Fokko Driesprong >> > -- Ryan Blue Tabular
