+1 for discontinue Hadoop 2 in the next release Jiwon Park <[email protected]> 于2026年9月20日周日 19:30写道：
> Hi all, > > Following Jinsong's suggestion on PR #4378, I would like to gather feedback > on the scope of Hadoop 2 support in Amoro. > > https://github.com/apache/amoro/pull/4378 > > The PR upgrades AMS and optimizers to Iceberg 1.10.2 while keeping > mixed-format connectors on Iceberg 1.8.1 to preserve existing engine > support, including Spark 3.3 and Flink 1.18. > > The new Parquet dependency requires Hadoop APIs unavailable in Hadoop 2, so > AMS and optimizers need a compatible Hadoop 3 runtime. The question is > whether we should also discontinue Hadoop 2 support for mixed-format > connectors. > > Amoro's hadoop2 Maven profile uses Hadoop 2.10.2. Although Hadoop 2.10 has > not been officially declared EOL, its last release was in May 2022, over > four years ago. Given this gap and the cost of maintaining separate build > and test configurations, I think it is worth considering retiring Hadoop 2 > support in Amoro. > > Either way, we need to retire Hadoop 2-based AMS distributions and update > optimizer environments, including the Hadoop 2.8.3-based bundle in the > Flink optimizer Docker image. > > For the connectors, I see two options: > > 1. Retain Hadoop 2 for mixed-format connectors only, maintaining both > Hadoop 2 and Hadoop 3 connector build and test configurations. > 2. Discontinue Hadoop 2 across Amoro in the next release, simplifying > maintenance but requiring affected connector users to migrate. > > This is separate from the Iceberg version split: neither option would drop > existing Spark or Flink versions, though their Hadoop 3 configurations > would need validation. > > Does anyone still require Hadoop 2 for mixed-format connectors? Which > option would the community prefer? > > I can update the implementation and migration documentation based on the > feedback. > > Thanks, > Jiwon >
