I remember that ./dev/make-distribution.sh in spark source allows people to specify Hadoop version.
> 2022年4月6日 下午4:31，Amin Borjian <borjianami...@outlook.com> 写道： > > From Spark version 3.1.0 onwards, the clients provided for Spark are built > with Hadoop 3 and placed in maven Repository. Unfortunately we use Hadoop > 2.7.7 in our infrastructure currently. > > 1) Does Spark have a plan to publish the Spark client dependencies for Hadoop > 2.x? > 2) Are the new Spark clients capable of connecting to the Hadoop 2.x cluster? > (According to a simple test, Spark client 3.2.1 had no problem with the > Hadoop 2.7 cluster but we wanted to know if there was any guarantee from > Spark?) > > Thank you very much in advance > Amin Borjian
