pan3793 commented on PR #57141: URL: https://github.com/apache/spark/pull/57141#issuecomment-4921038335
Requires Hadoop 3.4.0+ makes sense according to the [Hadoop supported-versions](https://github.com/apache/hadoop/blob/trunk/SECURITY.md#supported-versions) declaration. > Older release lines are end-of-life and do not receive security updates; the remedy for a vulnerability in an old line is to upgrade. Refer to the [Apache Hadoop release and download policy](https://hadoop.apache.org/releases.html) for which lines are currently maintained. Moving to the Hadoop shaded client largely simplified our dependency management of Hadoop. For official distribution building without `-Phadoop-cloud`, I think it's fine to just change the Hadoop version in the build command via `-Phadoop.version=3.4.0`, but I think users might need to tune the `pom.xml` when they want to build with both `-Phadoop-cloud` and `-Phadoop.version=x.x.x`, not sure if this is worth a document section. One side issue, we currently also release `spark-<x.y.z>-bin-without-hadoop.tgz`; is anyone using it? I think it has not been well maintained and is likely broken. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
