bjornjorgensen commented on PR #43844: URL: https://github.com/apache/spark/pull/43844#issuecomment-3468731350
Hi @Dzeri96 It's a valid point, and I'd like to provide some context on why the fix for the vulnerability was applied to Spark 4.0.0 and not backported to the 3.x releases. The decision is rooted in the Apache Spark community's release and maintenance policies, as well as the practical considerations of managing a large and active open-source project. Apache Spark's Maintenance Policy: The Apache Spark project generally maintains feature release branches with bug fixes for a period of 18 months. While critical security fixes are a high priority, the process for backporting them to older release lines involves careful consideration by the community. It is not always guaranteed that dependency updates, even for security reasons, will be backported to all supported versions. The focus is often on ensuring the stability of maintenance releases. Severity of the Vulnerability: The vulnerability in question, CVE-2023-44981, is indeed a critical issue in Apache ZooKeeper, with a CVSS score of 9.1. It allows for an authorization bypass if SASL Quorum Peer authentication is enabled. However, this is not a default configuration, (to my knowledge) which may have factored into the urgency and backporting decision within the Spark community. The recommended fix is to upgrade to ZooKeeper versions 3.9.1, 3.8.3, or 3.7.2. Focus on Future Releases: The Apache Spark community, like many open-source projects, prioritizes the stability and security of the upcoming major and minor releases. The upgrade to ZooKeeper 3.9.1 was merged into the master branch for the upcoming Spark 4.0.0 release, ensuring that the latest version of Spark would not be affected. API Compatibility and the Precautionary Principle: While the upgrade from ZooKeeper 3.6.x to 3.9.x is expected to be API-compatible for client libraries, any dependency change in a maintenance release of a project as large as Spark carries a risk of introducing unexpected regressions or breaking changes for some users' environments. The Spark community tends to be conservative about making non-essential changes to older, stable release branches to avoid any potential disruption. Community Resources: Backporting and thoroughly testing a dependency upgrade across all supported 3.x versions of Spark requires significant effort from the community contributors and committers. These resources are often focused on developing and stabilizing the next major version. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
