> On Mar 10, 2025, at 10:14 PM, Andrew Melo <andrew.m...@gmail.com> wrote: >> >> This config was released to "Apache" Spark 3.5.4, so this is NO LONGER just >> a problem with vendor distribution. The breakage will happen even if someone >> does not even know about Databricks Runtime at all and keeps using Apache >> Spark. It just happens when users in Apache Spark 3.5.4 directly upgrade to >> Apache Spark 4.0.0+ (rather than upgrading once to Apache Spark 3.5.5+). >> (Again, from the vendor codebase, it is not a problem at all to have such a >> config and there is a much easier way to deal with it - migration logic is >> not needed at all.) > > This is code that will be carried for sometime, and regardless of what > you responded to, I asked about what *users* would be affected. I > think it's fine to let your response speak for itself.
Andrew, your question about what users are affected is answered directly by Jungtaek in the part you quoted just above. To restate: It affects open source users of Apache Spark who try to upgrade directly from 3.5.4 to 4.0.0 without first upgrading to 3.5.5.
