What is the problem with the existence of the migration logic? I understand not keeping the misnamed config. But the migration logic does no harm other than taking up a couple lines in the code, no? Unless someone offers any reason this is an issue... what are we even talking about.
Is the idea that the existence of the string 'databricks' in the code is a problem? it is not. A simple test: Let us say we introduced a config called "spark.tubro.enabled" and fixed it to "spark.turbo.enabled". Would we be having the same conversation about retaining migration logic for way more than like 1 version? if not and the vendor name is the issue then I think that is the misunderstanding of... well I don't know what, still? On Fri, Mar 7, 2025 at 2:34 AM Jungtaek Lim <kabhwan.opensou...@gmail.com> wrote: > Sean, it's about how long we keep the vendor name in the codebase (since > the migration logic has the problematic config name as a string) for users. > > We agreed in a previous discussion thread that "this is generally bad", so > we fixed the incorrect config name immediately in 3.5. Now it's just a > string somewhere in the codebase, so I really don't get why this has to be > immediately removed in 4.0.0, having a risk of breaking users' queries. > > It'd be awesome if you can confirm you understand the issue and still be > supportive to retain the migration logic as it is. Thanks! > >>
