Github user markhamstra commented on the issue:
https://github.com/apache/spark/pull/21598
> so we can't just change the default value in a feature release
Agreed. Once a particular interface and behavior is in our released public
API, then we effectively have a contract not to change that behavior. If we are
going to provide another behavior before making a new major-number release
(e.g. spark-3.0.0), then we have to provide a user configuration option to
select that new behavior, and the default behavior if a user doesn't change
configuration must be the same as before the optional new behavior.
If there is a clear, severe bug (such as data loss or corruption), only
then we can consider changing the public API before making a new major-number
release -- but even then we are likely to either go immediately to a new
major-number or to at least preserve the old, buggy behavior with a
configuration option.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
