Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21427
2.3.1 wouldn't have this behaviour change and we marked this as
experimental. So, on the other hand, it probably will give more time to expose
that this is discouraged in production and there might be a bit of behaviour
changes. Actually, It isn't long time comparing to other APIs we have as well
on the other hand ...
> it turns runnable code into failure, and the old behavior is kind of
self-consistent(by-position match). it's not like turning failures into
runnable or fix a correctness bug.
It still sounds like we treat this API as a old stable API. It doesn't
replace the self-consistent way completely. This PR partially fixes its
behaviour to make it more sense, causing some corner behaviour changes which
are quite unlikely and making no sense (IMHO).
We should be relatively less conservative for new and experimental APIs to
promote to make it more stable and coherent as soon as possible until we remove
the experimental note ..
The only special reason I see is that it's not a correctness bug but it
changes the existing behaviour (which I actually don't completely agree but I
get what you mean at least). But then what can we do for experimental APIs
specifically .. ?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
