Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/19720
It's not about running slower. This PR solves the problem which makes the
user facing an exception if there are a lot of arguments in `coalesce` (or
`AtLestNNonNulls`), but what I am doing in the `coalesce` function here is that
I am adding to variables for each coalesce function. If there is a query with a
lot of coalesce function (instead of a coalesce with a lot of parameters), this
might result in having much more variables than before. This can cause the
problem and the exception described in SPARK-18016. Thus a query that was
previously running can fail.
The same thing is true for all the other PRs similar to this one submitted
by @kiszk. Then, we should keep all these changes only on master, where part of
SPARK-18016 is landing and hopefully soon it will be completely solved.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
