Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21060
> The callback works for collect in R and Scala but Python doesn't. I think
we should at least match the behaviour. I wonder why it's hard to say a bug
when collect is detected in some APIs but not in some APIs.
The behavior consistency among Python/Scala/R/JAVA does not mean a bug,
right?
> That's because the change was big and invasive. I wouldn't backport it
too; however, this fix is relatively small.
This is the reason why we did not backport that PR. We still can backport
the minimal changes to the previous releases.
> I think we usually use committer's judgement when we make an exception. I
already have been seeing many backports that actually causes behaviour changes
and I did this because it looks being backported in general. This is the reason
why we should formally document it if this is actually the rule.
I am not against this specific PR. All the committers need to be really
careful when they make a decision to backport a behavior change. If any
committer does it, we should jump in and stop the backport. This is what we
should do.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
