Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21537
I know and agree it ended up with many regressions and bugs (strictly I
meant those by behaviour changes, which was my mistake). I have been following
that and I am glad that it has been reverted by another way. But I am pretty
sure AnalysisBarrier motivated another way to solve it and we found introducing
AnalysisBarrier might be not a good idea after getting that in, in practice.
We should of course have the sufficient discussion ahead but I mean it
doesn't not necessarily is a disaster. We just tried one way, found problem /
better alternative and fixed that by the original try. It shouldn't be
frequent but I think it would've needed more time or never been fixed without
the original try.
This PR is relatively less critical comparing to that, and I usually merge
PRs if that looks less risky or similar cases are merged. It has been open more
than one month and that's why I checked this one over a week. Ideally, it's
good if we have sufficient discussion for every change but in practice we
can't. I think we need more active committers or reviewers, rather then just
leaving those PR open.
Also, we should consider revert https://github.com/apache/spark/pull/21193
too if this one is considered to be reverted. I wonder why this one was
specially being considered to be reverted, and you say that in this PR.
@viirya and @mgaido91 have been taking looks into these areas as far as I
can tell, and anyone could lead if they are willing to do that I believe. The
works and efforts were mostly done by @viirya and in this case it looks
reasonably it makes sense to ask him. It just looked to me weird because it
sounds to me who's an expert and better.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
