Github user squito commented on the issue:
https://github.com/apache/spark/pull/17297
@sitalkedia This change is pretty contentious, there are lot of questions
about whether or not this is a good change. I don't think discussing this here
in github comments on a PR is the best form. I think of PR comments as being
more about code details -- clarity, tests, whether the implementation is
correct, etc. But here we're discussing whether the behavior is even
desirable, as well as trying to discuss this in relation to other changes. I
think a better format would be for you to open a jira and submit a design
document (maybe a shared google doc at first), where we can focus more on the
desired behavior and consider all the changes, even if the PRs are smaller to
make them easier to review.
I'm explicitly *not* making a judgement on whether or not this is a good
change. Also I do appreciate you having the code changes ready, as a POC, as
that can help folks consider the complexity of the change. But it seems clear
to me that first we need to come to a decision about the end goal.
Also, assuming we do decide this is desirable behavior, there is also a
question about how we can get changes like this in without risking breaking
things -- I have started a thread on dev@ related to that topic in general, but
we should figure that for these changes in particular as well.
@kayousterhout @tgravescs @markhamstra makes sense?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
