Github user markhamstra commented on the issue:
https://github.com/apache/spark/pull/15213
@scwf I understand that you were trying to make the least invasive fix
possible to deal with the problem. That's usually a good thing to do, but even
when that kind of fix is getting to the root of the problem it can still result
in layers of patches that are hard to make sense of. That's not really the
fault of any one patch; rather, the blame lies more with those of us who often
didn't produce clear, maintainable code in the first place. When it's possible
to see re-organizing principles that will make the code clearer, reduce
duplication, make future maintenance less error prone, etc., then it's usually
a good idea to do a little larger refactoring instead of just a minimally
invasive fix.
I think this is a small example of where that kind of refactoring makes
sense, so that's why I made my code suggestion. If you can see ways to make
things even clearer, then feel free to suggest them. I'm sure that Kay, Imran
and others who also have been trying to make these kinds of clarifying changes
in the DAGScheduler will also chime in if they have further suggestions.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
