Github user kayousterhout commented on the pull request:
https://github.com/apache/spark/pull/3029#issuecomment-62458897
This is definitely a cool/useful feature! A few things:
(1) As Patrick alluded to, I think it could and should be able to
implemented as a SparkListener, which is definitely preferable to adding, as
@aarondav wisely said, "a bunch of random code in the middle of
TaskSetManager". The scheduling code is already quite complex and we should
try to avoid adding unnecessary complexity there.
(2) I tried this out and and it doesn't seem to work properly when a job
has multiple concurrent stages, because of the way lines get overwritten (only
one of the stages gets shown).
(3) This bar doesn't show up if any other messages get logged to the
console (e.g., if info logging is turned on). Is there a way to make this so
that it's always the bottom thing shown in the console, after other messages?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
