Github user kayousterhout commented on the pull request:
https://github.com/apache/spark/pull/3009#issuecomment-62508540
This sounds great. I do think it would be good to get this job page change
in ASAP, since it's a major, user-visible change and it would be good to
have it in master so folks can play with it for a few weeks before the
release is cut (assuming you're trying to get this in 1.2?). This is
already definitely pushing it, I think, in terms of what should be merged
in late in the release cycle. This is all to say that, at this point, I
think it might be better to get this in as-is (without the progress bar)
and add the progress bar later.
On Mon, Nov 10, 2014 at 7:42 PM, Josh Rosen <[email protected]>
wrote:
> One approach, which I'm exploring now, is to show a combined progress bar
> that just tracks "total number of tasks" summed across all stages involved
> in that job. To prevent weird "progress bar travels back in time" issues,
> I'm planning to extend JobUIData to track these task counts. I'll add a
> stage attempt -> job id mapping to track which job is associated with the
> current stage and use this to figure out which counters need to be updated
> when tasks finish.
>
> â
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/spark/pull/3009#issuecomment-62497811>.
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
