Github user mgummelt commented on the issue:
https://github.com/apache/spark/pull/14414
> a. Its not convenient to have only one option to add the history server
uri. I had to create a properties file to pass it to the dispatcher process. I
would expect to pass it with --conf
I agree, and we should change this at some point, but that's not the goal
of this PR.
> when you have one driver starting and stopping contexts then history link
can only point to the very first context. I guess that is a limitation.
Yea, it's a limitation. The dispatcher knows about drivers, but the
history server knows about apps (i.e. contexts), and this relationship is one
to many. This is a simple way to connect the two in the common case of 1
driver having 1 context. To solve the 1-to-many case, we'd need some way of
the driver communicating back to the dispatcher for every context that it
creates. That's a more difficult problem, and I don't think it's worth solving
right now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
