SaurabhChawla100 commented on pull request #31974:
URL: https://github.com/apache/spark/pull/31974#issuecomment-808404665
> so one downside to this and its true even for the driver, is this is the
executors page. yarn-am and driver aren't really executor sand it gets added to
the top level page and can be confusing to say there are 3 executors now when
only one. Yes we already have that issue with driver but this makes it worse.
is this showing cores and memory that you requested the AM to have as well? I'm
not sure I want to make this worse by adding an extra yarn-am container here,
especially if the only thing useful is the log link.
Right now I am not adding the cores and memory for yarn-am. Adding only the
hostname , Log-links for the yarn-am. I can try adding that also.
Since this extra yarn-am will have only one entry in the executor page and
that is also for only one scenario when deploy mode is client in yarn, for
other Resource Manger this entry will not be there.So thats the reason for
using the existing way for adding the yarn-am in the executors table. Also name
of the column executor id shows yarn-am which is fixed separates it from the
executors, if we try adding it to some other table like Miscellaneous
Container Info, we need to store it in some other variable and there is need to
make one more api call to fetch and display it in to the table.
We can also add the yarn-am log link at the top of executors page after show
Additional Metrics in that page for Client mode having info related to Yarn AM
info and its log link.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
