abhishekd0907 commented on pull request #35683:
URL: https://github.com/apache/spark/pull/35683#issuecomment-1084095300
> To clarify, I can see a need for supporting node decomissioning in yarn -
for both on-prem clusters and for cloud deployments - but the solution will
need to take into account what the cluster manager supports. As I detailed
above, yarn supports shuffle services and executors come and go on different
nodes.
>
> Decomissioning will need to account for it, which is a different model
compared to k8s support.
@mridulm I get your point. That's why I wanted to ask what are the
expectations from this PR for changes in External Shuffle Service. I noticed
**Part 8 (Optional): When there is a node-local external shuffle service** of
the design DOC talks to some extent about this and can work according to it.
But Do I need to include changes pertaining to External Shuffle Service in this
PR itself or mark this PR as a sub-task of the SPARK-30835 feature and pursue
ShuffleService side changes in a separate PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
