Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/19954
> I also don't fully understand all the abstractions being created here. It
seems there are three separate concepts (a "bootstrap", a "configuration step",
and an "orchestrator") and they're not used consistently.
A configuration step is a logical unit that applies an additive
transformation to the input. A steps orchestrator selects which configuration
steps to apply based on the configuration of the application. A bootstrap is a
component that can be shared by one or more configuration steps and the driver,
since a lot of times the submission client and the executor will share code for
configuring the driver and the executor pods, respectively. We discuss this a
bit more in here:
https://github.com/apache-spark-on-k8s/spark/blob/branch-2.2-kubernetes/resource-managers/kubernetes/architecture-docs/submission-client.md
- but we don't cover the bootstrap abstraction. We're open to different
representations and architectures as well.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
