Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19468
@skonto, there was some discussion about this on the SPIP. We see them as
separate and independent issues with the pluggable API being a long term goal.
It would involve a working group of people drawn from all cluster manager
integration maintainers - because the changes are extensive and require
scalability/performance testing in a variety of environments - probably over
2-3 releases before we can iron out the kinks.
In the short term however, adding the package here enables supporting the
growing number of K8s users of Spark that currently rely on our fork, and the
integration testing makes us confident about not impacting the release process
or adding complexity to the maintainers workflow. The K8s community will set up
all the requisite testing, and ensure health of this package. The integration
testing would also carry forward to when we have pluggable APIs in the future.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
