attilapiros commented on a change in pull request #34924:
URL: https://github.com/apache/spark/pull/34924#discussion_r793930121
##########
File path:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala
##########
@@ -381,4 +382,32 @@ object KubernetesUtils extends Logging {
}
}
}
+
+ @Since("3.3.0")
+ def loadFeatureStep(conf: KubernetesConf, className: String):
KubernetesFeatureConfigStep = {
+ val constructors = Utils.classForName(className).getConstructors
Review comment:
I was thinking about how we could avoid reflections here and my idea is
the following:
1) Extending `KubernetesFeatureConfigStep` with a new method `def
init(config: KubernetesConf) {}`
2) Introducing two new traits
(`KubernetesDriverFeatureConfigStep`/`KubernetesExecutorFeatureConfigStep`)
where the `init` accepts `KubernetesDriverConf` and `KubernetesExecutorConf`
respectively. The `init` in the derived classes this way is not overriding the
init of `KubernetesFeatureConfigStep`. So we cannot create a common
`loadFeatureStep` where `KubernetesConf` is passed but it can be a function
with generic parameter.
And finally we can extend the doc of `KUBERNETES_DRIVER_POD_FEATURE_STEPS`
and `KUBERNETES_EXECUTOR_POD_FEATURE_STEPS` to mention those two specific
traits.
This way the custom feature step's API will contain a clue that configs can
be used to initialize.
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
