Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22392#discussion_r218178934
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala ---
@@ -54,6 +54,10 @@ class Checkpoint(ssc: StreamingContext, val
checkpointTime: Time)
"spark.driver.bindAddress",
"spark.driver.port",
"spark.master",
+ "spark.kubernetes.driver.pod.name",
+ "spark.kubernetes.driver.limit.cores",
+ "spark.kubernetes.executor.limit.cores",
--- End diff --
I'm fine deferring consideration for the other properties to a later time.
But given that the the core limit ones are included, it makes no sense to
exclude the core request ones. If the purpose is to make it work first for 2.4,
I would suggest just keeping the ones that are not deterministic, e.g., driver
pod name and executor pod name prefix.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
