TQJADE commented on code in PR #749: URL: https://github.com/apache/spark-kubernetes-operator/pull/749#discussion_r3562238269
########## docs/config_properties.md: ########## @@ -35,12 +35,14 @@ | spark.kubernetes.operator.metrics.sanitizePrometheusMetricsNameEnabled | Boolean | true | false | Whether or not to enable automatic name sanitizing for all metrics based on best-practice guide from Prometheus https://prometheus.io/docs/practices/naming/ | | spark.kubernetes.operator.name | String | spark-kubernetes-operator | false | Name of the operator. | | spark.kubernetes.operator.namespace | String | default | false | Namespace that operator is deployed within. | - | spark.kubernetes.operator.periodicGC.intervalSeconds | Long | 1800 | false | Interval (in seconds) between periodic System.gc() invocations. Set to 0 or a negative value to disable. Note that System.gc() is a no-op if JVM is started with -XX:+DisableExplicitGC. | + | spark.kubernetes.operator.periodicGC.intervalSeconds | Long | 1800 | true | Interval (in seconds) between periodic System.gc() invocations. Set to 0 or a negative value to disable. Note that System.gc() is a no-op if JVM is started with -XX:+DisableExplicitGC. | | spark.kubernetes.operator.reconciler.appStatusListenerClassNames | String | | false | Comma-separated names of SparkAppStatusListener class implementations | | spark.kubernetes.operator.reconciler.clusterStatusListenerClassNames | String | | false | Comma-separated names of SparkClusterStatusListener class implementations | | spark.kubernetes.operator.reconciler.foregroundRequestTimeoutSeconds | Long | 60 | true | Timeout (in seconds) for requests made to API server. This applies only to foreground requests. | | spark.kubernetes.operator.reconciler.intervalSeconds | Long | 120 | true | Interval (in seconds, non-negative) to reconcile Spark applications. Note that reconciliation is always expected to be triggered when app spec / status is updated. This interval controls the reconcile behavior of operator reconciliation even when there's no update on SparkApplication, e.g. to determine whether a hanging app needs to be proactively terminated. Thus this is recommended to set to above 2 minutes to avoid unnecessary no-op reconciliation. | | spark.kubernetes.operator.reconciler.labelSelector | String | | false | A label selector that identifies a set of Spark custom resources for the operator to reconcile. If not set or set to empty string, operator would reconcile all Spark Apps and Clusters. | + | spark.kubernetes.operator.reconciler.missingDriverGracePeriodSeconds | Integer | 60 | true | Grace period (in seconds) the operator waits for the pod informer cache to catch up after the driver pod is observed missing, before doing a live API verification against the Kubernetes apiserver. This protects healthy applications from being failed when the informer cache is briefly stale (e.g., after a watch reconnect). Set to 0 to verify on every cache miss. | + | spark.kubernetes.operator.reconciler.missingDriverRequeueIntervalSeconds | Integer | 15 | true | Requeue interval (in seconds) used to re-check a driver pod that is missing from the informer cache, while waiting for the cache to catch up during the missing-driver grace period. Specific to the missing-driver flow; this is not the operator's default requeue cadence. The steady-state periodic reconcile interval is spark.kubernetes.operator.reconciler.intervalSeconds. | Review Comment: updated. ########## spark-operator/src/main/java/org/apache/spark/k8s/operator/config/SparkOperatorConf.java: ########## @@ -220,6 +199,47 @@ public final class SparkOperatorConf { .defaultValue(120L) .build(); + /** + * Grace period (in seconds) the operator waits for the pod informer cache to catch up + * after the driver pod is observed missing, before doing a live API verification. Brief + * cache lag during pod creation/update event propagation is normal; only sustained absence + * triggers an apiserver lookup. Set to 0 to verify on every cache miss. + */ + public static final ConfigOption<Integer> MISSING_DRIVER_GRACE_PERIOD_SECONDS = + ConfigOption.<Integer>builder() + .key("spark.kubernetes.operator.reconciler.missingDriverGracePeriodSeconds") + .enableDynamicOverride(true) + .description( + "Grace period (in seconds) the operator waits for the pod informer cache to " + + "catch up after the driver pod is observed missing, before doing a live " + + "API verification against the Kubernetes apiserver. This protects healthy " + + "applications from being failed when the informer cache is briefly stale " + + "(e.g., after a watch reconnect). Set to 0 to verify on every cache miss.") + .typeParameterClass(Integer.class) + .defaultValue(60) + .build(); + + /** + * Requeue interval (in seconds) used to re-check a driver pod that is missing from the informer + * cache, while waiting for the cache to catch up during the missing-driver grace period. This is + * specific to the missing-driver flow and is not the operator's default requeue cadence; the + * steady-state periodic reconcile interval is {@link #RECONCILER_INTERVAL_SECONDS}. + */ + public static final ConfigOption<Integer> MISSING_DRIVER_REQUEUE_INTERVAL_SECONDS = + ConfigOption.<Integer>builder() + .key("spark.kubernetes.operator.reconciler.missingDriverRequeueIntervalSeconds") + .enableDynamicOverride(true) + .description( + "Requeue interval (in seconds) used to re-check a driver pod that is missing from " + + "the informer cache, while waiting for the cache to catch up during the " + + "missing-driver grace period. Specific to the missing-driver flow; this is " + + "not the operator's default requeue cadence. The steady-state periodic " + + "reconcile interval is spark.kubernetes.operator.reconciler.intervalSeconds.") + .typeParameterClass(Integer.class) + .defaultValue(15) + .build(); + + Review Comment: removed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
