yalindogusahin commented on code in PR #794:
URL:
https://github.com/apache/spark-kubernetes-operator/pull/794#discussion_r4015306148
##########
spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkClusterResourceSpec.java:
##########
@@ -467,35 +469,67 @@ private static Optional<PodDisruptionBudget>
buildPodDisruptionBudget(
* resource and does not carry the cluster label, yet the driver must reach
the executors' block
* manager to fetch task results larger than {@code
spark.task.maxDirectResultSize}.
*
+ * <p>If both {@link WorkerNetworkPolicySpec#getMetricsPort()} and {@link
+ * WorkerNetworkPolicySpec#getMetricsIngress()} are set, a separate rule
admits the listed peers
+ * on that port only. Both are required: the port alone would open the
endpoint to every source in
+ * the cluster, and the peers alone would grant them every port. The worker
web UI port is
+ * deliberately not usable as the metrics port: it is served by the same
embedded HTTP server as
+ * the UI itself, so admitting it would also expose the full UI, not just a
metrics endpoint. A
+ * dedicated metrics port (e.g. a JMX-to-Prometheus exporter agent) keeps
metrics scraping
+ * separate from the UI.
+ *
* @param clusterName The name of the SparkCluster.
* @param namespace The namespace of the SparkApplication.
+ * @param workerSpec The WorkerSpec, used to look up the optional metrics
port and its peers.
* @return A NetworkPolicy object.
+ * @throws IllegalArgumentException if a metrics ingress rule is requested
on the worker web UI
+ * port.
*/
- private NetworkPolicy buildWorkerNetworkPolicy(String clusterName, String
namespace) {
- return new NetworkPolicyBuilder()
- .withNewMetadata()
- .withName(clusterName + "-worker")
- .withNamespace(namespace)
- .addToLabels(LABEL_SPARK_CLUSTER_NAME, clusterName)
- .endMetadata()
- .withNewSpec()
- .withNewPodSelector()
- .addToMatchLabels(LABEL_SPARK_ROLE_NAME, LABEL_SPARK_ROLE_WORKER_VALUE)
- .addToMatchLabels(LABEL_SPARK_CLUSTER_NAME, clusterName)
- .endPodSelector()
- .addNewIngress()
- .addNewFrom()
- .withNewPodSelector()
- .addToMatchLabels(LABEL_SPARK_CLUSTER_NAME, clusterName)
- .endPodSelector()
- .endFrom()
- .addNewFrom()
- .withNewPodSelector()
- .addToMatchLabels(LABEL_SPARK_ROLE_NAME, LABEL_SPARK_ROLE_DRIVER_VALUE)
- .endPodSelector()
- .endFrom()
- .endIngress()
- .endSpec()
- .build();
+ private NetworkPolicy buildWorkerNetworkPolicy(
+ String clusterName, String namespace, WorkerSpec workerSpec) {
+ var builder =
Review Comment:
Removed the reassignment and added an early return when the policy is absent
or the peer list is empty.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
