[
https://issues.apache.org/jira/browse/SPARK-57584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-57584:
-----------------------------------
Labels: pull-request-available (was: )
> Add standard gRPC health checking support to Spark Connect server
> -----------------------------------------------------------------
>
> Key: SPARK-57584
> URL: https://issues.apache.org/jira/browse/SPARK-57584
> Project: Spark
> Issue Type: Improvement
> Components: Connect
> Affects Versions: 4.1.2, kubernetes-operator-0.9.0
> Reporter: Shekhar Prasad Rajak
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.2.0, kubernetes-operator-1.0.0
>
>
> Spark Connect currently starts a Netty gRPC server and registers
> SparkConnectService, plus optional ProtoReflectionService in debug mode. It
> does not appear to register the standard grpc.health.v1.Health service.
>
> SparkConnectService.startGRPCService() registers Spark Connect service:
> sql/connect/server/.../SparkConnectService.scala
> Kubernetes support exposes the Spark Connect port with appProtocol: grpc, but
> that does not imply standard gRPC health checking is available.
>
> Motivation
> Spark Connect is often deployed as a long-running service on Kubernetes.
> Operators and SDKs need a reliable readiness signal before routing clients or
> returning a SparkSession.
> TCP readiness only proves the port is open. It does not prove that the
> Spark Connect gRPC service is ready to accept requests. Standard gRPC health
> checking would provide a portable
> readiness contract for:
> - Kubernetes grpc readiness/startup probes
> - Spark Kubernetes Operator implementations
> - Kubeflow Spark Operator SparkConnect readiness
> - SDKs and gateways that wait for Spark Connect before connecting clients
> - service meshes and load balancers that understand gRPC health
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
