Haiyang Sun created SPARK-59888:
-----------------------------------
Summary: shade grpc library used in the udf module
Key: SPARK-59888
URL: https://issues.apache.org/jira/browse/SPARK-59888
Project: Spark
Issue Type: Sub-task
Components: UDF
Affects Versions: 4.4.0
Reporter: Haiyang Sun
The UDF worker gRPC module currently uses unshaded dependencies. Its POM
explicitly defers shading until the transport is integrated into Spark’s
runtime classpath.
Including this module in Spark distributions could introduce dependency
conflicts involving gRPC, protobuf, and Netty. Reflective construction and
feature flags do not isolate packaged dependencies.
This is preventive compatibility work, not a report of a reproduced regression.
1. Audit the transport’s dependencies and their interaction with existing Spark
and Spark Connect packaging.
2. Define and implement a consistent shading boundary covering the client,
generated service stubs, and required dependencies. Ensure protobuf types
remain compatible across the core/client boundary.
3. Update Maven/SBT and distribution packaging, including published dependency
metadata. Preserve service-provider metadata and native transport loading.
4. Add regression coverage using the packaged artifacts, not only
development/test classpaths.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
