[
https://issues.apache.org/jira/browse/SPARK-59435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-59435:
-----------------------------------
Labels: pull-request-available (was: )
> Qualify UDF worker protobuf import paths
> ----------------------------------------
>
> Key: SPARK-59435
> URL: https://issues.apache.org/jira/browse/SPARK-59435
> Project: Spark
> Issue Type: Sub-task
> Components: UDF
> Affects Versions: 4.4.0
> Reporter: Haiyang Sun
> Priority: Major
> Labels: pull-request-available
>
> The UDF worker protobuf definitions currently use bare imports such as
> `common.proto` and `udf_message.proto`. These filenames are generic and do not
> identify the UDF worker protocol namespace.
> This works while the protobuf definitions are compiled in isolation, but it
> can
> become ambiguous when Spark protobuf sources are collected, indexed, or
> compiled alongside other protocol definitions containing similarly named
> files.
> The import paths should therefore be stable and globally meaningful.
> Proposed approach:
> Move the UDF worker protobuf definitions under a namespaced directory beneath
> the protobuf source root and update intra-proto imports to use fully qualified
> paths:
> - `udf/worker/proto/common.proto`
> - `udf/worker/proto/udf_message.proto`
> - `udf/worker/proto/udf_service.proto`
> - `udf/worker/proto/worker_spec.proto`
> For example, replace `import "common.proto";` with
> `import "udf/worker/proto/common.proto";`.
> This preserves the protocol schema, generated Java package, and wire format.
> It does change protobuf descriptor filenames, so the implementation should
> verify that no Spark code depends on the old descriptor names.
> Testing:
> - Compile the UDF worker protobuf module.
> - Compile the UDF worker gRPC module, which consumes the service definition.
> - Confirm generated sources and existing UDF worker tests continue to pass.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
