[
https://issues.apache.org/jira/browse/SPARK-53593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza resolved SPARK-53593.
--------------------------------
Fix Version/s: 4.1.0
Resolution: Fixed
Issue resolved by pull request 52328
[https://github.com/apache/spark/pull/52328]
> [SDP] Return qualified dataset name for DefineDataset and DefineFlow RPCs
> -------------------------------------------------------------------------
>
> Key: SPARK-53593
> URL: https://issues.apache.org/jira/browse/SPARK-53593
> Project: Spark
> Issue Type: Sub-task
> Components: Declarative Pipelines
> Affects Versions: 4.1.0
> Reporter: Yuheng Chang
> Assignee: Jessie Luo
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.1.0
>
>
> Currently, when the SC client defining a dataset or flow, the SC client use
> partially or maybe fully qualified name in DefineFlow / DefineDataset RPC. We
> want the server response back the fully qualified dataset / flow name to the
> client and this info is needed for LDP SC frontend graph resolution algorithm.
>
> Jobs needs to be done:
> 1. Add the response field in the DefineDataset / DefineFlow proto in
> {{spark}} repo.
> 2. For SDP, update the RPC handler to response back the fully qualified name.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
