SCHJonathan commented on code in PR #52685:
URL: https://github.com/apache/spark/pull/52685#discussion_r2452881375
##########
sql/connect/common/src/main/protobuf/spark/connect/pipelines.proto:
##########
@@ -280,3 +280,9 @@ message SourceCodeLocation {
message PipelineQueryFunctionExecutionSignal {
repeated string flow_names = 1;
}
+
+// Provides context for pipeline analysis during Spark Connect query execution.
+message PipelineAnalysisContext {
+ // Defines the path assigned at runtime when the pipeline initializes
+ optional string definition_path = 1;
Review Comment:
@sryza @cookiedough77 what do yall think about the following:
In SDP, `PipelineAnalysisContext` contains dataflow graph ID and a
`extension` field
In Runtime, we add `definition_path` to `PipelineAnalysisContext` as an
extension. We also add `definition_path` to `SourceCodeLocation`'s `extension`
and in the proto comment, mentioned these two are correlated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
