This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 7d0f0832ae1a [SPARK-47373][SQL] Match FileSourceScanLike to get
metadata instead of FileSourceScanExec
7d0f0832ae1a is described below
commit 7d0f0832ae1a222bd9c2492587b37fc1939a51e5
Author: zwangsheng <binjiey...@apache.org>
AuthorDate: Wed Mar 13 00:28:26 2024 -0700
[SPARK-47373][SQL] Match FileSourceScanLike to get metadata instead of
FileSourceScanExec
### What changes were proposed in this pull request?
When get Spark Plan info, we should match basic trait `FileSourceScanLike`
to get metadata instead of matching
subclass `FileSourceScanExec`.
So that user-define file scan operators(which extend `FileSourceScanLike`)
can be matched.
### Why are the changes needed?
Match user-define file scan operators.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Exists Unit Test
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #45491 from zwangsheng/SPARK-47373.
Authored-by: zwangsheng <binjiey...@apache.org>
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
.../src/main/scala/org/apache/spark/sql/execution/SparkPlanInfo.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlanInfo.scala
b/sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlanInfo.scala
index 9b699801c97a..7c45b02ee846 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlanInfo.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlanInfo.scala
@@ -66,7 +66,7 @@ private[execution] object SparkPlanInfo {
// dump the file scan metadata (e.g file path) to event log
val metadata = plan match {
- case fileScan: FileSourceScanExec => fileScan.metadata
+ case fileScan: FileSourceScanLike => fileScan.metadata
case _ => Map[String, String]()
}
new SparkPlanInfo(
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
