This is an automated email from the ASF dual-hosted git repository.
yangjie01 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new ca274fe15cf [SPARK-44607][SQL] Remove unused function
`containsNestedColumn` from `Filter`
ca274fe15cf is described below
commit ca274fe15cf9f621394759339b6c8d826692d1a1
Author: yangjie01 <[email protected]>
AuthorDate: Wed Aug 2 11:17:59 2023 +0800
[SPARK-44607][SQL] Remove unused function `containsNestedColumn` from
`Filter`
### What changes were proposed in this pull request?
This pr aims remove `private[sql] `function `containsNestedColumn` from
`org.apache.spark.sql.sources.Filter`.
This function was introduced by https://github.com/apache/spark/pull/27728
to avoid nested predicate pushdown for Orc.
After https://github.com/apache/spark/pull/28761, Orc also support nested
column predicate pushdown, so this function become unused.
### Why are the changes needed?
Remove unused `private[sql] ` function `containsNestedColumn`.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Pass GitHub Actions
Closes #42239 from LuciferYang/SPARK-44607.
Authored-by: yangjie01 <[email protected]>
Signed-off-by: yangjie01 <[email protected]>
---
.../src/main/scala/org/apache/spark/sql/sources/filters.scala | 7 -------
1 file changed, 7 deletions(-)
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/sources/filters.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/sources/filters.scala
index af5e4f5ef5a..a52bca10660 100644
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/sources/filters.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/sources/filters.scala
@@ -64,13 +64,6 @@ sealed abstract class Filter {
this.references.map(parseColumnPath(_).toArray)
}
- /**
- * If any of the references of this filter contains nested column
- */
- private[sql] def containsNestedColumn: Boolean = {
- this.v2references.exists(_.length > 1)
- }
-
/**
* Converts V1 filter to V2 filter
*/
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
