voonhous commented on code in PR #19458:
URL: https://github.com/apache/hudi/pull/19458#discussion_r3699406437
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/BaseHoodieCatalystPlanUtils.scala:
##########
@@ -22,24 +22,17 @@ import org.apache.hudi.SparkAdapterSupport
import org.apache.spark.sql.catalyst.TableIdentifier
import org.apache.spark.sql.catalyst.analysis.{ResolvedTable,
TableOutputResolver}
import org.apache.spark.sql.catalyst.catalog.CatalogStorageFormat
-import org.apache.spark.sql.catalyst.expressions.{Attribute, AttributeSet,
Expression, ProjectionOverSchema}
+import org.apache.spark.sql.catalyst.expressions.{Attribute, Expression}
import org.apache.spark.sql.catalyst.plans.JoinType
import org.apache.spark.sql.catalyst.plans.logical.{CreateIndex, DropIndex,
HoodieShowIndexes, InsertIntoStatement, Join, JoinHint, LogicalPlan,
RefreshIndex}
import org.apache.spark.sql.connector.catalog.{Identifier, Table, TableCatalog}
import org.apache.spark.sql.execution.{ExtendedMode, SimpleMode}
import org.apache.spark.sql.execution.command.{CreateTableLikeCommand,
ExplainCommand, RepairTableCommand}
import org.apache.spark.sql.execution.datasources.LogicalRelation
Review Comment:
Same nonexistent-class premise as the thread above, checked and it does not
hold:
- GitHub code search over `apache/hudi` returns exactly 2 hits for
`projectOverSchema`: the declaration in this trait and the call in
`HoodieNestedSchemaPruning.scala`, both deleted together by this PR.
- No `HoodieSparkXXCatalystPlanUtils` declares an `override def
projectOverSchema` on `master` (`git grep projectOverSchema` across all
`hudi-spark-datasource` per-version modules: 0 hits outside the two above). The
method was introduced by HUDI-4899 (#6755) purely to absorb the Spark 3.2.2
`ProjectionOverSchema` constructor change; the per-version need disappeared
with Spark 3.2 support.
- `BaseHoodieNestedSchemaPruning` (the claimed caller) does not exist in the
repo, see the reply on the `HoodieBaseRelation` thread.
- The branch compiles under `-Dspark3.3`, `-Dspark3.5` and `-Dspark4.2` with
the method removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
