[
https://issues.apache.org/jira/browse/HUDI-4466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shawn Chang updated HUDI-4466:
------------------------------
Description:
During Spark 3.3 upgrade. Many classes are copied from hudi-spark3 to
hudi-spark3.3.x without any modification.
We should reuse those code to reduce code duplication.
Classes copied:
Spark33HoodieVectorizedParquetRecordReader
Spark3DefaultSource
hudi-spark-datasource/hudi-spark3.3.x/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/TimeTravelRelation.scala
hudi-spark-datasource/hudi-spark3.3.x/src/main/scala/org/apache/spark/sql/connector/catalog/HoodieIdentifier.scala
Spark33DataSourceUtils
Spark33HoodieParquetFileFormat
HoodieSpark3_3AvroDeserializer
HoodieSpark3_3AvroSerializer
hudi-spark-datasource/hudi-spark3.3.x/src/main/scala/org/apache/spark/sql/hudi/analysis/HoodieSpark3Analysis.scala
'Classes under
hudi-spark-datasource/hudi-spark3.3.x/src/main/scala/org/apache/spark/sql/hudi/catalog'
hudi-spark-datasource/hudi-spark3.3.x/src/main/scala/org/apache/spark/sql/hudi/command/AlterTableCommand.scala
HoodieSpark33CatalystExpressionUtils
HoodieSpark33CatalystPlanUtils
was:
During Spark 3.3 upgrade. Many classes are copied from hudi-spark3 to
hudi-spark3.3.x without any modification.
We should reuse those code to reduce code duplication.
Classes copied:
Spark33HoodieVectorizedParquetRecordReader
Spark3DefaultSource
hudi-spark-datasource/hudi-spark3.3.x/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/TimeTravelRelation.scala
hudi-spark-datasource/hudi-spark3.3.x/src/main/scala/org/apache/spark/sql/connector/catalog/HoodieIdentifier.scala
Spark33DataSourceUtils
Spark33HoodieParquetFileFormat
hudi-spark-datasource/hudi-spark3.3.x/src/main/scala/org/apache/spark/sql/hudi/analysis/HoodieSpark3Analysis.scala
'Classes under
hudi-spark-datasource/hudi-spark3.3.x/src/main/scala/org/apache/spark/sql/hudi/catalog'
hudi-spark-datasource/hudi-spark3.3.x/src/main/scala/org/apache/spark/sql/hudi/command/AlterTableCommand.scala
HoodieSpark33CatalystExpressionUtils
HoodieSpark33CatalystPlanUtils
> Reuse common classes between Spark 3.2/3.3
> ------------------------------------------
>
> Key: HUDI-4466
> URL: https://issues.apache.org/jira/browse/HUDI-4466
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: Shawn Chang
> Priority: Major
>
> During Spark 3.3 upgrade. Many classes are copied from hudi-spark3 to
> hudi-spark3.3.x without any modification.
> We should reuse those code to reduce code duplication.
>
> Classes copied:
> Spark33HoodieVectorizedParquetRecordReader
> Spark3DefaultSource
> hudi-spark-datasource/hudi-spark3.3.x/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/TimeTravelRelation.scala
> hudi-spark-datasource/hudi-spark3.3.x/src/main/scala/org/apache/spark/sql/connector/catalog/HoodieIdentifier.scala
> Spark33DataSourceUtils
> Spark33HoodieParquetFileFormat
> HoodieSpark3_3AvroDeserializer
> HoodieSpark3_3AvroSerializer
> hudi-spark-datasource/hudi-spark3.3.x/src/main/scala/org/apache/spark/sql/hudi/analysis/HoodieSpark3Analysis.scala
> 'Classes under
> hudi-spark-datasource/hudi-spark3.3.x/src/main/scala/org/apache/spark/sql/hudi/catalog'
> hudi-spark-datasource/hudi-spark3.3.x/src/main/scala/org/apache/spark/sql/hudi/command/AlterTableCommand.scala
> HoodieSpark33CatalystExpressionUtils
> HoodieSpark33CatalystPlanUtils
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
