This is an automated email from the ASF dual-hosted git repository.
xushiyan pushed a change to branch release-0.11.0
in repository https://gitbox.apache.org/repos/asf/hudi.git
from 6fccca6c04 [HUDI-3935] Adding config to fallback to enabled Partition
Values extraction from Partition path (#5377)
new f8ce1c53ec [MINOR] Increase azure CI timeout to 120m (#5384)
new 304c5bf20e [HUDI-3940] Fix retry count increment in lock manager
(#5387)
new 0fb2d5bae1 [HUDI-3921] Fixed schema evolution cannot work with
HUDI-3855 (#5376)
new 5bd7a677b2 [HUDI-3936] Fix projection for a nested field as
pre-combined key (#5379)
new 25501c99e9 [HUDI-3934] Fix `Spark32HoodieParquetFileFormat` not being
compatible w/ Spark 3.2.0 (#5378)
The 5 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
azure-pipelines.yml | 10 +-
.../hudi/client/transaction/lock/LockManager.java | 3 +-
.../java/org/apache/hudi/io/HoodieWriteHandle.java | 12 +-
.../table/action/commit/HoodieMergeHelper.java | 14 +-
.../scala/org/apache/hudi/HoodieSparkUtils.scala | 8 +-
.../java/org/apache/hudi/avro/HoodieAvroUtils.java | 102 ++++++++++---
.../table/log/AbstractHoodieLogRecordReader.java | 3 +-
.../schema/action/InternalSchemaMerger.java | 26 +++-
.../internal/schema/utils/InternalSchemaUtils.java | 16 +++
.../org/apache/hudi/avro/TestHoodieAvroUtils.java | 7 +
.../schema/utils/TestAvroSchemaEvolutionUtils.java | 4 +-
.../org/apache/hudi/BaseFileOnlyRelation.scala | 6 +-
.../scala/org/apache/hudi/HoodieBaseRelation.scala | 19 ++-
.../hudi/MergeOnReadIncrementalRelation.scala | 2 +-
.../apache/hudi/MergeOnReadSnapshotRelation.scala | 2 +-
.../hudi/functional/TestMORDataSourceStorage.scala | 28 ++--
.../functional/TestParquetColumnProjection.scala | 4 +-
.../org/apache/spark/sql/hudi/TestSpark3DDL.scala | 31 ++--
.../apache/spark/sql/adapter/Spark3_1Adapter.scala | 4 +-
....scala => Spark31HoodieParquetFileFormat.scala} | 31 ++--
.../parquet/Spark32DataSourceUtils.scala | 77 ++++++++++
.../parquet/Spark32HoodieParquetFileFormat.scala | 157 +++++++++++++++++----
22 files changed, 439 insertions(+), 127 deletions(-)
rename
hudi-spark-datasource/hudi-spark3.1.x/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/{Spark312HoodieParquetFileFormat.scala
=> Spark31HoodieParquetFileFormat.scala} (95%)
create mode 100644
hudi-spark-datasource/hudi-spark3/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/Spark32DataSourceUtils.scala
