This is an automated email from the ASF dual-hosted git repository.
vhs pushed a change to branch minor-add-configurable-hfile-config
in repository https://gitbox.apache.org/repos/asf/hudi.git
discard 634d72e29d66 Ensure proper wiring in write and read paths
discard cb7123487dff Fix scalastyle issue
omit 1efd26a14938 Fix config usage typo
omit ddf7f9a9c6a6 Minor add configurable hfile config
add 4cefaa226d0f chore: Update PR template and compliance check after
deprecating JIRA (#13883)
add 2efb90c0a7e9 feat(merger): [RFC-101] Remove shouldFlush and outer
merge from record merger interface (#13904)
add 888263fad601 chore: update issue templates post JIRA migration (#13907)
add be2814511c1f chore: Change issue template to use `value` instead of
`placeholder` (#13913)
add 9031935b3bf8 chore: change label type for community support issue
(#13920)
add 042a0c3b8ec0 [MINOR] Reducing few noisy logs (#13895)
add b4cf65e20c67 [HUDI-9731] Add index definition for Record level Index
(#13741)
add a9c101125948 fix: adding back PRECOMBINE field Config property in
DataSourceReadOptions (#13900)
add ff00d212c404 feat(merger): [RFC-101] Update HoodieRecordMerger
interface and implementations (#13742)
add 6916c309cef9 [HUDI-9666] Fix the record key encoding with a single
record key field and add a guard for complex key generator (#13650)
add 36619c8a2ae2 [HUDI-9788] Fix the incremental query when records from
multi commits are from the same file (#13840)
add e3e8b7347940 [HUDI-4985] Update ARM64 Docker demo images to use latest
Hudi version (#13902)
add 7a7ad4b35333 fix: [HUDI-9611] Metadata writer tries to initialize
invalid partitions (#13583)
add f64cd6655f73 perf(spark): Write handle perf fix for prepending meta
fields (#13860)
add c50f5604e8b8 fix(storage-format)!: Fix HFile writer to properly encode
checksum for compatibility (#13897)
add 1e0513d1585a [MINOR] Incremental and Time Travel Query Testing (#13934)
add c2b564dbbe42 [HUDI-9778] Remove HoodieRecord wrapper from MDT reads
(#13796)
add 50a75f27ba69 fix(ingest): HudiIncrSource - Handle nullable partition
filter (#13945)
add 99e04da76c7a perf: Update the hfile prefetch size threshold to 50
(#13946)
add 76bb580ae149 [HUDI-7915] Spark 4 support (#12772)
add 0e30bd69d82b feat(merger): Payload deprecation bug fixes (#13947)
add 2817ccf8a31f test: fix flink flaky test
TestStreamReadMonitoringFunction#testCheckpointRestoreWithLimit (#13936)
add 66d2780d8983 [HUDI-9777] Avoid redundant file length fetch in
HoodieNativeHFileReader (#13797)
add b3b4658709d2 fix: issue when reading a Hudi table with nested
partition fields (#13943)
add 24f1be9a7070 [HUDI-9776] Reduce redundant config lookups and minor
FSView optimizations (#13803)
add 19961e77d539 feat(merger): Make HoodieRecord delete handling
consistent with merger code path (#13951)
add bf50dd438fe3 [HUDI-9158] Rename heartbeat poll secs to lock renewal
secs (#13968)
add 9a5467afe9d8 perf: Avoid repeated work in HFile reads and writes
(#13962)
add fb4dc84faa47 refactor: Move FlinkRecordContext to hudi-flink-client
module (#13958)
add d10310f4d9f3 fix: fixing hivesync tool in docker (#13957)
add be6f08d56643 feat(metadata): Improve Logical Type Handling on Col
Stats (#13711)
add 0685e8bbebee [HUDI-9757] Support TLS authentication for Datahub sync
(#13861)
add 6377edb97a6e fix: Fixing schema for Merge Into writes for COW (#13967)
add ff452bb1dd23 Minor add configurable hfile config
add 9244d706b5a6 Fix config usage typo
add ae8c9b304f2b Fix scalastyle issue
add 9b3a60b5b74f Ensure proper wiring in write and read paths
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (634d72e29d66)
\
N -- N -- N refs/heads/minor-add-configurable-hfile-config
(9b3a60b5b74f)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/ISSUE_TEMPLATE/SUPPORT_REQUEST.md | 56 --
.github/ISSUE_TEMPLATE/config.yml | 14 +-
.github/ISSUE_TEMPLATE/hudi_bug.yml | 58 ++
.github/ISSUE_TEMPLATE/hudi_community_support.yml | 88 +++
.github/ISSUE_TEMPLATE/hudi_feature.yml | 61 ++
.github/ISSUE_TEMPLATE/hudi_improvement.yml | 59 ++
.github/PULL_REQUEST_TEMPLATE.md | 31 +-
.github/workflows/bot.yml | 95 ++-
.github/workflows/pr_compliance.yml | 9 +-
.github/workflows/pr_title_validation.yml | 68 +++
.github/workflows/update_pr_compliance.yml | 9 +-
azure-pipelines-20230430.yml | 2 +
...r-compose_hadoop334_hive313_spark353_arm64.yml} | 194 +++---
docker/hoodie/hadoop/base/Dockerfile | 4 +-
docker/hoodie/hadoop/datanode/Dockerfile | 2 +-
docker/hoodie/hadoop/historyserver/Dockerfile | 17 +-
docker/hoodie/hadoop/hive_base/Dockerfile | 5 +-
docker/hoodie/hadoop/hive_base/conf/hive-env.sh | 2 +-
docker/hoodie/hadoop/hive_base/conf/hive-site.xml | 46 +-
docker/hoodie/hadoop/namenode/Dockerfile | 2 +-
docker/hoodie/hadoop/spark_base/Dockerfile | 29 +-
docker/hoodie/hadoop/sparkadhoc/Dockerfile | 10 +-
docker/hoodie/hadoop/sparkmaster/Dockerfile | 4 +-
docker/hoodie/hadoop/sparkworker/Dockerfile | 4 +-
docker/setup_demo.sh | 4 +-
docker/stop_demo.sh | 4 +-
hudi-aws/pom.xml | 4 +
.../org/apache/hudi/cli/commands/SparkMain.java | 6 +-
.../hudi/cli/utils/SparkTempViewProvider.java | 2 +-
.../cli/functional/CLIFunctionalTestHarness.java | 3 +-
.../apache/hudi/client/BaseHoodieWriteClient.java | 9 +-
.../transaction/lock/StorageBasedLockProvider.java | 2 +-
.../org/apache/hudi/config/HoodieWriteConfig.java | 40 ++
.../apache/hudi/config/StorageBasedLockConfig.java | 17 +-
.../org/apache/hudi/index/HoodieIndexUtils.java | 37 +-
.../apache/hudi/index/bloom/HoodieBloomIndex.java | 7 +-
.../java/org/apache/hudi/io/BaseCreateHandle.java | 2 +-
.../hudi/io/FileGroupReaderBasedMergeHandle.java | 10 -
.../org/apache/hudi/io/HoodieAppendHandle.java | 21 +-
.../org/apache/hudi/io/HoodieBinaryCopyHandle.java | 7 +-
.../hudi/io/HoodieMergeHandleWithChangeLog.java | 11 +-
.../apache/hudi/io/HoodieSortedMergeHandle.java | 6 +-
.../io/HoodieSortedMergeHandleWithChangeLog.java | 6 +-
.../java/org/apache/hudi/io/HoodieWriteHandle.java | 9 +-
.../org/apache/hudi/io/HoodieWriteMergeHandle.java | 84 +--
.../hudi/io/SecondaryIndexStreamingTracker.java | 11 +-
.../hudi/keygen/ComplexAvroKeyGenerator.java | 18 +-
.../java/org/apache/hudi/keygen/KeyGenUtils.java | 35 ++
.../metadata/HoodieBackedTableMetadataWriter.java | 19 +-
.../hudi/table/action/commit/BaseWriteHelper.java | 14 +-
.../table/action/commit/HoodieWriteHelper.java | 4 +-
.../table/upgrade/EightToNineUpgradeHandler.java | 39 +-
.../upgrade/EightToSevenDowngradeHandler.java | 6 +
.../table/upgrade/NineToEightDowngradeHandler.java | 23 +-
.../table/upgrade/SevenToEightUpgradeHandler.java | 7 +
.../hudi/table/upgrade/UpgradeDowngradeUtils.java | 19 +-
.../hudi/client/TestBaseHoodieWriteClient.java | 151 +++++
.../lock/StorageBasedLockProviderTestBase.java | 2 +-
.../lock/TestStorageBasedLockProvider.java | 12 +-
.../hudi/config/TestStorageBasedLockConfig.java | 14 +-
.../org/apache/hudi/io/TestHoodieWriteHandle.java | 2 +-
.../io/TestSecondaryIndexStreamingTracker.java | 25 +-
.../hudi/keygen/TestComplexAvroKeyGenerator.java | 62 +-
.../org/apache/hudi/keygen/TestKeyGenUtils.java | 115 ++++
.../upgrade/TestEightToNineUpgradeHandler.java | 110 ++--
.../upgrade/TestNineToEightDowngradeHandler.java | 27 +-
.../java/org/apache/hudi/testutils/Assertions.java | 9 +
.../hudi/testutils/HoodieMergeOnReadTestUtils.java | 4 +-
.../hudi/utils/HoodieWriterClientTestHarness.java | 3 +-
hudi-client/hudi-flink-client/pom.xml | 4 +
.../client/model/CommitTimeFlinkRecordMerger.java | 18 +-
.../client/model/EventTimeFlinkRecordMerger.java | 36 +-
.../hudi/client/model/HoodieFlinkRecord.java | 18 +-
.../model/PartialUpdateFlinkRecordMerger.java | 61 +-
.../FlinkIncrementalMergeHandleWithChangeLog.java | 9 +-
.../hudi/io/FlinkMergeHandleWithChangeLog.java | 8 +-
.../io/log/block/HoodieFlinkParquetDataBlock.java | 7 +-
.../apache/hudi/io/v2/RowDataLogWriteHandle.java | 14 +-
.../hudi/table/action/commit/FlinkWriteHelper.java | 4 +-
.../hudi/table/format/FlinkRecordContext.java | 4 +-
.../hudi/util/RecordKeyToRowDataConverter.java | 0
.../hudi/merge/TestHoodieFlinkRecordMerger.java | 78 +--
.../merge/TestPartialUpdateFlinkRecordMerger.java | 107 ++--
hudi-client/hudi-java-client/pom.xml | 4 +
.../hudi/table/action/commit/JavaWriteHelper.java | 4 +-
hudi-client/hudi-spark-client/pom.xml | 6 +
.../org/apache/hudi/DefaultSparkRecordMerger.java | 58 +-
.../org/apache/hudi/HoodieSparkRecordMerger.java | 44 --
.../hudi/OverwriteWithLatestSparkRecordMerger.java | 24 +-
.../HoodieSparkBootstrapSchemaProvider.java | 2 +-
.../MultipleSparkJobExecutionStrategy.java | 7 +-
.../client/common/SparkReaderContextFactory.java | 2 +-
.../hudi/client/model/HoodieInternalRow.java | 40 +-
.../client/utils/SparkInternalSchemaConverter.java | 11 +-
.../client/utils/SparkMetadataWriterUtils.java | 38 +-
.../hudi/client/utils/SparkPartitionUtils.java | 4 -
.../hudi/client/utils/SparkValidatorUtils.java | 2 +-
.../validator/SqlQueryPreCommitValidator.java | 2 +-
.../hudi/common/model/HoodieSparkRecord.java | 36 +-
.../bulkinsert/RDDBucketIndexPartitioner.java | 11 +-
.../RDDCustomColumnsSortPartitioner.java | 8 +-
.../apache/hudi/index/HoodieSparkIndexClient.java | 28 +-
.../hudi/index/SparkMetadataTableRecordIndex.java | 5 +-
.../hudi/io/storage/row/HoodieRowCreateHandle.java | 14 +-
.../storage/row/HoodieRowParquetWriteSupport.java | 14 +-
.../apache/hudi/keygen/BuiltinKeyGenerator.java | 40 +-
.../apache/hudi/keygen/ComplexKeyGenerator.java | 8 +-
.../hudi/keygen/GlobalDeleteKeyGenerator.java | 4 +-
.../apache/hudi/merge/SparkRecordMergingUtils.java | 42 +-
.../SparkHoodieBackedTableMetadataWriter.java | 2 +-
.../apache/hudi/stats/SparkValueMetadataUtils.java | 157 +++++
.../bootstrap/ParquetBootstrapMetadataHandler.java | 4 +-
.../commit/BaseSparkCommitActionExecutor.java | 6 +-
.../org/apache/hudi/AvroConversionUtils.scala | 15 +-
.../hudi/BaseSparkInternalRecordContext.java | 68 ++-
.../hudi/HoodieDatasetBulkInsertHelper.scala | 58 +-
.../hudi/HoodiePartitionCDCFileGroupMapping.scala | 9 +-
.../hudi/HoodiePartitionFileSliceMapping.scala | 10 +-
.../HoodiePartitionValues.scala} | 12 +-
.../scala/org/apache/hudi/HoodieSparkUtils.scala | 110 +++-
.../scala/org/apache/hudi/HoodieUTF8String.scala | 28 +-
.../org/apache/hudi/SparkAdapterSupport.scala | 4 +-
.../org/apache/hudi/SparkConversionUtils.scala | 4 +-
.../SparkFileFormatInternalRecordContext.scala | 13 +-
.../SparkFileFormatInternalRowReaderContext.scala | 2 +-
.../scala/org/apache/hudi/SparkJdbcUtils.scala | 64 --
.../main/scala/org/apache/hudi/SparkRowSerDe.scala | 6 +-
.../apache/hudi/spark/internal/ReflectUtil.java | 0
.../apache/spark/HoodieSparkKryoRegistrar.scala | 2 +-
.../apache/spark/sql/BucketPartitionUtils.scala | 5 +-
.../scala/org/apache/spark/sql/DataFrameUtil.scala | 13 +-
.../org/apache/spark/sql/HoodieSchemaUtils.scala | 18 +
.../apache/spark/sql/HoodieUTF8StringFactory.scala | 20 +-
.../org/apache/spark/sql/HoodieUnsafeUtils.scala | 51 +-
.../datasources/SparkParsePartitionUtil.scala | 265 +++++++-
.../org/apache/spark/sql/hudi/SparkAdapter.scala | 167 ++++--
.../spark/sql/hudi/execution/RangeSample.scala | 6 +-
.../org/apache/hudi/client/TestWriteStatus.java | 5 +-
.../common/TestSparkReaderContextFactory.java | 4 +-
.../functional/TestExternalPathHandling.java | 7 +-
.../functional/TestHoodieBackedTableMetadata.java | 43 +-
.../hudi/keygen/KeyGeneratorTestUtilities.java | 200 +++++++
.../hudi/keygen/TestComplexKeyGenerator.java | 226 +++++++
.../apache/hudi/keygen/TestCustomKeyGenerator.java | 165 +++--
.../keygen/TestGlobalDeleteRecordGenerator.java | 81 +--
.../keygen/TestNonpartitionedKeyGenerator.java | 96 +--
.../apache/hudi/keygen/TestSimpleKeyGenerator.java | 56 +-
.../keygen/TestTimestampBasedKeyGenerator.java | 129 ++--
.../TestCreateKeyGeneratorByTypeWithFactory.java | 24 +-
.../TestHoodieSparkKeyGeneratorFactory.java | 17 +-
.../commit/TestCopyOnWriteActionExecutor.java | 22 +-
.../hudi/testutils/DisableOnSpark4Condition.java | 20 +-
.../apache/hudi/testutils/DisabledOnSpark4.java | 18 +-
.../testutils/HoodieSparkClientTestHarness.java | 2 +-
.../testutils/HoodieSparkDeleteRecordMerger.java | 13 +-
.../SparkClientFunctionalTestHarness.java | 2 +-
hudi-common/src/main/avro/HoodieMetadata.avsc | 32 +-
.../main/java/org/apache/hudi/ParquetAdapter.java | 31 +-
.../org/apache/hudi/avro/AvroRecordContext.java | 4 +-
.../java/org/apache/hudi/avro/AvroSchemaUtils.java | 55 +-
.../apache/hudi/avro/ConvertingGenericData.java | 24 +-
.../apache/hudi/avro/HoodieAvroReaderContext.java | 63 +-
.../java/org/apache/hudi/avro/HoodieAvroUtils.java | 147 ++++-
.../apache/hudi/avro/HoodieAvroWrapperUtils.java | 154 ++++-
.../hudi/avro/HoodieBloomFilterWriteSupport.java | 8 +-
.../org/apache/hudi/avro/JoinedGenericRecord.java | 90 +++
.../org/apache/hudi/avro/processors/Parser.java | 10 +-
.../apache/hudi/common/config/ConfigProperty.java | 40 +-
.../hudi/common/config/HoodieIndexingConfig.java | 3 +-
.../hudi/common/config/HoodieMetadataConfig.java | 7 +-
.../hudi/common/engine/HoodieReaderContext.java | 2 +-
.../apache/hudi/common/engine/RecordContext.java | 19 +-
.../java/org/apache/hudi/common/fs/FSUtils.java | 27 +-
.../common/model/DefaultHoodieRecordPayload.java | 24 +-
.../hudi/common/model/EventTimeAvroPayload.java | 11 +-
.../hudi/common/model/HoodieAvroIndexedRecord.java | 53 +-
.../apache/hudi/common/model/HoodieAvroRecord.java | 16 +-
.../hudi/common/model/HoodieAvroRecordMerger.java | 82 ++-
.../hudi/common/model/HoodieEmptyRecord.java | 3 +-
.../apache/hudi/common/model/HoodieLogFile.java | 2 +-
.../common/model/HoodieMetadataRecordMerger.java | 51 --
.../model/HoodiePreCombineAvroRecordMerger.java | 44 +-
.../org/apache/hudi/common/model/HoodieRecord.java | 9 +-
.../hudi/common/model/HoodieRecordMerger.java | 67 ++-
.../apache/hudi/common/model/HoodieWriteStat.java | 1 +
.../common/model/OverwriteWithLatestMerger.java | 10 +-
.../hudi/common/table/HoodieTableConfig.java | 23 +-
.../hudi/common/table/HoodieTableMetaClient.java | 80 ++-
.../table/log/HoodieMergedLogRecordScanner.java | 22 +-
.../table/log/HoodieUnMergedLogRecordScanner.java | 2 +-
.../table/read/BufferedRecordMergerFactory.java | 239 ++------
.../hudi/common/table/read/BufferedRecords.java | 13 +-
.../hudi/common/table/read/DeleteContext.java | 21 +-
.../table/read/FileGroupReaderSchemaHandler.java | 3 +-
.../common/table/read/HoodieFileGroupReader.java | 7 +-
.../table/read/buffer/FileGroupRecordBuffer.java | 2 +-
.../read/buffer/KeyBasedFileGroupRecordBuffer.java | 1 -
.../buffer/PositionBasedFileGroupRecordBuffer.java | 3 +-
.../StreamingFileGroupRecordBufferLoader.java | 15 +-
.../table/view/AbstractTableFileSystemView.java | 12 +-
.../org/apache/hudi/common/util/ConfigUtils.java | 7 +-
.../org/apache/hudi/common/util/DateTimeUtils.java | 44 ++
.../apache/hudi/common/util/FileFormatUtils.java | 30 +-
.../org/apache/hudi/common/util/HFileUtils.java | 21 +-
.../apache/hudi/common/util/HoodieDataUtils.java | 27 +
.../apache/hudi/common/util/HoodieRecordUtils.java | 13 +-
.../org/apache/hudi/common/util/SortUtils.java | 36 ++
.../org/apache/hudi/expression/Comparators.java | 3 +
.../record/HoodieRecordIndex.java} | 22 +-
.../java/org/apache/hudi/internal/schema/Type.java | 11 +-
.../org/apache/hudi/internal/schema/Types.java | 211 ++++++-
.../convert/AvroInternalSchemaConverter.java | 69 ++-
.../hudi/internal/schema/utils/Conversions.java | 2 +
.../internal/schema/utils/SchemaChangeUtils.java | 79 ++-
.../hudi/internal/schema/utils/SerDeHelper.java | 25 +
.../io/storage/ColumnRangeMetadataProvider.java | 5 +-
.../apache/hudi/io/storage/HFileReaderFactory.java | 27 +-
.../io/storage/HoodieAvroHFileReaderImplBase.java | 41 +-
.../hudi/io/storage/HoodieFileReaderFactory.java | 20 +
.../io/storage/HoodieNativeAvroHFileReader.java | 43 +-
.../java/org/apache/hudi/keygen/KeyGenerator.java | 7 -
.../hudi/keygen/constant/KeyGeneratorType.java | 16 +
.../apache/hudi/metadata/BaseTableMetadata.java | 57 +-
.../hudi/metadata/HoodieBackedTableMetadata.java | 38 +-
.../apache/hudi/metadata/HoodieIndexVersion.java | 49 +-
.../hudi/metadata/HoodieMetadataPayload.java | 16 +-
.../hudi/metadata/HoodieTableMetadataUtil.java | 350 +++++++----
.../hudi/metadata/MetadataPartitionType.java | 32 +-
.../SecondaryIndexRecordGenerationUtils.java | 4 +-
.../model => stats}/HoodieColumnRangeMetadata.java | 73 ++-
.../java/org/apache/hudi/stats/ValueMetadata.java | 266 +++++++++
.../main/java/org/apache/hudi/stats/ValueType.java | 665 +++++++++++++++++++++
.../parquet/schema/LogicalTypeParquetAdapter.java | 92 +++
.../parquet/schema/OriginalTypeParquetAdapter.java | 83 +++
.../org/apache/hudi/avro/TestHoodieAvroUtils.java | 51 ++
.../hudi/common/config/TestConfigProperty.java | 12 +
.../model/TestDefaultHoodieRecordPayload.java | 18 +-
.../common/model/TestHoodieCommitMetadata.java | 3 +-
.../model/TestHoodieMetadataRecordMerger.java | 65 --
.../hudi/common/model/TestHoodieWriteStat.java | 4 +-
.../common/table/read/SchemaHandlerTestBase.java | 2 +-
.../read/buffer/BaseTestFileGroupRecordBuffer.java | 29 +-
.../versioning/BaseTestCommitMetadataSerDe.java | 7 +-
.../common/testutils/HoodieTestDataGenerator.java | 215 ++++++-
.../hudi/common/testutils/HoodieTestUtils.java | 2 +-
.../reader/HoodieAvroRecordTestMerger.java | 89 ---
.../apache/hudi/common/util/TestBaseFileUtils.java | 27 +-
.../apache/hudi/common/util/TestConfigUtils.java | 13 +-
.../hudi/common/util/TestHoodieRecordUtils.java | 8 +-
.../schema/utils/TestAvroSchemaEvolutionUtils.java | 2 +-
.../org/apache/hudi/keygen/TestKeyGenerator.java | 2 +-
.../hudi/keygen/constant/TestKeyGeneratorType.java | 92 +++
.../hudi/metadata/TestHoodieIndexVersion.java | 6 +-
.../hudi/metadata/TestMetadataPartitionType.java | 37 +-
.../org/apache/hudi/stats/TestValueMetadata.java | 60 ++
.../java/org/apache/hudi/stats/TestValueType.java | 57 ++
hudi-examples/hudi-examples-flink/pom.xml | 4 +
.../examples/quickstart/HoodieSparkQuickstart.java | 1 -
hudi-flink-datasource/hudi-flink/pom.xml | 4 +
.../apache/hudi/configuration/OptionsResolver.java | 8 +
.../org/apache/hudi/sink/StreamWriteFunction.java | 3 +-
.../apache/hudi/sink/bulk/AutoRowDataKeyGen.java | 10 +-
.../org/apache/hudi/sink/bulk/RowDataKeyGen.java | 29 +-
.../hudi/table/format/cdc/CdcInputFormat.java | 10 +-
.../apache/hudi/sink/bulk/TestRowDataKeyGen.java | 34 +-
.../source/TestStreamReadMonitoringFunction.java | 4 +-
.../apache/hudi/table/ITTestSchemaEvolution.java | 8 +-
.../apache/hudi/utils/TestAvroSchemaConverter.java | 4 +-
.../common/table/ParquetTableSchemaResolver.java | 12 +-
.../java/org/apache/hudi/common/util/OrcUtils.java | 5 +-
.../org/apache/hudi/common/util/ParquetUtils.java | 76 ++-
.../org/apache/hudi/hadoop/fs/HadoopFSUtils.java | 4 +-
.../io/hadoop/HoodieAvroFileReaderFactory.java | 10 +
.../io/hadoop/HoodieAvroFileWriterFactory.java | 5 +-
.../avro/AvroSchemaConverterWithTimestampNTZ.java | 598 ++++++++++++++++++
.../avro/HoodieAvroParquetSchemaConverter.java | 57 ++
.../apache/parquet/avro/HoodieAvroReadSupport.java | 47 ++
.../parquet/avro/NativeAvroSchemaConverter.java} | 30 +-
.../apache/parquet/conf/ParquetConfiguration.java | 178 ++++++
.../org/apache/hudi/common/fs/TestFSUtils.java | 2 +-
.../common/functional/TestHoodieLogFormat.java | 4 +-
.../common/model/TestHoodieAvroIndexedRecord.java | 18 +-
.../hudi/common/table/TestHoodieTableConfig.java | 8 +-
.../common/table/TestHoodieTableMetaClient.java | 3 +-
.../common/table/read/TestCustomRecordMerger.java | 40 +-
.../TestHoodieFileGroupReaderInflightCommit.java | 4 +-
.../apache/hudi/common/util/TestParquetUtils.java | 5 +-
.../hudi/metadata/TestHoodieMetadataPayload.java | 24 +-
.../hudi/metadata/TestHoodieTableMetadataUtil.java | 66 +-
.../hudi/hadoop/DefaultHiveRecordMerger.java | 42 +-
.../org/apache/hudi/hadoop/HiveRecordContext.java | 14 +-
.../org/apache/hudi/hadoop/HoodieHiveRecord.java | 16 +-
.../OverwriteWithLatestHiveRecordMerger.java | 11 +-
.../apache/hudi/hadoop/SchemaEvolutionContext.java | 6 +
.../hudi/hadoop/avro/HoodieAvroParquetReader.java | 5 +-
.../realtime/AbstractRealtimeRecordReader.java | 3 +-
.../realtime/RealtimeCompactedRecordReader.java | 27 +-
.../utils/HoodieRealtimeRecordReaderUtils.java | 3 +-
.../reader/DFSHoodieDatasetInputReader.java | 6 +-
.../integ/testsuite/reader/SparkBasedReader.java | 9 +-
.../schema/TestSuiteFileBasedSchemaProvider.java | 4 +-
hudi-io/pom.xml | 5 +
.../java/org/apache/hudi/io/hfile/HFileBlock.java | 22 +-
.../org/apache/hudi/io/hfile/TestHFileWriter.java | 2 +-
.../org/apache/hudi/io/hfile/TestHfileBlock.java | 5 -
.../src/test/resources/hfile/hudi-generated.hfile | Bin 4654 -> 4674 bytes
hudi-spark-datasource/README.md | 6 +-
...va => HoodieVectorizedParquetRecordReader.java} | 5 +-
...org.apache.spark.sql.sources.DataSourceRegister | 2 +-
...DefaultSource.scala => BaseDefaultSource.scala} | 2 +-
.../org/apache/hudi/ColumnStatsIndexSupport.scala | 46 +-
.../scala/org/apache/hudi/DataSourceOptions.scala | 3 +
.../org/apache/hudi/ExpressionIndexSupport.scala | 38 +-
.../scala/org/apache/hudi/HoodieBaseRelation.scala | 4 +-
.../scala/org/apache/hudi/HoodieCDCFileIndex.scala | 2 +-
.../scala/org/apache/hudi/HoodieFileIndex.scala | 2 +-
.../org/apache/hudi/HoodieSparkSqlWriter.scala | 2 +
.../src/main/scala/org/apache/hudi/Iterators.scala | 55 +-
.../apache/hudi/PartitionDirectoryConverter.scala | 2 +-
.../apache/hudi/PartitionStatsIndexSupport.scala | 6 +-
.../org/apache/hudi/RecordLevelIndexSupport.scala | 5 +-
.../org/apache/hudi/SecondaryIndexSupport.scala | 4 +-
.../org/apache/hudi/SparkBaseIndexSupport.scala | 15 +-
.../apache/hudi/SparkHoodieTableFileIndex.scala | 5 +-
.../org/apache/hudi/cdc/CDCFileGroupIterator.scala | 4 +-
...ils.scala => BaseHoodieCatalystPlanUtils.scala} | 7 +-
.../sql/FileFormatUtilsForFileGroupReader.scala | 9 +-
.../spark/sql/HoodieSpark3CatalogUtils.scala | 46 --
.../spark/sql/HoodieSparkCatalogUtils.scala} | 17 +-
.../apache/spark/sql/ResolveInsertionBase.scala | 5 +-
.../spark/sql/adapter/BaseSpark3Adapter.scala | 114 ----
.../apache/spark/sql/avro/SchemaConverters.scala | 5 +-
.../sql/catalyst/catalog/HoodieCatalogTable.scala | 11 +-
.../HoodieFileSystemViewTableValuedFunction.scala | 4 +-
.../HoodieMetadataTableValuedFunction.scala | 4 +-
.../sql/catalyst/plans/logical/HoodieQuery.scala | 4 +-
.../plans/logical/HoodieTableChanges.scala | 6 +-
.../HoodieTimelineTableValuedFunction.scala | 4 +-
.../BaseHoodieNestedSchemaPruning.scala} | 46 +-
.../datasources/Spark3ParsePartitionUtil.scala | 283 ---------
.../HoodieFileGroupReaderBasedFileFormat.scala | 14 +-
...ils.scala => ParquetSchemaEvolutionUtils.scala} | 21 +-
.../parquet/SparkParquetReaderBase.scala | 2 +-
.../apache/spark/sql/hudi/DataSkippingUtils.scala | 14 +-
.../spark/sql/hudi/HoodieSqlCommonUtils.scala | 11 +-
.../spark/sql/hudi/ProvidesHoodieConfig.scala | 16 +-
...nalysis.scala => HoodieSparkBaseAnalysis.scala} | 15 +-
.../spark/sql/hudi/catalog/HoodieCatalog.scala | 7 +-
.../AlterHoodieTableAddColumnsCommand.scala | 7 +-
.../AlterHoodieTableAddPartitionCommand.scala | 9 +-
.../AlterHoodieTableChangeColumnCommand.scala | 9 +-
.../AlterHoodieTableDropPartitionCommand.scala | 5 +-
.../spark/sql/hudi/command/AlterTableCommand.scala | 2 +-
.../hudi/command/CreateHoodieTableCommand.scala | 7 +-
.../command/CreateHoodieTableLikeCommand.scala | 7 +-
.../sql/hudi/command/HoodieCommandMetrics.scala | 0
...odieSparkValidateDuplicateKeyRecordMerger.scala | 11 +-
.../hudi/command/RepairHoodieTableCommand.scala | 9 +-
.../hudi/command/TruncateHoodieTableCommand.scala | 7 +-
.../exception/HoodieAnalysisException.scala | 12 +-
.../hudi/command/payload/ExpressionPayload.scala | 8 +-
.../sql/hudi/streaming/HoodieStreamSourceV1.scala | 17 +-
.../sql/hudi/streaming/HoodieStreamSourceV2.scala | 18 +-
.../hudi/testutils/LogFileColStatsTestUtil.java | 2 +-
.../hudi/testutils/SparkDatasetTestUtils.java | 2 +-
...stSparkFileFormatInternalRowReaderContext.scala | 9 +-
.../datasources/orc/TestSparkOrcReaderFormat.scala | 2 +-
.../parquet/TestSparkParquetReaderFormat.scala | 2 +-
.../spark/sql/hudi/TestProvidesHoodieConfig.scala | 1 -
.../org/apache/spark/sql/hudi/SparkHelpers.scala | 4 +-
.../spark/sql/hudi/analysis/HoodieAnalysis.scala | 55 +-
.../command/procedures/HiveSyncProcedure.scala | 2 +-
.../procedures/HoodieProcedureFilterUtils.scala | 15 +
.../procedures/PartitionBucketIndexManager.scala | 9 +-
.../command/procedures/RunBootstrapProcedure.scala | 2 +-
.../procedures/TruncateTableProcedure.scala | 5 +-
.../spark/sql/parser/HoodieCommonSqlParser.scala | 6 +-
.../org/apache/hudi/ColumnStatsIndexHelper.java | 8 +-
.../java/org/apache/hudi/TestDataSourceUtils.java | 5 +-
.../apache/hudi/TestDefaultSparkRecordMerger.java | 93 ++-
.../hudi/TestHoodieMergeHandleWithSparkMerger.java | 22 +-
.../TestPositionBasedFileGroupRecordBuffer.java | 8 +-
.../TestMetadataUtilRLIandSIRecordGeneration.java | 12 +-
.../hudi/client/model/TestHoodieInternalRow.java | 81 ++-
.../TestBulkInsertInternalPartitioner.java | 49 +-
.../TestRDDSimpleBucketBulkInsertPartitioner.java | 0
.../hudi/functional/TestBufferedRecordMerger.java | 114 +++-
.../TestColStatsRecordWithMetadataRecord.java | 40 +-
.../functional/TestHiveTableSchemaEvolution.java | 2 +-
.../hudi/functional/TestHoodieBackedMetadata.java | 5 +-
.../TestHoodieDatasetBulkInsertHelper.java | 3 +-
.../hudi/functional/TestIncrementalQueries.java | 200 +++++++
.../functional/TestSparkSortAndSizeClustering.java | 19 +-
.../test/java/org/apache/hudi/io/CustomMerger.java | 62 ++
.../java/org/apache/hudi/io/CustomPayload.java | 79 +++
.../java/org/apache/hudi/io/TestMergeHandle.java | 128 +---
.../hudi/keygen/TestComplexKeyGenerator.java | 183 ------
...dieSparkMergeOnReadTableInsertUpdateDelete.java | 2 +-
.../hudi/table/upgrade/TestUpgradeDowngrade.java | 78 ++-
.../hudi/testutils/KeyGeneratorTestUtilities.java | 104 ----
.../test/resources/schema_with_logical_types.avsc | 5 +-
.../resources/upgrade-downgrade-fixtures/README.md | 21 +
.../generate-fixtures.sh | 103 ++--
.../mor-tables/hudi-v6-table-complex-keygen.zip | Bin 0 -> 67342 bytes
.../mor-tables/hudi-v8-table-complex-keygen.zip | Bin 0 -> 136082 bytes
.../mor-tables/hudi-v9-table-complex-keygen.zip | Bin 0 -> 137899 bytes
...la => generate-fixture-complex-keygen-v9.scala} | 32 +-
...scala => generate-fixture-complex-keygen.scala} | 39 +-
...ate-fixture.scala => generate-fixture-v9.scala} | 0
.../scala-templates/generate-fixture.scala | 7 +-
.../org/apache/hudi/TestDataSkippingUtils.scala | 13 +-
.../org/apache/hudi/TestDataSourceDefaults.scala | 127 +++-
.../apache/hudi/TestHoodieDataSourceHelper.scala | 17 +-
.../org/apache/hudi/TestHoodieSparkUtils.scala | 2 +-
.../hudi/TestPartitionDirectoryConverter.scala | 16 +-
.../org/apache/hudi/TestSparkFilterHelper.scala | 2 +-
.../model/TestHoodieRecordSerialization.scala | 7 +-
.../read/TestHoodieFileGroupReaderOnSpark.scala | 4 +-
.../hudi/functional/ColumnStatIndexTestBase.scala | 148 +++--
.../hudi/functional/RecordLevelIndexTestBase.scala | 2 +-
.../hudi/functional/TestBasicSchemaEvolution.scala | 10 +-
.../hudi/functional/TestBucketIndexSupport.scala | 20 +-
.../apache/hudi/functional/TestCOWDataSource.scala | 293 ++++++++-
.../hudi/functional/TestColumnStatsIndex.scala | 19 +-
.../functional/TestColumnStatsIndexWithSQL.scala | 9 +-
.../hudi/functional/TestEightToNineUpgrade.scala | 44 +-
.../TestIncrementalReadWithFullTableScan.scala | 22 +-
.../apache/hudi/functional/TestMORDataSource.scala | 270 ++++++++-
.../TestMetadataTableWithSparkDataSource.scala | 8 +-
.../functional/TestNestedPartitionPathSchema.scala | 147 +++++
.../functional/TestParquetColumnProjection.scala | 75 +--
.../TestPartitionBucketIndexSupport.scala | 54 +-
.../functional/TestPartitionStatsPruning.scala | 13 +-
.../TestPartitionedRecordLevelIndex.scala | 11 +-
.../functional/TestPayloadDeprecationFlow.scala | 265 +++++---
.../hudi/functional/TestRecordLevelIndex.scala | 6 +-
.../functional/TestRecordLevelIndexWithSQL.scala | 26 +-
.../functional/TestSecondaryIndexDataTypes.scala | 11 +-
.../functional/cdc/TestCDCStreamingSuite.scala | 24 +-
.../benchmark/CreateHandleBenchmark.scala | 167 ++++++
.../sql/hudi/common/HoodieSparkSqlTestBase.scala | 25 +-
.../sql/hudi/common/TestHoodieDataUtils.scala | 35 +-
.../hudi/common/TestHoodieInternalRowUtils.scala | 20 +-
.../spark/sql/hudi/ddl/TestCreateTable.scala | 242 +++++++-
.../sql/hudi/dml/insert/TestInsertTable.scala | 2 +-
.../sql/hudi/dml/others/TestMergeIntoTable.scala | 71 +++
.../sql/hudi/dml/others/TestMergeIntoTable2.scala | 12 +-
.../hudi/feature/index/TestExpressionIndex.scala | 99 +--
.../TestHoodieBackedTableMetadataIndexLookup.scala | 4 +-
.../sql/hudi/feature/index/TestIndexSyntax.scala | 2 +-
.../hudi/feature/index/TestSecondaryIndex.scala | 3 +-
.../sql/hudi/procedure/TestCallCommandParser.scala | 56 +-
.../sql/hudi/procedure/TestCallProcedure.scala | 1 +
.../hudi/procedure/TestSavepointsProcedure.scala | 1 +
.../hudi-spark3-common}/pom.xml | 211 +++----
.../hudi/client/model/Spark3HoodieInternalRow.java | 24 +-
...Spark3HoodiePartitionCDCFileGroupMapping.scala} | 9 +-
.../Spark3HoodiePartitionFileSliceMapping.scala} | 11 +-
.../apache/hudi/Spark3HoodiePartitionValues.scala} | 4 +-
.../org/apache/hudi/Spark3HoodieUTF8String.scala} | 11 +-
.../org/apache/spark/sql/Spark3DataFrameUtil.scala | 2 +-
.../spark/sql/Spark3HoodieUTF8StringFactory.scala | 14 +-
.../apache/spark/sql/Spark3HoodieUnsafeUtils.scala | 5 +-
.../spark/sql/adapter/BaseSpark3Adapter.scala | 184 ++++++
.../Spark3HoodiePruneFileSourcePartitions.scala} | 6 +-
.../command/CreateHoodieTableAsSelectCommand.scala | 8 +-
.../hudi/command/DeleteHoodieTableCommand.scala | 5 +-
.../command/InsertIntoHoodieTableCommand.scala | 2 +-
.../hudi/command/MergeIntoHoodieTableCommand.scala | 28 +-
.../hudi/command/UpdateHoodieTableCommand.scala | 11 +-
hudi-spark-datasource/hudi-spark3.3.x/pom.xml | 7 +
...org.apache.spark.sql.sources.DataSourceRegister | 19 -
.../spark/sql/HoodieSpark33CatalystPlanUtils.scala | 2 +-
.../spark/sql/HoodieSpark33SchemaUtils.scala | 12 +
.../apache/spark/sql/adapter/Spark3_3Adapter.scala | 5 +-
.../datasources/Spark33NestedSchemaPruning.scala | 179 +-----
.../Spark33LegacyHoodieParquetFileFormat.scala | 2 +-
.../datasources/parquet/Spark33ParquetReader.scala | 2 +-
.../parser/HoodieSpark3_3ExtendedSqlParser.scala | 2 +-
hudi-spark-datasource/hudi-spark3.4.x/pom.xml | 7 +
...org.apache.spark.sql.sources.DataSourceRegister | 19 -
.../spark/sql/HoodieSpark34CatalystPlanUtils.scala | 2 +-
.../spark/sql/HoodieSpark34SchemaUtils.scala | 12 +
.../apache/spark/sql/adapter/Spark3_4Adapter.scala | 2 -
.../datasources/Spark34NestedSchemaPruning.scala | 179 +-----
.../Spark34LegacyHoodieParquetFileFormat.scala | 2 +-
.../datasources/parquet/Spark34ParquetReader.scala | 2 +-
.../parser/HoodieSpark3_4ExtendedSqlParser.scala | 2 +-
hudi-spark-datasource/hudi-spark3.5.x/pom.xml | 7 +
...org.apache.spark.sql.sources.DataSourceRegister | 19 -
.../spark/sql/HoodieSpark35CatalystPlanUtils.scala | 2 +-
.../spark/sql/HoodieSpark35SchemaUtils.scala | 12 +
.../apache/spark/sql/adapter/Spark3_5Adapter.scala | 2 -
.../datasources/Spark35NestedSchemaPruning.scala | 178 +-----
.../Spark35LegacyHoodieParquetFileFormat.scala | 2 +-
.../datasources/parquet/Spark35ParquetReader.scala | 2 +-
.../parser/HoodieSpark3_5ExtendedSqlParser.scala | 2 +-
.../hudi-spark4-common}/pom.xml | 211 +++----
.../hudi/client/model/Spark4HoodieInternalRow.java | 47 ++
...Spark4HoodiePartitionCDCFileGroupMapping.scala} | 9 +-
.../Spark4HoodiePartitionFileSliceMapping.scala} | 11 +-
.../apache/hudi/Spark4HoodiePartitionValues.scala} | 10 +-
.../org/apache/hudi/Spark4HoodieUTF8String.scala} | 12 +-
.../spark/sql/ExpressionColumnNodeWrapper.scala | 12 +-
.../sql/HoodieSpark4CatalystExpressionUtils.scala} | 3 +-
.../org/apache/spark/sql/Spark4DataFrameUtil.scala | 6 +-
.../spark/sql/Spark4HoodieUTF8StringFactory.scala | 13 +-
.../apache/spark/sql/Spark4HoodieUnsafeUtils.scala | 15 +-
.../spark/sql/adapter/BaseSpark4Adapter.scala | 189 ++++++
.../Spark4HoodiePruneFileSourcePartitions.scala} | 8 +-
.../command/CreateHoodieTableAsSelectCommand.scala | 11 +-
.../hudi/command/DeleteHoodieTableCommand.scala | 10 +-
.../command/InsertIntoHoodieTableCommand.scala | 2 +-
.../hudi/command/MergeIntoHoodieTableCommand.scala | 36 +-
.../hudi/command/UpdateHoodieTableCommand.scala | 16 +-
.../{hudi-spark3.5.x => hudi-spark4.0.x}/pom.xml | 33 +-
.../src/main/antlr4/imports/SqlBase.g4 | 0
.../apache/hudi/spark/sql/parser/HoodieSqlBase.g4 | 0
.../apache/hudi/Spark40HoodieFileScanRDD.scala} | 2 +-
.../HoodieSpark40CatalystExpressionUtils.scala} | 2 +-
.../sql/HoodieSpark40CatalystPlanUtils.scala} | 6 +-
.../spark/sql/HoodieSpark40SchemaUtils.scala} | 14 +-
.../spark/sql/adapter/Spark4_0Adapter.scala} | 36 +-
.../apache/spark/sql/avro/AvroDeserializer.scala | 0
.../org/apache/spark/sql/avro/AvroSerializer.scala | 0
.../org/apache/spark/sql/avro/AvroUtils.scala | 0
.../sql/avro/HoodieSpark4_0AvroDeserializer.scala} | 2 +-
.../sql/avro/HoodieSpark4_0AvroSerializer.scala} | 2 +-
.../HoodieSpark40PartitionedFileUtils.scala} | 4 +-
.../datasources/Spark40NestedSchemaPruning.scala | 61 ++
.../datasources/orc/Spark40OrcReader.scala} | 8 +-
.../parquet/Spark40DataSourceUtils.scala} | 2 +-
.../Spark40LegacyHoodieParquetFileFormat.scala} | 48 +-
.../parquet/Spark40ParquetReader.scala} | 14 +-
.../Spark40ResolveHudiAlterTableCommand.scala} | 5 +-
.../sql/hudi/analysis/HoodieSpark40Analysis.scala} | 19 +-
.../HoodieSpark4_0ExtendedSqlAstBuilder.scala} | 16 +-
.../parser/HoodieSpark4_0ExtendedSqlParser.scala} | 19 +-
.../sync/datahub/config/DataHubSyncConfig.java | 72 ++-
.../config/TlsEnabledDataHubEmitterSupplier.java | 176 ++++++
.../sync/datahub/config/TestDataHubSyncConfig.java | 88 +++
.../TestTlsEnabledDataHubEmitterSupplier.java | 460 ++++++++++++++
.../src/test/resources/multi-ca-cert.pem | 60 ++
.../src/test/resources/test-ca-cert.pem | 21 +
.../src/test/resources/test-keystore.p12 | Bin 0 -> 2706 bytes
.../src/test/resources/test-truststore.p12 | Bin 0 -> 2706 bytes
hudi-tests-common/pom.xml | 4 +
.../utilities/HoodieMetadataTableValidator.java | 17 +-
.../hudi/utilities/HoodieSnapshotExporter.java | 2 +-
.../org/apache/hudi/utilities/UtilHelpers.java | 8 +-
.../utilities/schema/KafkaOffsetPostProcessor.java | 4 +-
.../DeleteSupportSchemaPostProcessor.java | 4 +-
.../DropColumnSchemaPostProcessor.java | 3 +-
.../add/AddPrimitiveColumnSchemaPostProcessor.java | 3 +-
.../hudi/utilities/sources/HoodieIncrSource.java | 6 +-
.../sources/SnapshotLoadQuerySplitter.java | 4 +-
.../apache/hudi/utilities/sources/SqlSource.java | 1 -
.../helpers/MercifulJsonToRowConverter.java | 95 +++
.../utilities/sources/helpers/QueryRunner.java | 4 +-
.../apache/hudi/utilities/streamer/StreamSync.java | 3 +
.../TestHoodieMetadataTableValidator.java | 27 +-
.../hudi/utilities/TestHoodieRepairTool.java | 2 +-
.../deltastreamer/HoodieDeltaStreamerTestBase.java | 2 +-
.../deltastreamer/TestHoodieDeltaStreamer.java | 258 +++++++-
.../deltastreamer/TestSourceFormatAdapter.java | 3 +-
.../TestHoodieMultiTableServicesMain.java | 2 +-
.../utilities/sources/TestHoodieIncrSource.java | 17 +-
.../helpers/TestMercifulJsonToRowConverter.java | 67 ++-
.../sources/helpers/TestSanitizationUtils.java | 2 +-
...hotQuerySplitterWithoutPartitionFilterImpl.java | 49 ++
.../testutils/ColStatsUpgradeTesting.java | 160 +++++
.../utilities/testutils/UtilitiesTestBase.java | 2 +-
.../testutils/sources/AbstractBaseTestSource.java | 14 +-
.../transform/TestSqlQueryBasedTransformer.java | 3 +-
.../src/test/resources/col-stats/README.md | 40 ++
.../col-stats/colstats-upgrade-test-v6.zip | Bin 0 -> 212182 bytes
.../col-stats/colstats-upgrade-test-v8.zip | Bin 0 -> 260900 bytes
.../{source.avsc => source-timestamp-millis.avsc} | 4 +-
packaging/bundle-validation/base/Dockerfile | 2 +-
...h => build_flink1200hive313spark400scala213.sh} | 9 +-
packaging/bundle-validation/ci_run.sh | 17 +-
.../docker_java17/docker_java17_test.sh | 17 +-
packaging/bundle-validation/run_docker_java17.sh | 10 +
packaging/bundle-validation/validate.sh | 35 +-
packaging/hudi-cli-bundle/hudi-cli-with-bundle.sh | 3 +-
packaging/hudi-flink-bundle/pom.xml | 8 +
packaging/hudi-hive-sync-bundle/pom.xml | 2 +
packaging/hudi-integ-test-bundle/pom.xml | 13 +
packaging/hudi-spark-bundle/pom.xml | 7 +
packaging/hudi-utilities-bundle/pom.xml | 7 +
pom.xml | 77 ++-
rfc/rfc-101/rfc-101.md | 4 +-
scripts/pr_compliance.py | 315 ++++++----
scripts/release/validate_source_copyright.sh | 2 +-
594 files changed, 14519 insertions(+), 6132 deletions(-)
delete mode 100644 .github/ISSUE_TEMPLATE/SUPPORT_REQUEST.md
create mode 100644 .github/ISSUE_TEMPLATE/hudi_bug.yml
create mode 100644 .github/ISSUE_TEMPLATE/hudi_community_support.yml
create mode 100644 .github/ISSUE_TEMPLATE/hudi_feature.yml
create mode 100644 .github/ISSUE_TEMPLATE/hudi_improvement.yml
create mode 100644 .github/workflows/pr_title_validation.yml
copy docker/compose/{docker-compose_hadoop284_hive233_spark353_arm64.yml =>
docker-compose_hadoop334_hive313_spark353_arm64.yml} (61%)
rename {hudi-flink-datasource/hudi-flink =>
hudi-client/hudi-flink-client}/src/main/java/org/apache/hudi/table/format/FlinkRecordContext.java
(97%)
rename {hudi-flink-datasource/hudi-flink =>
hudi-client/hudi-flink-client}/src/main/java/org/apache/hudi/util/RecordKeyToRowDataConverter.java
(100%)
create mode 100644
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/stats/SparkValueMetadataUtils.java
copy
hudi-sync/hudi-sync-common/src/main/java/org/apache/hudi/sync/common/util/TableUtils.java
=>
hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/HoodiePartitionCDCFileGroupMapping.scala
(79%)
copy
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/procedures/ProcedureArgs.scala
=>
hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/HoodiePartitionFileSliceMapping.scala
(78%)
copy
hudi-client/hudi-spark-client/src/main/scala/org/apache/{spark/sql/HoodieCatalogUtils.scala
=> hudi/HoodiePartitionValues.scala} (76%)
copy
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/package.scala
=>
hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/HoodieUTF8String.scala
(59%)
delete mode 100644
hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/SparkJdbcUtils.scala
rename
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/Spark3RowSerDe.scala
=>
hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/SparkRowSerDe.scala
(87%)
rename {hudi-spark-datasource/hudi-spark-common/src/main/java =>
hudi-client/hudi-spark-client/src/main/scala}/org/apache/hudi/spark/internal/ReflectUtil.java
(100%)
rename
hudi-spark-datasource/hudi-spark3.5.x/src/main/scala/org/apache/spark/sql/HoodieSpark35CatalogUtils.scala
=>
hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/HoodieUTF8StringFactory.scala
(63%)
create mode 100644
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/keygen/KeyGeneratorTestUtilities.java
create mode 100644
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/keygen/TestComplexKeyGenerator.java
rename {hudi-spark-datasource/hudi-spark =>
hudi-client/hudi-spark-client}/src/test/java/org/apache/hudi/keygen/TestCustomKeyGenerator.java
(70%)
rename {hudi-spark-datasource/hudi-spark =>
hudi-client/hudi-spark-client}/src/test/java/org/apache/hudi/keygen/TestGlobalDeleteRecordGenerator.java
(50%)
rename {hudi-spark-datasource/hudi-spark =>
hudi-client/hudi-spark-client}/src/test/java/org/apache/hudi/keygen/TestNonpartitionedKeyGenerator.java
(54%)
rename {hudi-spark-datasource/hudi-spark =>
hudi-client/hudi-spark-client}/src/test/java/org/apache/hudi/keygen/TestSimpleKeyGenerator.java
(79%)
rename {hudi-spark-datasource/hudi-spark =>
hudi-client/hudi-spark-client}/src/test/java/org/apache/hudi/keygen/TestTimestampBasedKeyGenerator.java
(79%)
rename {hudi-spark-datasource/hudi-spark =>
hudi-client/hudi-spark-client}/src/test/java/org/apache/hudi/keygen/factory/TestCreateKeyGeneratorByTypeWithFactory.java
(87%)
rename {hudi-spark-datasource/hudi-spark =>
hudi-client/hudi-spark-client}/src/test/java/org/apache/hudi/keygen/factory/TestHoodieSparkKeyGeneratorFactory.java
(91%)
rename
hudi-spark-datasource/hudi-spark3.3.x/src/main/scala/org/apache/spark/sql/HoodieSpark33CatalogUtils.scala
=>
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/testutils/DisableOnSpark4Condition.java
(56%)
copy
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/procedures/ProcedureArgs.scala
=>
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/testutils/DisabledOnSpark4.java
(65%)
copy
packaging/hudi-gcp-bundle/src/main/java/org/apache/hudi/gcp/bigquery/bundle/Main.java
=> hudi-common/src/main/java/org/apache/hudi/ParquetAdapter.java (52%)
create mode 100644
hudi-common/src/main/java/org/apache/hudi/avro/JoinedGenericRecord.java
delete mode 100644
hudi-common/src/main/java/org/apache/hudi/common/model/HoodieMetadataRecordMerger.java
copy hudi-common/src/main/java/org/apache/hudi/{common/util/MapUtils.java =>
index/record/HoodieRecordIndex.java} (56%)
rename hudi-common/src/main/java/org/apache/hudi/{common/model =>
stats}/HoodieColumnRangeMetadata.java (69%)
create mode 100644
hudi-common/src/main/java/org/apache/hudi/stats/ValueMetadata.java
create mode 100644
hudi-common/src/main/java/org/apache/hudi/stats/ValueType.java
create mode 100644
hudi-common/src/main/java/org/apache/parquet/schema/LogicalTypeParquetAdapter.java
create mode 100644
hudi-common/src/main/java/org/apache/parquet/schema/OriginalTypeParquetAdapter.java
delete mode 100644
hudi-common/src/test/java/org/apache/hudi/common/model/TestHoodieMetadataRecordMerger.java
delete mode 100644
hudi-common/src/test/java/org/apache/hudi/common/testutils/reader/HoodieAvroRecordTestMerger.java
create mode 100644
hudi-common/src/test/java/org/apache/hudi/keygen/constant/TestKeyGeneratorType.java
create mode 100644
hudi-common/src/test/java/org/apache/hudi/stats/TestValueMetadata.java
create mode 100644
hudi-common/src/test/java/org/apache/hudi/stats/TestValueType.java
create mode 100644
hudi-hadoop-common/src/main/java/org/apache/parquet/avro/AvroSchemaConverterWithTimestampNTZ.java
create mode 100644
hudi-hadoop-common/src/main/java/org/apache/parquet/avro/HoodieAvroParquetSchemaConverter.java
copy
hudi-hadoop-common/src/{test/java/org/apache/hudi/storage/hadoop/TestHadoopStorageConfiguration.java
=> main/java/org/apache/parquet/avro/NativeAvroSchemaConverter.java} (58%)
create mode 100644
hudi-hadoop-common/src/main/java/org/apache/parquet/conf/ParquetConfiguration.java
rename
hudi-spark-datasource/hudi-spark-common/src/main/java/org/apache/spark/sql/execution/datasources/parquet/{Spark3HoodieVectorizedParquetRecordReader.java
=> HoodieVectorizedParquetRecordReader.java} (97%)
rename
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/{Spark3DefaultSource.scala
=> BaseDefaultSource.scala} (94%)
rename
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/{HoodieSpark3CatalystPlanUtils.scala
=> BaseHoodieCatalystPlanUtils.scala} (94%)
delete mode 100644
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/HoodieSpark3CatalogUtils.scala
rename
hudi-spark-datasource/{hudi-spark3.4.x/src/main/scala/org/apache/spark/sql/HoodieSpark34CatalogUtils.scala
=>
hudi-spark-common/src/main/scala/org/apache/spark/sql/HoodieSparkCatalogUtils.scala}
(69%)
delete mode 100644
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/adapter/BaseSpark3Adapter.scala
copy
hudi-spark-datasource/{hudi-spark3.4.x/src/main/scala/org/apache/spark/sql/execution/datasources/Spark34NestedSchemaPruning.scala
=>
hudi-spark-common/src/main/scala/org/apache/spark/sql/execution/datasources/BaseHoodieNestedSchemaPruning.scala}
(80%)
delete mode 100644
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/execution/datasources/Spark3ParsePartitionUtil.scala
rename
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/{Spark3ParquetSchemaEvolutionUtils.scala
=> ParquetSchemaEvolutionUtils.scala} (93%)
rename
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/analysis/{HoodieSpark3Analysis.scala
=> HoodieSparkBaseAnalysis.scala} (96%)
rename hudi-spark-datasource/{hudi-spark =>
hudi-spark-common}/src/main/scala/org/apache/spark/sql/hudi/command/HoodieCommandMetrics.scala
(100%)
copy
hudi-aws/src/main/java/org/apache/hudi/aws/sync/util/GluePartitionFilterGenerator.java
=>
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/command/exception/HoodieAnalysisException.scala
(70%)
rename hudi-spark-datasource/{hudi-spark =>
hudi-spark-common}/src/main/scala/org/apache/spark/sql/hudi/command/payload/ExpressionPayload.scala
(98%)
rename {hudi-client/hudi-spark-client =>
hudi-spark-datasource/hudi-spark}/src/test/java/org/apache/hudi/client/model/TestHoodieInternalRow.java
(80%)
rename {hudi-client/hudi-spark-client =>
hudi-spark-datasource/hudi-spark}/src/test/java/org/apache/hudi/execution/bulkinsert/TestBulkInsertInternalPartitioner.java
(86%)
rename {hudi-client/hudi-spark-client =>
hudi-spark-datasource/hudi-spark}/src/test/java/org/apache/hudi/execution/bulkinsert/TestRDDSimpleBucketBulkInsertPartitioner.java
(100%)
create mode 100644
hudi-spark-datasource/hudi-spark/src/test/java/org/apache/hudi/functional/TestIncrementalQueries.java
create mode 100644
hudi-spark-datasource/hudi-spark/src/test/java/org/apache/hudi/io/CustomMerger.java
create mode 100644
hudi-spark-datasource/hudi-spark/src/test/java/org/apache/hudi/io/CustomPayload.java
delete mode 100644
hudi-spark-datasource/hudi-spark/src/test/java/org/apache/hudi/keygen/TestComplexKeyGenerator.java
delete mode 100644
hudi-spark-datasource/hudi-spark/src/test/java/org/apache/hudi/testutils/KeyGeneratorTestUtilities.java
create mode 100644
hudi-spark-datasource/hudi-spark/src/test/resources/upgrade-downgrade-fixtures/mor-tables/hudi-v6-table-complex-keygen.zip
create mode 100644
hudi-spark-datasource/hudi-spark/src/test/resources/upgrade-downgrade-fixtures/mor-tables/hudi-v8-table-complex-keygen.zip
create mode 100644
hudi-spark-datasource/hudi-spark/src/test/resources/upgrade-downgrade-fixtures/mor-tables/hudi-v9-table-complex-keygen.zip
copy
hudi-spark-datasource/hudi-spark/src/test/resources/upgrade-downgrade-fixtures/scala-templates/{generate-fixture.scala
=> generate-fixture-complex-keygen-v9.scala} (79%)
copy
hudi-spark-datasource/hudi-spark/src/test/resources/upgrade-downgrade-fixtures/scala-templates/{generate-fixture.scala
=> generate-fixture-complex-keygen.scala} (76%)
copy
hudi-spark-datasource/hudi-spark/src/test/resources/upgrade-downgrade-fixtures/scala-templates/{generate-fixture.scala
=> generate-fixture-v9.scala} (100%)
create mode 100644
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestNestedPartitionPathSchema.scala
create mode 100644
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/execution/benchmark/CreateHandleBenchmark.scala
copy {hudi-examples/hudi-examples-spark =>
hudi-spark-datasource/hudi-spark3-common}/pom.xml (64%)
rename
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/utils/SparkRowSerDe.java
=>
hudi-spark-datasource/hudi-spark3-common/src/main/java/org/apache/hudi/client/model/Spark3HoodieInternalRow.java
(54%)
copy
hudi-spark-datasource/{hudi-spark-common/src/main/scala/org/apache/hudi/HoodiePartitionCDCFileGroupMapping.scala
=>
hudi-spark3-common/src/main/scala/org/apache/hudi/Spark3HoodiePartitionCDCFileGroupMapping.scala}
(74%)
copy
hudi-spark-datasource/{hudi-spark-common/src/main/scala/org/apache/hudi/HoodiePartitionFileSliceMapping.scala
=>
hudi-spark3-common/src/main/scala/org/apache/hudi/Spark3HoodiePartitionFileSliceMapping.scala}
(73%)
copy
hudi-spark-datasource/{hudi-spark-common/src/main/scala/org/apache/hudi/HoodiePartitionValues.scala
=>
hudi-spark3-common/src/main/scala/org/apache/hudi/Spark3HoodiePartitionValues.scala}
(95%)
copy
hudi-spark-datasource/{hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/procedures/ProcedureArgs.scala
=>
hudi-spark3-common/src/main/scala/org/apache/hudi/Spark3HoodieUTF8String.scala}
(75%)
copy
hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/DataFrameUtil.scala
=>
hudi-spark-datasource/hudi-spark3-common/src/main/scala/org/apache/spark/sql/Spark3DataFrameUtil.scala
(96%)
copy
hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/HoodieCatalogUtils.scala
=>
hudi-spark-datasource/hudi-spark3-common/src/main/scala/org/apache/spark/sql/Spark3HoodieUTF8StringFactory.scala
(73%)
copy
hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/HoodieUnsafeUtils.scala
=>
hudi-spark-datasource/hudi-spark3-common/src/main/scala/org/apache/spark/sql/Spark3HoodieUnsafeUtils.scala
(98%)
create mode 100644
hudi-spark-datasource/hudi-spark3-common/src/main/scala/org/apache/spark/sql/adapter/BaseSpark3Adapter.scala
copy
hudi-spark-datasource/{hudi-spark/src/main/scala/org/apache/spark/sql/hudi/analysis/HoodiePruneFileSourcePartitions.scala
=>
hudi-spark3-common/src/main/scala/org/apache/spark/sql/hudi/analysis/Spark3HoodiePruneFileSourcePartitions.scala}
(93%)
copy hudi-spark-datasource/{hudi-spark =>
hudi-spark3-common}/src/main/scala/org/apache/spark/sql/hudi/command/CreateHoodieTableAsSelectCommand.scala
(95%)
copy hudi-spark-datasource/{hudi-spark =>
hudi-spark3-common}/src/main/scala/org/apache/spark/sql/hudi/command/DeleteHoodieTableCommand.scala
(95%)
copy hudi-spark-datasource/{hudi-spark =>
hudi-spark3-common}/src/main/scala/org/apache/spark/sql/hudi/command/InsertIntoHoodieTableCommand.scala
(99%)
copy hudi-spark-datasource/{hudi-spark =>
hudi-spark3-common}/src/main/scala/org/apache/spark/sql/hudi/command/MergeIntoHoodieTableCommand.scala
(97%)
copy hudi-spark-datasource/{hudi-spark =>
hudi-spark3-common}/src/main/scala/org/apache/spark/sql/hudi/command/UpdateHoodieTableCommand.scala
(92%)
delete mode 100644
hudi-spark-datasource/hudi-spark3.3.x/src/main/resources/META-INF/services/org.apache.spark.sql.sources.DataSourceRegister
delete mode 100644
hudi-spark-datasource/hudi-spark3.4.x/src/main/resources/META-INF/services/org.apache.spark.sql.sources.DataSourceRegister
delete mode 100644
hudi-spark-datasource/hudi-spark3.5.x/src/main/resources/META-INF/services/org.apache.spark.sql.sources.DataSourceRegister
copy {hudi-examples/hudi-examples-spark =>
hudi-spark-datasource/hudi-spark4-common}/pom.xml (64%)
create mode 100644
hudi-spark-datasource/hudi-spark4-common/src/main/java/org/apache/hudi/client/model/Spark4HoodieInternalRow.java
rename
hudi-spark-datasource/{hudi-spark-common/src/main/scala/org/apache/hudi/HoodiePartitionCDCFileGroupMapping.scala
=>
hudi-spark4-common/src/main/scala/org/apache/hudi/Spark4HoodiePartitionCDCFileGroupMapping.scala}
(74%)
rename
hudi-spark-datasource/{hudi-spark-common/src/main/scala/org/apache/hudi/HoodiePartitionFileSliceMapping.scala
=>
hudi-spark4-common/src/main/scala/org/apache/hudi/Spark4HoodiePartitionFileSliceMapping.scala}
(73%)
rename
hudi-spark-datasource/{hudi-spark-common/src/main/scala/org/apache/hudi/HoodiePartitionValues.scala
=>
hudi-spark4-common/src/main/scala/org/apache/hudi/Spark4HoodiePartitionValues.scala}
(91%)
copy
hudi-spark-datasource/{hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/procedures/ProcedureArgs.scala
=>
hudi-spark4-common/src/main/scala/org/apache/hudi/Spark4HoodieUTF8String.scala}
(69%)
copy
hudi-common/src/main/java/org/apache/hudi/common/HoodieSchemaNotFoundException.java
=>
hudi-spark-datasource/hudi-spark4-common/src/main/scala/org/apache/spark/sql/ExpressionColumnNodeWrapper.scala
(72%)
copy
hudi-spark-datasource/{hudi-spark-common/src/main/scala/org/apache/spark/sql/HoodieSpark3CatalystExpressionUtils.scala
=>
hudi-spark4-common/src/main/scala/org/apache/spark/sql/HoodieSpark4CatalystExpressionUtils.scala}
(96%)
copy
hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/DataFrameUtil.scala
=>
hudi-spark-datasource/hudi-spark4-common/src/main/scala/org/apache/spark/sql/Spark4DataFrameUtil.scala
(83%)
rename
hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/HoodieCatalogUtils.scala
=>
hudi-spark-datasource/hudi-spark4-common/src/main/scala/org/apache/spark/sql/Spark4HoodieUTF8StringFactory.scala
(73%)
copy
hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/HoodieUnsafeUtils.scala
=>
hudi-spark-datasource/hudi-spark4-common/src/main/scala/org/apache/spark/sql/Spark4HoodieUnsafeUtils.scala
(87%)
create mode 100644
hudi-spark-datasource/hudi-spark4-common/src/main/scala/org/apache/spark/sql/adapter/BaseSpark4Adapter.scala
rename
hudi-spark-datasource/{hudi-spark/src/main/scala/org/apache/spark/sql/hudi/analysis/HoodiePruneFileSourcePartitions.scala
=>
hudi-spark4-common/src/main/scala/org/apache/spark/sql/hudi/analysis/Spark4HoodiePruneFileSourcePartitions.scala}
(92%)
rename hudi-spark-datasource/{hudi-spark =>
hudi-spark4-common}/src/main/scala/org/apache/spark/sql/hudi/command/CreateHoodieTableAsSelectCommand.scala
(93%)
rename hudi-spark-datasource/{hudi-spark =>
hudi-spark4-common}/src/main/scala/org/apache/spark/sql/hudi/command/DeleteHoodieTableCommand.scala
(90%)
rename hudi-spark-datasource/{hudi-spark =>
hudi-spark4-common}/src/main/scala/org/apache/spark/sql/hudi/command/InsertIntoHoodieTableCommand.scala
(99%)
rename hudi-spark-datasource/{hudi-spark =>
hudi-spark4-common}/src/main/scala/org/apache/spark/sql/hudi/command/MergeIntoHoodieTableCommand.scala
(96%)
rename hudi-spark-datasource/{hudi-spark =>
hudi-spark4-common}/src/main/scala/org/apache/spark/sql/hudi/command/UpdateHoodieTableCommand.scala
(90%)
copy hudi-spark-datasource/{hudi-spark3.5.x => hudi-spark4.0.x}/pom.xml (92%)
copy hudi-spark-datasource/{hudi-spark3.3.x =>
hudi-spark4.0.x}/src/main/antlr4/imports/SqlBase.g4 (100%)
copy hudi-spark-datasource/{hudi-spark3.3.x =>
hudi-spark4.0.x}/src/main/antlr4/org/apache/hudi/spark/sql/parser/HoodieSqlBase.g4
(100%)
copy
hudi-spark-datasource/{hudi-spark3.4.x/src/main/scala/org/apache/hudi/Spark34HoodieFileScanRDD.scala
=>
hudi-spark4.0.x/src/main/scala/org/apache/hudi/Spark40HoodieFileScanRDD.scala}
(96%)
copy
hudi-spark-datasource/{hudi-spark3.5.x/src/main/scala/org/apache/spark/sql/HoodieSpark35CatalystExpressionUtils.scala
=>
hudi-spark4.0.x/src/main/scala/org/apache/spark/sql/HoodieSpark40CatalystExpressionUtils.scala}
(98%)
copy
hudi-spark-datasource/{hudi-spark3.5.x/src/main/scala/org/apache/spark/sql/HoodieSpark35CatalystPlanUtils.scala
=>
hudi-spark4.0.x/src/main/scala/org/apache/spark/sql/HoodieSpark40CatalystPlanUtils.scala}
(97%)
copy
hudi-spark-datasource/{hudi-spark3.5.x/src/main/scala/org/apache/spark/sql/HoodieSpark35SchemaUtils.scala
=>
hudi-spark4.0.x/src/main/scala/org/apache/spark/sql/HoodieSpark40SchemaUtils.scala}
(73%)
copy
hudi-spark-datasource/{hudi-spark3.5.x/src/main/scala/org/apache/spark/sql/adapter/Spark3_5Adapter.scala
=>
hudi-spark4.0.x/src/main/scala/org/apache/spark/sql/adapter/Spark4_0Adapter.scala}
(87%)
copy hudi-spark-datasource/{hudi-spark3.5.x =>
hudi-spark4.0.x}/src/main/scala/org/apache/spark/sql/avro/AvroDeserializer.scala
(100%)
copy hudi-spark-datasource/{hudi-spark3.5.x =>
hudi-spark4.0.x}/src/main/scala/org/apache/spark/sql/avro/AvroSerializer.scala
(100%)
copy hudi-spark-datasource/{hudi-spark3.4.x =>
hudi-spark4.0.x}/src/main/scala/org/apache/spark/sql/avro/AvroUtils.scala (100%)
copy
hudi-spark-datasource/{hudi-spark3.5.x/src/main/scala/org/apache/spark/sql/avro/HoodieSpark3_5AvroDeserializer.scala
=>
hudi-spark4.0.x/src/main/scala/org/apache/spark/sql/avro/HoodieSpark4_0AvroDeserializer.scala}
(95%)
copy
hudi-spark-datasource/{hudi-spark3.5.x/src/main/scala/org/apache/spark/sql/avro/HoodieSpark3_5AvroSerializer.scala
=>
hudi-spark4.0.x/src/main/scala/org/apache/spark/sql/avro/HoodieSpark4_0AvroSerializer.scala}
(94%)
copy
hudi-spark-datasource/{hudi-spark3.5.x/src/main/scala/org/apache/spark/sql/execution/datasources/HoodieSpark35PartitionedFileUtils.scala
=>
hudi-spark4.0.x/src/main/scala/org/apache/spark/sql/execution/datasources/HoodieSpark40PartitionedFileUtils.scala}
(96%)
create mode 100644
hudi-spark-datasource/hudi-spark4.0.x/src/main/scala/org/apache/spark/sql/execution/datasources/Spark40NestedSchemaPruning.scala
copy
hudi-spark-datasource/{hudi-spark3.5.x/src/main/scala/org/apache/spark/sql/execution/datasources/orc/Spark35OrcReader.scala
=>
hudi-spark4.0.x/src/main/scala/org/apache/spark/sql/execution/datasources/orc/Spark40OrcReader.scala}
(95%)
copy
hudi-spark-datasource/{hudi-spark3.5.x/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/Spark35DataSourceUtils.scala
=>
hudi-spark4.0.x/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/Spark40DataSourceUtils.scala}
(99%)
copy
hudi-spark-datasource/{hudi-spark3.5.x/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/Spark35LegacyHoodieParquetFileFormat.scala
=>
hudi-spark4.0.x/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/Spark40LegacyHoodieParquetFileFormat.scala}
(93%)
copy
hudi-spark-datasource/{hudi-spark3.5.x/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/Spark35ParquetReader.scala
=>
hudi-spark4.0.x/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/Spark40ParquetReader.scala}
(96%)
copy
hudi-spark-datasource/{hudi-spark3.4.x/src/main/scala/org/apache/spark/sql/hudi/Spark34ResolveHudiAlterTableCommand.scala
=>
hudi-spark4.0.x/src/main/scala/org/apache/spark/sql/hudi/Spark40ResolveHudiAlterTableCommand.scala}
(93%)
copy
hudi-spark-datasource/{hudi-spark3.5.x/src/main/scala/org/apache/spark/sql/hudi/analysis/HoodieSpark35Analysis.scala
=>
hudi-spark4.0.x/src/main/scala/org/apache/spark/sql/hudi/analysis/HoodieSpark40Analysis.scala}
(96%)
copy
hudi-spark-datasource/{hudi-spark3.5.x/src/main/scala/org/apache/spark/sql/parser/HoodieSpark3_5ExtendedSqlAstBuilder.scala
=>
hudi-spark4.0.x/src/main/scala/org/apache/spark/sql/parser/HoodieSpark4_0ExtendedSqlAstBuilder.scala}
(99%)
copy
hudi-spark-datasource/{hudi-spark3.5.x/src/main/scala/org/apache/spark/sql/parser/HoodieSpark3_5ExtendedSqlParser.scala
=>
hudi-spark4.0.x/src/main/scala/org/apache/spark/sql/parser/HoodieSpark4_0ExtendedSqlParser.scala}
(92%)
create mode 100644
hudi-sync/hudi-datahub-sync/src/main/java/org/apache/hudi/sync/datahub/config/TlsEnabledDataHubEmitterSupplier.java
create mode 100644
hudi-sync/hudi-datahub-sync/src/test/java/org/apache/hudi/sync/datahub/config/TestTlsEnabledDataHubEmitterSupplier.java
create mode 100644
hudi-sync/hudi-datahub-sync/src/test/resources/multi-ca-cert.pem
create mode 100644
hudi-sync/hudi-datahub-sync/src/test/resources/test-ca-cert.pem
create mode 100644
hudi-sync/hudi-datahub-sync/src/test/resources/test-keystore.p12
create mode 100644
hudi-sync/hudi-datahub-sync/src/test/resources/test-truststore.p12
create mode 100644
hudi-utilities/src/test/java/org/apache/hudi/utilities/sources/helpers/TestSnapshotQuerySplitterWithoutPartitionFilterImpl.java
create mode 100644
hudi-utilities/src/test/java/org/apache/hudi/utilities/testutils/ColStatsUpgradeTesting.java
create mode 100644 hudi-utilities/src/test/resources/col-stats/README.md
create mode 100644
hudi-utilities/src/test/resources/col-stats/colstats-upgrade-test-v6.zip
create mode 100644
hudi-utilities/src/test/resources/col-stats/colstats-upgrade-test-v8.zip
copy hudi-utilities/src/test/resources/streamer-config/{source.avsc =>
source-timestamp-millis.avsc} (98%)
copy
packaging/bundle-validation/base/{build_flink1200hive313spark351scala213.sh =>
build_flink1200hive313spark400scala213.sh} (79%)
