This is an automated email from the ASF dual-hosted git repository.
yihua pushed a change to branch branch-0.x
in repository https://gitbox.apache.org/repos/asf/hudi.git
from fbc4f4dd3ed [HUDI-7447] Fix not bootstrap when subTask restart when
OPCoordinator handle CheckPointComplete not finished (#10767)
new e09f39aceb3 [HUDI-9424] Support using local timezone when writing
flink TIMESTAMP data (#10594)
new 2deb00f3147 [HUDI-7465] Split tests in CI further to reduce total CI
elapsed time (#10795)
new 5179da7d24b [HUDI-6089] Handle default insert behaviour to ingest
duplicates (#10728)
new 2401ccf0362 [HUDI-7469] Reduce redundant tests with Hudi record types
(#10800)
new f358bb89488 [HUDI-6953] Adding test for composite keys with bulk
insert row writer (#10214)
new e2e7c96bb39 [HUDI-3625] Update RFC-60 (#9462)
new 39135e707fa [MINOR] Clean code of FileSystemViewManager (#10797)
new 4f2d60dd387 [HUDI-7471] Use existing util method to get Spark conf in
tests (#10802)
The 8 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/bot.yml | 139 ++++++++++-
azure-pipelines-20230430.yml | 58 ++++-
.../org/apache/hudi/config/HoodieWriteConfig.java | 2 +-
.../hudi/metadata/HoodieMetadataWriteUtils.java | 1 +
.../java/org/apache/hudi/table/HoodieTable.java | 15 +-
.../apache/hudi/config/TestHoodieWriteConfig.java | 1 +
.../TestRemoteFileSystemViewWithMetadataTable.java | 2 +-
.../TestTimelineServerBasedWriteMarkers.java | 5 +-
.../hudi/testutils/HoodieClientTestUtils.java | 5 +-
.../hudi/testutils/providers/SparkProvider.java | 2 +-
.../hudi/common/table/HoodieTableMetaClient.java | 6 +-
.../common/table/view/FileSystemViewManager.java | 102 ++++----
.../apache/hudi/configuration/FlinkOptions.java | 11 +-
.../sink/transform/RowDataToHoodieFunction.java | 3 +-
.../apache/hudi/streamer/FlinkStreamerConfig.java | 2 +-
.../org/apache/hudi/table/HoodieTableSource.java | 2 +-
.../table/format/mor/MergeOnReadInputFormat.java | 4 +-
.../apache/hudi/util/RowDataToAvroConverters.java | 54 ++++-
.../apache/hudi/table/ITTestHoodieDataSource.java | 53 +++++
.../hudi/utils/TestRowDataToAvroConverters.java | 84 +++++++
.../datasources/TestHoodieInMemoryFileIndex.scala | 5 +-
.../org/apache/hudi/TestHoodieSparkSqlWriter.scala | 9 +-
.../org/apache/hudi/TestHoodieSparkUtils.scala | 13 +-
.../apache/hudi/functional/TestCOWDataSource.scala | 93 ++++----
.../apache/hudi/functional/TestMORDataSource.scala | 20 +-
.../sql/hudi/TestAlterTableDropPartition.scala | 4 +-
.../spark/sql/hudi/TestCompactionTable.scala | 4 +-
.../apache/spark/sql/hudi/TestInsertTable.scala | 261 ++++++++++-----------
.../apache/spark/sql/hudi/TestMergeIntoTable.scala | 24 +-
.../spark/sql/hudi/TestMergeIntoTable2.scala | 22 +-
.../TestMergeIntoTableWithNonRecordKeyField.scala | 8 +-
.../org/apache/spark/sql/hudi/TestSpark3DDL.scala | 16 +-
.../spark/sql/hudi/TestTimeTravelTable.scala | 12 +-
.../apache/spark/sql/hudi/TestUpdateTable.scala | 6 +-
.../hudi/timeline/service/TimelineService.java | 6 +-
.../TestRemoteHoodieTableFileSystemView.java | 4 +-
.../deltastreamer/TestHoodieDeltaStreamer.java | 65 +++--
.../deltastreamer/TestSourceFormatAdapter.java | 5 +-
.../sources/helpers/TestSanitizationUtils.java | 6 +-
.../utilities/testutils/UtilitiesTestBase.java | 2 +-
.../transform/TestSqlQueryBasedTransformer.java | 4 +-
rfc/rfc-60/read_flow.png | Bin 0 -> 176856 bytes
rfc/rfc-60/rfc-60.md | 99 ++++++--
rfc/rfc-60/wrapper_fs.png | Bin 0 -> 148392 bytes
44 files changed, 814 insertions(+), 425 deletions(-)
create mode 100644
hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/utils/TestRowDataToAvroConverters.java
create mode 100644 rfc/rfc-60/read_flow.png
create mode 100644 rfc/rfc-60/wrapper_fs.png
