This is an automated email from the ASF dual-hosted git repository.
yihua pushed a change to branch branch-0.x
in repository https://gitbox.apache.org/repos/asf/hudi.git
omit a00ef9dd01a [HUDI-7651] Add util methods for creating meta client
(#11081)
omit e358279a860 [MINOR] Fix LoggerName for JDBCExecutor (#11063)
omit 0f6bc8cbd58 [HUDI-6386] Enable testArchivalWithMultiWriters back as
they are passing (#9085)
omit 92425585e59 [HUDI-7647] READ_UTC_TIMEZONE doesn't affect log files for
MOR tables (#11066)
omit 48890500ef6 [MINOR] Fixe naming of methods in HoodieMetadataConfig
(#11076)
omit 9a2a61c64f2 [MINOR] Fix incorrect catch of ClassCastException using
HoodieSparkKeyGeneratorFactory (#11062)
omit 278f57828fd [HUDI-7608] Fix Flink table creation configuration not
taking effect when writing to Spark (#11005)
new 4669a4cad5d [HUDI-7608] Fix Flink table creation configuration not
taking effect when writing to Spark (#11005)
new d14b6401d2e [MINOR] Fix incorrect catch of ClassCastException using
HoodieSparkKeyGeneratorFactory (#11062)
new 63ec2431743 [MINOR] Fixe naming of methods in HoodieMetadataConfig
(#11076)
new 94a67e420a5 [HUDI-7647] READ_UTC_TIMEZONE doesn't affect log files for
MOR tables (#11066)
new 0a556cb3dcd [HUDI-6386] Enable testArchivalWithMultiWriters back as
they are passing (#9085)
new 55598858969 [MINOR] Fix LoggerName for JDBCExecutor (#11063)
new 586898b9256 [HUDI-7651] Add util methods for creating meta client
(#11081)
new 226c2c7f895 [HUDI-7632] Remove FileSystem usage in
HoodieLogFormatWriter (#11082)
new b99e87db0d0 [HUDI-7650] Remove FileSystem argument in TestHelpers
methods (#11072)
new 84a98695ea6 [MINOR] Remove unused util methods in LogReaderUtils
(#11086)
new 2a66bc3fb0f [HUDI-7660] Fix excessive object creation in RowDataKeyGen
(#11084)
new 6a84e7cca10 [HUDI-7235] Fix checkpoint bug for S3/GCS Incremental
Source (#10336)
new 5ee71fd7f34 [HUDI-7645] Optimize BQ sync tool for MDT (#11065)
new e8494df5704 [HUDI-7666] Fix serializable implementation of
StorageConfiguration class (#11091)
new c95ebfb47ab [MINOR] Make KafkaSource abstraction public and more
flexible (#11093)
new 8eb7518711d [HUDI-7658] Add time to meta sync failure log (#11080)
new 63a3e2fe28e [HUDI-7511] Fixing offset range calculation for kafka
(#10875)
new ed5e8a64e56 [HUDI-7672] Fix the Hive server scratch dir for tests in
hudi-utilities (#11097)
new 90c429eb9dc [HUDI-7575] Avoid repeated fetching of pending replace
instants (#10976)
new 3ef0b7a5d95 [HUDI-7676] Fix serialization in Spark DAG in
HoodieBackedTableMetadataWriter (#11103)
new c32beae1443 [HUDI-7664] Remove Hadoop dependency from hudi-io module
(#11089)
new 0bda958007f [MINOR] Streamer test setup performance (#10806)
new f4ba12c10e7 [HUDI-7670] Return StorageConfiguration from getConf() in
HoodieStorage (#11096)
new 878d0d92c05 [HUDI-7668] Add and rename APIs in StorageConfiguration
(#11102)
new 40f2c765d0d [HUDI-7675] Don't set default value for primary key when
get schema from hms (#11101)
new 3a0be9cafe9 [HUDI-7674] Fix Hudi CLI Command "metadata validate-files"
to use file listing to validate (#11100)
new 46b666a7d83 [HUDI-7681] Remove Hadoop Path usage in a few classes in
hudi-common module (#11108)
new 70155da8ff5 [HUDI-7683] Make HoodieMetadataMetrics log level debug ro
reduce noise (#11114)
new 055df60db80 [HUDI-7682] Remove the files copy in Azure CI tests report
(#11110)
new af9eb516dd9 [MINOR] Remove the redundant log in HFileBootstrapIndex
(#11115)
new 2c5af864cd5 [HUDI-7667] Created util method to get offset range for
fetching new data (#11092)
new 971d0b89247 [HUDI-7684] Sort the records for Flink metadata table
bulk_insert (#11116)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (a00ef9dd01a)
\
N -- N -- N refs/heads/branch-0.x (971d0b89247)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 32 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
azure-pipelines-20230430.yml | 26 ++--
.../apache/hudi/cli/commands/MetadataCommand.java | 14 +-
.../metadata/HoodieBackedTableMetadataWriter.java | 5 +-
.../hudi/table/marker/DirectWriteMarkers.java | 4 +-
.../table/upgrade/SixToFiveDowngradeHandler.java | 10 +-
.../apache/hudi/client/HoodieFlinkWriteClient.java | 2 +
.../FlinkHoodieBackedTableMetadataWriter.java | 2 +-
.../hudi/client/TestJavaHoodieBackedMetadata.java | 1 +
.../testutils/HoodieJavaClientTestHarness.java | 2 +-
.../bootstrap/index/HFileBootstrapIndex.java | 3 +-
.../java/org/apache/hudi/common/fs/FSUtils.java | 14 +-
.../hudi/common/model/HoodiePartitionMetadata.java | 2 +-
.../table/log/AbstractHoodieLogRecordReader.java | 2 +-
.../hudi/common/table/log/HoodieLogFileReader.java | 4 +-
.../common/table/log/HoodieLogFormatWriter.java | 30 ++--
.../hudi/common/table/log/LogReaderUtils.java | 57 --------
.../table/timeline/HoodieActiveTimeline.java | 3 +-
.../table/timeline/HoodieDefaultTimeline.java | 32 +++--
.../clean/CleanMetadataV1MigrationHandler.java | 9 +-
.../clean/CleanMetadataV2MigrationHandler.java | 5 +-
.../clean/CleanPlanV1MigrationHandler.java | 5 +-
.../clean/CleanPlanV2MigrationHandler.java | 5 +-
.../compaction/CompactionV1MigrationHandler.java | 9 +-
.../compaction/CompactionV2MigrationHandler.java | 8 +-
.../java/org/apache/hudi/common/util/OrcUtils.java | 2 +-
.../hudi/metadata/HoodieMetadataMetrics.java | 2 +-
.../main/java/org/apache/hudi/metrics/Metrics.java | 12 +-
.../org/apache/hudi/common/fs/TestFSUtils.java | 12 +-
.../hudi/common/testutils/FileCreateUtils.java | 3 +-
.../hudi/common/testutils/HoodieTestUtils.java | 2 +-
.../hudi/common/testutils/RawTripTestPayload.java | 7 +-
.../hudi/common/testutils/SchemaTestUtil.java | 4 +-
.../util/TestDFSPropertiesConfiguration.java | 13 +-
.../storage/TestHoodieHFileReaderWriterBase.java | 2 +-
.../org/apache/hudi/sink/bulk/RowDataKeyGen.java | 45 +++---
.../apache/hudi/table/catalog/HiveSchemaUtils.java | 6 +-
.../hudi/table/catalog/HoodieHiveCatalog.java | 9 +-
.../apache/hudi/table/ITTestHoodieDataSource.java | 29 ++++
.../test/java/org/apache/hudi/utils/TestSQL.java | 12 ++
.../apache/hudi/gcp/bigquery/BigQuerySyncTool.java | 12 +-
.../org/apache/hudi/hadoop/fs/HadoopFSUtils.java | 12 +-
.../storage/hadoop/HadoopStorageConfiguration.java | 50 +++++--
.../hudi/storage/hadoop/HoodieHadoopStorage.java | 24 +++-
.../hudi/hadoop/HoodieROTablePathFilter.java | 2 +-
.../hudi/hadoop/utils/HoodieInputFormatUtils.java | 2 +-
.../testsuite/HoodieDeltaStreamerWrapper.java | 2 +-
hudi-io/pom.xml | 6 -
.../org/apache/hudi/common/util/FileIOUtils.java | 40 ++----
.../java/org/apache/hudi/common/util/Option.java | 4 +
.../org/apache/hudi/storage/HoodieStorage.java | 37 ++++-
.../apache/hudi/storage/StorageConfiguration.java | 44 +++---
.../io/storage/BaseTestStorageConfiguration.java | 37 ++++-
.../main/scala/org/apache/hudi/DefaultSource.scala | 2 +-
.../org/apache/hudi/HoodieDataSourceHelpers.java | 2 +-
.../org/apache/spark/sql/hudi/DedupeSparkJob.scala | 10 +-
.../hudi/functional/TestStructuredStreaming.scala | 2 +-
.../hudi/sync/common/util/ManifestFileWriter.java | 18 ++-
.../apache/hudi/utilities/HoodieRepairTool.java | 15 +-
.../hudi/utilities/sources/AvroKafkaSource.java | 4 +-
.../sources/GcsEventsHoodieIncrSource.java | 4 +-
.../hudi/utilities/sources/JsonKafkaSource.java | 4 +-
.../apache/hudi/utilities/sources/KafkaSource.java | 55 ++++---
.../hudi/utilities/sources/ProtoKafkaSource.java | 4 +-
.../sources/S3EventsHoodieIncrSource.java | 4 +-
.../sources/helpers/IncrSourceHelper.java | 18 ++-
.../utilities/sources/helpers/KafkaOffsetGen.java | 10 ++
.../hudi/utilities/streamer/HoodieStreamer.java | 4 +-
.../apache/hudi/utilities/streamer/StreamSync.java | 26 ++--
.../deltastreamer/HoodieDeltaStreamerTestBase.java | 35 +++--
.../deltastreamer/TestHoodieDeltaStreamer.java | 158 +++++++++++----------
.../TestHoodieDeltaStreamerWithMultiWriter.java | 22 +--
.../offlinejob/HoodieOfflineJobTestBase.java | 7 +-
.../offlinejob/TestHoodieClusteringJob.java | 14 +-
.../offlinejob/TestHoodieCompactorJob.java | 8 +-
.../utilities/sources/BaseTestKafkaSource.java | 8 +-
.../sources/TestGcsEventsHoodieIncrSource.java | 2 +-
.../utilities/sources/TestJsonKafkaSource.java | 6 +-
.../utilities/sources/TestProtoKafkaSource.java | 8 +-
.../sources/TestS3EventsHoodieIncrSource.java | 6 +-
.../sources/helpers/TestIncrSourceHelper.java | 14 +-
.../sources/helpers/TestKafkaOffsetGen.java | 26 ++++
.../utilities/testutils/UtilitiesTestBase.java | 17 ++-
scripts/ci/move_surefire_reports.sh | 58 --------
83 files changed, 691 insertions(+), 566 deletions(-)
delete mode 100755 scripts/ci/move_surefire_reports.sh
