This is an automated email from the ASF dual-hosted git repository.
yihua pushed a change to branch branch-0.x
in repository https://gitbox.apache.org/repos/asf/hudi.git
omit b0e0b148b32 [MINOR] Remove redundant lines in StreamSync and
TestStreamSyncUnitTests (#11027)
omit 926f5ad4163 [HUDI-6762] Removed usages of
MetadataRecordsGenerationParams (#10962)
omit 8a1b6f3845c [HUDI-7619] Removed code duplicates in
HoodieTableMetadataUtil (#11022)
omit 35b330e587b [HUDI-7584] Always read log block lazily and remove
readBlockLazily argument (#11015)
omit 0bb9ef9b5b8 [HUDI-7615] Mark a few write configs with the correct
sinceVersion (#11012)
omit d0f47ebe80e [HUDI-7606] Unpersist RDDs after table services, mainly
compaction and clustering (#11000)
omit 0e96b874c9c [HUDI-7616] Avoid multiple cleaner plans and deprecate
hoodie.clean.allow.multiple (#11013)
omit c10866b967f [HUDI-7378] Fix Spark SQL DML with custom key generator
(#10615)
omit dfe903d4880 [HUDI-7601] Add heartbeat mechanism to refresh lock
(#10994)
omit 5bae2c68bb4 [HUDI-7290] Don't assume ReplaceCommits are always
Clustering (#10479)
omit a5f4e8f1894 [HUDI-7605] Allow merger strategy to be set in spark sql
writer (#10999)
omit 67f6477b495 [HUDI-6441] Passing custom Headers with Hudi Callback URL
(#10970)
omit 4b338ff0ce9 [HUDI-7391] HoodieMetadataMetrics should use Metrics
instance for metrics registry (#10635)
omit 46374ff47d4 [HUDI-7600] Shutdown ExecutorService when
HiveMetastoreBasedLockProvider is closed (#10993)
omit 6ab10d71f3f [MINOR] Fix BUG: HoodieLogFormatWriter: unable to close
output stream for log file HoodieLogFile{xxx} (#10989)
omit eca574a9fda [HUDI-7597] Add logs of Kafka offsets when the checkpoint
is out of bound (#10987)
omit a2f043512ea [HUDI-7583] Read log block header only for the schema and
instant time (#10984)
omit d37bbed20fa [HUDI-7556] Fixing false positive validation with MDT
validator (#10986)
new 1ed67a94fd4 [HUDI-7556] Fixing false positive validation with MDT
validator (#10986)
new 92ee83c0c7c [HUDI-7583] Read log block header only for the schema and
instant time (#10984)
new a1d601c5866 [HUDI-7597] Add logs of Kafka offsets when the checkpoint
is out of bound (#10987)
new 271a22712c2 [MINOR] Fix BUG: HoodieLogFormatWriter: unable to close
output stream for log file HoodieLogFile{xxx} (#10989)
new 604c8fb7c81 [HUDI-7600] Shutdown ExecutorService when
HiveMetastoreBasedLockProvider is closed (#10993)
new 0caed098c35 [HUDI-7391] HoodieMetadataMetrics should use Metrics
instance for metrics registry (#10635)
new 804b88d212c [HUDI-6441] Passing custom Headers with Hudi Callback URL
(#10970)
new ca6c4f96dcc [HUDI-7605] Allow merger strategy to be set in spark sql
writer (#10999)
new 1013610ee8b [HUDI-7290] Don't assume ReplaceCommits are always
Clustering (#10479)
new 50caab4ee11 [HUDI-7601] Add heartbeat mechanism to refresh lock
(#10994)
new 3e6bb0d77bc [HUDI-7378] Fix Spark SQL DML with custom key generator
(#10615)
new b5b04af9d1e [HUDI-7616] Avoid multiple cleaner plans and deprecate
hoodie.clean.allow.multiple (#11013)
new a5745c6d550 [HUDI-7606] Unpersist RDDs after table services, mainly
compaction and clustering (#11000)
new ba521081ff6 [HUDI-7615] Mark a few write configs with the correct
sinceVersion (#11012)
new 3a9ca9ecd0a [HUDI-7584] Always read log block lazily and remove
readBlockLazily argument (#11015)
new 38701e37637 [HUDI-7619] Removed code duplicates in
HoodieTableMetadataUtil (#11022)
new 21a1b7806db [HUDI-6762] Removed usages of
MetadataRecordsGenerationParams (#10962)
new 862575a546b [MINOR] Remove redundant lines in StreamSync and
TestStreamSyncUnitTests (#11027)
new 6b0c67a5c9f [MINOR] Rename location to path in `makeQualified` (#11037)
new 9bca6153adc [HUDI-7578] Avoid unnecessary rewriting to improve
performance (#11028)
new 1c2e2b5341c [HUDI-7625] Avoid unnecessary rewrite for metadata table
(#11038)
new 6df621aec1f [HUDI-7626] Propagate UserGroupInformation from the main
thread to the new thread of timeline service threadpool (#11039)
new 74f91c70474 [HUDI-4228] Clean up literal usage in Hudi CLI argument
check (#11042)
new 07120a64e81 [HUDI-7633] Use try with resources for AutoCloseable
(#11045)
new 6238a9dd9a3 [MINOR] Remove redundant TestStringUtils in hudi-common
(#11046)
new 6ac417341eb [HUDI-7636] Make StoragePath Serializable (#11049)
new ab3741cd528 [HUDI-7635] Add default block size and openSeekable APIs
to HoodieStorage (#11048)
new b71f03776ba [HUDI-7637] Make StoragePathInfo Comparable (#11050)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (b0e0b148b32)
\
N -- N -- N refs/heads/branch-0.x (b71f03776ba)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 28 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../hudi/cli/commands/ArchivedCommitsCommand.java | 104 ++++++-----
.../apache/hudi/cli/commands/ExportCommand.java | 93 +++++-----
.../hudi/cli/commands/HoodieLogFileCommand.java | 104 +++++------
.../org/apache/hudi/cli/commands/SparkMain.java | 201 +++++++--------------
.../org/apache/hudi/cli/commands/TableCommand.java | 6 +-
.../java/org/apache/hudi/io/HoodieMergeHandle.java | 19 +-
.../hudi/io/HoodieMergeHandleWithChangeLog.java | 2 +-
.../apache/hudi/io/HoodieSortedMergeHandle.java | 4 +-
.../metadata/HoodieBackedTableMetadataWriter.java | 8 +-
.../FlinkMergeAndReplaceHandleWithChangeLog.java | 2 +-
.../hudi/io/FlinkMergeHandleWithChangeLog.java | 2 +-
.../java/org/apache/hudi/avro/HoodieAvroUtils.java | 4 +
.../java/org/apache/hudi/common/fs/FSUtils.java | 30 +--
.../hudi/common/model/HoodiePartitionMetadata.java | 8 +-
.../hudi/common/table/log/HoodieLogFileReader.java | 75 +-------
.../hudi/common/table/log/LogReaderUtils.java | 22 +--
.../table/log/block/HoodieAvroDataBlock.java | 135 +++++++-------
.../hudi/common/util/SerializationUtils.java | 6 +-
.../hudi/metadata/HoodieBackedTableMetadata.java | 24 +--
.../apache/hudi/common/util/TestStringUtils.java | 124 -------------
.../java/HoodieJavaWriteClientExample.java | 70 +++----
.../examples/spark/HoodieWriteClientExample.java | 90 ++++-----
.../org/apache/hudi/hadoop/fs/HadoopFSUtils.java | 90 +++++++++
.../hudi/storage/hadoop/HoodieHadoopStorage.java | 13 ++
.../org/apache/hudi/common/util/FileIOUtils.java | 14 +-
.../org/apache/hudi/storage/HoodieStorage.java | 30 +++
.../java/org/apache/hudi/storage/StoragePath.java | 14 +-
.../org/apache/hudi/storage/StoragePathInfo.java | 7 +-
.../hudi/io/storage/TestHoodieStorageBase.java | 43 +++++
.../apache/hudi/io/storage/TestStoragePath.java | 28 ++-
.../hudi/io/storage/TestStoragePathInfo.java | 13 ++
.../org/apache/hudi/cli/ArchiveExecutorUtils.java | 2 +-
.../hudi/timeline/service/RequestHandler.java | 128 +++++++------
.../hudi/utilities/HoodieCompactionAdminTool.java | 9 +-
.../utilities/HoodieMetadataTableValidator.java | 4 +-
.../utilities/streamer/SchedulerConfGenerator.java | 6 +-
.../TestHoodieMetadataTableValidator.java | 5 +-
37 files changed, 748 insertions(+), 791 deletions(-)
delete mode 100644
hudi-common/src/test/java/org/apache/hudi/common/util/TestStringUtils.java
