This is an automated email from the ASF dual-hosted git repository.
satish pushed a change to branch release-0.12.2
in repository https://gitbox.apache.org/repos/asf/hudi.git
at 1d47f95024 [HUDI-5283] Replace deprecated method Schema.parse with
Schema.Parser (#7308)
This branch includes the following new commits:
new eaa24f2193 [HUDI-4769] Option read.streaming.skip_compaction skips
delta commit (#6848)
new 923a85ad95 [HUDI-4966] Add a partition extractor to handle partition
values with slashes (#6851)
new abd1f69725 [MINOR] Fix testUpdateRejectForClustering (#6852)
new 3b6b0c9532 [HUDI-4962] Move cloud dependencies to cloud modules (#6846)
new 0a3665346c [HUDI-4966] Add a partition extractor to handle partition
values with slashes (#6851)
new 2fb2df9441 [MINOR] Fix testUpdateRejectForClustering (#6852)
new 39e99d1752 [HUDI-4962] Move cloud dependencies to cloud modules (#6846)
new 8bbac01a38 [HOTFIX] Fix source release validate script (#6865)
new 0e75565e20 [HUDI-4980] Calculate avg record size using commit only
(#6864)
new 978a5dd238 shade protobuf dependency
new 7949af812f Revert "[HUDI-4915] improve avro serializer/deserializer
(#6788)" (#6809)
new 77d62f7d3f [HUDI-4970] Update kafka-connect readme and refactor
HoodieConfig#create (#6857)
new 45f08d50db Enhancing README for multi-writer tests (#6870)
new f00c4e17bc [MINOR] Fix deploy script for flink 1.15 (#6872)
new 08aa68915d Revert "shade protobuf dependency"
new 95839af584 [HUDI-4972] Fixes to make unit tests work on m1 mac (#6751)
new 4f97952fd6 [HUDI-2786] Docker demo on mac aarch64 (#6859)
new 6db4594117 [HUDI-4971] Fix shading kryo-shaded with reusing configs
(#6873)
new 012f5b8278 Relocate apache http package (#6874)
new 73c1ee2a16 [HUDI-4975] Fix datahub bundle dependency (#6896)
new c5f965b647 [HUDI-4999] Refactor FlinkOptions#allOptions and
CatalogOptions#allOptions (#6901)
new 7e77e7b6e6 [MINOR] Update GitHub setting for merge button (#6922)
new 6260a8b793 [HUDI-4993] Make DataPlatform name and Dataset env
configurable in DatahubSyncTool (#6885)
new 22f9853150 [HUDI-4754] Add compliance check in github actions (#6575)
new 6b5ad3b1bb [HUDI-4994] Fix bug that prevents re-ingestion of
soft-deleted Datahub entities (#6886)
new c8d20c64d5 [MINOR] Moved readme from .github to the workflows folder
(#6932)
new c8aef6305f [HUDI-4952] Fixing reading from metadata table when there
are no inflight commits (#6836)
new ee1557b3bd [HUDI-5006] Use the same wrapper for timestamp type
metadata for parquet and log files (#6918)
new 6048c4d0a5 [HUDI-5016] Flink clustering does not reserve commit
metadata (#6929)
new 56fcf51a46 [HUDI-3900] Fixing hdfs setup and tear down in tests to
avoid flakiness (#6912)
new 8b1d5c2797 [HUDI-5002] Remove deprecated API usage in
SparkHoodieHBaseIndex#generateStatement (#6909)
new a9723257b8 [HUDI-5010] Fix flink hive catalog external config not work
(#6923)
new 267862336a [HUDI-5030] Fix
TestPartialUpdateAvroPayload.testUseLatestRecordMetaValue(#6948)
new beaa41f771 [HUDI-5033] Fix Broken Link In
MultipleSparkJobExecutionStrategy (#6951)
new a8ea6bb447 [HUDI-5037] Upgrade org.apache.thrift:libthrift to 0.14.0
(#6941)
new c226947bb6 [MINOR] Fixing verbosity of docker set up (#6944)
new 2d964fdd8e [HUDI-5022] Make better error messages for pr compliance
(#6934)
new 64a3631bdb [HUDI-5003] Fix the type of InLineFileSystem`startOffset to
long (#6916)
new 9ccc6c361d [HUDI-4855] Add missing table configs for bootstrap in
Deltastreamer (#6694)
new 59082ce797 [MINOR] Handling null event time (#6876)
new 0b059ff557 [MINOR] Update DOAP with 0.12.1 Release (#6988)
new 5012c882e5 [MINOR] Increase maxParameters size in scalastyle (#6987)
new 07bbc68107 [HUDI-3900] Closing resources in TestHoodieLogRecord (#6995)
new af2f7e0044 [MINOR] Test case for
hoodie.merge.allow.duplicate.on.inserts (#6949)
new cca41eea02 [HUDI-4982] Add validation job for spark bundles in GitHub
Actions (#6954)
new 857877bafe [HUDI-5041] Fix lock metric register confict error (#6968)
new 3e84320c17 [HUDI-4998] Infer partition extractor class first from meta
sync partition fields (#6899)
new bcf257883f [HUDI-4781] Allow omit metadata fields for hive sync (#6471)
new f662d81bcc [HUDI-4997] Use jackson-v2 import instead of jackson-v1
(#6893)
new bc3ce82115 [HUDI-3900] Fixing tempDir usage in TestHoodieLogFormat
(#6981)
new 126bb81c5e [HUDI-4995] Relocate httpcomponents (#6906)
new 632cab9c1e [MINOR] Update GitHub setting for branch protection (#7008)
new dac2f38edc [HUDI-4960] Upgrade jetty version for timeline server
(#6844)
new 320e5131ce [HUDI-5046] Support all the hive sync options for flink sql
(#6985)
new a0533743d0 [MINOR] Remove redundant space in PR compliance check
(#7022)
new e413545f52 [HUDI-5063] Enabling run time stats to be serialized with
commit metadata (#7006)
new d19df2a683 [HUDI-5070] Adding lock provider to testCleaner tests since
async cleaning is invoked (#7023)
new 4edfa1d440 [HUDI-5070] Move flaky cleaner tests to separate class
(#7034)
new effbc0cf5b [HUDI-4971] Remove direct use of kryo from `SerDeUtils`
(#7014)
new b9483b3497 [HUDI-5081] Tests clean up in hudi-utilities (#7033)
new 29efe2e559 [HUDI-5027] Replace hardcoded hbase config keys with
constant variables (#6946)
new 052037cbe8 [MINOR] add commit_action output in show_commits (#7012)
new 06f99f0f40 [HUDI-5061] bulk insert operation don't throw other
exception except IOE Exception (#7001)
new 3ace87f3b1 [MINOR] Skip loading last completed txn for single writer
(#6660)
new 316ce4e445 [HUDI-4281] Using hudi to build a large number of tables in
spark on hive causes OOM (#5903)
new aeca0457df [HUDI-5042] Fix clustering schedule problem in flink when
enable schedule clustering and disable async clustering (#6976)
new 41c6de142d [HUDI-4753] more accurate record size estimation for log
writing and spillable map (#6632)
new aeec3da008 [HUDI-4201] Cli tool to get warned about empty
non-completed instants from timeline (#6867)
new 8657b21076 [HUDI-5038] Increase default num_instants to fetch for
incremental source (#6955)
new 40446b3911 [HUDI-5057] Fix msck repair hudi table (#6999)
new bcf6a732f2 [HUDI-4959] Fixing Avro's `Utf8` serialization in Kryo
(#7024)
new ef8ea9dac6 temp_view_support (#6990)
new 3afd9aedb4 [HUDI-4982] Add Utilities and Utilities Slim + Spark Bundle
testing to GH Actions (#7005)
new 431f3bbdd3 [HUDI-5085]When a flink job has multiple sink tables, the
index loading status is abnormal (#7051)
new 489545b0d2 [HUDI-5089] Refactor HoodieCommitMetadata deserialization
(#7055)
new 56316944d1 [HUDI-5058] Fix flink catalog read spark table error :
primary key col can not be nullable (#7009)
new 9ff798c0e5 [HUDI-5087] Fix incorrect merging sequence for Column Stats
Record in `HoodieMetadataPayload` (#7053)
new eed21378ad [HUDI-4946] fix merge into with no preCombineField having
dup row by only insert (#6824)
new c1261f6961 [HUDI-5072] Extract `ExecutionStrategy#transform` duplicate
code (#7030)
new 7fed00d47e [HUDI-3287] Remove hudi-spark dependencies from
hudi-kafka-connect-bundle (#6079)
new cc6c3f892a [HUDI-4716] Avoid parquet-hadoop-bundle in hudi-hadoop-mr
(#6930)
new 2cd4b5303c [HUDI-5035] Remove usage of deprecated HoodieTimer
constructor (#6952)
new 01806ae202 [HUDI-5083]Fixed a bug when schema evolution (#7045)
new 8c5c634b9e [HUDI-5102] source operator(monitor and reader) support
user uid (#7085)
new 0b3836511c [HUDI-5057] Fix msck repair external hudi table (#7084)
new 1cdddd2163 [MINOR] Fix typos in Spark client related classes (#7083)
new 3db0981a7f [HUDI-4741] hotfix to avoid partial failover cause restored
subtask timeout (#6796)
new 2dcbe5ca16 [MINOR] use default maven version since it already fix the
warnings recently (#6863)
new f107e2d161 Revert "[HUDI-4741] hotfix to avoid partial failover cause
restored subtask timeout (#6796)" (#7090)
new a707f011a6 [MINOR] Fix doc of
org.apache.hudi.sink.meta.CkpMetadata#bootstrap (#7048)
new c796022a9a [HUDI-4799] improve analyzer exception tip when cannot
resolve expression (#6625)
new 7976b93c0c [HUDI-5096] Upgrade jcommander to 1.78 (#7068)
new a2228f7943 [HUDI-5105] Add Call show_commit_extra_metadata for spark
sql (#7091)
new eb86d5d098 [HUDI-5107] Fix hadoop config in DirectWriteMarkers,
HoodieFlinkEngineContext and StreamerUtil are not consistent issue (#7094)
new 7bae457f0f [MINOR] Fix OverwriteWithLatestAvroPayload full class name
(#7096)
new 39089c1e88 [HUDI-5074] Warn if table for metastore sync has capitals
in it (#7077)
new 04fb5bf554 [HUDI-5124] Fix HoodieInternalRowFileWriter#canWrite error
return tag. (#7107)
new c93da976ec [MINOR] update commons-codec:commons-codec 1.4 to 1.13
(#6959)
new 036f10254e [HUDI-5065] Call close on SparkRDDWriteClient in
HoodieCleaner (#7101)
new fb979fa730 [HUDI-4624] Implement Closable for S3EventsSource (#7086)
new 25e308fe2f [HUDI-5045] Adding support to configure index type with
integ tests (#6982)
new 57a926ceaf [HUDI-5076] Fixing non serializable path used in
engineContext with metadata table intialization (#7036)
new df5ae63b38 [HUDI-5032] Add archive to cli (#7076)
new ec17fc46bc [HUDI-4880] Fix corrupted parquet file issue left over by
cancelled compaction task (#6733)
new 0a3ad91fc9 [HUDI-5147] Flink data skipping doesn't work when
HepPlanner calls copy()… (#7113)
new 6de89cd8f3 [MINOR] Fixing broken test (#7123)
new 6751939052 [HUDI-4898] presto/hive respect payload during merge
parquet file and logfile when reading mor table (#6741)
new 9a3d63fd1b [HUDI-5126] Delete duplicate configuration items
PAYLOAD_CLASS_NAME (#7103)
new b131dd32be [HUDI-4989] Fixing deltastreamer init failures (#6862)
new e7a9685f6d [MINOR] Fix flaky test in ITTestHoodieDataSource (#7134)
new 9f6adad0f3 [HUDI-4071] Remove default value for mandatory record key
field (#6681)
new eed88083f0 [HUDI-5088]Fix bug:Failed to synchronize the hive metadata
of the Flink table (#7056)
new 89246b7ac3 [MINOR] Removing spark2 scala12 combinations from readme
(#7112)
new 7ff52baa07 [HUDI-5066] Support flink hoodie source metaclient cache
(#7017)
new b7aacc439d [HUDI-5132] Add hadoop-mr bundle validation (#7157)
new 70a180d74e [HUDI-2673] Add kafka connect bundle to validation test
(#7131)
new db91f60f39 [HUDI-5154] Improve hudi-spark-client Lambada writing
(#7127)
new 4e93c39dfa [HUDI-5178] Add Call show_table_properties for spark sql
(#7161)
new 74c2270261 [HUDI-5067] Merge the columns stats of multiple log blocks
from the same log file (#7018)
new 53dc58ff82 [HUDI-5025] Rollback failed with log file not found when
rollOver in rollback process (#6939)
new d66a17f8f3 [HUDI-4526] Improve spillableMapBasePath when disk
directory is full (#6284)
new 0013571d49 [minor] Refactor the code for CkpMetadata (#7166)
new 390fddf850 [HUDI-5111] Improve integration test coverage (#7092)
new 0ae9ebfa33 [HUDI-5187] Remove the preCondition check of BucketAssigner
assign state (#7170)
new bebadf81df [HUDI-5145] Avoid starting HDFS in hudi-utilities tests
(#7171)
new 53f424663a [MINOR] Performance improvement of flink ITs with reused
miniCluster (#7151)
new da032e5390 [HUDI-5171] Ensure validateTableConfig also checks for
partition path field value switch (#7163)
new a882deb487 [HUDI-5056] Allow wildcards in partition paths for
DELETE_PARTITIONS (#7142)
new 84fd11debd [HUDI-4888] Throw exception if COW table and consistent
hashing bucket index (#7172)
new 8c45876cc9 [HUDI-5176] Fix incremental source to consider inflight
commits before completed commits (#7160)
new 0e27912b54 [HUDI-5184] Remove unnecessary line from pyspark example
(#7178)
new 1167ace18d [MINOR] Balance CI jobs (#6838)
new 5e5cc4e11f [HUDI-5185] Fix CLI run compaction failing with
--hoodieConfigs (#7168)
new cc379795be [HUDI-5198] Reduce test run time in hudi-utilities and
locking related tests (#7180)
new ceb21f2c7b [HUDI-5191] Fix compatibility with avro 1.10 (#7175)
new 24eee05790 [HUDI-4496] Fix Orc support broken for Spark 3.x and more
(#6227)
new 446a0c6330 [HUDI-5200] Clean up resources in hudi common UT (#7190)
new 51c97060bc [HUDI-5201] Add totalRecordsDeleted metric (#7181)
new b6effa7e4b [HUDI-5206] RowColumnReader should not return null value
for certain null child columns (#7194)
new 5d22039de3 [MINOR] Make sure Dictionary Encoding in Parquet enabled by
default (#7052)
new 0c0e4eb010 [HUDI-5221] Make the decision for flink sql bucket index
case-insensitive (#7207)
new 9acd94bc74 [HUDI-5223] Partial failover for flink (#7208)
new 8b2b036cb2 [HUDI-5228] Flink table service job fs view conf overwrites
the one of writing job (#7214)
new dfce5568a8 [HUDI-5227] Bump Javalin to 4.6.7 and Jetty to 9.4.48
(#7211)
new 1019f854df Use as.of.instant for IncrementalRelation (#6921)
new 6d90801bac [HUDI-5203] Handle null fields in debezium avro payloads
(#7193)
new 7aba4629fa [HUDI-5233] Fix bug when
InternalSchemaUtils.collectTypeChangedCols returns all columns (#7228)
new e606d08f13 [HUDI-5162] Allow user specified start offset for streaming
query (#7138)
new d143ca965e [HUDI-5070] Move flaky cleaner tests to separate class
(#7251)
new c9792b89bc [HUDI-5247] Clean up java client tests (#7250)
new 63668f9548 [HUDI-5237] Support for HoodieUnMergedLogRecordScanner with
InternalSchema (#7237)
new 321451d4fc [HUDI-5244] Fix bugs in schema evolution client with lost
operation field and not found schema (#7248)
new 123b8cd527 [MINOR] Fix `TestSchemaEvolutionClient` compilation (#7256)
new 2573f83123 [HUDI-712] Improve exporter file listing and copy perf
(#7267)
new 72e8b91639 [HUDI-5157] Support dropping all meta fields from source
hudi table with hudi incr source (#7132)
new 83c8c90edc [MINOR] Fix typos in HoodieTimelineArchiver (#7268)
new 83fbe16ddf [MINOR] Use direct marker for spark engine when timeline
server is disabled (#7272)
new b74a1028be [HUDI-5252] ClusteringCommitSink supports to rollback
clustering (#7263)
new 668fd0f25a [HUDI-5258] Fix checkstyle issues in hudi-common (#7270)
new 189aa8f63b [HUDI-5260] Fix insert into sql command with strict sql
insert mode (#7269)
new 29c63555c5 [HUDI-5234] Streaming read skip clustering (#7296)
new 89a4e9648f [HUDI-5277] Close HoodieWriteClient before exiting
RunClusteringProcedure (#7300)
new 1d47f95024 [HUDI-5283] Replace deprecated method Schema.parse with
Schema.Parser (#7308)
The 163 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
