This is an automated email from the ASF dual-hosted git repository.
lzljs3620320 pushed a change to branch release-0.6
in repository https://gitbox.apache.org/repos/asf/incubator-paimon.git
from 19aaeff3c [core] Add "snapshot.expire.clean-empty-directories" to
determine whether to clean empty directories when expiring snapshots (#2522)
new ecdcd58e4 [core] Primary key table only max level can convert to raw
files (#2586)
new 55957987e [core] fix streaming read empty table in latest mode (#2589)
new a679273d1 [core][spark] Fix comment consistency in the schema (#2600)
new 4da02b8ae [flink] Fix mongo primary key field not objectId type (#2606)
new 23a6f0754 [NOTICE] Add 2024 on NOTICE (#2613)
new f14e7a458 [flink] Ignore PartitionNotExistException on drop if exists
partition statement (#2617)
new b96974975 [hive] Improve hive database is exists yet create/drop
database (#2595)
new 770334b3e [hive] Compatible with upper and lower case when use hivesql
queries paimon external table (#2639)
new 824954e42 [notice] Copyright 2023-2024
new 6fc6a5a73 [flink] Add paimon procedures in flink generic catalog
(#2651)
new 3d9686d76 [hive] Fix hive to be compatible with uppercase fields when
using predicates to query paimon external tables. (#2659)
new eb0d030b3 [flink] StaticFileStoreSplitEnumerator should ignore
ReaderConsumeProgressEvent (#2664)
new 87d3c14db [core] Don't delete file out of Snapshot expire (#2665)
new 8378a1777 [core] Fix index file checkFilesExistence in TableCommitImpl
(#2668)
new 99c2200e0 [parquet] Fix timestamp type write parquet file not have
logicaltype (#2558)
new b83b68053 [core] auto-create should be false when using create table
sql (#2538)
new d51417cf7 [core] Optimize snapshot expire loop for consumers (#2546)
new 05a2836e9 [Bug] Fix the IndexOutOfBoundsException when lookup empty
sortedRun (#2530)
new 0e609d70d [core] Fix creation failure caused by savepoint tag (#2515)
new bdb206807 [core] Use isAutoTag in TagAutoCreation
The 20 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
NOTICE | 2 +-
paimon-codegen/src/main/resources/META-INF/NOTICE | 2 +-
paimon-common/src/main/resources/META-INF/NOTICE | 2 +-
.../org/apache/paimon/catalog/AbstractCatalog.java | 19 +++-
.../apache/paimon/catalog/FileSystemCatalog.java | 4 +-
.../apache/paimon/manifest/IndexManifestFile.java | 4 -
.../org/apache/paimon/mergetree/LookupUtils.java | 3 +
.../paimon/operation/FileStoreExpireImpl.java | 39 ++++----
.../main/java/org/apache/paimon/schema/Schema.java | 2 +-
.../org/apache/paimon/schema/SchemaChange.java | 38 ++++++++
.../org/apache/paimon/schema/SchemaManager.java | 7 +-
.../org/apache/paimon/schema/SchemaSerializer.java | 5 +-
.../java/org/apache/paimon/schema/TableSchema.java | 10 +-
.../apache/paimon/table/sink/TableCommitImpl.java | 4 +-
.../snapshot/ContinuousLatestStartingScanner.java | 12 ++-
.../table/source/snapshot/SnapshotReaderImpl.java | 11 +--
.../org/apache/paimon/tag/TagAutoCreation.java | 13 ++-
.../java/org/apache/paimon/utils/TagManager.java | 26 +++++-
.../org/apache/paimon/catalog/CatalogTestBase.java | 80 +++++++++++++++-
.../apache/paimon/mergetree/LookupLevelsTest.java | 16 ++++
.../paimon/operation/FileStoreExpireTestBase.java | 2 +-
.../apache/paimon/table/sink/TableCommitTest.java | 22 ++++-
.../table/source/snapshot/SnapshotReaderTest.java | 13 +--
.../org/apache/paimon/tag/TagAutoCreationTest.java | 31 +++++++
.../src/main/resources/META-INF/NOTICE | 2 +-
.../src/main/resources/META-INF/NOTICE | 2 +-
.../src/main/resources/META-INF/NOTICE | 2 +-
.../cdc/mongodb/strategy/MongoVersionStrategy.java | 4 +-
.../cdc/mongodb/MongoDBSyncTableActionITCase.java | 37 +++++++-
.../mongodb/table/defaultid/defaultId-1.js | 8 +-
.../defaultid/{defaultId-1.js => defaultId-2.js} | 2 +-
.../java/org/apache/paimon/flink/FlinkCatalog.java | 4 +-
.../apache/paimon/flink/FlinkGenericCatalog.java | 24 +++++
.../source/StaticFileStoreSplitEnumerator.java | 6 ++
.../apache/paimon/flink/CatalogTableITCase.java | 5 +
.../paimon/flink/ContinuousFileStoreITCase.java | 17 ++++
.../flink/SchemaChangeSerializationTest.java | 1 +
.../format/parquet/ParquetSchemaConverter.java | 23 +++--
paimon-format/src/main/resources/META-INF/NOTICE | 2 +-
.../java/org/apache/paimon/hive/HiveCatalog.java | 51 +++++++----
.../hive/SearchArgumentToPredicateConverter.java | 20 ++--
.../paimon/hive/mapred/PaimonRecordReader.java | 19 +++-
.../org/apache/paimon/hive/utils/HiveUtils.java | 6 +-
.../paimon/hive/FlinkGenericCatalogITCase.java | 8 ++
.../org/apache/paimon/hive/HiveReadITCase.java | 102 +++++++++++++++++++++
.../org/apache/paimon/hive/HiveWriteITCase.java | 28 +-----
.../java/org/apache/paimon/spark/SparkCatalog.java | 19 +++-
.../java/org/apache/paimon/spark/SparkTable.java | 3 +
.../paimon/spark/sql/DescribeTableTest.scala | 83 +++++++++++++++++
49 files changed, 688 insertions(+), 157 deletions(-)
copy
paimon-flink/paimon-flink-cdc/src/test/resources/mongodb/table/defaultid/{defaultId-1.js
=> defaultId-2.js} (96%)
create mode 100644
paimon-hive/paimon-hive-connector-common/src/test/java/org/apache/paimon/hive/HiveReadITCase.java
create mode 100644
paimon-spark/paimon-spark-common/src/test/scala/org/apache/paimon/spark/sql/DescribeTableTest.scala
