This is an automated email from the ASF dual-hosted git repository.
dependabot[bot] pushed a change to branch
dependabot/github_actions/codeql-action-9d88a12d2b
in repository https://gitbox.apache.org/repos/asf/iceberg.git
omit d83e370aea Build: Bump the codeql-action group across 1 directory with
3 updates
add 0b31766290 Kafka Connect: Preserve explicit null values instead of
replacing them with schema defaults (#17653)
add 32f0914f9a Build: Bump joda-time:joda-time from 2.14.4 to 2.15.0
(#18451)
add c4e71162dd Build: Bump software.amazon.awssdk:bom from 2.55.1 to
2.55.11 (#18452)
add 87fed79bf0 Build: Bump guava from 33.7.1-jre to 33.7.2-jre (#18453)
add 3d13e4e26c Build: Bump gradle/actions/setup-gradle from 6.3.0 to 6.4.0
(#18454)
add 0cd2faaf45 Build: Bump org.xerial.snappy:snappy-java from 1.1.10.8 to
1.1.10.10 (#18455)
add ef7171b69a Build: Bump com.aliyun:credentials-java from 1.0.3 to 1.0.8
(#18457)
add 6dca1cd504 Build: Bump com.google.cloud:libraries-bom from 26.89.0 to
26.90.0 (#18461)
add 72c18ed57a Core: Accept manifest format version >= 4 in manifest info
(#18436)
add eee5457984 BigQuery: Deduplicate retry handling in
BigQueryMetastoreClientImpl (#17904)
add b2dac7b3d7 Spark 4.1: Add MergingSortedRowDataReader for k-way merge
of sorted files (#14948)
add 3869163581 Spark: Don't reapply stream-from-timestamp when a stream
resumes (#18348)
add 561a4e326f Kafka Connect: Split route field path once instead of per
record (#16655)
add b505f93581 Build: Bump Parquet 1.18.1 (#18347)
add 256f542d3e Core, Flink: Refactor InputFilesDecryptor and use
EncryptingFileIO.bulkDecrypt() (#18198)
add 0f070dee3a GCP: Support CMEK (kms-key-name) for GCS server-side
encryption (#18151)
add 9961f9609a Spec: require monotonically increasing snapshot
timestamp-ms in v4 (#16294)
add cd3ae3c805 Core: Sweep only the transaction's own snapshots after
commit (#18095)
add 843d36de69 Parquet: Prune nested structs to the projected subtree
(#18336)
add 9747b14056 Spark: Backport: Don't reapply stream-from-timestamp when a
stream resumes (#18348) (#18469)
add db43b88a6c Nessie: Don't log catalog options when warehouse is unset
(#18474)
add c411138af5 Build: Remove stale parquet-jackson CVEs from
kafka-connect-runtime trivyignore (#18483)
add 8b08b9ea73 Docs: Fix broken links in docs and site (#18460)
add 2234c549de Spec: Use relative links between spec documents and fix
broken links (#18441)
add c131e75958 Core: Fix file cleanup logic to retain Puffin files shared
by live deletion vectors (#18408)
add cf6e703804 Docs: Track spec versions and changes since adoption in the
specs (#18379)
add 1e39a61f26 Spark 4.2: Fix rewrite failure on tables created with a
sort order (#18493) (#17814)
add fd3a81b15c Spec: V4 Adaptive Metadata Tree Spec Changes for Entry
Structures (#16025)
add 53a86256e2 Spark 4.2: Support CREATE TABLE ... LIKE ... (#17954)
add 0bfdb323c3 Spark 4.2: Implement SupportsSchemaEvolution for automatic
schema evolution (#17957)
add c3239b8622 Build: Print failed test summary after suite completes
(#16537)
add f324b54920 Spark 4.2: Apply session snapshot properties to maintenance
actions (#18406)
add 39e35d1a87 GCP: Run integration tests as part of gradle check (#18462)
add 9120cfa789 Kafka Connect: Update runtime distribution LICENSE and
NOTICE (#18019)
add acee558a84 Spark 4.0, 4.1: Backport: Apply session snapshot properties
to actions and procedures (#18524)
add 417985375c Core: Fix ArrayIndexOutOfBoundsException in
RewriteTablePathUtil (#17629)
add 2cf68d9515 Flink: Deep copy RowData in
DynamicRecordInternalSerializer#copy (#18497)
add 524fe9ef1b Flink: Remap equality field ids on table update (#18507)
add 06ec3c02fe Build: Bump the codeql-action group across 1 directory with
3 updates
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (d83e370aea)
\
N -- N -- N
refs/heads/dependabot/github_actions/codeql-action-9d88a12d2b (06ec3c02fe)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../trivyignore/kafka-connect-runtime.trivyignore | 18 +-
.github/workflows/api-binary-compatibility.yml | 2 +-
.github/workflows/codeql.yml | 4 +-
.github/workflows/cve-scan.yml | 4 +-
.github/workflows/delta-conversion-ci.yml | 4 +-
.github/workflows/flink-ci.yml | 2 +-
.github/workflows/hive-ci.yml | 2 +-
.github/workflows/java-ci.yml | 8 +-
.github/workflows/jmh-benchmarks.yml | 2 +-
.github/workflows/kafka-connect-ci.yml | 2 +-
.../publish-iceberg-rest-fixture-docker.yml | 2 +-
.github/workflows/publish-snapshot.yml | 2 +-
.github/workflows/recurring-jmh-benchmarks.yml | 2 +-
.github/workflows/source-release-ci.yml | 2 +-
.github/workflows/spark-ci.yml | 2 +-
.../gcp/bigquery/BigQueryMetastoreClientImpl.java | 110 +-
build.gradle | 23 +
.../java/org/apache/iceberg/BaseTransaction.java | 5 +-
.../org/apache/iceberg/IncrementalFileCleanup.java | 65 +-
.../org/apache/iceberg/ManifestInfoStruct.java | 2 +-
.../org/apache/iceberg/RewriteTablePathUtil.java | 2 +-
.../iceberg/encryption/InputFilesDecryptor.java | 81 +-
.../org/apache/iceberg/TestManifestInfoStruct.java | 6 +-
.../org/apache/iceberg/TestRemoveSnapshots.java | 83 +
.../apache/iceberg/TestRewriteTablePathUtil.java | 89 +
.../java/org/apache/iceberg/TestTransaction.java | 111 ++
.../encryption/TestInputFilesDecryptor.java | 180 +++
.../iceberg/data/TestMetricsRowGroupFilter.java | 12 +-
docs/docs/aws.md | 2 +-
docs/docs/flink.md | 2 +-
docs/docs/hive.md | 2 +-
docs/docs/kafka-connect.md | 30 +-
docs/docs/nessie.md | 4 +-
docs/docs/spark-configuration.md | 17 +-
docs/docs/spark-ddl.md | 15 +-
flink/v1.20/flink-runtime/runtime-deps.txt | 16 +-
.../apache/iceberg/flink/source/DataIterator.java | 4 +-
.../flink/source/RowDataFileScanTaskReader.java | 4 +-
.../operator/TestDataFileRewriteRunner.java | 2 +-
flink/v2.2/flink-runtime/runtime-deps.txt | 16 +-
.../apache/iceberg/flink/source/DataIterator.java | 4 +-
.../flink/source/RowDataFileScanTaskReader.java | 4 +-
.../operator/TestDataFileRewriteRunner.java | 2 +-
flink/v2.3/flink-runtime/runtime-deps.txt | 16 +-
.../dynamic/DynamicRecordInternalSerializer.java | 8 +-
.../sink/dynamic/DynamicTableUpdateOperator.java | 26 +-
.../apache/iceberg/flink/source/DataIterator.java | 4 +-
.../flink/source/RowDataFileScanTaskReader.java | 4 +-
.../operator/TestDataFileRewriteRunner.java | 2 +-
.../flink/sink/dynamic/TestDynamicIcebergSink.java | 178 +-
.../TestDynamicRecordInternalSerializer.java | 68 +
.../dynamic/TestDynamicTableUpdateOperator.java | 38 +
format/expressions-spec.md | 6 +-
format/spec.md | 344 +++-
format/udf-spec.md | 4 +-
gcp-bundle/runtime-deps.txt | 58 +-
.../org/apache/iceberg/gcp/gcs/TestGcsFileIO.java | 5 +
.../java/org/apache/iceberg/gcp/GCPProperties.java | 18 +
.../apache/iceberg/gcp/gcs/GCSOutputStream.java | 3 +
.../org/apache/iceberg/gcp/TestGCPProperties.java | 29 +
.../iceberg/gcp/gcs/TestGCSOutputStream.java | 29 +-
gradle/libs.versions.toml | 12 +-
kafka-connect/build.gradle | 40 +-
kafka-connect/kafka-connect-runtime/main/LICENSE | 1708 ++++++++++++++++++--
kafka-connect/kafka-connect-runtime/main/NOTICE | 754 ++++++++-
.../kafka-connect-runtime/runtime-deps.txt | 117 +-
.../iceberg/connect/transforms/CopyValue.java | 5 +-
.../connect/transforms/DebeziumTransform.java | 3 +-
.../connect/transforms/KafkaMetadataTransform.java | 3 +-
.../connect/transforms/MongoDebeziumTransform.java | 3 +-
.../iceberg/connect/transforms/TestCopyValue.java | 29 +
.../connect/transforms/TestDebeziumTransform.java | 22 +-
.../transforms/TestKafkaMetadataTransform.java | 31 +
.../transforms/TestMongoDebeziumTransform.java | 33 +
.../apache/iceberg/connect/IcebergSinkConfig.java | 12 +
.../iceberg/connect/data/RecordConverter.java | 37 +-
.../apache/iceberg/connect/data/RecordUtils.java | 20 +-
.../apache/iceberg/connect/data/SinkWriter.java | 19 +-
.../iceberg/connect/TestIcebergSinkConfig.java | 1 +
.../iceberg/connect/data/TestRecordConverter.java | 173 ++
.../iceberg/connect/data/TestRecordUtils.java | 90 +-
.../iceberg/connect/data/TestSinkWriter.java | 110 ++
.../org/apache/iceberg/nessie/NessieCatalog.java | 5 +-
.../iceberg/nessie/TestCustomNessieClient.java | 17 +
.../apache/iceberg/parquet/ParquetSchemaUtil.java | 10 +
.../org/apache/iceberg/parquet/PruneColumns.java | 12 +-
.../apache/iceberg/parquet/TestPruneColumns.java | 376 +++++
.../apache/iceberg/parquet/TestVariantMetrics.java | 38 +-
site/docs/hive-quickstart.md | 2 +-
site/docs/integrations/risingwave.md | 2 +-
site/docs/multi-engine-support.md | 2 +-
site/docs/releases.md | 77 +-
site/docs/spark-quickstart.md | 2 +-
site/docs/vendors.md | 6 +-
site/overrides/home.html | 2 +-
spark/v3.5/spark-runtime/runtime-deps.txt | 16 +-
.../spark/source/AsyncSparkMicroBatchPlanner.java | 4 +-
.../spark/source/BaseSparkMicroBatchPlanner.java | 12 +
.../spark/source/SparkMicroBatchPlanner.java | 9 +
.../spark/source/SparkMicroBatchStream.java | 19 +-
.../spark/source/SyncSparkMicroBatchPlanner.java | 8 +-
.../spark/source/TestStructuredStreamingRead3.java | 249 +++
.../extensions/TestRewriteManifestsProcedure.java | 17 +
spark/v4.0/spark-runtime/runtime-deps.txt | 16 +-
.../java/org/apache/iceberg/spark/SparkUtil.java | 14 +
.../org/apache/iceberg/spark/SparkWriteConf.java | 6 +-
.../actions/BaseSnapshotUpdateSparkAction.java | 8 +
.../actions/RemoveDanglingDeletesSparkAction.java | 6 +-
.../spark/actions/RewriteDataFilesSparkAction.java | 4 +-
.../apache/iceberg/spark/actions/SparkActions.java | 3 +-
.../spark/source/AsyncSparkMicroBatchPlanner.java | 4 +-
.../spark/source/BaseSparkMicroBatchPlanner.java | 12 +
.../spark/source/SparkMicroBatchPlanner.java | 9 +
.../spark/source/SparkMicroBatchStream.java | 19 +-
.../spark/source/SyncSparkMicroBatchPlanner.java | 8 +-
.../actions/TestRemoveDanglingDeleteAction.java | 3 +-
.../spark/actions/TestRewriteDataFilesAction.java | 47 +
.../spark/actions/TestRewriteManifestsAction.java | 26 +
.../spark/source/TestStructuredStreamingRead3.java | 249 +++
.../extensions/TestRewriteManifestsProcedure.java | 17 +
spark/v4.1/spark-runtime/runtime-deps.txt | 16 +-
.../java/org/apache/iceberg/spark/SparkUtil.java | 14 +
.../org/apache/iceberg/spark/SparkWriteConf.java | 6 +-
.../actions/BaseSnapshotUpdateSparkAction.java | 8 +
.../actions/RemoveDanglingDeletesSparkAction.java | 6 +-
.../spark/actions/RewriteDataFilesSparkAction.java | 4 +-
.../apache/iceberg/spark/actions/SparkActions.java | 3 +-
.../spark/source/AsyncSparkMicroBatchPlanner.java | 4 +-
.../spark/source/BaseSparkMicroBatchPlanner.java | 12 +
.../spark/source/MergingSortedRowDataReader.java | 365 +++++
.../spark/source/SparkMicroBatchPlanner.java | 9 +
.../spark/source/SparkMicroBatchStream.java | 19 +-
.../spark/source/SyncSparkMicroBatchPlanner.java | 8 +-
.../actions/TestRemoveDanglingDeleteAction.java | 3 +-
.../spark/actions/TestRewriteDataFilesAction.java | 47 +
.../spark/actions/TestRewriteManifestsAction.java | 26 +
.../source/TestMergingSortedRowDataReader.java | 1085 +++++++++++++
.../spark/source/TestStructuredStreamingRead3.java | 249 +++
.../spark/extensions/TestMergeSchemaEvolution.java | 30 +
.../extensions/TestRewriteManifestsProcedure.java | 17 +
spark/v4.2/spark-runtime/runtime-deps.txt | 16 +-
.../org/apache/iceberg/spark/SparkCatalog.java | 139 +-
.../apache/iceberg/spark/SparkSessionCatalog.java | 28 +
.../java/org/apache/iceberg/spark/SparkUtil.java | 14 +
.../org/apache/iceberg/spark/SparkWriteConf.java | 11 +-
.../actions/BaseSnapshotUpdateSparkAction.java | 8 +
.../actions/RemoveDanglingDeletesSparkAction.java | 6 +-
.../spark/actions/RewriteDataFilesSparkAction.java | 4 +-
.../apache/iceberg/spark/actions/SparkActions.java | 3 +-
.../spark/source/AsyncSparkMicroBatchPlanner.java | 4 +-
.../spark/source/BaseSparkMicroBatchPlanner.java | 12 +
.../spark/source/SparkMicroBatchPlanner.java | 9 +
.../spark/source/SparkMicroBatchStream.java | 19 +-
.../iceberg/spark/source/SparkSchemaEvolution.java | 170 ++
.../apache/iceberg/spark/source/SparkTable.java | 20 +-
.../spark/source/SyncSparkMicroBatchPlanner.java | 8 +-
.../iceberg/spark/TestSparkSessionCatalog.java | 129 ++
.../apache/iceberg/spark/TestSparkWriteConf.java | 26 +
.../actions/TestRemoveDanglingDeleteAction.java | 3 +-
.../spark/actions/TestRewriteDataFilesAction.java | 78 +
.../spark/actions/TestRewriteManifestsAction.java | 26 +
.../iceberg/spark/source/TestSparkTable.java | 243 +++
.../spark/source/TestStructuredStreamingRead3.java | 249 +++
.../apache/iceberg/spark/sql/TestCreateTable.java | 324 ++++
.../spark/sql/TestInsertSchemaEvolution.java | 178 ++
165 files changed, 9094 insertions(+), 913 deletions(-)
create mode 100644
core/src/test/java/org/apache/iceberg/encryption/TestInputFilesDecryptor.java
create mode 100644
spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/MergingSortedRowDataReader.java
create mode 100644
spark/v4.1/spark/src/test/java/org/apache/iceberg/spark/source/TestMergingSortedRowDataReader.java
create mode 100644
spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/source/SparkSchemaEvolution.java
create mode 100644
spark/v4.2/spark/src/test/java/org/apache/iceberg/spark/sql/TestInsertSchemaEvolution.java
