This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/gradle/com.azure-azure-sdk-bom-1.2.27
in repository https://gitbox.apache.org/repos/asf/iceberg.git
discard ee936e304c Build: Bump com.azure:azure-sdk-bom from 1.2.25 to 1.2.27
add 153b070695 Spark 3.3, 3.4: Fix incorrect catalog loaded in
TestCreateActions (#11049)
add 41d00ae64d Core: Prevent incremental file cleanup when expiring
specified snapshots (#10983)
add 4873b4b753 Core, Kafka, Spark: Use AssertJ instead of JUnit assertions
(#11102)
add 026166f229 Docs: Add blogs written by rmoff (#11069)
add e40fe4cdbe Build: Upgrade to Gradle 8.10.1 (#11104)
add a5c8f9cd45 Build: Remove unused variables, fields and parameters
(#11101)
add 8d97d54756 OpenAPI: Add Scan Planning Endpoints to REST spec (#9695)
add 5439cbdb27 Kafka Connect: Docs on configuring the sink (#10746)
add 0747b60441 Kafka Connect: Terminate commits on coordinator stop
(#10814)
add 34cd01ba2e Build: Upgrade google-java-format to 1.22.0 (#11050)
add 6ff7a6ec2e Flink: Maintenance - Lock remover (#11010)
add 8b4b2c1975 Docs: Update Project links to includ contributing and REST
spec (#11114)
add ab0594bf71 Flink: Port #10484 to v1.19 (#11010) (#11117)
add e3d3f8845a OpenAPI: Fix YAML example and value json formatting (#11119)
add d2087a04bd Core: Parallelize manifest writing for many new files
(#11086)
add 799120636e API, Core: Add manifestLocation API to ContentFile (#11044)
add a2b8008da7 Core: Allow servers to express supported endpoints via
endpoint field in ConfigResponse (#10929)
add e449d3405c Hive: Add View support for HIVE catalog (#9852)
add 5582b0ca67 Spark 3.4: Action to compute table stats (#11106)
add 2e4d5b5b21 Docs: Fix missing options for remove_orphan_files procedure
(#11080)
add 5ce7c3091b Flink: Increase the number of checkpoints from 4 to 6 to
fix flakiness. (#11121)
add d5b21d82e3 Spark 3.4: Add utility to load table state reliably (#11115)
add 06ed235f97 Build: switch to slf4j-simple 2.x for test implementation
dependency because avro 1.12.0 brings in slf4j-api dependency to 2.x (#11001)
add f71c7dfb0f Core: Update metadata location without updating
lastUpdatedMillis (#11151)
add 40ffcb9ad4 Kafka Connect: separate CI workflow (#11075)
add bbeadea75e Core: Move internal struct projection to
SupportsIndexProjection (#11132)
add e3088bc098 Core: Add explicit JSON parser for LoadTableResponse
(#11148)
add ffa13b12eb Build: Add .java-version to gitignore (#11167)
add e5d9a1594b Docs: Clarify Partition Transform (#8337)
add 82cedbbb19 API, Core: Enable removing rewritten delete files in
RowDelta (#11166)
add 79fd977f67 Docs: `field_id` in name serialisation spec should read
`field-id` (#11135)
add 60f61c3dd0 AWS: Bump AWS SDK to version 2.28.5 (#11170)
add d4af40c9bc Build: Bump org.xerial.snappy:snappy-java from 1.1.10.6 to
1.1.10.7 (#11140)
add 2d3485e0db Build: Bump com.azure:azure-sdk-bom from 1.2.25 to 1.2.27
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (ee936e304c)
\
N -- N -- N
refs/heads/dependabot/gradle/com.azure-azure-sdk-bom-1.2.27 (2d3485e0db)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.baseline/checkstyle/checkstyle.xml | 5 +
.github/workflows/delta-conversion-ci.yml | 6 +-
.github/workflows/flink-ci.yml | 4 +-
.github/workflows/hive-ci.yml | 6 +-
.github/workflows/java-ci.yml | 3 +-
.../{flink-ci.yml => kafka-connect-ci.yml} | 15 +-
.github/workflows/publish-snapshot.yml | 2 +-
.github/workflows/spark-ci.yml | 4 +-
.gitignore | 3 +
.../main/java/org/apache/iceberg/ContentFile.java | 8 +
api/src/main/java/org/apache/iceberg/DataFile.java | 5 +-
api/src/main/java/org/apache/iceberg/RowDelta.java | 11 +
.../org/apache/iceberg/UpdatePartitionSpec.java | 2 +-
.../org/apache/iceberg/encryption/KmsClient.java | 4 +-
...eption.java => NoSuchIcebergViewException.java} | 8 +-
.../apache/iceberg/expressions/ExpressionUtil.java | 1 -
.../org/apache/iceberg/transforms/Transforms.java | 4 +-
.../org/apache/iceberg/transforms/Truncate.java | 4 +-
.../java/org/apache/iceberg/types/TypeUtil.java | 1 -
.../test/java/org/apache/iceberg/TestHelpers.java | 1 +
aws-bundle/LICENSE | 72 +-
aws-bundle/NOTICE | 76 +-
.../aws/TestAssumeRoleAwsClientFactory.java | 4 -
.../org/apache/iceberg/aws/glue/GlueTestBase.java | 8 +-
.../aws/glue/TestGlueCatalogCommitFailure.java | 3 +-
.../iceberg/aws/glue/TestGlueCatalogTable.java | 2 -
.../aws/lakeformation/LakeFormationTestBase.java | 8 +-
.../TestLakeFormationAwsClientFactory.java | 1 -
.../iceberg/aws/s3/TestS3FileIOIntegration.java | 7 +-
.../java/org/apache/iceberg/aws/AwsProperties.java | 5 -
.../apache/iceberg/aws/HttpClientProperties.java | 12 +
.../iceberg/aws/s3/S3FileIOAwsClientFactory.java | 1 +
.../org/apache/iceberg/aws/s3/TestS3FileIO.java | 3 +-
baseline.gradle | 17 +-
build.gradle | 2 +-
.../org/apache/iceberg/common/DynConstructors.java | 4 +-
.../java/org/apache/iceberg/common/DynMethods.java | 4 +-
.../java/org/apache/iceberg/AppendBenchmark.java | 13 +-
.../org/apache/iceberg/ManifestReadBenchmark.java | 3 +-
...hmark.java => ReplaceDeleteFilesBenchmark.java} | 53 +-
.../src/main/java/org/apache/iceberg/BaseFile.java | 109 ++-
.../main/java/org/apache/iceberg/BaseRowDelta.java | 6 +
.../java/org/apache/iceberg/CatalogProperties.java | 1 +
.../main/java/org/apache/iceberg/CatalogUtil.java | 18 +
.../main/java/org/apache/iceberg/FastAppend.java | 9 +-
.../java/org/apache/iceberg/GenericDataFile.java | 5 +
.../java/org/apache/iceberg/GenericDeleteFile.java | 5 +
.../org/apache/iceberg/GenericManifestEntry.java | 17 +-
.../org/apache/iceberg/GenericManifestFile.java | 65 +-
.../apache/iceberg/InheritableMetadataFactory.java | 11 +-
.../java/org/apache/iceberg/ManifestEntry.java | 1 +
.../org/apache/iceberg/ManifestFilterManager.java | 4 +-
.../java/org/apache/iceberg/ManifestReader.java | 4 +-
.../java/org/apache/iceberg/ManifestWriter.java | 3 +-
.../apache/iceberg/MergingSnapshotProducer.java | 73 +-
.../java/org/apache/iceberg/RemoveSnapshots.java | 11 +
.../java/org/apache/iceberg/SnapshotProducer.java | 124 +++
.../java/org/apache/iceberg/SystemConfigs.java | 4 +-
.../java/org/apache/iceberg/TableMetadata.java | 9 +
.../java/org/apache/iceberg/TableProperties.java | 16 +-
.../org/apache/iceberg/UpdateRequirements.java | 10 +-
.../main/java/org/apache/iceberg/V1Metadata.java | 5 +
.../main/java/org/apache/iceberg/V2Metadata.java | 5 +
.../main/java/org/apache/iceberg/V3Metadata.java | 5 +
.../apache/iceberg/actions/RewriteFileGroup.java | 2 +-
.../actions/RewritePositionDeletesGroup.java | 2 +-
.../iceberg/avro/SupportsIndexProjection.java | 85 ++
.../java/org/apache/iceberg/rest/Endpoint.java | 161 ++++
.../org/apache/iceberg/rest/RESTSerializers.java | 23 +-
.../apache/iceberg/rest/RESTSessionCatalog.java | 156 ++-
.../apache/iceberg/rest/RESTTableOperations.java | 13 +-
.../apache/iceberg/rest/RESTViewOperations.java | 11 +-
.../org/apache/iceberg/rest/ResourcePaths.java | 14 +
.../org/apache/iceberg/rest/auth/OAuth2Util.java | 2 -
.../iceberg/rest/responses/ConfigResponse.java | 29 +-
.../rest/responses/ConfigResponseParser.java | 16 +
.../iceberg/rest/responses/LoadTableResponse.java | 8 +-
...nseParser.java => LoadTableResponseParser.java} | 49 +-
.../org/apache/iceberg/util/ParallelIterable.java | 4 +-
.../org/apache/iceberg/DataTableScanTestBase.java | 65 +-
.../src/test/java/org/apache/iceberg/TestBase.java | 25 +-
.../java/org/apache/iceberg/TestFastAppend.java | 19 +
.../org/apache/iceberg/TestManifestReader.java | 31 +-
.../java/org/apache/iceberg/TestMergeAppend.java | 21 +
.../apache/iceberg/TestMetadataTableFilters.java | 4 +-
.../org/apache/iceberg/TestRemoveSnapshots.java | 25 +-
.../test/java/org/apache/iceberg/TestRowDelta.java | 147 +++
.../org/apache/iceberg/TestSnapshotProducer.java | 77 ++
.../java/org/apache/iceberg/TestTableMetadata.java | 59 +-
.../apache/iceberg/hadoop/HadoopFileIOTest.java | 6 +-
.../apache/iceberg/rest/RESTCatalogAdapter.java | 103 +-
.../java/org/apache/iceberg/rest/TestEndpoint.java | 108 +++
.../apache/iceberg/rest/TestRESTViewCatalog.java | 8 +-
.../TestRESTViewCatalogWithAssumedViewSupport.java | 95 ++
.../rest/responses/TestConfigResponseParser.java | 79 ++
.../responses/TestLoadTableResponseParser.java | 203 ++++
.../iceberg/dell/ecs/EcsSeekableInputStream.java | 1 +
dev/stage-binaries.sh | 3 +-
docs/docs/kafka-connect.md | 352 +++++++
docs/docs/partitioning.md | 2 +-
docs/docs/spark-procedures.md | 38 +
docs/mkdocs.yml | 1 +
.../org/apache/iceberg/flink/FlinkSchemaUtil.java | 4 +-
.../apache/iceberg/flink/data/StructRowData.java | 4 +-
.../sink/shuffle/AggregatedStatisticsTracker.java | 4 +-
.../sink/shuffle/RequestGlobalStatisticsEvent.java | 4 +-
.../apache/iceberg/flink/source/FlinkSource.java | 4 +-
.../apache/iceberg/flink/source/IcebergSource.java | 4 +-
.../iceberg/flink/source/RowDataRewriter.java | 6 -
.../enumerator/ContinuousIcebergEnumerator.java | 1 +
.../sink/shuffle/TestMapRangePartitioner.java | 4 +-
.../enumerator/TestContinuousSplitPlannerImpl.java | 4 +-
.../org/apache/iceberg/flink/FlinkSchemaUtil.java | 4 +-
.../apache/iceberg/flink/data/StructRowData.java | 4 +-
.../maintenance/operator/JdbcLockFactory.java | 4 +
.../flink/maintenance/operator/LockRemover.java | 144 +++
.../operator/TableMaintenanceMetrics.java | 5 +
.../flink/maintenance/operator/TaskResult.java} | 55 +-
.../flink/maintenance/operator/TriggerManager.java | 14 +-
.../sink/shuffle/AggregatedStatisticsTracker.java | 4 +-
.../sink/shuffle/RequestGlobalStatisticsEvent.java | 4 +-
.../apache/iceberg/flink/source/FlinkSource.java | 4 +-
.../apache/iceberg/flink/source/IcebergSource.java | 4 +-
.../iceberg/flink/source/RowDataRewriter.java | 6 -
.../enumerator/ContinuousIcebergEnumerator.java | 1 +
.../flink/maintenance/operator/ManualSource.java | 16 +-
.../maintenance/operator/TestLockRemover.java | 378 ++++++++
...FlinkIcebergSinkRangeDistributionBucketing.java | 2 +-
.../iceberg/flink/sink/TestIcebergCommitter.java | 2 +-
.../sink/shuffle/TestMapRangePartitioner.java | 4 +-
.../enumerator/TestContinuousSplitPlannerImpl.java | 4 +-
.../org/apache/iceberg/flink/FlinkSchemaUtil.java | 4 +-
.../apache/iceberg/flink/data/StructRowData.java | 4 +-
.../maintenance/operator/JdbcLockFactory.java | 4 +
.../flink/maintenance/operator/LockRemover.java | 144 +++
.../operator/TableMaintenanceMetrics.java | 5 +
.../operator/{Trigger.java => TaskResult.java} | 55 +-
.../flink/maintenance/operator/TriggerManager.java | 14 +-
.../sink/shuffle/AggregatedStatisticsTracker.java | 4 +-
.../sink/shuffle/RequestGlobalStatisticsEvent.java | 4 +-
.../apache/iceberg/flink/source/FlinkSource.java | 4 +-
.../apache/iceberg/flink/source/IcebergSource.java | 4 +-
.../iceberg/flink/source/RowDataRewriter.java | 6 -
.../enumerator/ContinuousIcebergEnumerator.java | 1 +
.../flink/maintenance/operator/ManualSource.java | 16 +-
.../maintenance/operator/TestLockRemover.java | 378 ++++++++
...FlinkIcebergSinkRangeDistributionBucketing.java | 2 +-
.../iceberg/flink/sink/TestIcebergCommitter.java | 2 +-
.../sink/shuffle/TestMapRangePartitioner.java | 4 +-
.../enumerator/TestContinuousSplitPlannerImpl.java | 4 +-
format/spec.md | 2 +-
.../java/org/apache/iceberg/gcp/GCPProperties.java | 1 +
gradle.properties | 2 +
gradle/libs.versions.toml | 4 +-
gradle/wrapper/gradle-wrapper.properties | 4 +-
gradlew | 2 +-
.../java/org/apache/iceberg/hive/HiveCatalog.java | 200 +++-
.../apache/iceberg/hive/HiveOperationsBase.java | 13 +
.../apache/iceberg/hive/HiveTableOperations.java | 6 +-
.../apache/iceberg/hive/HiveViewOperations.java | 389 ++++++++
.../org/apache/iceberg/hive/TestHiveCatalog.java | 15 +
.../apache/iceberg/hive/TestHiveViewCatalog.java | 323 +++++++
...stHiveCommits.java => TestHiveViewCommits.java} | 376 ++++----
.../org/apache/hadoop/hive/ql/io/orc/OrcSplit.java | 1 +
kafka-connect/build.gradle | 2 +-
.../iceberg/connect/IntegrationTestBase.java | 18 +-
.../org/apache/iceberg/connect/TestContext.java | 12 +-
.../iceberg/connect/channel/Coordinator.java | 19 +-
.../iceberg/connect/channel/CoordinatorThread.java | 10 +-
.../iceberg/connect/data/RecordConverter.java | 9 +-
.../iceberg/connect/channel/CoordinatorTest.java | 46 +-
.../main/java/org/apache/iceberg/mr/Catalogs.java | 10 +-
.../iceberg/mr/hive/HiveIcebergMetaHook.java | 4 +-
open-api/rest-catalog-open-api.py | 195 +++-
open-api/rest-catalog-open-api.yaml | 1014 +++++++++++++++-----
orc/src/main/java/org/apache/iceberg/orc/ORC.java | 4 +-
.../java/org/apache/iceberg/orc/ORCSchemaUtil.java | 1 +
.../org/apache/iceberg/orc/OrcFileAppender.java | 3 +
.../java/org/apache/iceberg/parquet/Parquet.java | 8 +-
settings.gradle | 34 +-
site/docs/blogs.md | 18 +
site/nav.yml | 4 +-
.../apache/iceberg/snowflake/SnowflakeCatalog.java | 1 +
.../extensions/TestSnapshotTableProcedure.java | 1 +
.../java/org/apache/iceberg/spark/SmokeTest.java | 2 +-
.../iceberg/spark/actions/TestCreateActions.java | 15 +
.../spark/source/TestIcebergSourceTablesBase.java | 9 +-
spark/v3.4/build.gradle | 2 +
.../extensions/TestSnapshotTableProcedure.java | 1 +
.../java/org/apache/iceberg/spark/SmokeTest.java | 2 +-
.../org/apache/iceberg/spark/SparkTableUtil.java | 19 +-
.../actions/ComputeTableStatsSparkAction.java | 0
.../iceberg/spark/actions/NDVSketchUtil.java | 0
.../apache/iceberg/spark/actions/SparkActions.java | 6 +
.../apache/spark/sql/stats/ThetaSketchAgg.scala | 0
.../spark/actions/TestComputeTableStatsAction.java | 208 ++--
.../iceberg/spark/actions/TestCreateActions.java | 15 +
.../spark/source/TestIcebergSourceTablesBase.java | 9 +-
.../extensions/TestSnapshotTableProcedure.java | 1 +
.../java/org/apache/iceberg/spark/SmokeTest.java | 2 +-
.../spark/actions/TestComputeTableStatsAction.java | 139 ++-
.../spark/source/TestIcebergSourceTablesBase.java | 9 +-
202 files changed, 6173 insertions(+), 1336 deletions(-)
copy .github/workflows/{flink-ci.yml => kafka-connect-ci.yml} (86%)
copy
api/src/main/java/org/apache/iceberg/exceptions/{NoSuchIcebergTableException.java
=> NoSuchIcebergViewException.java} (76%)
copy core/src/jmh/java/org/apache/iceberg/{AppendBenchmark.java =>
ReplaceDeleteFilesBenchmark.java} (69%)
create mode 100644
core/src/main/java/org/apache/iceberg/avro/SupportsIndexProjection.java
create mode 100644 core/src/main/java/org/apache/iceberg/rest/Endpoint.java
copy
core/src/main/java/org/apache/iceberg/rest/responses/{LoadViewResponseParser.java
=> LoadTableResponseParser.java} (55%)
create mode 100644
core/src/test/java/org/apache/iceberg/TestSnapshotProducer.java
create mode 100644 core/src/test/java/org/apache/iceberg/rest/TestEndpoint.java
create mode 100644
core/src/test/java/org/apache/iceberg/rest/TestRESTViewCatalogWithAssumedViewSupport.java
create mode 100644
core/src/test/java/org/apache/iceberg/rest/responses/TestLoadTableResponseParser.java
create mode 100644 docs/docs/kafka-connect.md
create mode 100644
flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/LockRemover.java
copy
flink/{v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/Trigger.java
=>
v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/TaskResult.java}
(52%)
create mode 100644
flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/maintenance/operator/TestLockRemover.java
create mode 100644
flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/LockRemover.java
copy
flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/{Trigger.java
=> TaskResult.java} (52%)
create mode 100644
flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/maintenance/operator/TestLockRemover.java
create mode 100644
hive-metastore/src/main/java/org/apache/iceberg/hive/HiveViewOperations.java
create mode 100644
hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveViewCatalog.java
copy
hive-metastore/src/test/java/org/apache/iceberg/hive/{TestHiveCommits.java =>
TestHiveViewCommits.java} (55%)
copy spark/{v3.5 =>
v3.4}/spark/src/main/java/org/apache/iceberg/spark/actions/ComputeTableStatsSparkAction.java
(100%)
copy spark/{v3.5 =>
v3.4}/spark/src/main/java/org/apache/iceberg/spark/actions/NDVSketchUtil.java
(100%)
copy spark/{v3.5 =>
v3.4}/spark/src/main/scala/org/apache/spark/sql/stats/ThetaSketchAgg.scala
(100%)
copy spark/{v3.5 =>
v3.4}/spark/src/test/java/org/apache/iceberg/spark/actions/TestComputeTableStatsAction.java
(74%)
