This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/gradle/com.diffplug.spotless-spotless-plugin-gradle-6.15.0
in repository https://gitbox.apache.org/repos/asf/iceberg.git
omit b79011fd42 Build: Bump com.diffplug.spotless:spotless-plugin-gradle
add 1521296dd2 Core: Support lazy snapshot loading in TableMetadata (#6811)
add a7a6d354b1 Flink: Support branch writes in Flink sink (#6660)
add 96dc0c6a6e Python: TypeVar for bounding TableScan (#6819)
add e5a93e83b5 API: Revert to using stephenc findbugs dependency for
Nullable (#6815)
add 1e57760394 API: Add new KeyManagementClient interface (#6485)
add 775b3ce3a7 Python: Add support for static table (#6644)
add ef7e20e12e Core: Refactor validation in TableScanUtil (#6791)
add 6cd3d240fb Flink: use tag or branch to scan data (#5029)
add 38d7602d86 Parquet: deprecate Decimal Metadata usage in favor of
DecimalLogicalTypeAnnotation (#6735)
add 72adc7c0ed Spark 3.3: SQL Extensions for DROP TAG (#6807)
add 634215fcd4 Spark 3.3: Return partition stats for AddFiles procedure
(#6797)
add becba79d85 Build: Bump
com.palantir.gradle.gitversion:gradle-git-version (#6812)
add 1ab43a9192 Python: Inline the PyArrow import (#6827)
add 3bbea4aee9 Python: Set PyArrow as the default FileIO (#6822)
add 18d45b42b6 Python: Support for DynamoDB Catalog (#6646)
add 14663ed633 Python: Fix the CI (#6842)
add 49d833aa83 API,Core,Spark: Add rewritten bytes to rewrite data files
procedure results (#6801)
add d79acd4d68 Add application identifier for Snowflake JDBC driver (#6740)
add ec85314266 Spark 3.3: Make manifest file names unique during imports
(#6818)
add 8c4fd98e15 Spark 3.2: Make manifest file names unique during imports
(#6845)
add a6ad1d1312 Spark 3.3: Change default distribution modes (#6828)
add 7fff07c6e2 Data: Add a select collections for IcebergGenerics (#6834)
add 630201e8da Build: Bump werkzeug from 2.2.2 to 2.2.3 in /python (#6854)
add 7d08f4f6fa Python: Add String to Boolean literal conversion (#6851)
add 27311cea11 Build: Bump Arrow from 10.0.1 to 11.0.0 (#6696)
add ab6ba6801c Build: Bump jackson-annotations from 2.14.1 to 2.14.2
(#6687)
add 370a43e26f [Python Legacy] Convert string to boolean if the binding
variable is Boolean (#6843)
add b0f55ce835 Spark 3.1: Add rewritten bytes to rewrite data files
procedure results (#6856)
add 085b556cca Spark 3.2: Add rewritten bytes to rewrite data files
procedure results (#6855)
add e1a09e78ef AWS: set lastEvaluatedKey for listTables in DynamoDb
Catalog (#6823)
add 5a4761c8fd Flink: backport PR #6764 to 1.14 and 1.15 for
elapsedSecondsSinceLastSuccessfulCommit sink metric (#6808)
add aa3cc32c98 Python: Add `default-catalog` option to the config (#6864)
add 59b94366ae Docs: Note partitions metadata table might show 'old'
partitions (#6771)
add bb95e8bb1a AWS: Fix warning in S3V4RestSignerClient (#6859)
add f0df880a80 Spark 3.3: Add SQL property to control distribution mode
(#6838)
add d67d06a89d Spark 3.1: Make manifest file names unique during imports
(#6846)
add bd58ca5dd7 Spark 3.3: Support reading position deletes table (#6716)
add f7470b84b3 Spark 3.3: Delete orphan files JMH benchmark (#5615)
add 2a9e653996 Python: Bump DuckDB to 0.7.0 (#6848)
add 66e2ffcb71 Build: Bump pydantic from 1.10.4 to 1.10.5 in /python
(#6881)
add 2dea41dffe Build: Bump typing-extensions from 4.4.0 to 4.5.0 in
/python (#6883)
add 42d209f491 Python: Add more pyiceberg docs (#6874)
add 237e991466 Core: Add REST support for lazy snapshot loading (#6850)
add cd7bd16ca9 AWS: OSS S3 Signer Updates (#6835)
add e16a51bd16 Build: Fix compile warning with Immutables subtype (#6869)
add ee317a58bd Spark 3.2: Change default distribution modes (#6877)
add ed2ad9dff8 Spark 3.2: Add SQL property to control distribution mode
(#6878)
add 2403067441 Build: Bump Gradle to 8.0.1 (#6826)
add 664fa0367f Build: Bump com.diffplug.spotless:spotless-plugin-gradle
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (b79011fd42)
\
N -- N -- N
refs/heads/dependabot/gradle/com.diffplug.spotless-spotless-plugin-gradle-6.15.0
(664fa0367f)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../apache/iceberg/actions/RewriteDataFiles.java | 8 +
.../org/apache/iceberg/encryption/KmsClient.java | 2 +
.../apache/iceberg/expressions/ExpressionUtil.java | 29 +
.../apache/iceberg/view/SQLViewRepresentation.java | 2 +-
.../apache/iceberg/view/ViewRepresentation.java | 3 -
.../iceberg/aws/dynamodb/DynamoDbCatalog.java | 3 +-
.../iceberg/aws/s3/signer/S3ObjectMapper.java | 9 +-
.../aws/s3/signer/S3V4RestSignerClient.java | 33 +-
build.gradle | 28 +-
.../java/org/apache/iceberg/BaseMetadataTable.java | 19 +
.../org/apache/iceberg/PositionDeletesTable.java | 23 +-
.../java/org/apache/iceberg/TableMetadata.java | 60 +-
.../org/apache/iceberg/TableMetadataParser.java | 1 +
.../actions/BaseFileGroupRewriteResult.java | 18 +
.../apache/iceberg/actions/RewriteFileGroup.java | 4 +-
.../iceberg/encryption/KeyManagementClient.java | 30 +-
.../apache/iceberg/rest/RESTSessionCatalog.java | 47 +-
.../org/apache/iceberg/util/TableScanUtil.java | 27 +-
.../org/apache/iceberg/TestSnapshotLoading.java | 241 ++++++
.../java/org/apache/iceberg/TestSplitPlanning.java | 6 +-
.../java/org/apache/iceberg/TestTableMetadata.java | 11 +
.../encryption/{kms => }/KeyStoreKmsClient.java | 8 +-
.../encryption/{kms => }/MemoryMockKMS.java | 16 +-
.../org/apache/iceberg/rest/TestRESTCatalog.java | 114 +++
.../org/apache/iceberg/data/IcebergGenerics.java | 9 +-
.../java/org/apache/iceberg/data/FileHelpers.java | 22 +
.../apache/iceberg/data/GenericAppenderHelper.java | 21 +-
deploy.gradle | 6 +-
docs/spark-queries.md | 4 +-
flink/v1.14/build.gradle | 2 +-
.../iceberg/flink/sink/IcebergFilesCommitter.java | 3 +
.../flink/sink/IcebergFilesCommitterMetrics.java | 31 +
flink/v1.15/build.gradle | 2 +-
.../iceberg/flink/sink/IcebergFilesCommitter.java | 3 +-
.../flink/sink/IcebergFilesCommitterMetrics.java | 31 +
flink/v1.16/build.gradle | 2 +-
.../org/apache/iceberg/flink/FlinkReadConf.java | 16 +
.../org/apache/iceberg/flink/FlinkReadOptions.java | 12 +
.../org/apache/iceberg/flink/FlinkWriteConf.java | 8 +
.../apache/iceberg/flink/FlinkWriteOptions.java | 5 +
.../org/apache/iceberg/flink/sink/FlinkSink.java | 8 +-
.../iceberg/flink/sink/IcebergFilesCommitter.java | 16 +-
.../apache/iceberg/flink/source/FlinkSource.java | 20 +
.../iceberg/flink/source/FlinkSplitPlanner.java | 29 +-
.../apache/iceberg/flink/source/IcebergSource.java | 20 +
.../apache/iceberg/flink/source/ScanContext.java | 89 ++-
.../flink/source/StreamingMonitorFunction.java | 27 +-
.../org/apache/iceberg/flink/SimpleDataUtil.java | 46 +-
.../iceberg/flink/sink/TestFlinkIcebergSink.java | 35 +-
.../flink/sink/TestFlinkIcebergSinkBase.java | 64 ++
.../flink/sink/TestFlinkIcebergSinkBranch.java | 137 ++++
.../iceberg/flink/sink/TestFlinkIcebergSinkV2.java | 310 +-------
.../flink/sink/TestFlinkIcebergSinkV2Base.java} | 505 +++++-------
.../flink/sink/TestFlinkIcebergSinkV2Branch.java | 134 ++++
.../flink/sink/TestIcebergFilesCommitter.java | 87 ++-
.../apache/iceberg/flink/source/TestFlinkScan.java | 144 ++++
.../iceberg/flink/source/TestFlinkSource.java | 4 +
.../iceberg/flink/source/TestStreamScanSql.java | 71 +-
.../flink/source/TestStreamingMonitorFunction.java | 36 +
gradle/wrapper/gradle-wrapper.properties | 5 +-
gradlew | 8 +-
hive-runtime/build.gradle | 2 +-
hive3-orc-bundle/build.gradle | 2 +-
hive3/build.gradle | 2 +-
mr/build.gradle | 2 +-
open-api/rest-catalog-open-api.yaml | 13 +
.../iceberg/data/parquet/BaseParquetReaders.java | 5 +-
.../iceberg/parquet/ParquetAvroValueReaders.java | 5 +-
.../apache/iceberg/parquet/ParquetAvroWriter.java | 5 +-
.../parquet/ParquetBloomRowGroupFilter.java | 5 +-
.../apache/iceberg/parquet/ParquetConversions.java | 5 +-
.../org/apache/iceberg/pig/PigParquetReader.java | 5 +-
python/mkdocs/docs/api.md | 29 +-
python/mkdocs/docs/configuration.md | 36 +
python/mkdocs/docs/index.md | 19 +-
python/poetry.lock | 587 ++++++++------
python/pyiceberg/catalog/__init__.py | 151 +++-
python/pyiceberg/catalog/dynamodb.py | 769 ++++++++++++++++++
python/pyiceberg/catalog/glue.py | 228 ++----
python/pyiceberg/catalog/hive.py | 76 +-
python/pyiceberg/catalog/rest.py | 6 +-
python/pyiceberg/cli/console.py | 4 +-
python/pyiceberg/exceptions.py | 12 +
python/pyiceberg/expressions/literals.py | 8 +
python/pyiceberg/io/__init__.py | 8 +-
python/pyiceberg/table/__init__.py | 46 +-
python/pyiceberg/utils/config.py | 16 +
python/pyproject.toml | 7 +-
...n_test_glue.py => integration_test_dynamodb.py} | 82 +-
python/tests/catalog/integration_test_glue.py | 39 +-
.../catalog/{test_glue.py => test_dynamodb.py} | 259 ++++---
python/tests/catalog/test_glue.py | 90 +--
python/tests/catalog/test_hive.py | 14 +-
python/tests/conftest.py | 76 +-
python/tests/expressions/test_literals.py | 16 +-
python/tests/io/test_io.py | 3 +-
python/tests/table/test_init.py | 19 +-
python_legacy/iceberg/api/expressions/literals.py | 3 +
.../expressions/test_string_literal_conversions.py | 10 +-
.../apache/iceberg/snowflake/SnowflakeCatalog.java | 8 +
spark/v2.4/build.gradle | 2 +-
spark/v3.1/build.gradle | 2 +-
.../spark/extensions/TestAddFilesProcedure.java | 42 +
.../extensions/TestRewriteDataFilesProcedure.java | 66 +-
.../org/apache/iceberg/spark/SparkTableUtil.java | 5 +-
.../iceberg/spark/data/SparkParquetReaders.java | 5 +-
.../iceberg/spark/data/SparkParquetWriters.java | 5 +-
.../procedures/RewriteDataFilesProcedure.java | 7 +-
.../apache/iceberg/spark/SparkCatalogConfig.java | 5 +-
.../org/apache/iceberg/spark/SparkTestBase.java | 2 +-
.../spark/actions/TestRewriteDataFilesAction.java | 49 ++
.../apache/iceberg/spark/sql/TestRefreshTable.java | 4 +-
spark/v3.2/build.gradle | 2 +-
.../spark/extensions/TestAddFilesProcedure.java | 35 +
.../extensions/TestRewriteDataFilesProcedure.java | 134 +++-
.../apache/iceberg/spark/SparkSQLProperties.java | 3 +
.../org/apache/iceberg/spark/SparkTableUtil.java | 5 +-
.../org/apache/iceberg/spark/SparkWriteConf.java | 26 +-
.../iceberg/spark/data/SparkParquetReaders.java | 5 +-
.../iceberg/spark/data/SparkParquetWriters.java | 5 +-
.../procedures/RewriteDataFilesProcedure.java | 7 +-
.../apache/iceberg/spark/SparkCatalogConfig.java | 5 +-
.../apache/iceberg/spark/SparkTestHelperBase.java | 2 +-
.../TestSparkDistributionAndOrderingUtil.java | 227 +++++-
.../apache/iceberg/spark/TestSparkWriteConf.java | 197 +++++
.../spark/actions/TestRewriteDataFilesAction.java | 55 ++
.../spark/actions/TestRewriteManifestsAction.java | 8 +-
.../spark/source/TestIcebergSourceTablesBase.java | 2 +
.../iceberg/spark/source/TestPartitionValues.java | 1 +
.../iceberg/spark/source/TestSparkDataWrite.java | 7 +-
.../apache/iceberg/spark/sql/TestRefreshTable.java | 4 +-
spark/v3.3/build.gradle | 2 +-
.../IcebergSqlExtensions.g4 | 1 +
.../IcebergSparkSqlExtensionsParser.scala | 3 +-
.../IcebergSqlExtensionsAstBuilder.scala | 8 +
.../logical/{DropBranch.scala => DropTag.scala} | 4 +-
.../v2/{DropBranchExec.scala => DropTagExec.scala} | 12 +-
.../v2/ExtendedDataSourceV2Strategy.scala | 4 +
.../spark/extensions/TestAddFilesProcedure.java | 189 +++--
.../iceberg/spark/extensions/TestBranchDDL.java | 22 +
.../extensions/TestRewriteDataFilesProcedure.java | 134 +++-
.../iceberg/spark/extensions/TestTagDDL.java | 114 ++-
.../spark/action/DeleteOrphanFilesBenchmark.java | 179 +++++
.../apache/iceberg/spark/SparkSQLProperties.java | 3 +
.../org/apache/iceberg/spark/SparkTableUtil.java | 5 +-
.../org/apache/iceberg/spark/SparkWriteConf.java | 26 +-
.../iceberg/spark/data/SparkParquetReaders.java | 5 +-
.../iceberg/spark/data/SparkParquetWriters.java | 5 +-
.../spark/procedures/AddFilesProcedure.java | 21 +-
.../procedures/RewriteDataFilesProcedure.java | 7 +-
.../spark/source/PositionDeletesRowReader.java | 108 +++
.../spark/source/SparkRowReaderFactory.java | 4 +
.../apache/iceberg/spark/SparkCatalogConfig.java | 5 +-
.../apache/iceberg/spark/SparkTestHelperBase.java | 2 +-
.../TestSparkDistributionAndOrderingUtil.java | 227 +++++-
.../apache/iceberg/spark/TestSparkWriteConf.java | 197 +++++
.../spark/actions/TestRewriteDataFilesAction.java | 55 ++
.../spark/actions/TestRewriteManifestsAction.java | 8 +-
.../spark/source/TestIcebergSourceTablesBase.java | 2 +
.../iceberg/spark/source/TestPartitionValues.java | 1 +
.../spark/source/TestPositionDeletesTable.java | 861 +++++++++++++++++++++
.../iceberg/spark/source/TestSparkDataWrite.java | 7 +-
.../apache/iceberg/spark/sql/TestRefreshTable.java | 4 +-
versions.props | 5 +-
164 files changed, 6658 insertions(+), 1882 deletions(-)
copy api/src/main/java/org/apache/iceberg/encryption/KmsClient.java =>
core/src/main/java/org/apache/iceberg/encryption/KeyManagementClient.java (75%)
create mode 100644
core/src/test/java/org/apache/iceberg/TestSnapshotLoading.java
rename core/src/test/java/org/apache/iceberg/encryption/{kms =>
}/KeyStoreKmsClient.java (93%)
rename core/src/test/java/org/apache/iceberg/encryption/{kms =>
}/MemoryMockKMS.java (75%)
create mode 100644
flink/v1.16/flink/src/test/java/org/apache/iceberg/flink/sink/TestFlinkIcebergSinkBase.java
create mode 100644
flink/v1.16/flink/src/test/java/org/apache/iceberg/flink/sink/TestFlinkIcebergSinkBranch.java
copy
flink/{v1.14/flink/src/test/java/org/apache/iceberg/flink/sink/TestFlinkIcebergSinkV2.java
=>
v1.16/flink/src/test/java/org/apache/iceberg/flink/sink/TestFlinkIcebergSinkV2Base.java}
(62%)
create mode 100644
flink/v1.16/flink/src/test/java/org/apache/iceberg/flink/sink/TestFlinkIcebergSinkV2Branch.java
create mode 100644 python/pyiceberg/catalog/dynamodb.py
copy python/tests/catalog/{integration_test_glue.py =>
integration_test_dynamodb.py} (80%)
copy python/tests/catalog/{test_glue.py => test_dynamodb.py} (67%)
create mode 100644
spark/v3.2/spark/src/test/java/org/apache/iceberg/spark/TestSparkWriteConf.java
copy
spark/v3.3/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/{DropBranch.scala
=> DropTag.scala} (87%)
copy
spark/v3.3/spark-extensions/src/main/scala/org/apache/spark/sql/execution/datasources/v2/{DropBranchExec.scala
=> DropTagExec.scala} (82%)
create mode 100644
spark/v3.3/spark/src/jmh/java/org/apache/iceberg/spark/action/DeleteOrphanFilesBenchmark.java
create mode 100644
spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/PositionDeletesRowReader.java
create mode 100644
spark/v3.3/spark/src/test/java/org/apache/iceberg/spark/TestSparkWriteConf.java
create mode 100644
spark/v3.3/spark/src/test/java/org/apache/iceberg/spark/source/TestPositionDeletesTable.java
