This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/gradle/com.azure-azure-sdk-bom-1.2.35
in repository https://gitbox.apache.org/repos/asf/iceberg.git
omit bb7f5f05f2 Build: Bump com.azure:azure-sdk-bom from 1.2.31 to 1.2.35
add b196aeec0d Build: Bump datamodel-code-generator from 0.30.2 to 0.31.0
(#13319)
add 24cb139b15 Build: Bump org.xerial:sqlite-jdbc from 3.49.1.0 to
3.50.1.0 (#13318)
add 077af5fec2 Build: Bump jackson-bom from 2.19.0 to 2.19.1 (#13315)
add d6b52149de Build: Bump io.delta:delta-spark_2.12 from 3.3.1 to 3.3.2
(#13205)
add d0b90c540d Build: Bump com.google.cloud:libraries-bom from 26.61.0 to
26.62.0 (#13317)
add 1456c69aa1 Build: Bump io.delta:delta-standalone_2.12 from 3.3.1 to
3.3.2 (#13202)
add 7ceb7402ad Build: Bump net.snowflake:snowflake-jdbc from 3.24.0 to
3.24.2 (#13200)
add 3ed8184387 Flink: If IcebergSink writeParallelism is not specified,
defaults to the input source parallelism (#13260)
add 67f550e7ee docs: Fix typo in `spark-procedures.md` (#13325)
add 7dc4a66b42 Flink: Backports #13260 to Flink 1.19 and 1.20 (#13326)
add 68651a3498 Flink: Dynamic Iceberg Sink: Add sink / core processing
logic / benchmarking (#13304)
add 0eb9728a46 AWS, GCP: Fix double-checked-locking pattern in S3FileIO,
GCSFileIO. (#13276)
add 5b50afe8f2 Core, REST: Add context aware response parsing (#13191)
add 81b74fc0b9 Flink: Backport dynamic Iceberg Sink: Add sink / core
processing logic / benchmarking to Flink 1.19 / 1.20 (#13341)
add cea7f9c829 Flink: Dynamic Iceberg Sink: Rename method in
DynamicRecordGenerator (#13342)
add fa162de015 Core: PartitionsTable#partitions returns incomplete list in
case of partition evolution and NULL partition values (#12528)
add e47351980f Flink: Backport dynamic Iceberg Sink: Rename method in
DynamicRecordGenerator to Flink 1.19 / 1.20 (#13346)
add cdd2b5bb3e Core: Clean expired metadata even if there is no snapshot
to expire (#13322)
add e4dc658c19 Docs: Add Firebolt to vendor-related documentation (#13350)
add feeb045a73 Spark 3.4: Backport #12836 for row lineage support in spark
parquet reader (#13353)
add 40c0a73c65 GCP: Add Google Authentication Support (#13212)
add c41fd6457f spark 4.0: SPJ: add bucket reducer using gcd (#13167)
add 56d81e72ff Spark 3.4: Backport UPDATE/MERGE logic for row lineage
(#13344)
add e5541de0fd Azure: Bump `AzuriteContainer` to 3.34.0 (#13321)
add ab6751d3a9 Build: Bump com.azure:azure-sdk-bom from 1.2.31 to 1.2.35
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (bb7f5f05f2)
\
N -- N -- N
refs/heads/dependabot/gradle/com.azure-azure-sdk-bom-1.2.35 (ab6751d3a9)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.palantir/revapi.yml | 6 +
.../org/apache/iceberg/util/StructProjection.java | 5 +
.../java/org/apache/iceberg/aws/s3/S3FileIO.java | 7 +-
.../iceberg/azure/adlsv2/AzuriteContainer.java | 2 +-
build.gradle | 1 +
.../java/org/apache/iceberg/PartitionsTable.java | 63 +-
.../java/org/apache/iceberg/RemoveSnapshots.java | 4 +-
.../org/apache/iceberg/rest/BaseHTTPClient.java | 32 +
.../java/org/apache/iceberg/rest/HTTPClient.java | 20 +-
.../org/apache/iceberg/rest/ParserContext.java | 64 ++
.../java/org/apache/iceberg/rest/RESTClient.java | 27 +
.../org/apache/iceberg/util/StructLikeMap.java | 31 +-
.../org/apache/iceberg/util/StructLikeWrapper.java | 13 +-
.../apache/iceberg/MetadataTableScanTestBase.java | 2 +-
.../src/test/java/org/apache/iceberg/TestBase.java | 12 +-
...stMetadataTableScansWithPartitionEvolution.java | 86 +++
.../org/apache/iceberg/TestRemoveSnapshots.java | 30 +
.../test/java/org/apache/iceberg/TestTables.java | 41 +-
.../apache/iceberg/rest/RESTCatalogAdapter.java | 11 +
docs/docs/spark-procedures.md | 2 +-
docs/mkdocs.yml | 1 +
...namicRecordSerializerDeserializerBenchmark.java | 138 ++++
.../org/apache/iceberg/flink/FlinkConfParser.java | 7 +
.../org/apache/iceberg/flink/FlinkWriteConf.java | 4 +
.../org/apache/iceberg/flink/sink/FlinkSink.java | 49 +-
.../org/apache/iceberg/flink/sink/IcebergSink.java | 65 +-
.../org/apache/iceberg/flink/sink/SinkUtil.java | 59 +-
.../flink/sink/dynamic/DynamicCommitter.java | 3 +-
.../flink/sink/dynamic/DynamicIcebergSink.java | 406 ++++++++++
...riteResult.java => DynamicRecordGenerator.java} | 28 +-
.../flink/sink/dynamic/DynamicRecordProcessor.java | 171 +++++
.../flink/sink/dynamic/HashKeyGenerator.java | 16 +-
.../org/apache/iceberg/flink/SimpleDataUtil.java | 22 +-
.../java/org/apache/iceberg/flink/TestHelpers.java | 2 +-
.../flink/sink/TestFlinkIcebergSinkBase.java | 5 +-
.../apache/iceberg/flink/sink/TestIcebergSink.java | 45 ++
.../flink/sink/dynamic/TestDynamicIcebergSink.java | 831 +++++++++++++++++++++
.../sink/dynamic/TestDynamicIcebergSinkPerf.java | 245 ++++++
...namicRecordSerializerDeserializerBenchmark.java | 138 ++++
.../org/apache/iceberg/flink/FlinkConfParser.java | 7 +
.../org/apache/iceberg/flink/FlinkWriteConf.java | 4 +
.../org/apache/iceberg/flink/sink/FlinkSink.java | 49 +-
.../org/apache/iceberg/flink/sink/IcebergSink.java | 65 +-
.../org/apache/iceberg/flink/sink/SinkUtil.java | 59 +-
.../flink/sink/dynamic/DynamicCommitter.java | 3 +-
.../flink/sink/dynamic/DynamicIcebergSink.java | 406 ++++++++++
.../sink/dynamic/DynamicRecordGenerator.java} | 28 +-
.../flink/sink/dynamic/DynamicRecordProcessor.java | 171 +++++
.../flink/sink/dynamic/HashKeyGenerator.java | 16 +-
.../org/apache/iceberg/flink/SimpleDataUtil.java | 22 +-
.../org/apache/iceberg/flink/TestFlinkFilters.java | 1 +
.../java/org/apache/iceberg/flink/TestHelpers.java | 2 +-
.../flink/sink/TestFlinkIcebergSinkBase.java | 5 +-
.../apache/iceberg/flink/sink/TestIcebergSink.java | 45 ++
.../flink/sink/dynamic/TestDynamicIcebergSink.java | 831 +++++++++++++++++++++
.../sink/dynamic/TestDynamicIcebergSinkPerf.java | 245 ++++++
...namicRecordSerializerDeserializerBenchmark.java | 138 ++++
.../org/apache/iceberg/flink/FlinkConfParser.java | 7 +
.../org/apache/iceberg/flink/FlinkWriteConf.java | 4 +
.../org/apache/iceberg/flink/sink/FlinkSink.java | 49 +-
.../org/apache/iceberg/flink/sink/IcebergSink.java | 65 +-
.../org/apache/iceberg/flink/sink/SinkUtil.java | 59 +-
.../flink/sink/dynamic/DynamicCommitter.java | 3 +-
.../flink/sink/dynamic/DynamicIcebergSink.java | 406 ++++++++++
.../sink/dynamic/DynamicRecordGenerator.java} | 28 +-
.../flink/sink/dynamic/DynamicRecordProcessor.java | 171 +++++
.../flink/sink/dynamic/HashKeyGenerator.java | 16 +-
.../org/apache/iceberg/flink/SimpleDataUtil.java | 22 +-
.../org/apache/iceberg/flink/TestFlinkFilters.java | 1 -
.../java/org/apache/iceberg/flink/TestHelpers.java | 2 +-
.../flink/sink/TestFlinkIcebergSinkBase.java | 5 +-
.../apache/iceberg/flink/sink/TestIcebergSink.java | 45 ++
.../flink/sink/dynamic/TestDynamicIcebergSink.java | 831 +++++++++++++++++++++
.../sink/dynamic/TestDynamicIcebergSinkPerf.java | 245 ++++++
.../apache/iceberg/gcp/auth/GoogleAuthManager.java | 146 ++++
.../apache/iceberg/gcp/auth/GoogleAuthSession.java | 94 +++
.../java/org/apache/iceberg/gcp/gcs/GCSFileIO.java | 7 +-
.../iceberg/gcp/auth/TestGoogleAuthManager.java | 193 +++++
.../iceberg/gcp/auth/TestGoogleAuthSession.java | 168 +++++
gradle/libs.versions.toml | 12 +-
open-api/requirements.txt | 2 +-
site/docs/blogs.md | 5 +
site/docs/vendors.md | 8 +
.../extensions/IcebergSparkSessionExtensions.scala | 6 +
.../RewriteMergeIntoTableForRowLineage.scala | 28 +-
.../analysis/RewriteOperationForRowLineage.scala | 2 +-
.../analysis/RewriteUpdateTableForRowLineage.scala | 6 +-
.../RemoveRowLineageOutputFromOriginalTable.scala | 8 +-
.../SparkRowLevelOperationsTestBase.java | 14 +-
.../extensions/TestCopyOnWriteWithLineage.java | 0
.../extensions/TestMergeOnReadWithLineage.java | 0
.../TestRowLevelOperationsWithLineage.java | 59 --
.../iceberg/spark/data/SparkParquetReaders.java | 60 +-
.../apache/iceberg/spark/source/SparkBatch.java | 5 +-
.../spark/source/SparkCopyOnWriteOperation.java | 20 +-
.../iceberg/spark/source/SparkCopyOnWriteScan.java | 40 +
.../spark/source/SparkPositionDeltaOperation.java | 17 +-
.../source/SparkPositionDeltaWriteBuilder.java | 13 +-
.../apache/iceberg/spark/source/SparkTable.java | 5 +-
.../iceberg/spark/source/SparkWriteBuilder.java | 24 +-
.../apache/iceberg/spark/SparkTestHelperBase.java | 7 +-
.../apache/iceberg/spark/data/AvroDataTest.java | 58 ++
.../apache/iceberg/spark/data/GenericsHelpers.java | 45 +-
.../iceberg/spark/data/TestSparkParquetReader.java | 45 +-
.../iceberg/spark/functions/BucketFunction.java | 38 +-
.../spark/sql/TestStoragePartitionedJoins.java | 146 ++++
106 files changed, 7407 insertions(+), 624 deletions(-)
create mode 100644
core/src/main/java/org/apache/iceberg/rest/ParserContext.java
create mode 100644
flink/v1.19/flink/src/jmh/java/org/apache/iceberg/flink/sink/dynamic/DynamicRecordSerializerDeserializerBenchmark.java
create mode 100644
flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicIcebergSink.java
copy
flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/{DynamicWriteResult.java
=> DynamicRecordGenerator.java} (61%)
create mode 100644
flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicRecordProcessor.java
create mode 100644
flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/sink/dynamic/TestDynamicIcebergSink.java
create mode 100644
flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/sink/dynamic/TestDynamicIcebergSinkPerf.java
create mode 100644
flink/v1.20/flink/src/jmh/java/org/apache/iceberg/flink/sink/dynamic/DynamicRecordSerializerDeserializerBenchmark.java
create mode 100644
flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicIcebergSink.java
copy
flink/{v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicWriteResult.java
=>
v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicRecordGenerator.java}
(61%)
create mode 100644
flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicRecordProcessor.java
create mode 100644
flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/sink/dynamic/TestDynamicIcebergSink.java
create mode 100644
flink/v1.20/flink/src/test/java/org/apache/iceberg/flink/sink/dynamic/TestDynamicIcebergSinkPerf.java
create mode 100644
flink/v2.0/flink/src/jmh/java/org/apache/iceberg/flink/sink/dynamic/DynamicRecordSerializerDeserializerBenchmark.java
create mode 100644
flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicIcebergSink.java
copy
flink/{v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicWriteResult.java
=>
v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicRecordGenerator.java}
(61%)
create mode 100644
flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicRecordProcessor.java
create mode 100644
flink/v2.0/flink/src/test/java/org/apache/iceberg/flink/sink/dynamic/TestDynamicIcebergSink.java
create mode 100644
flink/v2.0/flink/src/test/java/org/apache/iceberg/flink/sink/dynamic/TestDynamicIcebergSinkPerf.java
create mode 100644
gcp/src/main/java/org/apache/iceberg/gcp/auth/GoogleAuthManager.java
create mode 100644
gcp/src/main/java/org/apache/iceberg/gcp/auth/GoogleAuthSession.java
create mode 100644
gcp/src/test/java/org/apache/iceberg/gcp/auth/TestGoogleAuthManager.java
create mode 100644
gcp/src/test/java/org/apache/iceberg/gcp/auth/TestGoogleAuthSession.java
copy spark/{v3.5 =>
v3.4}/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/RewriteMergeIntoTableForRowLineage.scala
(72%)
copy spark/{v3.5 =>
v3.4}/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/RewriteOperationForRowLineage.scala
(97%)
copy spark/{v3.5 =>
v3.4}/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/RewriteUpdateTableForRowLineage.scala
(89%)
copy spark/{v3.5 =>
v3.4}/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RemoveRowLineageOutputFromOriginalTable.scala
(88%)
copy spark/{v3.5 =>
v3.4}/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestCopyOnWriteWithLineage.java
(100%)
copy spark/{v3.5 =>
v3.4}/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestMergeOnReadWithLineage.java
(100%)
copy spark/{v3.5 =>
v3.4}/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRowLevelOperationsWithLineage.java
(89%)
