dependabot[bot] opened a new pull request, #17780: URL: https://github.com/apache/iceberg/pull/17780
Bumps [io.delta:delta-spark_2.12](https://github.com/delta-io/delta) from 3.3.2 to 3.3.3. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/delta-io/delta/releases";>io.delta:delta-spark_2.12's releases</a>.</em></p> <blockquote> <h2>Delta Lake 3.3.3</h2> <p>We are pleased to announce the release of Delta Lake 3.3.3, a patch release on top of <a href="https://github.com/delta-io/delta/releases/tag/v3.3.2";>3.3.2</a> with targeted fixes for Delta Sharing cache correctness, transaction log retention safety, and a Kernel Parquet-reader configuration bug, plus a Delta Sharing client upgrade.</p> <p><strong>Note:</strong> this patch release does not publish <code>delta-iceberg</code> (UniForm/Iceberg support) — see <a href="https://github.com/delta-io/delta/blob/HEAD/#delta-uniform";>Delta UniForm</a> below for details and follow-up plan.</p> <h2>Highlights</h2> <ul> <li><strong>[Spark] Fix transaction log retention during metadata cleanup</strong>: Log cleanup could previously delete transaction log files still needed to reconstruct table versions within the retention window on tables with infrequent commits/checkpoints, breaking time travel and CDF reads.</li> <li><strong>[Delta Sharing] Fix deletion vector cache refresh bug</strong>: Cache refresh could drop deletion-vector fileId→URL mappings, causing long-running queries against shared DV tables to fail with "file id not found" errors.</li> <li><strong>[Delta Sharing] Upgrade <code>delta-sharing-client</code> 1.2.2 → 1.2.8</strong>: Picks up more resilient OAuth token parsing and improved retry/error-reporting behavior.</li> <li><strong>[Spark] Add <code>RANDOMIZE_FILE_PREFIXES</code> support</strong>: Wires up the previously-declared <code>delta.randomizeFilePrefixes</code> / <code>delta.randomPrefixLength</code> table properties on the write path, to spread S3 object keys for high-throughput workloads. Opt-in; default behavior unchanged.</li> <li><strong>[Spark] Fix Spark 3.5.6 compatibility for <code>TRUNCATE TABLE</code></strong>: Spark 3.5.6 tightened <code>V2Writes</code> validation; Delta tables now correctly declare <code>TRUNCATE</code>/<code>OVERWRITE_BY_FILTER</code> capability.</li> <li><strong>[Kernel] Fix missing Hadoop configuration in Parquet reader</strong>: The Hadoop <code>Configuration</code> was not being passed when Kernel opened Parquet footers, so custom filesystem/credential settings could be silently ignored.</li> </ul> <h2>Delta Spark</h2> <p>Delta Spark 3.3.3 is built on <a href="https://spark.apache.org/releases/spark-release-3-5-6.html";>Apache Spark 3.5.6</a>. Maven artifacts are published for both Scala 2.12 and 2.13.</p> <ul> <li>Maven artifacts: <ul> <li><a href="https://repo1.maven.org/maven2/io/delta/delta-spark_2.12/3.3.3/";>delta-spark_2.12</a> / <a href="https://repo1.maven.org/maven2/io/delta/delta-spark_2.13/3.3.3/";>delta-spark_2.13</a></li> <li><a href="https://repo1.maven.org/maven2/io/delta/delta-contribs_2.12/3.3.3/";>delta-contribs_2.12</a> / <a href="https://repo1.maven.org/maven2/io/delta/delta-contribs_2.13/3.3.3/";>delta-contribs_2.13</a></li> <li><a href="https://repo1.maven.org/maven2/io/delta/delta-storage/3.3.3/";>delta-storage</a></li> <li><a href="https://repo1.maven.org/maven2/io/delta/delta-storage-s3-dynamodb/3.3.3/";>delta-storage-s3-dynamodb</a></li> </ul> </li> <li>Python artifact: <ul> <li><a href="https://pypi.org/project/delta-spark/3.3.3/";>delta-spark</a></li> </ul> </li> </ul> <p>Fixes and changes in this release:</p> <ul> <li><strong>Fix transaction log retention during metadata cleanup</strong>: Delete eligible log files only if a newer checkpoint exists before the retention cutoff, so time travel and CDF reads to retained versions keep working. Tables with a low commit/checkpoint rate may retain log files slightly beyond the nominal cutoff as a result.</li> <li><strong>Add <code>RANDOMIZE_FILE_PREFIXES</code> support</strong>: <code>delta.randomizeFilePrefixes</code> / <code>delta.randomPrefixLength</code> table properties are now honored on writes, placing new data files under a random subdirectory prefix to spread S3 request load. Opt-in; default write path is unchanged.</li> <li><strong>Fix Spark 3.5.6 compatibility for <code>TRUNCATE TABLE</code></strong>: <code>StagedDeltaTableV2</code> now implements <code>SupportsTruncate</code> and declares <code>TRUNCATE</code>/<code>OVERWRITE_BY_FILTER</code> capabilities, fixing <code>TRUNCATE TABLE</code> under Spark 3.5.6's stricter <code>V2Writes</code> validation.</li> </ul> <h2>Delta Kernel</h2> <p>The <a href="https://delta.io/blog/delta-kernel/";>Delta Kernel</a> project is a set of Java libraries for building Delta connectors without needing to implement the <a href="https://github.com/delta-io/delta/blob/master/PROTOCOL.md";>Delta protocol</a> directly.</p> <ul> <li>Maven artifacts: <ul> <li><a href="https://repo1.maven.org/maven2/io/delta/delta-kernel-api/3.3.3/";>delta-kernel-api</a></li> <li><a href="https://repo1.maven.org/maven2/io/delta/delta-kernel-defaults/3.3.3/";>delta-kernel-defaults</a></li> </ul> </li> </ul> <p>Fixes in this release:</p> <ul> <li><strong>Fix missing Hadoop configuration parameter when creating the Parquet reader</strong>: A follow-up to an earlier <code>IllegalAccessError</code> fix had dropped the Hadoop <code>Configuration</code> when constructing the <code>ParquetReader</code> used to read footers, so custom Hadoop filesystem settings could be ignored during Kernel reads.</li> </ul> <h2>Delta UniForm</h2> <p>Delta UniForm's <code>delta-iceberg</code> and <code>delta-hudi</code> modules keep Apache Iceberg and Apache Hudi metadata in sync with Delta commits.</p> <ul> <li><strong><code>delta-iceberg</code> is not published in 3.3.3.</strong> There will be no delta-iceberg artifact in this release. There are no changes to delta-iceberg from the previous version. Existing <code>delta-iceberg_2.12</code>/<code>delta-iceberg_2.13</code> v3.3.2 artifacts remain available and are forward-compatible with <code>delta-spark</code> 3.3.3.</li> <li>Maven artifacts:</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/delta-io/delta/commit/0ce5dd9f7d02cbc1cd74b7333045784f440636a1";><code>0ce5dd9</code></a> [Release] Bump version to 3.3.3 for release (<a href="https://redirect.github.com/delta-io/delta/issues/7424";>#7424</a>)</li> <li><a href="https://github.com/delta-io/delta/commit/6cba1d7ca7a1e144da059fb3a00f0f730712954e";><code>6cba1d7</code></a> [Branch-3.3] Fix javadoc HTML errors + add -DskipDeltaIceberg build flag (<a href="https://redirect.github.com/delta-io/delta/issues/7411";>#7411</a>)</li> <li><a href="https://github.com/delta-io/delta/commit/73cc8cb1c830281bc56e3c16f83230e6acf41a85";><code>73cc8cb</code></a> [Branch-3.3][Delta Sharing] Upgrade delta-sharing-client to 1.2.8 (<a href="https://redirect.github.com/delta-io/delta/issues/7419";>#7419</a>)</li> <li><a href="https://github.com/delta-io/delta/commit/7e7b8ce0086f68261e2964289f5c8553e462509a";><code>7e7b8ce</code></a> [Branch-3.3] Bump build dependency versions for JDK 17 and Maven Central comp...</li> <li><a href="https://github.com/delta-io/delta/commit/86435544b6a0906830200e391e7cbf3386204832";><code>8643554</code></a> [Spark][CherryPick][3.3] Fix metadata cleanup by retaining files required for...</li> <li><a href="https://github.com/delta-io/delta/commit/23609bafb55926cd407f26325f88ad9ccf0f8149";><code>23609ba</code></a> [Branch-3.3][Delta Sharing] Ensure deletion vector fileId-URL entries are pre...</li> <li><a href="https://github.com/delta-io/delta/commit/37f4f36882b9e5c5f7fef2e441a0b082ad83f46c";><code>37f4f36</code></a> [CI] Add branch filter to branch-3.3 workflows (<a href="https://redirect.github.com/delta-io/delta/issues/6418";>#6418</a>)</li> <li><a href="https://github.com/delta-io/delta/commit/a7f222c4cc89e10203c9a4c586c26d367777da97";><code>a7f222c</code></a> [Spark][CherryPick][3.3] Support RANDOMIZE_FILE_PREFIXES (<a href="https://redirect.github.com/delta-io/delta/issues/5263";>#5263</a>)</li> <li><a href="https://github.com/delta-io/delta/commit/80b9ad92117425853aaf151364f8784aa2a2b87a";><code>80b9ad9</code></a> Fix Spark 3.5.6 compatibility: Add truncate with overwrite support to StagedD...</li> <li><a href="https://github.com/delta-io/delta/commit/4070076ff7011395513d6506c715f4bf3c4d3607";><code>4070076</code></a> [Spark] [Test] [3.3] Fix curl call to follow redirects (<a href="https://redirect.github.com/delta-io/delta/issues/5133";>#5133</a>)</li> <li>Additional commits viewable in <a href="https://github.com/delta-io/delta/compare/v3.3.2...v3.3.3";>compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
