dependabot[bot] opened a new pull request, #120: URL: https://github.com/apache/beam-starter-java/pull/120
Bumps [org.apache.beam:beam-runners-direct-java](https://github.com/apache/beam) from 2.59.0 to 2.60.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/apache/beam/releases";>org.apache.beam:beam-runners-direct-java's releases</a>.</em></p> <blockquote> <h2>Beam 2.60.0 release</h2> <p>We are happy to present the new 2.60.0 release of Beam. This release includes both improvements and new functionality.</p> <p>For more information on changes in 2.60.0, check out the <a href="https://github.com/apache/beam/milestone/24";>detailed release notes</a>.</p> <h2>Highlights</h2> <ul> <li>Added support for using vLLM in the RunInference transform (Python) (<a href="https://redirect.github.com/apache/beam/issues/32528";>#32528</a>)</li> <li>[Managed Iceberg] Added support for streaming writes (<a href="https://redirect.github.com/apache/beam/pull/32451";>#32451</a>)</li> <li>[Managed Iceberg] Added auto-sharding for streaming writes (<a href="https://redirect.github.com/apache/beam/pull/32612";>#32612</a>)</li> <li>[Managed Iceberg] Added support for writing to dynamic destinations (<a href="https://redirect.github.com/apache/beam/pull/32565";>#32565</a>)</li> </ul> <h2>New Features / Improvements</h2> <ul> <li>Dataflow worker can install packages from Google Artifact Registry Python repositories (Python) (<a href="https://redirect.github.com/apache/beam/issues/32123";>#32123</a>).</li> <li>Added support for Zstd codec in SerializableAvroCodecFactory (Java) (<a href="https://redirect.github.com/apache/beam/issues/32349";>#32349</a>)</li> <li>Added support for using vLLM in the RunInference transform (Python) (<a href="https://redirect.github.com/apache/beam/issues/32528";>#32528</a>)</li> <li>Prism release binaries and container bootloaders are now being built with the latest Go 1.23 patch. (<a href="https://redirect.github.com/apache/beam/pull/32575";>#32575</a>)</li> <li>Prism <ul> <li>Prism now supports Bundle Finalization. (<a href="https://redirect.github.com/apache/beam/pull/32425";>#32425</a>)</li> </ul> </li> <li>Significantly improved performance of Kafka IO reads that enable <a href="https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/kafka/KafkaIO.Read.html#commitOffsetsInFinalize--";>commitOffsetsInFinalize</a> by removing the data reshuffle from SDF implementation. (<a href="https://redirect.github.com/apache/beam/pull/31682";>#31682</a>).</li> <li>Added support for dynamic writing in MqttIO (Java) (<a href="https://redirect.github.com/apache/beam/issues/19376";>#19376</a>)</li> <li>Optimized Spark Runner parDo transform evaluator (Java) (<a href="https://redirect.github.com/apache/beam/issues/32537";>#32537</a>)</li> <li>[Managed Iceberg] More efficient manifest file writes/commits (<a href="https://redirect.github.com/apache/beam/issues/32666";>#32666</a>)</li> </ul> <h2>Breaking Changes</h2> <ul> <li>In Python, assert_that now throws if it is not in a pipeline context instead of silently succeeding (<a href="https://redirect.github.com/apache/beam/pull/30771";>#30771</a>)</li> <li>In Python and YAML, ReadFromJson now override the dtype from None to an explicit False. Most notably, string values like <code>"123"</code> are preserved as strings rather than silently coerced (and possibly truncated) to numeric values. To retain the old behavior, pass <code>dtype=True</code> (or any other value accepted by <code>pandas.read_json</code>).</li> <li>Users of KafkaIO Read transform that enable <a href="https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/kafka/KafkaIO.Read.html#commitOffsetsInFinalize--";>commitOffsetsInFinalize</a> might encounter pipeline graph compatibility issues when updating the pipeline. To mitigate, set the <code>updateCompatibilityVersion</code> option to the SDK version used for the original pipeline, example <code>--updateCompatabilityVersion=2.58.1</code></li> </ul> <h2>Deprecations</h2> <ul> <li>Python 3.8 is reaching EOL and support is being removed in Beam 2.61.0. The 2.60.0 release will warn users when running on 3.8. (<a href="https://redirect.github.com/apache/beam/issues/31192";>#31192</a>)</li> </ul> <h2>Bugfixes</h2> <ul> <li>(Java) Fixed custom delimiter issues in TextIO (<a href="https://redirect.github.com/apache/beam/issues/32249";>#32249</a>, <a href="https://redirect.github.com/apache/beam/issues/32251";>#32251</a>).</li> <li>(Java, Python, Go) Fixed PeriodicSequence backlog bytes reporting, which was preventing Dataflow Runner autoscaling from functioning properly (<a href="https://redirect.github.com/apache/beam/issues/32506";>#32506</a>).</li> <li>(Java) Fix improper decoding of rows with schemas containing nullable fields when encoded with a schema with equal encoding positions but modified field order. (<a href="https://redirect.github.com/apache/beam/issues/32388";>#32388</a>).</li> </ul> <h2>Known Issues</h2> <p>N/A</p> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/apache/beam/blob/master/CHANGES.md";>org.apache.beam:beam-runners-direct-java's changelog</a>.</em></p> <blockquote> <h1>[2.60.0] - Unreleased</h1> <h2>Highlights</h2> <ul> <li>Added support for using vLLM in the RunInference transform (Python) (<a href="https://redirect.github.com/apache/beam/issues/32528";>#32528</a>)</li> <li>[Managed Iceberg] Added support for streaming writes (<a href="https://redirect.github.com/apache/beam/pull/32451";>#32451</a>)</li> <li>[Managed Iceberg] Added auto-sharding for streaming writes (<a href="https://redirect.github.com/apache/beam/pull/32612";>#32612</a>)</li> <li>[Managed Iceberg] Added support for writing to dynamic destinations (<a href="https://redirect.github.com/apache/beam/pull/32565";>#32565</a>)</li> </ul> <h2>I/Os</h2> <ul> <li>PubsubIO can validate that the Pub/Sub topic exists before running the Read/Write pipeline (Java) (<a href="https://redirect.github.com/apache/beam/pull/32465";>#32465</a>)</li> </ul> <h2>New Features / Improvements</h2> <ul> <li>Dataflow worker can install packages from Google Artifact Registry Python repositories (Python) (<a href="https://redirect.github.com/apache/beam/issues/32123";>#32123</a>).</li> <li>Added support for Zstd codec in SerializableAvroCodecFactory (Java) (<a href="https://redirect.github.com/apache/beam/issues/32349";>#32349</a>)</li> <li>Added support for using vLLM in the RunInference transform (Python) (<a href="https://redirect.github.com/apache/beam/issues/32528";>#32528</a>)</li> <li>Prism release binaries and container bootloaders are now being built with the latest Go 1.23 patch. (<a href="https://redirect.github.com/apache/beam/pull/32575";>#32575</a>)</li> <li>Prism <ul> <li>Prism now supports Bundle Finalization. (<a href="https://redirect.github.com/apache/beam/pull/32425";>#32425</a>)</li> </ul> </li> <li>Significantly improved performance of Kafka IO reads that enable <a href="https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/kafka/KafkaIO.Read.html#commitOffsetsInFinalize--";>commitOffsetsInFinalize</a> by removing the data reshuffle from SDF implementation. (<a href="https://redirect.github.com/apache/beam/pull/31682";>#31682</a>).</li> <li>Added support for dynamic writing in MqttIO (Java) (<a href="https://redirect.github.com/apache/beam/issues/19376";>#19376</a>)</li> <li>Optimized Spark Runner parDo transform evaluator (Java) (<a href="https://redirect.github.com/apache/beam/issues/32537";>#32537</a>)</li> <li>[Managed Iceberg] More efficient manifest file writes/commits (<a href="https://redirect.github.com/apache/beam/issues/32666";>#32666</a>)</li> </ul> <h2>Breaking Changes</h2> <ul> <li>In Python, assert_that now throws if it is not in a pipeline context instead of silently succeeding (<a href="https://redirect.github.com/apache/beam/pull/30771";>#30771</a>)</li> <li>In Python and YAML, ReadFromJson now override the dtype from None to an explicit False. Most notably, string values like <code>"123"</code> are preserved as strings rather than silently coerced (and possibly truncated) to numeric values. To retain the old behavior, pass <code>dtype=True</code> (or any other value accepted by <code>pandas.read_json</code>).</li> <li>Users of KafkaIO Read transform that enable <a href="https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/kafka/KafkaIO.Read.html#commitOffsetsInFinalize--";>commitOffsetsInFinalize</a> might encounter pipeline graph compatibility issues when updating the pipeline. To mitigate, set the <code>updateCompatibilityVersion</code> option to the SDK version used for the original pipeline, example <code>--updateCompatabilityVersion=2.58.1</code></li> </ul> <h2>Deprecations</h2> <ul> <li>Python 3.8 is reaching EOL and support is being removed in Beam 2.61.0. The 2.60.0 release will warn users when running on 3.8. (<a href="https://redirect.github.com/apache/beam/issues/31192";>#31192</a>)</li> </ul> <h2>Bugfixes</h2> <ul> <li>(Java) Fixed custom delimiter issues in TextIO (<a href="https://redirect.github.com/apache/beam/issues/32249";>#32249</a>, <a href="https://redirect.github.com/apache/beam/issues/32251";>#32251</a>).</li> <li>(Java, Python, Go) Fixed PeriodicSequence backlog bytes reporting, which was preventing Dataflow Runner autoscaling from functioning properly (<a href="https://redirect.github.com/apache/beam/issues/32506";>#32506</a>).</li> <li>(Java) Fix improper decoding of rows with schemas containing nullable fields when encoded with a schema with equal encoding positions but modified field order. (<a href="https://redirect.github.com/apache/beam/issues/32388";>#32388</a>).</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/apache/beam/commit/02208fc59887f22d5130310c902bb422f1f0be33";><code>02208fc</code></a> Set version for 2.60.0 RC2</li> <li><a href="https://github.com/apache/beam/commit/83a883d6cf6b239436a4df1187495de2b1dd2441";><code>83a883d</code></a> fix: skip close on bundles (<a href="https://redirect.github.com/apache/beam/issues/32661";>#32661</a>) (<a href="https://redirect.github.com/apache/beam/issues/32759";>#32759</a>)</li> <li><a href="https://github.com/apache/beam/commit/30c109faf3132a32a6f07d9bc184b873df318eb4";><code>30c109f</code></a> Merge pull request <a href="https://redirect.github.com/apache/beam/issues/32747";>#32747</a> from apache/pydocUpdate (<a href="https://redirect.github.com/apache/beam/issues/32762";>#32762</a>)</li> <li><a href="https://github.com/apache/beam/commit/7d0d9c65cb6d27910eb6bcaf695a19c94b5d8bb4";><code>7d0d9c6</code></a> Fix build_wheels and build_release_candidate (<a href="https://redirect.github.com/apache/beam/issues/32725";>#32725</a>) (<a href="https://redirect.github.com/apache/beam/issues/32761";>#32761</a>)</li> <li><a href="https://github.com/apache/beam/commit/20a45bb6eaa271074f8b1e1a3c4e21b9f8bd97ce";><code>20a45bb</code></a> Fix the FlinkRunner for Dataproc (<a href="https://redirect.github.com/apache/beam/issues/32742";>#32742</a>)</li> <li><a href="https://github.com/apache/beam/commit/75422cb58acf68f6323fb5c270ee56886a9e11bf";><code>75422cb</code></a> Force kafka 3.1.2 for expansion-service jar (<a href="https://redirect.github.com/apache/beam/issues/32703";>#32703</a>) (<a href="https://redirect.github.com/apache/beam/issues/32708";>#32708</a>)</li> <li><a href="https://github.com/apache/beam/commit/f61c8e13047bfe3c0d0332bcd2114ea7dbec3189";><code>f61c8e1</code></a> Enforce a size limit on StringSetData (<a href="https://redirect.github.com/apache/beam/issues/32650";>#32650</a>) (<a href="https://redirect.github.com/apache/beam/issues/32707";>#32707</a>)</li> <li><a href="https://github.com/apache/beam/commit/96a96daa011d38fd58ad5dbcd8e6c1478d948c40";><code>96a96da</code></a> Report File Lineage on directory (<a href="https://redirect.github.com/apache/beam/issues/32662";>#32662</a>) (<a href="https://redirect.github.com/apache/beam/issues/32706";>#32706</a>)</li> <li><a href="https://github.com/apache/beam/commit/64854bb4a04f5c6becc6f254776b5754f947ebd5";><code>64854bb</code></a> Revert "Set setFailIfPoolExhausted in SessionPoolOptions for SpannerAccessor"...</li> <li><a href="https://github.com/apache/beam/commit/8eef4a3c3a3a676761e4f6748fcc933e70f6ffc0";><code>8eef4a3</code></a> [Managed Iceberg] add GiB autosharding (<a href="https://redirect.github.com/apache/beam/issues/32612";>#32612</a>) (<a href="https://redirect.github.com/apache/beam/issues/32663";>#32663</a>)</li> <li>Additional commits viewable in <a href="https://github.com/apache/beam/compare/v2.59.0...v2.60.0";>compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
