dependabot[bot] opened a new pull request, #64: URL: https://github.com/apache/beam-starter-python/pull/64
Bumps [apache-beam](https://github.com/apache/beam) from 2.75.0 to 2.77.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/apache/beam/releases";>apache-beam's releases</a>.</em></p> <blockquote> <h2>Beam 2.77.0 release</h2> <p>We are happy to present the new 2.77.0 release of Beam. This release includes both improvements and new functionality. See the <a href="https://github.com/apache/beam/blob/HEAD/get-started/downloads/#2770-2026-10-07";>download page</a> for this release.</p> <!-- raw HTML omitted --> <p>For more information on changes in 2.77.0, check out the <a href="https://github.com/apache/beam/milestone/44";>detailed release notes</a>.</p> <h2>Highlights</h2> <h3>I/Os</h3> <ul> <li>Added <code>schema_update_options</code> to <code>WriteToBigQuery</code> file loads, allowing BigQuery load jobs to add nullable fields or relax required fields when appending data (Python) (<a href="https://redirect.github.com/apache/beam/issues/21141";>#21141</a>).</li> <li>BigQueryIO now supports reading BigQuery Lakehouse runtime catalog (BigLake metastore) Iceberg tables with the Storage Read API, using 4-part <code>project.catalog.namespace.table</code> identifiers (or a <code>TableReference</code> with a composite <code>catalog.namespace</code> dataset id). Previously such references were silently mis-parsed (Java) (<a href="https://redirect.github.com/apache/beam/issues/39597";>#39597</a>) .</li> <li>SolaceIO now supports reading and writing binary and text content data payload (Java) (<a href="https://redirect.github.com/apache/beam/issues/39875";>#39875</a>).</li> <li>ClickHouseIO: support writing <code>Decimal(P, S)</code> / <code>Decimal32/64/128/256</code> columns (Java) (<a href="https://redirect.github.com/apache/beam/issues/39840";>#39840</a>).</li> <li>SolaceIO now supports reading and writing user properties (message metadata) (Java) (<a href="https://redirect.github.com/apache/beam/issues/40099";>#40099</a>).</li> <li>[IcebergIO] AddFiles (<code>IcebergAddFiles</code> in YAML) can evolve the table schema before registering files, with <code>schema_evolution_options</code>, <code>required_columns</code>, <code>incompatible_schema_handling</code> and <code>unverifiable_file_handling</code> (Java/YAML, batch only) (<a href="https://redirect.github.com/apache/beam/issues/40144";>#40144</a>).</li> </ul> <h3>New Features / Improvements</h3> <ul> <li>(Java/Python) <code>Watch</code> can bound its deduplication state by event time, retiring an output key once the greatest emitted timestamp has moved more than the allowed lateness past it. Java adds <code>Watch.growthOf(...).withTimestampCursor()</code>. Python adds <code>allowed_lateness</code> for the existing <code>timestamp_cursor</code> option (<a href="https://redirect.github.com/apache/beam/issues/18459";>#18459</a>).</li> <li>(Java) Spark Structured Streaming runner: stateful ParDo with state, timers, <code>@RequiresTimeSortedInput</code> and tagged outputs is now supported in batch mode (<a href="https://redirect.github.com/apache/beam/issues/39779";>#39779</a>).</li> <li>(Python) Added support for Vertex AI Model Monitoring V2 in RunInference (<a href="https://redirect.github.com/apache/beam/issues/39738";>#39738</a>).</li> <li>[Flink Runner] Added opt-in static round-robin split assignment for small bounded sources via the new <code>sourceStaticSplitThresholdMb</code> pipeline option. The default of 0 keeps the existing lazy pull-based assignment (<a href="https://redirect.github.com/apache/beam/issues/39873";>#39873</a>).</li> <li>Added automatic caching of bounded, single-pane side-input views for classic Java Flink DataStream execution (<a href="https://redirect.github.com/apache/beam/issues/39866";>#39866</a>).</li> <li>(Python) Added <code>Sample.Any</code>, the Python equivalent of Java's <code>Sample.any</code>, which returns up to n arbitrary elements from a PCollection (<a href="https://redirect.github.com/apache/beam/issues/18552";>#18552</a>).</li> </ul> <h3>Breaking Changes</h3> <ul> <li>Portable Java SDK now encodes SchemaCoders in a portable way (<a href="https://redirect.github.com/apache/beam/issues/34672";>#34672</a>). <ul> <li>Original custom Java coder encoding can still be obtained using <a href="https://github.com/apache/beam/blob/2cf0930e7ae1aa389c26ce6639b584877a3e31d9/sdks/java/core/src/main/java/org/apache/beam/sdk/options/StreamingOptions.java#L47";>StreamingOptions.setUpdateCompatibilityVersion("2.76")</a> (<a href="https://redirect.github.com/apache/beam/issues/34672";>#34672</a>).</li> <li>Fixes (<a href="https://redirect.github.com/apache/beam/issues/36496";>#36496</a>), (<a href="https://redirect.github.com/apache/beam/issues/30276";>#30276</a>), (<a href="https://redirect.github.com/apache/beam/issues/29245";>#29245</a>).</li> </ul> </li> <li>(Python) <code>TensorRTEngineHandlerNumPy</code> now requires TensorRT 10 or later. TensorRT 8.x is no longer supported, since TensorRT 10 removed the engine binding API the handler was written against (<a href="https://redirect.github.com/apache/beam/issues/36306";>#36306</a>). <ul> <li>Engines serialized by TensorRT 8.x must be rebuilt, as an engine can only be deserialized by the major version that built it.</li> <li>TensorRT 10 and later require a GPU with compute capability 7.5 or higher, which excludes NVIDIA Pascal and Volta GPUs.</li> <li>If dropping TensorRT 8.x support is a hard blocker for you, please comment on (<a href="https://redirect.github.com/apache/beam/issues/36306";>#36306</a>).</li> </ul> </li> </ul> <h3>Bugfixes</h3> <ul> <li>(Java) Fixed the Spark runner firing processing-time timers in reverse timestamp order (<a href="https://redirect.github.com/apache/beam/issues/39824";>#39824</a>).</li> <li>(Java) Fixed the Spark runner dropping the stored watermark of a streaming source with no update in a batch (<a href="https://redirect.github.com/apache/beam/issues/39822";>#39822</a>).</li> <li>(Python) Fixed incorrect profiler options handling on portable runners (<a href="https://redirect.github.com/apache/beam/issues/39613";>#39613</a>).</li> <li>(Java) KafkaIO dynamic reads no longer require the obsolete <code>beam_fn_api</code> experiment (<a href="https://redirect.github.com/apache/beam/issues/29998";>#29998</a>).</li> <li>(Prism) Self-checkpointing splittable DoFns now resume after their requested delay instead of immediately, so polling SDFs no longer busy-spin (<a href="https://redirect.github.com/apache/beam/issues/39848";>#39848</a>).</li> <li>(Java) MongoDbIO read splitting now preserves non-ObjectId <code>_id</code> types (e.g. string ids) instead of failing to parse the generated range filters (<a href="https://redirect.github.com/apache/beam/issues/39900";>#39900</a>).</li> <li>(Go) Fixed GCS glob matching silently dropping objects when the glob pattern contains multi-byte characters (<a href="https://redirect.github.com/apache/beam/issues/39969";>#39969</a>).</li> <li>(Python) Fixed <code>TensorRTEngineHandlerNumPy</code> failing with <code>CUDA_ERROR_INVALID_VALUE</code> on models with a single-element input or output tensor (<a href="https://redirect.github.com/apache/beam/issues/36306";>#36306</a>).</li> <li>(Python) Fixed <code>PickleCoder</code>/<code>_MemoizingPickleCoder.as_deterministic_coder()</code> raising <code>TypeError</code> instead of returning a working deterministic coder (<a href="https://redirect.github.com/apache/beam/issues/28558";>#28558</a>).</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/apache/beam/blob/master/CHANGES.md";>apache-beam's changelog</a>.</em></p> <blockquote> <h1>[2.77.0] - 2026-10-07</h1> <h2>I/Os</h2> <ul> <li>Added <code>schema_update_options</code> to <code>WriteToBigQuery</code> file loads, allowing BigQuery load jobs to add nullable fields or relax required fields when appending data (Python) (<a href="https://redirect.github.com/apache/beam/issues/21141";>#21141</a>).</li> <li>BigQueryIO now supports reading BigQuery Lakehouse runtime catalog (BigLake metastore) Iceberg tables with the Storage Read API, using 4-part <code>project.catalog.namespace.table</code> identifiers (or a <code>TableReference</code> with a composite <code>catalog.namespace</code> dataset id). Previously such references were silently mis-parsed (Java) (<a href="https://redirect.github.com/apache/beam/issues/39597";>#39597</a>) .</li> <li>SolaceIO now supports reading and writing binary and text content data payload (Java) (<a href="https://redirect.github.com/apache/beam/issues/39875";>#39875</a>).</li> <li>ClickHouseIO: support writing <code>Decimal(P, S)</code> / <code>Decimal32/64/128/256</code> columns (Java) (<a href="https://redirect.github.com/apache/beam/issues/39840";>#39840</a>).</li> <li>SolaceIO now supports reading and writing user properties (message metadata) (Java) (<a href="https://redirect.github.com/apache/beam/issues/40099";>#40099</a>).</li> <li>[IcebergIO] AddFiles (<code>IcebergAddFiles</code> in YAML) can evolve the table schema before registering files, with <code>schema_evolution_options</code>, <code>required_columns</code>, <code>incompatible_schema_handling</code> and <code>unverifiable_file_handling</code> (Java/YAML, batch only) (<a href="https://redirect.github.com/apache/beam/issues/40144";>#40144</a>).</li> <li>[IcebergIO] Added batch and streaming CDC writes that applies INSERT/UPDATE_BEFORE/UPDATE_AFTER/DELETE changes to Iceberg V2+ tables by primary key. Invoke with <code>IcebergIO.writeCdcRows</code> (Java) or by setting <code>mode: merge-on-read</code> on the Managed <code>ICEBERG</code> write (Java, Python, YAML) (<a href="https://redirect.github.com/apache/beam/issues/39979";>#39979</a>).</li> <li>[IcebergIO] Added an optional side-input table cache for writes to significantly reduce catalog and table requests for large pipelines. A single worker polls the table and broadcasts it to other workers in the pipeline. Enable with <code>IcebergIO.writeRows(...).withSideInputTableCache()</code> (Java) or by setting <code>use_side_input_table_cache: true</code> on the Managed <code>ICEBERG</code> write (Java, Python) (<a href="https://redirect.github.com/apache/beam/issues/39723";>#39723</a>).</li> </ul> <h2>New Features / Improvements</h2> <ul> <li>(Java/Python) <code>Watch</code> can bound its deduplication state by event time, retiring an output key once the greatest emitted timestamp has moved more than the allowed lateness past it. Java adds <code>Watch.growthOf(...).withTimestampCursor()</code>. Python adds <code>allowed_lateness</code> for the existing <code>timestamp_cursor</code> option (<a href="https://redirect.github.com/apache/beam/issues/18459";>#18459</a>).</li> <li>(Java) Spark Structured Streaming runner: stateful ParDo with state, timers, <code>@RequiresTimeSortedInput</code> and tagged outputs is now supported in batch mode (<a href="https://redirect.github.com/apache/beam/issues/39779";>#39779</a>).</li> <li>(Python) Added support for Vertex AI Model Monitoring V2 in RunInference (<a href="https://redirect.github.com/apache/beam/issues/39738";>#39738</a>).</li> <li>[Flink Runner] Added opt-in static round-robin split assignment for small bounded sources via the new <code>sourceStaticSplitThresholdMb</code> pipeline option. The default of 0 keeps the existing lazy pull-based assignment (<a href="https://redirect.github.com/apache/beam/issues/39873";>#39873</a>).</li> <li>Added automatic caching of bounded, single-pane side-input views for classic Java Flink DataStream execution (<a href="https://redirect.github.com/apache/beam/issues/39866";>#39866</a>).</li> <li>(Python) Added <code>Sample.Any</code>, the Python equivalent of Java's <code>Sample.any</code>, which returns up to n arbitrary elements from a PCollection (<a href="https://redirect.github.com/apache/beam/issues/18552";>#18552</a>).</li> </ul> <h2>Breaking Changes</h2> <ul> <li>Portable Java SDK now encodes SchemaCoders in a portable way (<a href="https://redirect.github.com/apache/beam/issues/34672";>#34672</a>). <ul> <li>Original custom Java coder encoding can still be obtained using <a href="https://github.com/apache/beam/blob/2cf0930e7ae1aa389c26ce6639b584877a3e31d9/sdks/java/core/src/main/java/org/apache/beam/sdk/options/StreamingOptions.java#L47";>StreamingOptions.setUpdateCompatibilityVersion("2.76")</a> (<a href="https://redirect.github.com/apache/beam/issues/34672";>#34672</a>).</li> <li>Fixes (<a href="https://redirect.github.com/apache/beam/issues/36496";>#36496</a>), (<a href="https://redirect.github.com/apache/beam/issues/30276";>#30276</a>), (<a href="https://redirect.github.com/apache/beam/issues/29245";>#29245</a>).</li> </ul> </li> <li>(Python) <code>TensorRTEngineHandlerNumPy</code> now requires TensorRT 10 or later. TensorRT 8.x is no longer supported, since TensorRT 10 removed the engine binding API the handler was written against (<a href="https://redirect.github.com/apache/beam/issues/36306";>#36306</a>). <ul> <li>Engines serialized by TensorRT 8.x must be rebuilt, as an engine can only be deserialized by the major version that built it.</li> <li>TensorRT 10 and later require a GPU with compute capability 7.5 or higher, which excludes NVIDIA Pascal and Volta GPUs.</li> <li>If dropping TensorRT 8.x support is a hard blocker for you, please comment on (<a href="https://redirect.github.com/apache/beam/issues/36306";>#36306</a>).</li> </ul> </li> </ul> <h2>Bugfixes</h2> <ul> <li>(Java) Fixed the Spark runner firing processing-time timers in reverse timestamp order (<a href="https://redirect.github.com/apache/beam/issues/39824";>#39824</a>).</li> <li>(Java) Fixed the Spark runner dropping the stored watermark of a streaming source with no update in a batch (<a href="https://redirect.github.com/apache/beam/issues/39822";>#39822</a>).</li> <li>(Python) Fixed incorrect profiler options handling on portable runners (<a href="https://redirect.github.com/apache/beam/issues/39613";>#39613</a>).</li> <li>(Python) Fixed <code>AsyncWrapper</code> mixing results between keys or windows whose values or custom identifiers match, and cancelling work belonging to other windows (<a href="https://redirect.github.com/apache/beam/pull/39996";>#39996</a>).</li> <li>(Java) KafkaIO dynamic reads no longer require the obsolete <code>beam_fn_api</code> experiment (<a href="https://redirect.github.com/apache/beam/issues/29998";>#29998</a>).</li> <li>(Prism) Self-checkpointing splittable DoFns now resume after their requested delay instead of immediately, so polling SDFs no longer busy-spin (<a href="https://redirect.github.com/apache/beam/issues/39848";>#39848</a>).</li> <li>(Java) MongoDbIO read splitting now preserves non-ObjectId <code>_id</code> types (e.g. string ids) instead of failing to parse the generated range filters (<a href="https://redirect.github.com/apache/beam/issues/39900";>#39900</a>).</li> <li>(Go) Fixed GCS glob matching silently dropping objects when the glob pattern contains multi-byte characters (<a href="https://redirect.github.com/apache/beam/issues/39969";>#39969</a>).</li> <li>(Python) Fixed <code>TensorRTEngineHandlerNumPy</code> failing with <code>CUDA_ERROR_INVALID_VALUE</code> on models with a single-element input or output tensor (<a href="https://redirect.github.com/apache/beam/issues/36306";>#36306</a>).</li> <li>(Python) Fixed <code>PickleCoder</code>/<code>_MemoizingPickleCoder.as_deterministic_coder()</code> raising <code>TypeError</code> instead of returning a working deterministic coder (<a href="https://redirect.github.com/apache/beam/issues/28558";>#28558</a>).</li> </ul> <h1>[2.76.0] - 2026-08-31</h1> <h2>Highlights</h2> <ul> <li>Added a full Iceberg batch and streaming changelog source (CDC) (<a href="https://redirect.github.com/apache/beam/issues/38831";>#38831</a>)</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/apache/beam/commit/1a6fce487c12193439eb69ca7b9a3114cce4d994";><code>1a6fce4</code></a> Set version for 2.77.0 RC4</li> <li><a href="https://github.com/apache/beam/commit/5cd677e3d14421c33a92cda7257a5c2be104fc30";><code>5cd677e</code></a> Merge pull request <a href="https://redirect.github.com/apache/beam/issues/40337";>#40337</a> from claudevdm/cp-40252</li> <li><a href="https://github.com/apache/beam/commit/be2778396fbc0a175de18f4fdd13ec6097566d7c";><code>be27783</code></a> Exposing evictWritersWhenFull in AvroIO (<a href="https://redirect.github.com/apache/beam/issues/40336";>#40336</a>) (<a href="https://redirect.github.com/apache/beam/issues/40338";>#40338</a>)</li> <li><a href="https://github.com/apache/beam/commit/8c9d31ed3e7e4f28b37cec164577da29281d30c0";><code>8c9d31e</code></a> Merge pull request <a href="https://redirect.github.com/apache/beam/issues/40259";>#40259</a> from reuvenlax/improve_gcp_presubmit_speed (<a href="https://redirect.github.com/apache/beam/issues/40342";>#40342</a>)</li> <li><a href="https://github.com/apache/beam/commit/c45e45ca099e75d54fd54b08474d26a85a1e4d73";><code>c45e45c</code></a> Addfiles partition bounds fixes (<a href="https://redirect.github.com/apache/beam/issues/40252";>#40252</a>)</li> <li><a href="https://github.com/apache/beam/commit/34e08cf5a12b9fa0863ce5676d4ce266ea8cafc2";><code>34e08cf</code></a> Merge pull request <a href="https://redirect.github.com/apache/beam/issues/40287";>#40287</a> from damccorm/cp-40205-40251-release-2.77</li> <li><a href="https://github.com/apache/beam/commit/e1b0794826786758a16836a97e90a88241f0b92a";><code>e1b0794</code></a> Merge pull request <a href="https://redirect.github.com/apache/beam/issues/40271";>#40271</a> from ahmedabu98/cp40253</li> <li><a href="https://github.com/apache/beam/commit/bcb44cffa54b64e3418ef2339cf8a5dfb5e7c092";><code>bcb44cf</code></a> Time out RestrictionTrackers.trySplit after 5 minutes on lock contention (<a href="https://redirect.github.com/apache/beam/issues/40";>#40</a>...</li> <li><a href="https://github.com/apache/beam/commit/84da653377b340bcf04dde2ea276754f7507a4e6";><code>84da653</code></a> Return last evaluated progress on lock timeout in RestrictionTrackers.getProg...</li> <li><a href="https://github.com/apache/beam/commit/396ae8e96ecccd820c83fbd4a65b8e9472538eab";><code>396ae8e</code></a> Merge branch 'release-2.77' of <a href="https://github.com/apache/beam";>https://github.com/apache/beam</a> into cp40253</li> <li>Additional commits viewable in <a href="https://github.com/apache/beam/compare/v2.75.0...v2.77.0";>compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
