dependabot[bot] opened a new pull request, #51: URL: https://github.com/apache/beam-starter-python/pull/51
Bumps [apache-beam](https://github.com/apache/beam) from 2.68.0 to 2.69.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/apache/beam/releases";>apache-beam's releases</a>.</em></p> <blockquote> <h2>Beam 2.69.0 release</h2> <p>We are happy to present the new 2.69.0 release of Beam. This release includes both improvements and new functionality. See the <a href="https://github.com/apache/beam/blob/HEAD/get-started/downloads/#2690-2025-10-??";>download page</a> for this release.</p> <!-- raw HTML omitted --> <p>For more information on changes in 2.69.0, check out the <a href="https://github.com/apache/beam/milestone/37?closed=1";>detailed release notes</a>.</p> <h2>Highlights</h2> <ul> <li>(Python) Add YAML Editor and Visualization Panel (<a href="https://redirect.github.com/apache/beam/issues/35772";>#35772</a>).</li> <li>(Java) Java 25 Support (<a href="https://redirect.github.com/apache/beam/issues/35627";>#35772</a>).</li> </ul> <h3>I/Os</h3> <ul> <li>Upgraded Iceberg dependency to 1.10.0 (<a href="https://redirect.github.com/apache/beam/issues/36123";>#36123</a>).</li> </ul> <h3>New Features / Improvements</h3> <ul> <li>Enhance JAXBCoder with XMLInputFactory support (Java) (<a href="https://redirect.github.com/apache/beam/issues/36446";>#36446</a>).</li> <li>Python examples added for CloudSQL enrichment handler on <a href="https://beam.apache.org/documentation/transforms/python/elementwise/enrichment-cloudsql/";>Beam website</a> (Python) (<a href="https://redirect.github.com/apache/beam/issues/36095";>#35473</a>).</li> <li>Support for batch mode execution in WriteToPubSub transform added (Python) (<a href="https://redirect.github.com/apache/beam/issues/35990";>#35990</a>).</li> <li>Added official support for Python 3.13 (<a href="https://redirect.github.com/apache/beam/issues/34869";>#34869</a>).</li> <li>Added an optional output_schema verification to all YAML transforms (<a href="https://redirect.github.com/apache/beam/issues/35952";>#35952</a>).</li> <li>Support for encryption when using GroupByKey added, along with <code>--gbek</code> pipeline option to automatically replace all GroupByKey transforms (Java/Python) (<a href="https://redirect.github.com/apache/beam/issues/36214";>#36214</a>).</li> </ul> <h3>Breaking Changes</h3> <ul> <li>(Python) <code>dill</code> is no longer a required, default dependency for Apache Beam (<a href="https://redirect.github.com/apache/beam/issues/21298";>#21298</a>). <ul> <li>This change only affects pipelines that explicitly use the <code>pickle_library=dill</code> pipeline option.</li> <li>While <code>dill==0.3.1.1</code> is still pre-installed on the official Beam SDK base images, it is no longer a direct dependency of the apache-beam Python package. This means it can be overridden by other dependencies in your environment.</li> <li>If your pipeline uses <code>pickle_library=dill</code>, you must manually ensure <code>dill==0.3.1.1</code> is installed in both your submission and runtime environments. <ul> <li>Submission environment: Install the dill extra in your local environment <code>pip install apache-beam[gcpdill]</code>.</li> <li>Runtime (worker) environment: Your action depends on how you manage your worker's environment. <ul> <li>If using default containers or custom containers with the official Beam base image e.g. <code>FROM apache/beam_python3.10_sdk:2.69.0</code> <ul> <li>Add <code>dill==0.3.1.1</code> to your worker's requirements file (e.g., requirements.txt)</li> <li>Pass this file to your pipeline using the --requirements_file requirements.txt pipeline option (For more details see <a href="https://cloud.google.com/dataflow/docs/guides/manage-dependencies#py-custom-containers";>managing Dataflow dependencies</a>).</li> </ul> </li> <li>If custom containers with a non-Beam base image e.g. <code>FROM python:3.9-slim</code> <ul> <li>Install apache-beam with the dill extra in your docker file e.g. <code>RUN pip install --no-cache-dir apache-beam[gcp,dill]</code></li> </ul> </li> </ul> </li> </ul> </li> <li>If there is a dill version mismatch between submission and runtime environments you might encounter unpickling errors like <code>Can't get attribute '_create_code' on <module 'dill._dill' from...</code>.</li> <li>If dill is not installed in the runtime environment you will see the error <code>ImportError: Pipeline option pickle_library=dill is set, but dill is not installed...</code></li> <li>Report any issues you encounter when using <code>pickle_library=dill</code> to the GitHub issue (<a href="https://redirect.github.com/apache/beam/issues/21298";>#21298</a>)</li> </ul> </li> <li>(Python) Added a <code>pickle_library=dill_unsafe</code> pipeline option. This allows overriding <code>dill==0.3.1.1</code> using dill as the pickle_library. Use with extreme caution. Other versions of dill has not been tested with Apache Beam (<a href="https://redirect.github.com/apache/beam/issues/21298";>#21298</a>).</li> <li>(Python) The deterministic fallback coder for complex types like NamedTuple, Enum, and dataclasses now normalizes filepaths for better determinism guarantees. This affects streaming pipelines updating from 2.68 to 2.69 that utilize this fallback coder. If your pipeline is affected, you may see a warning like: "Using fallback deterministic coder for type X...". To update safely sepcify the pipeline option <code>--update_compatibility_version=2.68.0</code> (<a href="https://redirect.github.com/apache/beam/pull/36345";>#36345</a>).</li> <li>(Python) Fixed transform naming conflict when executing DataTransform on a dictionary of PColls (<a href="https://redirect.github.com/apache/beam/issues/30445";>#30445</a>). This may break update compatibility if you don't provide a <code>--transform_name_mapping</code>.</li> <li>Removed deprecated Hadoop versions (2.10.2 and 3.2.4) that are no longer supported for <a href="https://redirect.github.com/apache/iceberg/issues/10940";>Iceberg</a> from IcebergIO (<a href="https://redirect.github.com/apache/beam/issues/36282";>#36282</a>).</li> <li>(Go) Coder construction on SDK side is more faithful to the specs from runners without stripping length-prefix. This may break streaming pipeline update as the underlying coder could be changed (<a href="https://redirect.github.com/apache/beam/issues/36387";>#36387</a>).</li> <li>Minimum Go version for Beam Go updated to 1.25.2 (<a href="https://redirect.github.com/apache/beam/issues/36461";>#36461</a>).</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/apache/beam/blob/master/CHANGES.md";>apache-beam's changelog</a>.</em></p> <blockquote> <h1>[2.69.0] - Unreleased</h1> <h2>Highlights</h2> <ul> <li>New highly anticipated feature X added to Python SDK (<a href="https://github.com/apache/beam/issues/X";>#X</a>).</li> <li>New highly anticipated feature Y added to Java SDK (<a href="https://github.com/apache/beam/issues/Y";>#Y</a>).</li> <li>(Python) Add YAML Editor and Visualization Panel (<a href="https://redirect.github.com/apache/beam/issues/35772";>#35772</a>).</li> <li>(Java) Java 25 Support (<a href="https://redirect.github.com/apache/beam/issues/35627";>#35772</a>).</li> </ul> <h2>I/Os</h2> <ul> <li>Support for X source added (Java/Python) (<a href="https://github.com/apache/beam/issues/X";>#X</a>).</li> <li>Upgraded Iceberg dependency to 1.10.0 (<a href="https://redirect.github.com/apache/beam/issues/36123";>#36123</a>).</li> </ul> <h2>New Features / Improvements</h2> <ul> <li>Enhance JAXBCoder with XMLInputFactory support (Java) (<a href="https://redirect.github.com/apache/beam/issues/36446";>#36446</a>).</li> <li>Python examples added for CloudSQL enrichment handler on <a href="https://beam.apache.org/documentation/transforms/python/elementwise/enrichment-cloudsql/";>Beam website</a> (Python) (<a href="https://redirect.github.com/apache/beam/issues/36095";>#35473</a>).</li> <li>Support for batch mode execution in WriteToPubSub transform added (Python) (<a href="https://redirect.github.com/apache/beam/issues/35990";>#35990</a>).</li> <li>Added official support for Python 3.13 (<a href="https://redirect.github.com/apache/beam/issues/34869";>#34869</a>).</li> <li>Added an optional output_schema verification to all YAML transforms (<a href="https://redirect.github.com/apache/beam/issues/35952";>#35952</a>).</li> <li>Support for encryption when using GroupByKey added, along with <code>--gbek</code> pipeline option to automatically replace all GroupByKey transforms (Java/Python) (<a href="https://redirect.github.com/apache/beam/issues/36214";>#36214</a>).</li> <li>In Python SDK, the <code>--element_processing_timeout_minutes</code> option will also interrupt the SDK process if slowness happens during DoFn initialization, for example in <code>DoFn.setup()</code> (<a href="https://redirect.github.com/apache/beam/issues/36518";>#36518</a>).</li> </ul> <h2>Breaking Changes</h2> <ul> <li>(Python) <code>dill</code> is no longer a required, default dependency for Apache Beam (<a href="https://redirect.github.com/apache/beam/issues/21298";>#21298</a>). <ul> <li>This change only affects pipelines that explicitly use the <code>pickle_library=dill</code> pipeline option.</li> <li>While <code>dill==0.3.1.1</code> is still pre-installed on the official Beam SDK base images, it is no longer a direct dependency of the apache-beam Python package. This means it can be overridden by other dependencies in your environment.</li> <li>If your pipeline uses <code>pickle_library=dill</code>, you must manually ensure <code>dill==0.3.1.1</code> is installed in both your submission and runtime environments. <ul> <li>Submission environment: Install the dill extra in your local environment <code>pip install apache-beam[gcpdill]</code>.</li> <li>Runtime (worker) environment: Your action depends on how you manage your worker's environment. <ul> <li>If using default containers or custom containers with the official Beam base image e.g. <code>FROM apache/beam_python3.10_sdk:2.69.0</code> <ul> <li>Add <code>dill==0.3.1.1</code> to your worker's requirements file (e.g., requirements.txt)</li> <li>Pass this file to your pipeline using the --requirements_file requirements.txt pipeline option (For more details see <a href="https://cloud.google.com/dataflow/docs/guides/manage-dependencies#py-custom-containers";>managing Dataflow dependencies</a>).</li> </ul> </li> <li>If custom containers with a non-Beam base image e.g. <code>FROM python:3.9-slim</code> <ul> <li>Install apache-beam with the dill extra in your docker file e.g. <code>RUN pip install --no-cache-dir apache-beam[gcp,dill]</code></li> </ul> </li> </ul> </li> </ul> </li> <li>If there is a dill version mismatch between submission and runtime environments you might encounter unpickling errors like <code>Can't get attribute '_create_code' on <module 'dill._dill' from...</code>.</li> <li>If dill is not installed in the runtime environment you will see the error <code>ImportError: Pipeline option pickle_library=dill is set, but dill is not installed...</code></li> <li>Report any issues you encounter when using <code>pickle_library=dill</code> to the GitHub issue (<a href="https://redirect.github.com/apache/beam/issues/21298";>#21298</a>)</li> </ul> </li> <li>(Python) Added a <code>pickle_library=dill_unsafe</code> pipeline option. This allows overriding <code>dill==0.3.1.1</code> using dill as the pickle_library. Use with extreme caution. Other versions of dill has not been tested with Apache Beam (<a href="https://redirect.github.com/apache/beam/issues/21298";>#21298</a>).</li> <li>(Python) The deterministic fallback coder for complex types like NamedTuple, Enum, and dataclasses now normalizes filepaths for better determinism guarantees. This affects streaming pipelines updating from 2.68 to 2.69 that utilize this fallback coder. If your pipeline is affected, you may see a warning like: "Using fallback deterministic coder for type X...". To update safely sepcify the pipeline option <code>--update_compatibility_version=2.68.0</code> (<a href="https://redirect.github.com/apache/beam/pull/36345";>#36345</a>).</li> <li>(Python) Fixed transform naming conflict when executing DataTransform on a dictionary of PColls (<a href="https://redirect.github.com/apache/beam/issues/30445";>#30445</a>). This may break update compatibility if you don't provide a <code>--transform_name_mapping</code>.</li> <li>Removed deprecated Hadoop versions (2.10.2 and 3.2.4) that are no longer supported for <a href="https://redirect.github.com/apache/iceberg/issues/10940";>Iceberg</a> from IcebergIO (<a href="https://redirect.github.com/apache/beam/issues/36282";>#36282</a>).</li> <li>(Go) Coder construction on SDK side is more faithful to the specs from runners without stripping length-prefix. This may break streaming pipeline update as the underlying coder could be changed (<a href="https://redirect.github.com/apache/beam/issues/36387";>#36387</a>).</li> <li>Minimum Go version for Beam Go updated to 1.25.2 (<a href="https://redirect.github.com/apache/beam/issues/36461";>#36461</a>).</li> <li>(Java) DoFn OutputReceiver now requires implementing a builder method as part of extended metadata support for elements (<a href="https://redirect.github.com/apache/beam/issues/34902";>#34902</a>).</li> <li>(Java) Removed ProcessContext outputWindowedValue introduced in 2.68 that allowed setting offset and record Id. Use OutputReceiver's builder to set those field ([#36523]<a href="https://redirect.github.com/apache/beam/pull/36523";>apache/beam#36523</a>).</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/apache/beam/commit/2ca7d2119dc151553b2aa6b02bae47ba3e6ec61a";><code>2ca7d21</code></a> Set version for 2.69.0 RC3</li> <li><a href="https://github.com/apache/beam/commit/976cbc32854fab7be9200d952a300528938ae8bb";><code>976cbc3</code></a> Revert "Add GRPC experiments to Python dockerfile (<a href="https://redirect.github.com/apache/beam/issues/36525";>#36525</a>)" (<a href="https://redirect.github.com/apache/beam/issues/36573";>#36573</a>)</li> <li><a href="https://github.com/apache/beam/commit/ecdc49be961538ce83f2d79d8ac6673e4cf4e321";><code>ecdc49b</code></a> Fix dependency version (<a href="https://redirect.github.com/apache/beam/issues/36570";>#36570</a>)</li> <li><a href="https://github.com/apache/beam/commit/5729522a0cc94d46a56db58c0dd674187bab8269";><code>5729522</code></a> Merge pull request <a href="https://redirect.github.com/apache/beam/issues/36567";>#36567</a> from Abacn/cp-36564</li> <li><a href="https://github.com/apache/beam/commit/7d85f18d163d52bf077777137b92d2f8e942f5a6";><code>7d85f18</code></a> Fix BigQueryIO File load validate runtime value provider (<a href="https://redirect.github.com/apache/beam/issues/36564";>#36564</a>)</li> <li><a href="https://github.com/apache/beam/commit/c6e2df051e9c1f92ef4354c1be506bbefc18ffa6";><code>c6e2df0</code></a> CP: Fix publishing of ml/distroless images (<a href="https://redirect.github.com/apache/beam/issues/36565";>#36565</a>)</li> <li><a href="https://github.com/apache/beam/commit/b48d832da1f4a1d69f5de73620a40bcb9f816955";><code>b48d832</code></a> [release-2.69] Cherrypick <a href="https://redirect.github.com/apache/beam/issues/36518";>#36518</a> to the branch. (<a href="https://redirect.github.com/apache/beam/issues/36551";>#36551</a>)</li> <li><a href="https://github.com/apache/beam/commit/8739e50dba031780f1c0b02b29f4bede1026e4fd";><code>8739e50</code></a> Set Dataflow container to release version.</li> <li><a href="https://github.com/apache/beam/commit/ed39503878ef8b6aff8439e0b757068f5f5493a7";><code>ed39503</code></a> Skip TestTimers_ProcessingTime_Unbounded for spark. (<a href="https://redirect.github.com/apache/beam/issues/36527";>#36527</a>)</li> <li><a href="https://github.com/apache/beam/commit/abf1904759e4d0368061e832e261a5a8c052de2b";><code>abf1904</code></a> Merge pull request <a href="https://redirect.github.com/apache/beam/issues/36523";>#36523</a>: revert outputWindowedValue changes from KafkaIO as...</li> <li>Additional commits viewable in <a href="https://github.com/apache/beam/compare/v2.68.0...v2.69.0";>compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
