gemini-code-assist[bot] commented on PR #37434: URL: https://github.com/apache/beam/pull/37434#issuecomment-3812728912
## Summary of Changes This pull request significantly advances the type hinting capabilities within the Apache Beam Python SDK by introducing robust support for tagged outputs. It enables developers to explicitly define types for all outputs of a transform, whether main or side outputs, through decorator, method chaining, or function annotation styles. This enhancement aims to improve code readability, facilitate static analysis, and reduce runtime type-related errors in complex Beam pipelines that utilize multiple outputs. ### Highlights * **Enhanced Tagged Output Type Hinting**: Introduced comprehensive support for type hints on tagged outputs across various Beam transforms (ParDo, Map, FlatMap, etc.). This allows users to specify types for both main and side outputs, improving type safety and static analysis. * **Updated `with_output_types` API**: The `with_output_types` method and decorator now accept keyword arguments to define type hints for specific tagged outputs (e.g., `errors=str`), making the API more expressive for multi-output scenarios. * **Automatic Type Hint Extraction from Annotations**: New internal helper functions were added to automatically parse and extract main and tagged output types from function annotations, including complex `Union` and `Iterable` types, simplifying type declaration for users. * **Improved `TaggedOutput` Class**: The `TaggedOutput` class has been made generic, allowing for more precise type annotations using `TypeVar` and `Generic`, which enhances the clarity and correctness of type definitions. * **Warning for Undeclared Tags**: The system now logs a warning if `with_outputs` requests tags that have not been explicitly declared in the type hints, helping to catch potential misconfigurations early. * **New Dedicated Test Suite**: A new test file (`tagged_output_typehints_test.py`) has been added to thoroughly validate the new functionality across all supported type hinting styles (decorator, method chain, and function annotation) and transforms. 🧠**New Feature in Public Preview:** You can now enable **Memory** to help **Gemini Code Assist** learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. **Click [here](https://codeassist.google/code-review/login) to enable Memory in your admin console.** <details> <summary><b>Changelog</b></summary> * **sdks/python/apache_beam/pvalue.py** * Added `_tagged_output_types` attribute to `DoOutputsTuple` to store type hints for tagged outputs. * Updated `PCollection.__getitem__` to use specific `element_type` from `_tagged_output_types` for tagged outputs, falling back to `Any`. * Made `TaggedOutput` generic (`TaggedOutput[TagType, ValueType]`) for more precise type annotations. * **sdks/python/apache_beam/transforms/core.py** * Implemented a warning in `with_outputs` if requested tags are not declared in type hints. * Modified `Map`, `MapTuple`, and `FlatMapTuple` to pass `tagged_output_types` to `with_output_types`. * **sdks/python/apache_beam/transforms/ptransform.py** * Updated `with_output_types` method to accept `**tagged_type_hints` for specifying types of tagged outputs. * Added documentation and an example for using `with_output_types` with tagged outputs. * Modified `type_check_inputs_or_outputs` to allow keyword arguments for output type hints, specifically for tagged outputs. * **sdks/python/apache_beam/typehints/decorators.py** * Imported `collections.abc`, `Literal`, `get_args`, `get_origin`, and `TaggedOutput`. * Introduced new helper functions: `_tag_and_type`, `_contains_tagged_output`, `_extract_main_and_tagged`, and `_extract_output_types` for parsing return annotations. * Updated `IOTypeHints.from_callable` to use `_extract_output_types` and store tagged output types. * Added `IOTypeHints.tagged_output_types` method and adjusted `simple_output_type`, `has_simple_output_type`, and `strip_iterable` to account for tagged outputs. * Modified `with_output_types` decorator to accept `**tagged_type_hints` and updated its documentation. * **sdks/python/apache_beam/typehints/decorators_test.py** * Added `TaggedOutput` import. * Added new test cases for `IOTypeHints.from_callable` to verify correct extraction of main and tagged output types. * Added a new test class `TaggedOutputExtractionTest` to specifically test the new helper functions for tagged output type hint extraction. * **sdks/python/apache_beam/typehints/tagged_output_typehints_test.py** * Added new file containing extensive tests for tagged output type hinting across various transforms and type hinting styles (decorator, method chain, and function annotation). * **sdks/python/apache_beam/typehints/typehints_test.py** * Updated the regex in `test_no_kwargs_accepted` to reflect the new behavior where kwargs are now accepted for tagged output types. </details> <details> <summary><b>Activity</b></summary> * A bot attempted to summarize the PR but encountered high traffic and failed. * The author, claudevdm, manually requested a summary from the bot. </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
