mtauha opened a new pull request, #37679:
URL: https://github.com/apache/beam/pull/37679
## Description
This PR adds unbounded streaming support to the native Python
`GenerateSequence` PTransform, building on top of the bounded implementation
introduced in #37675.
Addresses #18088
### Motivation
Phase 1 (#37675) introduced the bounded `GenerateSequence`. This PR
completes
Phase 2 by adding unbounded support — allowing Python pipelines to generate
infinite integer sequences with optional rate limiting and custom timestamps,
equivalent to Java's `UnboundedCountingSource`.
The existing external Flink-only version at
`apache_beam/io/external/generate_sequence.py` remains untouched.
### Changes
- Updated `sdks/python/apache_beam/io/generate_sequence.py`:
- `GenerateSequence` now supports unbounded mode (when `stop` is not
provided)
- Added `_UnboundedCountingSource` backed by `UnboundedSource`
- Added rate limiting support via `elements_per_second` parameter
- Added custom timestamp support via `timestamp_fn` parameter
- Added checkpointing via `_CounterMark` (equivalent to Java's
`CounterMark`)
- Updated `sdks/python/apache_beam/io/generate_sequence_test.py` with tests
covering unbounded mode, rate limiting, and checkpointing behavior
### Notes
- This is **Phase 2 (unbounded)**. The bounded implementation from Phase 1
remains fully intact and untouched.
- Works on **all runners** without a Java expansion service.
- Modelled after Java's `UnboundedCountingSource` and `CounterMark`.
### Testing
```bash
cd sdks/python
python -m pytest apache_beam/io/generate_sequence_test.py -v
```
## Checklist
- [X] Addresses #18088
- [X] Updated `CHANGES.md` with noteworthy changes
- [X] All existing and new tests pass locally
- [X] No pylint errors (`pylint apache_beam/io/generate_sequence.py`)
- [X] Formatted with yapf (`yapf --in-place --parallel --recursive
apache_beam/io/generate_sequence.py`)
- [X] Apache license header present in all new/modified files
R: @shunping @damccorm @robertwb @claudevdm @kennknowles
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
