Eliaaazzz commented on code in PR #40091: URL: https://github.com/apache/beam/pull/40091#discussion_r3986058424
########## website/www/site/content/en/blog/python-unboundedsource-watch.md: ########## @@ -0,0 +1,163 @@ +--- +title: "UnboundedSource and the Watch Transform in the Apache Beam Python SDK" +date: 2026-09-10T00:00:00+10:00 +categories: + - blog + - gsoc +authors: + - eliaaazzz +--- +<!-- +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +The Apache Beam Python SDK now includes an `UnboundedSource` API for custom Review Comment: Thanks, this was the right thing to push on. The section now leads with the use case: I wanted someone to be able to read their own message queue or database change feed without dropping into Java. Writing such a source in Python was already possible through an unbounded splittable DoFn, so the honest gap was that every author had to work out the stream-reading part themselves, the reader position as a restriction, when to stop and hand progress back, and how the watermark moves. `UnboundedSource` settles that once. `Watch` is the same idea for an input that keeps growing, where Python had `PeriodicImpulse` composed with `MatchAll` and per-file state that never shrank. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
