Eliaaazzz commented on code in PR #40091: URL: https://github.com/apache/beam/pull/40091#discussion_r3986058204
########## website/www/site/data/authors.yml: ########## @@ -60,6 +60,8 @@ dhalperi: name: Dan Halperin email: [email protected] twitter: +eliaaazzz: + name: Elia Liu Review Comment: Happy to include it. Added [email protected] to the author entry. ########## website/www/site/content/en/blog/python-unboundedsource-watch.md: ########## @@ -0,0 +1,163 @@ +--- +title: "UnboundedSource and the Watch Transform in the Apache Beam Python SDK" +date: 2026-09-10T00:00:00+10:00 +categories: + - blog + - gsoc +authors: + - eliaaazzz +--- +<!-- +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +The Apache Beam Python SDK now includes an `UnboundedSource` API for custom Review Comment: Good idea, thank you. The opening paragraph now says I built both APIs during my Google Summer of Code 2026 project with Beam, mentored by Yi Hu. ########## website/www/site/content/en/blog/python-unboundedsource-watch.md: ########## @@ -0,0 +1,163 @@ +--- +title: "UnboundedSource and the Watch Transform in the Apache Beam Python SDK" +date: 2026-09-10T00:00:00+10:00 +categories: + - blog + - gsoc +authors: + - eliaaazzz +--- +<!-- +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +The Apache Beam Python SDK now includes an `UnboundedSource` API for custom Review Comment: Added a Motivation section right after the intro. It names the gaps this project started from: Python had no `UnboundedSource` API, so reading from a streaming system with no existing connector meant either a cross-language Java connector or writing an unbounded splittable DoFn directly; there was no Python `Watch` transform; and `fileio.MatchContinuously` kept one state entry per matched file path for the life of the pipeline, so its state grew with every file the pattern had ever seen. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
