Hello, If you use Java, I recommend the Building Big Data Pipelines with Apache Beam book - https://www.oreilly.com/library/view/building-big-data/9781800564930/.
If Python, I wrote two blog posts about building Beam pipelines using batch and streaming splittable DoRn earlier. They are translated from the above book. - Apache Beam Python Examples - Part 9 Develop Batch File Reader and PiSampler Using Splittable DoFn ( https://jaehyeon.me/blog/2024-12-05-beam-examples-9/) - Apache Beam Python Examples - Part 10 Develop Streaming File Reader Using Splittable DoFn ( https://jaehyeon.me/blog/2024-12-19-beam-examples-10/) Also, you'd find the following workshop useful. - Splittable DoFns in Python: a hands-on workshop: https://2022.beamsummit.org/sessions/splittable-dofns-in-python/ - GitHub: https://github.com/iht/splittable-dofns-python Cheers, Jaehyeon On Wed, 9 Jul 2025 at 12:37, Zack Culberson <[email protected]> wrote: > I have tried but the python examples take you to the source code from the > looks of it. > > I'll look more at this page but if you have other forums or articles that > I could look at I would appreciate it ! > > Thank you, > Zack Culberson > ------------------------------ > *From:* XQ Hu via user <[email protected]> > *Sent:* Tuesday, July 8, 2025 7:35 PM > *To:* [email protected] <[email protected]> > *Cc:* XQ Hu <[email protected]> > *Subject:* EXTERNAL Email: Re: Splitable DoFn > > How you checked > https://beam.apache.org/blog/splittable-do-fn-is-available/ > <https://urldefense.com/v3/__https://beam.apache.org/blog/splittable-do-fn-is-available/__;!!K1aXqncla1X7G90AkdLmCg!u95zpGTtg5oGdVUekbzjOmZQJZfmN3y8UdyOgJFqeBKvn_k8IS61Z8AmMdo8ILvuqvCBuAWEcyaJe_tn_ffhdsZL$>? > It lists some real world examples. > > On Tue, Jul 8, 2025 at 6:59 PM Zack Culberson < > [email protected]> wrote: > > Hi all, > > I have been looking for a working example of a splitable DoFn and what is > needed to implement one. I have added some of the restrictions and other > functions that are needed but wanted to see if there was a good example to > look at to build a splitable DoFn. > > Thank you , > Zack > ------------------------------ > Warning: All e-mail sent to this address will be received by the corporate > e-mail system, and is subject to archival and review by someone other than > the recipient. This e-mail may contain proprietary information and is > intended only for the use of the intended recipient(s). If the reader of > this message is not the intended recipient(s), you are notified that you > have received this message in error and that any review, dissemination, > distribution or copying of this message is strictly prohibited. If you have > received this message in error, please notify the sender immediately. > ------------------------------ > > ------------------------------ > Warning: All e-mail sent to this address will be received by the corporate > e-mail system, and is subject to archival and review by someone other than > the recipient. This e-mail may contain proprietary information and is > intended only for the use of the intended recipient(s). If the reader of > this message is not the intended recipient(s), you are notified that you > have received this message in error and that any review, dissemination, > distribution or copying of this message is strictly prohibited. If you have > received this message in error, please notify the sender immediately. > ------------------------------ >
