Adding to XQ's list, there are also some docs discussing the execution model in addition to the programming model - https://cloud.google.com/dataflow/docs/pipeline-lifecycle - https://cloud.google.com/dataflow/docs/concepts/beam-programming-model & https://beam.apache.org/documentation/programming-guide/
On Sat, Mar 8, 2025 at 9:56 AM XQ Hu via user <[email protected]> wrote: > We probably have many online resources that cover these topics but they > are scattered. For example, Beam Summit and College talks on Youtube: > https://www.youtube.com/@ApacheBeamYT (Beam Summit slides can be found > here: https://beamsummit.org/) and https://www.youtube.com/@BeamCollege ( > https://beamcollege.dev/sessions/ has the slides). > > We also have been collecting any design or discussion doc under > https://github.com/apache/beam/tree/master/contributor-docs/discussion-docs > > It will be great if you can share more with the community with your > learning. > > Anyway, not particular links for you but I hope they can give you some > places to check out. > > > > On Sat, Mar 8, 2025 at 12:45 PM Joey Tran <[email protected]> > wrote: > >> Hey all, >> >> We're starting to adopt Beam more widely amongst our engineers so we're >> trying to put together a workshop to teach Beam as it's proven a little bit >> difficult for some developers to get started on their own. Just wanted to >> see if there are any slide decks in the community for this kind of thing. >> >> Some of the troubles we've had so far when bringing new people up to >> speed: >> - The deferred execution (pipeline constructions vs pipeline execution >> time) of Beam pipelines/runners >> - How aggregations works, e.g. `CombineGlobally` is met with the >> question of "globally relative to what?" >> - Grouping by key in general as a way of partitioning data >> - Reshuffle. This one really trips people up because they're now >> suddenly having to understand both the Beam model and how a runner might >> work >> >> Anyone have any tips or advice to smooth these things out? >> >> Cheers, >> Joey >> >
