FWIW, there's already a python-implemented "dot runner" https://github.com/apache/beam/blob/eac11fed1d45bdfd50c9499b8c4a75f36dfbdfe7/sdks/python/apache_beam/runners/render.py
It can generate dot graphs from a portable pipeline protos already. On Tue, Feb 24, 2026 at 1:36 PM Robert Burke <[email protected]> wrote: > That's aligned with what I was thinking. > > I would caution against in premature-abstraction of course. Since this is > all iterable locally, it's easy enough to first target Dot with the initial > abstraction, get that working then try to get a 2nd working, allowing the > abstraction to evolve to support both, and not be overly coupled to either. > > Robert Burke > > On Tue, Feb 24, 2026 at 10:28 AM Yousuf Ansari <[email protected]> > wrote: > >> Subject: Re: [GSoC 2026] Go SDK Project Idea – Pipeline Visualization in >> Prism Runner >> >> Hi Robert, >> >> Thank you for the clarification, that perspective helps a lot. >> >> I agree that the real value is introducing a clean abstraction between >> the portable pipeline proto and the rendering layer, rather than coupling >> directly to DOT. >> >> As a next step, I’m planning to explore defining an intermediate graph >> representation derived from the portable proto, so that DOT becomes one >> pluggable renderer, with room for future formats and Prism integration. >> >> Does that direction sound reasonable from a package/layout perspective >> within the Go SDK? >> >> Thanks again for the guidance. >> >> Best, >> Yousuf >> >> On Tue, 24 Feb 2026 at 23:32, Robert Burke <[email protected]> wrote: >> >>> As a rule, I'm in favour of this project, but outside of code reviews >>> I'm not sure how much time I can spend on mentoring beyond that. (But I >>> promise I'll look at all PRs tagged with @lostluck.) >>> >>> While the proprosal around "dot" is largely just a continuation of the >>> pre-existing "dot" runner, the important thing is that it gives a basic >>> scaffolding of going from something that currently works (Go pipeline to >>> some other format for rendering), not the specific outcome (dot graphs). >>> It's however a useful starting point for later incorporating the same code >>> into Prism. >>> >>> The ideal case is using the dot runner as the initial proof of concept >>> for converting it to use the portable pipeline proto format instead of the >>> existing Raw Go SDK structure. Then somehow from there abstract what it >>> produces from dot graphs, to some other format (mermaid flow charts, some >>> fun HTML thing directly that can be made responsive in the UI with live >>> metrics and the like, etc.). >>> >>> Incorporating it into the prism runner enables Python and Java (or other >>> SDKs) to take advantage of it as well, which is where the value of this >>> proposal is. >>> >>> Robert >>> >>> On Mon, Feb 23, 2026 at 12:18 AM Yousuf Ansari < >>> [email protected]> wrote: >>> >>>> Hi Beam community, >>>> >>>> My name is Yousuf. I'm a student who has been contributing to Apache >>>> projects for a while now - I have 4 merged PRs in Apache Superset and >>>> recently opened PR #37673 in Beam, which rewrites the Go SDK dot runner to >>>> generate DOT graphs from the portable pipeline proto. >>>> >>>> The PR notes that the new portable-proto-based approach enables future >>>> reuse in Prism and other portable tooling. I'd like to propose a Go SDK >>>> GSoC project built around that direction: >>>> >>>> - Integrating the portable-proto DOT generation into the Prism >>>> runner's web UI, so developers can visualize their pipeline while it >>>> runs >>>> locally >>>> - Adding per-transform metrics display on the graph nodes >>>> - Potentially surfacing watermark progress per PCollection for >>>> streaming pipelines >>>> >>>> This feels like a natural 350-hour project that directly extends the >>>> work already started in #37673 and aligns with the Go SDK roadmap's focus >>>> on streaming features and portable runner improvements. >>>> >>>> I'd love to know: >>>> >>>> 1. Is there mentor interest in a Go SDK project for GSoC 2026? >>>> 2. Does this direction align with the priorities you have in mind? >>>> 3. Are there related issues I should pick up to strengthen my >>>> proposal before the application window opens? >>>> >>>> Happy to discuss scope and adjust based on your feedback. >>>> >>>> Thanks, Yousuf >>>> >>>
