Another +1 to support your research into this Chad. Thank you.
Trying to understand where a beam process is in the Spark DAG is... not
easy. A UI that helped would be a great addition.
On Wed, Jun 26, 2019 at 3:30 PM Ismaël Mejía wrote:
> +1 don't hesitate to create a JIRA + PR. You may be
+1 don't hesitate to create a JIRA + PR. You may be interested in [1].
This is a simple util class that takes a proto pipeline object and
converts it into its graph representation in .dot format. You can
easily reuse the code or the idea as a first approach to show what the
pipeline is about.
[1]
Yes, offering a way to get a pipeline from the job service directly
would be a completely reasonable thing to do (and likely not hard at
all). We welcome pull requests.
Alternative UIs built on top of this abstraction would be an
interesting project to explore.
On Wed, Jun 26, 2019 at 8:44 AM
Hi all,
I've been poking around the beam source code trying to determine whether
it's possible to get the definition of a pipeline via beam's gPRC-based
services. It looks like the message types are there for describing a
Pipeline
Hi everyone!
Happy to announce 2 upcoming Beam Summits:
One in *Paris *on *01/07/2019 *with JB talking about Beam IOs, Konstantin
Knauf talking about Beam on Flink and Pablo Estrada talking about SQL in
Beam!
More info and RSVP:
https://www.meetup.com/Paris-Apache-Beam-Meetup/events/261775884/
