Re: External resources in Beam pipelines

2018-08-14 Thread Andrea Foegler
Here's the raw doc content, for your convenience.Beam I/O representation In the current Beam programming model, sources and sinks are virtually indistinguishable from other transforms. From a composition point of view, this is great. But from an integration point of view, special handling of

External resources in Beam pipelines

2018-08-14 Thread Andrea Foegler
Hi folks - Many of you don't know me, as I don't contribute directly to Beam. But I do a lot of work around the periphery, in particular considering how to manage and monitor Beam pipelines. I think there's room in Beam to greatly improve both the management and monitoring story, especially