Re: Adding back PipelineRunner#apply method

thank you! Shen On Tue, Aug 15, 2017 at 1:29 PM, Robert Bradshaw < rober...@google.com.invalid> wrote: > On Tue, Aug 15, 2017 at 10:21 AM, Shen Li wrote: > > Hi Thomas, > > > > Does it mean future Pipeline implementations would allow applications to > > set the runner

Re: Adding back PipelineRunner#apply method

On Tue, Aug 15, 2017 at 10:21 AM, Shen Li wrote: > Hi Thomas, > > Does it mean future Pipeline implementations would allow applications to > set the runner after a pipeline has been constructed? Correct, that's the intent. > > Thanks, > Shen > > On Tue, Aug 15, 2017 at

Re: Adding back PipelineRunner#apply method

Hi Thomas, Does it mean future Pipeline implementations would allow applications to set the runner after a pipeline has been constructed? Thanks, Shen On Tue, Aug 15, 2017 at 12:36 PM, Thomas Groh wrote: > This style of method doesn't fit with the current approach of

Re: Adding back PipelineRunner#apply method

This style of method doesn't fit with the current approach of pipeline construction, where the PipelineRunner need not be specified until the pipeline is run; as such, the runner can't observe the construction of the Pipeline, as it may not exist during the construction of the Pipeline. On Tue,

Re: Adding back PipelineRunner#apply method

... And remember it and make available inside PCollection (which application produced this collection). On Tue, Aug 15, 2017, 8:39 AM Eugene Kirpichov wrote: > In general, no - but the implementation of PAssertionSite exemplifies the > approach. I guess it could be useful

Re: Adding back PipelineRunner#apply method

In general, no - but the implementation of PAssertionSite exemplifies the approach. I guess it could be useful to make this a general beam feature and remember it for all transforms. It would probably be best to implement inside Pipeline.apply(). On Tue, Aug 15, 2017, 7:02 AM Shen Li

Re: Adding back PipelineRunner#apply method

Hi Eugene, Thanks for sharing the info. That PAssertionSite tracks where an assertion error occurred. Do you know if it is possible to get the class name and line number where a PTransform was added? Thanks, Shen On Mon, Aug 14, 2017 at 10:54 PM, Eugene Kirpichov < kirpic...@google.com.invalid>

Re: Adding back PipelineRunner#apply method

Hi Shen, Responding just to one part of your message - "remember the line at which the PTransform was added": take a look at https://github.com/apache/beam/pull/2247 which does this for PAssert. On Mon, Aug 14, 2017 at 7:32 PM Shen Li wrote: > In 0.5.0 or earlier releases,