This style of method doesn't fit with the current approach of pipeline construction, where the PipelineRunner need not be specified until the pipeline is run; as such, the runner can't observe the construction of the Pipeline, as it may not exist during the construction of the Pipeline.
On Tue, Aug 15, 2017 at 8:41 AM, Eugene Kirpichov < kirpic...@google.com.invalid> wrote: > ... And remember it and make available inside PCollection (which > application produced this collection). > > On Tue, Aug 15, 2017, 8:39 AM Eugene Kirpichov <kirpic...@google.com> > wrote: > > > In general, no - but the implementation of PAssertionSite exemplifies the > > approach. I guess it could be useful to make this a general beam feature > > and remember it for all transforms. It would probably be best to > implement > > inside Pipeline.apply(). > > > > On Tue, Aug 15, 2017, 7:02 AM Shen Li <cs.she...@gmail.com> wrote: > > > >> Hi Eugene, > >> > >> Thanks for sharing the info. That PAssertionSite tracks where an > assertion > >> error occurred. Do you know if it is possible to get the class name and > >> line number where a PTransform was added? > >> > >> Thanks, > >> Shen > >> > >> On Mon, Aug 14, 2017 at 10:54 PM, Eugene Kirpichov < > >> kirpic...@google.com.invalid> wrote: > >> > >> > Hi Shen, > >> > Responding just to one part of your message - "remember the line at > >> which > >> > the PTransform was added": take a look at > >> > https://github.com/apache/beam/pull/2247 which does this for PAssert. > >> > > >> > On Mon, Aug 14, 2017 at 7:32 PM Shen Li <cs.she...@gmail.com> wrote: > >> > > >> > > In 0.5.0 or earlier releases, PipelineRunner provides an > >> > > apply(PTransform<InputT, OutputT>, InputT) method which allows > runner > >> > > implementation to perform actions when the PTransform is added to > the > >> > > pipeline. In later releases, that apply(...) method has been > replaced > >> by > >> > a > >> > > Pipeline#replaceAll() method, where the runner can only get involved > >> > after > >> > > the pipeline has been fully constructed. In terms of override > >> > PTransforms, > >> > > these two APIs are identical. But, the early API could still be > >> helpful. > >> > > For example, the runner could remember the line at which the > >> PTransform > >> > was > >> > > added, and provide that info to users to assist debugging. Is it > >> possible > >> > > to add that API back? Or is there any other way to involve the > runner > >> > when > >> > > a PTransform is added? > >> > > > >> > > Thanks, > >> > > Shen > >> > > > >> > > >> > > >
