... And remember it and make available inside PCollection (which application produced this collection).
On Tue, Aug 15, 2017, 8:39 AM Eugene Kirpichov <kirpic...@google.com> wrote: > In general, no - but the implementation of PAssertionSite exemplifies the > approach. I guess it could be useful to make this a general beam feature > and remember it for all transforms. It would probably be best to implement > inside Pipeline.apply(). > > On Tue, Aug 15, 2017, 7:02 AM Shen Li <cs.she...@gmail.com> wrote: > >> Hi Eugene, >> >> Thanks for sharing the info. That PAssertionSite tracks where an assertion >> error occurred. Do you know if it is possible to get the class name and >> line number where a PTransform was added? >> >> Thanks, >> Shen >> >> On Mon, Aug 14, 2017 at 10:54 PM, Eugene Kirpichov < >> kirpic...@google.com.invalid> wrote: >> >> > Hi Shen, >> > Responding just to one part of your message - "remember the line at >> which >> > the PTransform was added": take a look at >> > https://github.com/apache/beam/pull/2247 which does this for PAssert. >> > >> > On Mon, Aug 14, 2017 at 7:32 PM Shen Li <cs.she...@gmail.com> wrote: >> > >> > > In 0.5.0 or earlier releases, PipelineRunner provides an >> > > apply(PTransform<InputT, OutputT>, InputT) method which allows runner >> > > implementation to perform actions when the PTransform is added to the >> > > pipeline. In later releases, that apply(...) method has been replaced >> by >> > a >> > > Pipeline#replaceAll() method, where the runner can only get involved >> > after >> > > the pipeline has been fully constructed. In terms of override >> > PTransforms, >> > > these two APIs are identical. But, the early API could still be >> helpful. >> > > For example, the runner could remember the line at which the >> PTransform >> > was >> > > added, and provide that info to users to assist debugging. Is it >> possible >> > > to add that API back? Or is there any other way to involve the runner >> > when >> > > a PTransform is added? >> > > >> > > Thanks, >> > > Shen >> > > >> > >> >
