Adding Andrea as she may have thoughts on this. On Wed, Aug 29, 2018 at 10:43 AM Robin Qiu <[email protected]> wrote:
> Hi everyone, > > I am writing a test [1] for the support of @RequiresStableInput annotation > in Java SDK [2]. In the test I need to have a ParDo make some side effects > and then fail. Currently, the only way to persist the side effect during > retries without depending on external states is using attempted metrics. > However, attempted metrics cannot be accessed from within a DoFn. So I am > wondering if we should make an improvement on this by, for example, > exposing a job handler to DoFn. > > As Luke mentioned in another email thread, this improvement will also be > useful since: > * cancelling a pipeline from within the pipeline is useful > * starting a new job against the existing runner from in a pipeline is > useful > * accessing attempted metrics to test DoFn's with side effects is useful > for error handling testing > > What do you think? All suggestions are appreciated. > > Best, > Robin > > [1] https://github.com/apache/beam/pull/6220 > [2] > https://docs.google.com/document/d/117yRKbbcEdm3eIKB_26BHOJGmHSZl1YNoF0RqWGtqAM/edit# > >
