When you say instance do you mean a different named PTransform class? Or just a separate instance of whatever "HandleAssertResult" PTransform is introduced? If the latter, I believe that makes sense. It also addresses the problems with committed vs. attempted metrics. Specifically:
Since we want to know that each assertion has run, we'd like to do something like "Number of successful assertions" = N. But, in the presence of retries, the number of successful assertions may be over counted. But, if we have a separate transform for each assertion, then we can say "for each transform, the number of successful assertions should be >= 1", which ensures that every assert has succeeded. This will work with any metric implementation that may overcount but never (or sufficiently rarely) undercount. On Thu, Mar 30, 2017 at 3:39 PM Kenneth Knowles <k...@google.com.invalid> wrote: > This sounds pretty good to me. I really like emphasizing the ability of a > runner to go ahead and use pipeline surgery to put in their own > verification transform. > > On Thu, Mar 30, 2017 at 3:12 PM, Pablo Estrada <pabl...@google.com.invalid > > > wrote: > > > * The current asserting DoFns in PAssert will go from being DoFn<ActualT, > > Void> to be DoFn<ActualT, SuccesOrFailure>, and pass the output > > SuccessOrFailure objects downstream to the checking > > PTransform<SuccessOrFailure, Void>. > > * This default PTransform<SuccessOrFailure, Void> will use metrics to > count > > successes, and fail immediately on failures (after incrementing the > failure > > counter). > > > > Could it possibly be useful to have a separate instance of a > PTransform<SuccessOrFailure> for each assertion? It might be able to proxy > for counting, for runners that don't yet have even rudimentary metrics. The > transform could set up triggering to fire just once and fail if it has not > received one Success. The runner might still just not execute that > transform, but we could set it up so that is extremely unlikely, and anyhow > it is better than no coverage. I could be missing something about the > implementation issues here, though. > > Kenn >
