Understood. I just wanted to pass that along because my gut sense was that it's not necessary for a MVP of the tool I'm hacking out, but I wanted to run it by you instead of assuming.
Here's the branch I created off your suggestion. It looks clean, and my tool was able to be refactored around it and run my test flows with more or less the same capability so it seems like a fit. https://github.com/MikeThomsen/nifi/tree/mp-nip46 Thanks, Mike On Thu, Sep 17, 2026 at 1:14 PM Mark Payne <[email protected]> wrote: > Thanks for the follow-up Mike. > > I think I’d need more details to really understand the concern there. A > general exception does not necessarily mean that there is a failure. If > there are true failures that somehow get swallowed and not propagated back > up to the caller, that sounds like a bug that we’d need to dig into. > > Thanks > -Mark > > > > On Sep 17, 2026, at 8:34 AM, Mike Thomsen <[email protected]> > wrote: > > > > To clarify… where an exception happened during an asynchronous commit. > > > > Sent from my iPhone > > > >> On Sep 17, 2026, at 8:33 AM, Mike Thomsen <[email protected]> > wrote: > >> > >> Mark, > >> > >> I am working on an integration test runner for flows, and that is the > core use case that made me look into this. I had Claude work out my > proposal, David’s and yours. I think yours is probably the cleanest for > capturing exceptions that escape onTrigger, but Claude warned that it > didn’t capture cases where an asynchronous commit happened nor did it > reliably capture exceptions inside callbacks in processsession read/write. > What are your thoughts on that? > >> > >> Sent from my iPhone > >> > >>> On Sep 16, 2026, at 10:22 AM, Mark Payne <[email protected]> wrote: > >>> > >>> Thanks, David, I’ve been meaning to take a minute to reply here so > good that you pinged. > >>> > >>> Currently the API has: > >>> > >>> ``` > >>> interface StatelessDataflow { > >>> … > >>> DataflowTrigger trigger(DataflowTriggerContext context); > >>> … > >>> } > >>> ``` > >>> > >>> And DataFlowTrigger has: > >>> ``` > >>> Optional<TriggerResult> getResult(long maxWaitTime, TimeUnit timeUnit) > throws InterruptedException; > >>> ``` > >>> > >>> And TriggerResult contains: > >>> ``` > >>> boolean isSuccessful(); > >>> Optional<Throwable> getFailureCause(); > >>> ``` > >>> > >>> So at present, the API does expose whether or not the run was > successful and if not, the Throwable that was the cause. > >>> In terms of the information that is available, compared to what is > proposed in NIP-46, I would say the differences are: > >>> > >>> - The API does NOT expose which component caused the failure. The type > is likely exposed in the stack trace, but not made available in a way that > is accessible programmatically, and it doesn’t expose the ID or the name of > the component. > >>> - The API is not a callback but rather this information is surfaced to > the caller. > >>> > >>> I do believe that surfacing this information to the caller via > TriggerResult is the more desirable API, rather than introducer a new > callback / listener type of API. > >>> > >>> If there’s a concrete use case where we think it makes sense to return > something like a `FailingComponent` object that contains the component id, > name, type, etc. I’m not opposed to surfacing that information. But IMO it > should be surfaced on the TriggerResult, such as a new > `Optional<FailingComponent> getFailingComponent()` method, rather than via > listeners. > >>> > >>> Would that scratch the itch here? > >>> > >>> Thanks > >>> -Mark > >>> > >>> > >>> > >>> On Sep 16, 2026, at 9:09 AM, Mike Thomsen <[email protected]> > wrote: > >>> > >>> David, > >>> > >>> That might actually be a better approach. I'll look into those two > >>> repositories and see if I can achieve a similar outcome without messing > >>> with stateless NiFi (or doing nothing more than forking the current > >>> repository code to make it more observable). > >>> > >>> Thanks, > >>> > >>> Mike > >>> > >>> On Tue, Sep 15, 2026 at 4:51 PM David Handermann < > >>> [email protected]> wrote: > >>> > >>> Mike, > >>> > >>> Thanks for proposing this addition to NiFi Stateless behavior. > >>> > >>> The concept of observing lower-level details in the Stateless flow > >>> makes sense at one level, but also raises a few more motivation > >>> questions. > >>> > >>> The CI/CD use case is a good one in general, and I can see why it > >>> could be useful to get more details on whether the flow might have > >>> failed. Although it would be a bit more indirect, did you consider > >>> some alternative type of Bulletin Repository or Provenance Repository > >>> that could surface this information? The Bulletins provide the > >>> component-level message and Provenance keeps track of processing > >>> outcomes, so building something in that direction seems to align more > >>> with the general approach of NiFi itself. > >>> > >>> The new listener surface seems focused enough to avoid significant > >>> changes, but I raise the question of alternatives to evaluate whether > >>> this approach aligns with Stateless in general, or whether it starts > >>> observability going in a different direction. > >>> > >>> I would also look for input from Mark Payne given his extensive work > >>> with Stateless and NiFi itself. > >>> > >>> Regards, > >>> David Handermann > >>> > >>> On Sat, Sep 12, 2026 at 5:31 AM Mike Thomsen <[email protected]> > >>> wrote: > >>> > >>> I have created a NIP to add some additional capabilities to Stateless > >>> NiFi > >>> that would make it much easier for calling applications to track > failure > >>> signals in NiFi components. > >>> > >>> https://issues.apache.org/jira/browse/NIP-46 > >>> > >>> The need for this NIP came about while I was working on a tool to run > >>> NiFi > >>> flows inside of a test harness, but I think it could also be useful in > >>> general for other applications of Stateless NiFi such as launching it > >>> from > >>> a stateful flow. > >>> > >>> Assuming no objections, I plan to start a lazy-consensus vote soon. > >>> > >>> Thanks, > >>> > >>> Mike > >>> > >>> > >
