Thanks for the follow-up Mike. I think I’d need more details to really understand the concern there. A general exception does not necessarily mean that there is a failure. If there are true failures that somehow get swallowed and not propagated back up to the caller, that sounds like a bug that we’d need to dig into.
Thanks -Mark > On Sep 17, 2026, at 8:34 AM, Mike Thomsen <[email protected]> wrote: > > To clarify… where an exception happened during an asynchronous commit. > > Sent from my iPhone > >> On Sep 17, 2026, at 8:33 AM, Mike Thomsen <[email protected]> wrote: >> >> Mark, >> >> I am working on an integration test runner for flows, and that is the core >> use case that made me look into this. I had Claude work out my proposal, >> David’s and yours. I think yours is probably the cleanest for capturing >> exceptions that escape onTrigger, but Claude warned that it didn’t capture >> cases where an asynchronous commit happened nor did it reliably capture >> exceptions inside callbacks in processsession read/write. What are your >> thoughts on that? >> >> Sent from my iPhone >> >>> On Sep 16, 2026, at 10:22 AM, Mark Payne <[email protected]> wrote: >>> >>> Thanks, David, I’ve been meaning to take a minute to reply here so good >>> that you pinged. >>> >>> Currently the API has: >>> >>> ``` >>> interface StatelessDataflow { >>> … >>> DataflowTrigger trigger(DataflowTriggerContext context); >>> … >>> } >>> ``` >>> >>> And DataFlowTrigger has: >>> ``` >>> Optional<TriggerResult> getResult(long maxWaitTime, TimeUnit timeUnit) >>> throws InterruptedException; >>> ``` >>> >>> And TriggerResult contains: >>> ``` >>> boolean isSuccessful(); >>> Optional<Throwable> getFailureCause(); >>> ``` >>> >>> So at present, the API does expose whether or not the run was successful >>> and if not, the Throwable that was the cause. >>> In terms of the information that is available, compared to what is proposed >>> in NIP-46, I would say the differences are: >>> >>> - The API does NOT expose which component caused the failure. The type is >>> likely exposed in the stack trace, but not made available in a way that is >>> accessible programmatically, and it doesn’t expose the ID or the name of >>> the component. >>> - The API is not a callback but rather this information is surfaced to the >>> caller. >>> >>> I do believe that surfacing this information to the caller via >>> TriggerResult is the more desirable API, rather than introducer a new >>> callback / listener type of API. >>> >>> If there’s a concrete use case where we think it makes sense to return >>> something like a `FailingComponent` object that contains the component id, >>> name, type, etc. I’m not opposed to surfacing that information. But IMO it >>> should be surfaced on the TriggerResult, such as a new >>> `Optional<FailingComponent> getFailingComponent()` method, rather than via >>> listeners. >>> >>> Would that scratch the itch here? >>> >>> Thanks >>> -Mark >>> >>> >>> >>> On Sep 16, 2026, at 9:09 AM, Mike Thomsen <[email protected]> wrote: >>> >>> David, >>> >>> That might actually be a better approach. I'll look into those two >>> repositories and see if I can achieve a similar outcome without messing >>> with stateless NiFi (or doing nothing more than forking the current >>> repository code to make it more observable). >>> >>> Thanks, >>> >>> Mike >>> >>> On Tue, Sep 15, 2026 at 4:51 PM David Handermann < >>> [email protected]> wrote: >>> >>> Mike, >>> >>> Thanks for proposing this addition to NiFi Stateless behavior. >>> >>> The concept of observing lower-level details in the Stateless flow >>> makes sense at one level, but also raises a few more motivation >>> questions. >>> >>> The CI/CD use case is a good one in general, and I can see why it >>> could be useful to get more details on whether the flow might have >>> failed. Although it would be a bit more indirect, did you consider >>> some alternative type of Bulletin Repository or Provenance Repository >>> that could surface this information? The Bulletins provide the >>> component-level message and Provenance keeps track of processing >>> outcomes, so building something in that direction seems to align more >>> with the general approach of NiFi itself. >>> >>> The new listener surface seems focused enough to avoid significant >>> changes, but I raise the question of alternatives to evaluate whether >>> this approach aligns with Stateless in general, or whether it starts >>> observability going in a different direction. >>> >>> I would also look for input from Mark Payne given his extensive work >>> with Stateless and NiFi itself. >>> >>> Regards, >>> David Handermann >>> >>> On Sat, Sep 12, 2026 at 5:31 AM Mike Thomsen <[email protected]> >>> wrote: >>> >>> I have created a NIP to add some additional capabilities to Stateless >>> NiFi >>> that would make it much easier for calling applications to track failure >>> signals in NiFi components. >>> >>> https://issues.apache.org/jira/browse/NIP-46 >>> >>> The need for this NIP came about while I was working on a tool to run >>> NiFi >>> flows inside of a test harness, but I think it could also be useful in >>> general for other applications of Stateless NiFi such as launching it >>> from >>> a stateful flow. >>> >>> Assuming no objections, I plan to start a lazy-consensus vote soon. >>> >>> Thanks, >>> >>> Mike >>> >>>
