Mark, I am working on an integration test runner for flows, and that is the core use case that made me look into this. I had Claude work out my proposal, David’s and yours. I think yours is probably the cleanest for capturing exceptions that escape onTrigger, but Claude warned that it didn’t capture cases where an asynchronous commit happened nor did it reliably capture exceptions inside callbacks in processsession read/write. What are your thoughts on that?
Sent from my iPhone > On Sep 16, 2026, at 10:22 AM, Mark Payne <[email protected]> wrote: > > Thanks, David, I’ve been meaning to take a minute to reply here so good that > you pinged. > > Currently the API has: > > ``` > interface StatelessDataflow { > … > DataflowTrigger trigger(DataflowTriggerContext context); > … > } > ``` > > And DataFlowTrigger has: > ``` > Optional<TriggerResult> getResult(long maxWaitTime, TimeUnit timeUnit) throws > InterruptedException; > ``` > > And TriggerResult contains: > ``` > boolean isSuccessful(); > Optional<Throwable> getFailureCause(); > ``` > > So at present, the API does expose whether or not the run was successful and > if not, the Throwable that was the cause. > In terms of the information that is available, compared to what is proposed > in NIP-46, I would say the differences are: > > - The API does NOT expose which component caused the failure. The type is > likely exposed in the stack trace, but not made available in a way that is > accessible programmatically, and it doesn’t expose the ID or the name of the > component. > - The API is not a callback but rather this information is surfaced to the > caller. > > I do believe that surfacing this information to the caller via TriggerResult > is the more desirable API, rather than introducer a new callback / listener > type of API. > > If there’s a concrete use case where we think it makes sense to return > something like a `FailingComponent` object that contains the component id, > name, type, etc. I’m not opposed to surfacing that information. But IMO it > should be surfaced on the TriggerResult, such as a new > `Optional<FailingComponent> getFailingComponent()` method, rather than via > listeners. > > Would that scratch the itch here? > > Thanks > -Mark > > > > On Sep 16, 2026, at 9:09 AM, Mike Thomsen <[email protected]> wrote: > > David, > > That might actually be a better approach. I'll look into those two > repositories and see if I can achieve a similar outcome without messing > with stateless NiFi (or doing nothing more than forking the current > repository code to make it more observable). > > Thanks, > > Mike > > On Tue, Sep 15, 2026 at 4:51 PM David Handermann < > [email protected]> wrote: > > Mike, > > Thanks for proposing this addition to NiFi Stateless behavior. > > The concept of observing lower-level details in the Stateless flow > makes sense at one level, but also raises a few more motivation > questions. > > The CI/CD use case is a good one in general, and I can see why it > could be useful to get more details on whether the flow might have > failed. Although it would be a bit more indirect, did you consider > some alternative type of Bulletin Repository or Provenance Repository > that could surface this information? The Bulletins provide the > component-level message and Provenance keeps track of processing > outcomes, so building something in that direction seems to align more > with the general approach of NiFi itself. > > The new listener surface seems focused enough to avoid significant > changes, but I raise the question of alternatives to evaluate whether > this approach aligns with Stateless in general, or whether it starts > observability going in a different direction. > > I would also look for input from Mark Payne given his extensive work > with Stateless and NiFi itself. > > Regards, > David Handermann > > On Sat, Sep 12, 2026 at 5:31 AM Mike Thomsen <[email protected]> > wrote: > > I have created a NIP to add some additional capabilities to Stateless > NiFi > that would make it much easier for calling applications to track failure > signals in NiFi components. > > https://issues.apache.org/jira/browse/NIP-46 > > The need for this NIP came about while I was working on a tool to run > NiFi > flows inside of a test harness, but I think it could also be useful in > general for other applications of Stateless NiFi such as launching it > from > a stateful flow. > > Assuming no objections, I plan to start a lazy-consensus vote soon. > > Thanks, > > Mike > >
