Thanks, David, I’ve been meaning to take a minute to reply here so good that
you pinged.
Currently the API has:
```
interface StatelessDataflow {
…
DataflowTrigger trigger(DataflowTriggerContext context);
…
}
```
And DataFlowTrigger has:
```
Optional<TriggerResult> getResult(long maxWaitTime, TimeUnit timeUnit) throws
InterruptedException;
```
And TriggerResult contains:
```
boolean isSuccessful();
Optional<Throwable> getFailureCause();
```
So at present, the API does expose whether or not the run was successful and if
not, the Throwable that was the cause.
In terms of the information that is available, compared to what is proposed in
NIP-46, I would say the differences are:
- The API does NOT expose which component caused the failure. The type is
likely exposed in the stack trace, but not made available in a way that is
accessible programmatically, and it doesn’t expose the ID or the name of the
component.
- The API is not a callback but rather this information is surfaced to the
caller.
I do believe that surfacing this information to the caller via TriggerResult is
the more desirable API, rather than introducer a new callback / listener type
of API.
If there’s a concrete use case where we think it makes sense to return
something like a `FailingComponent` object that contains the component id,
name, type, etc. I’m not opposed to surfacing that information. But IMO it
should be surfaced on the TriggerResult, such as a new
`Optional<FailingComponent> getFailingComponent()` method, rather than via
listeners.
Would that scratch the itch here?
Thanks
-Mark
On Sep 16, 2026, at 9:09 AM, Mike Thomsen <[email protected]> wrote:
David,
That might actually be a better approach. I'll look into those two
repositories and see if I can achieve a similar outcome without messing
with stateless NiFi (or doing nothing more than forking the current
repository code to make it more observable).
Thanks,
Mike
On Tue, Sep 15, 2026 at 4:51 PM David Handermann <
[email protected]> wrote:
Mike,
Thanks for proposing this addition to NiFi Stateless behavior.
The concept of observing lower-level details in the Stateless flow
makes sense at one level, but also raises a few more motivation
questions.
The CI/CD use case is a good one in general, and I can see why it
could be useful to get more details on whether the flow might have
failed. Although it would be a bit more indirect, did you consider
some alternative type of Bulletin Repository or Provenance Repository
that could surface this information? The Bulletins provide the
component-level message and Provenance keeps track of processing
outcomes, so building something in that direction seems to align more
with the general approach of NiFi itself.
The new listener surface seems focused enough to avoid significant
changes, but I raise the question of alternatives to evaluate whether
this approach aligns with Stateless in general, or whether it starts
observability going in a different direction.
I would also look for input from Mark Payne given his extensive work
with Stateless and NiFi itself.
Regards,
David Handermann
On Sat, Sep 12, 2026 at 5:31 AM Mike Thomsen <[email protected]>
wrote:
I have created a NIP to add some additional capabilities to Stateless
NiFi
that would make it much easier for calling applications to track failure
signals in NiFi components.
https://issues.apache.org/jira/browse/NIP-46
The need for this NIP came about while I was working on a tool to run
NiFi
flows inside of a test harness, but I think it could also be useful in
general for other applications of Stateless NiFi such as launching it
from
a stateful flow.
Assuming no objections, I plan to start a lazy-consensus vote soon.
Thanks,
Mike
