You could add an UpdateAttribute processor first in the failure path to add a
new attribute which contains the time the error occurred by using the ${now()}
or ${now():toNumber()} expression language function.
https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#now
Then later on in the flow you can compare current time to the saved error time
to see how much time has elapsed.
— Jim
> On Feb 15, 2024, at 9:44 AM, James McMahon <[email protected]> wrote:
>
> As it turns out lineageStartDate and Queued Duration are very different.
> Without being able to get at Queued Duration as an attribute, it appears we
> cannot RouteOnAttribute to filter thousands in a queue by anything like hours
> they have been in queue.
> Why would this be helpful? Let us say we have an InvokeHttp processor making
> calls to a REST endpoint. We leave for a weekend and return to find 5000
> files in the Failure queue from this processor. It would be most helpful to
> identify the start time and end time of these 5000 failures. We can't do that
> reviewing only the first 100 flowfiles in the queue from the UI.
> One can make an assumption that all of these 5000 flowfiles that failed
> InvokeHttp share a similar range of lineageStartDate, but that will not
> necessarily be true depending on flow complexity.
>
> On Wed, Feb 14, 2024 at 9:49 AM James McMahon <[email protected]
> <mailto:[email protected]>> wrote:
>> What a great workaround, thank you once again Mike. I'll put this in and use
>> it now.
>> Jim
>>
>> On Tue, Feb 13, 2024 at 4:41 PM Michael Moser <[email protected]
>> <mailto:[email protected]>> wrote:
>>> Hello James,
>>>
>>> I'm not aware of a way to access Queued Duration using expression language,
>>> but you can access the Lineage Duration information. The Getting Started
>>> Guide mentions both entryDate and lineageStartDate as immutable attributes
>>> on all flowfiles. These are numbers of milliseconds since epoch. If you
>>> need them in a readable format, you can use the format() function.
>>>
>>> simple examples:
>>> ${entryDate} = 1707859943778
>>> ${lineageStartDate} = 1707859943778
>>> ${lineageStartDate:format("yyyy-MM-dd HH:mm:ss.SSS")} = 2024-02-13
>>> 21:32:23.778
>>>
>>> -- Mike
>>>
>>>
>>> On Mon, Feb 12, 2024 at 11:38 AM James McMahon <[email protected]
>>> <mailto:[email protected]>> wrote:
>>>> When we examine the contents of a queue through the UI and select a
>>>> flowfile from the resulting list, we see FlowFile Details in the Details
>>>> tab. Are those key/values accessible from nifi expression language? I
>>>> would like to access Queued Duration. I have a queue that holds flowfiles
>>>> with non-successful return codes for calls to REST services, and I want to
>>>> route depending on how long these flowfiles have been sitting in my error
>>>> queue to isolate the window when the REST service was unavailable.
>>>> Thank you for any examples that show how we can access these keys and
>>>> values.
