As it turns out lineageStartDate and Queued Duration are very different. Without being able to get at Queued Duration as an attribute, it appears we cannot RouteOnAttribute to filter thousands in a queue by anything like hours they have been in queue. Why would this be helpful? Let us say we have an InvokeHttp processor making calls to a REST endpoint. We leave for a weekend and return to find 5000 files in the Failure queue from this processor. It would be most helpful to identify the start time and end time of these 5000 failures. We can't do that reviewing only the first 100 flowfiles in the queue from the UI. One can make an assumption that all of these 5000 flowfiles that failed InvokeHttp share a similar range of lineageStartDate, but that will not necessarily be true depending on flow complexity.
On Wed, Feb 14, 2024 at 9:49 AM James McMahon <[email protected]> wrote: > What a great workaround, thank you once again Mike. I'll put this in and > use it now. > Jim > > On Tue, Feb 13, 2024 at 4:41 PM Michael Moser <[email protected]> wrote: > >> Hello James, >> >> I'm not aware of a way to access Queued Duration using expression >> language, but you can access the Lineage Duration information. The Getting >> Started Guide mentions both entryDate and lineageStartDate as immutable >> attributes on all flowfiles. These are numbers of milliseconds since >> epoch. If you need them in a readable format, you can use the format() >> function. >> >> simple examples: >> ${entryDate} = 1707859943778 >> ${lineageStartDate} = 1707859943778 >> ${lineageStartDate:format("yyyy-MM-dd HH:mm:ss.SSS")} = 2024-02-13 >> 21:32:23.778 >> >> -- Mike >> >> >> On Mon, Feb 12, 2024 at 11:38 AM James McMahon <[email protected]> >> wrote: >> >>> When we examine the contents of a queue through the UI and select a >>> flowfile from the resulting list, we see FlowFile Details in the Details >>> tab. Are those key/values accessible from nifi expression language? I would >>> like to access Queued Duration. I have a queue that holds flowfiles with >>> non-successful return codes for calls to REST services, and I want to route >>> depending on how long these flowfiles have been sitting in my error queue >>> to isolate the window when the REST service was unavailable. >>> Thank you for any examples that show how we can access these keys and >>> values. >>> >>
