Hi,
Simple suggestion to detect lag would be (imagining 1 single flow handling
the 15 inbound streams):
*Processor UpdateAttribute*
Store State= store state locally
add one attribute per stream to keep last ts
last_stream_1_ts if stream=1 then ${now:toNumber()} else ${getStateValue("
last_stream_1_ts")}
processor RouteOnAttribute
stream1_lag if ${now:toNumber()} - ${last_stream_1_ts} > 86400000 then
'yes' else 'no'.
Sorry i used pseudo code -im in a restaurant :) - but just wanted to share
how you generate Boolean attributes that one can then use to do whatever
action needed.
Hope this helps,
Emanuel O.
On Fri 7 Feb 2020, 18:47 Marroquin, Stuardo X. -ND, <
[email protected]> wrote:
> Good afternoon,
>
>
>
> I was wondering if there was a way to monitor the data flow into NiFi and
> also from NiFi to our storage device.
>
> We have approximately 15 streams of files coming into NiFi at a rate of a
> file per minute, 24/7, 365 days, varying in size from 4MB to 12MB each. We
> need a way to ensure that we are notified if one of those streams stops.
>
> I saw that you have PutEmail but there is not a whole lot of documentation
> on setting it up and I don’t even know if that will meet our needs.
>
> Any help you can provide on this would be greatly appreciated.
>
>
>
> Kind Regards,
>
> Stu Marroquin
>
> [image: blue_strip]
>
> *Stu Marroquin*
>
> Information Management Senior Specialist
>
> The Walt Disney Account
>
> Atos North America
>
> 407-619-2498
>
> [email protected]
>
> (DISC) 5601 Center Drive
> Orlando, FL 32830
>
> www.na.atos.net
>
>
>
> [image: Atos]
>
>
>
> This e-mail and the documents attached are confidential and intended
> solely for the addressee; it may also be privileged. If you receive this
> e-mail in error, please notify the sender immediately and destroy it. As
> its integrity cannot be secured on the Internet, the Atos group liability
> cannot be triggered for the message content. Although the sender endeavors
> to maintain a computer virus-free network, the sender does not warrant that
> this transmission is virus-free and will not be liable for any damages
> resulting from any virus transmitted
>
>
>
