Nirmal, NiFi's orientation is around continuous streams of data rather than starting and completing jobs.
When you look at processors via the UI and you see the green started arrow or the red stopped square it is best to think about that as "this component is (or is not) scheduled to execute" rather than "this is running or not running". It is ok that they're scheduled to run. They're still not going to actually get triggered unless they need to be. If PutHDFS has not data to send then it won't actually be triggered to execute. The status information you mention, particularly the rolling 5 min window information, is to help operators and flow managers understand behavior in the recent past and is not likely to be useful for an external process to decide when to schedule and unschedule components to be eligible to run. Does this make sense? Does this approach create some challenge for your use case? If so please describe a bit more on what you're trying to accomplish so we can see if we can help. Thanks Joe On Tue, Nov 22, 2016 at 7:12 AM, Nirmal Kumar <[email protected]> wrote: > Hi All, > > > > How do I know when exactly the Process Group(and its processors) has > actually finished the processing? > > > > Consider a simple Nifi use case: > > GetFile -> PutHDFS > > > > I want to execute the above use case and on completion want to stop it. > > > > From the Rest API endpoint > http://nifi-host:port/nifi-api/process-groups/$processGroupId I can get the > below statistics(aggregateSnapshot). > > Currently I check these aggregateSnapshot 25 metrics for zeros and > runningCount as 2. > > But somehow seems like these statistics by design shows last 5 mins metrics > that means I will need to wait at least for 5 mins. > > My requirement is to get the status of Nifi graph / Process Group as soon as > the use case is done processing. > > > > I also tried looking if this 5 min time period can be tweaked/changed > somewhere but seems it CANNOT. > > > > { > > revision: + {... }, > > id: "4e26e718-0158-1000-90e2-7b10528dc41d", > > uri: > "http://xxx.xxx.xxx.xxx:8080/nifi-api/process-groups/4e26e718-0158-1000-90e2-7b10528dc41d";, > > position: + {... }, > > permissions: + {... }, > > bulletins:[], > > component: + {... }, > > status: - { > > id: > "4e26e718-0158-1000-90e2-7b10528dc41d", > > name: "TestGetFileToHDFS", > > statsLastRefreshed: > "17:20:49 IST", > > aggregateSnapshot: - { > > id: > "4e26e718-0158-1000-90e2-7b10528dc41d", > > name: > "TestGetFileToHDFS", > > flowFilesIn: > 0, > > bytesIn: 0, > > input: "0 (0 > bytes)", > > > flowFilesQueued: 0, > > bytesQueued: > 0, > > queued: "0 > (0 bytes)", > > queuedCount: > "0", > > queuedSize: > "0 bytes", > > bytesRead: > 0, > > read: "0 > bytes", > > > bytesWritten: 0, > > written: "0 > bytes", > > > flowFilesOut: 0, > > bytesOut: 0, > > output: "0 > (0 bytes)", > > > flowFilesTransferred: 0, > > > bytesTransferred: 0, > > transferred: > "0 (0 bytes)", > > > bytesReceived: 0, > > > flowFilesReceived: 0, > > received: "0 > (0 bytes)", > > bytesSent: > 0, > > > flowFilesSent: 0, > > sent: "0 (0 > bytes)", > > > activeThreadCount: 0 > > } > > }, > > runningCount: 2, > > stoppedCount: 0, > > invalidCount: 0, > > disabledCount: 0, > > activeRemotePortCount: 0, > > inactiveRemotePortCount: 0, > > inputPortCount: 0, > > outputPortCount: 0 > > } > > > > Thanks, > > -Nirmal > > > > > ________________________________ > > > > > > > NOTE: This message may contain information that is confidential, > proprietary, privileged or otherwise protected by law. The message is > intended solely for the named addressee. If received in error, please > destroy and notify the sender. Any use of this email is prohibited when > received in error. Impetus does not represent, warrant and/or guarantee, > that the integrity of this communication has been maintained nor that the > communication is free of errors, virus, interception or interference.
