Hi Eric, The REST API provides a great deal of power for manipulating flows, by creating/updating/deleting components and for obtaining information such as metrics about the components on the graph.
The REST API doesn't provide any ability, though, to access FlowFiles from outside of NiFi. FlowFile access (content and attributes) is intended to be done only through the Processors that are added to the flow. Currently, the capability to impact a FlowFile based on other FlowFiles (e.g., children) isn't present. However, this is something that is being worked on ticket NIFI-190 [1]. Once this ticket is complete, you should be able to "hold" a FlowFile while you process its children, and then once the children have completed processing, the flow can be notified to release the original FlowFile, optionally passing some attributes to the original, so that you can make routing decisions, etc. There are certainly mechanisms for packaging FlowFiles with their attributes and contents. If you are transferring the data directly from one NiFi to another, this can be accomplished by using site-to-site to transfer the data between instances. Unfortunately, it looks like the site-to-site information is not documented much at all in the User Guide, so I have created a ticket for that, as well [2]. This is really the preferred method for transferring data between two NiFi instances, as it will automatically detect clusters and perform load balancing. Alternatively, the PostHTTP/ListenHTTP processors to pass the data and configuring the PostHTTP to include attributes. If you are not sending the data directly to another NiFi but have some sort of intermediary step, you can package the FlowFiles with their contents by using MergeContent. If you need each FlowFile packaged separately, you can do that by setting both the min and max entries to 1. Otherwise, you can package multiple together into a single file. For the Merge Strategy, use the Bin-Packing Algorithm and for the Merge Format, choose the "FlowFile Stream, v3". Sorry, these are some fairly long-winded answers, but I wanted to ensure that I didn't leave you with more questions than answers :) If anything isn't clear, please let me know! Thanks -Mark [1] https://issues.apache.org/jira/browse/NIFI-190 [2] https://issues.apache.org/jira/browse/NIFI-832 ---------------------------------------- > Date: Sun, 9 Aug 2015 22:30:49 -0400 > Subject: NiFi Flow Usage questions > From: [email protected] > To: [email protected] > > Do you have an example workflow of a REST API or external program that > reads the flowfile contents and update attributes with the output? > > Can you have processors for child flows impact original flow? > -Compressed file is input into system > -Unpack file > -Run all children files through external program > -Make decision on original based on children attributes > > Last and not least- > Is there a way to export and import attributes between different Nifi > instances? You could have uuid.attributes and uuid.file to match them on > the distance end. > > Thoughts? > > Thank you, > Eric
