Hello,
I just found Apache Nifi recently and I decided to use it to take care of our
ftp transfer system, which basically means to monitor the incoming data and
generate Flowfiles for each incoming file.
The basic flow I have in mind is:
1. GetFile to monitor the incoming directory
2. Lookup the actions for each Flowfile based on their path and filename
attributes in a database and store them as an attribute
3. Figure out the next action to do by looking at that attribute and route the
Flowfile into the correct flow implementing that action (e.g. execute a command
or send an email)
4. After that action has completed, send the Flowfile back to step 3 if there
are more actions that need to be done
I'm somewhat stumped on how to implement that in a sensible way with Nifi. Step
2 will result in something like the following json and I guess I have to write
my own processor for that:
[ { "unzip": null }, {"email": {"to": "[email protected]", "cc":
"[email protected]", "withAttachment": false}, { "execute": "/bin/true" }, {
"execute": "/bin/anothercommand " } ]
Now I can store that as an attribute, but apparently EvaluateJsonPath like all
other Evaluate* processors are unable to parse attributes and storing them as
content is not possible, as the existing content is what I have to work on
later.
I could also store just the next action type (unzip, email, execute) as an
Attribute for the RouteOnAttribute processor, but that means I need to run the
Flowfile through yet another script and even more scripts once it's been routed
to the flow implementing the action.
Due to the amount of scripts involved I'm beginning to question if Nifi is the
right tool for this job. Am I missing an obvious solution to this?
Regards,
Daniel Frank
