Hey Krish,
Welcome to NiFi! Storm is how I got my start with data flow as well and NiFi
has been awesome. NiFi's "processors" are equivalent to Storm's "spouts" and
"bolts".
NiFi can certainly handle the logging/auditing use case among others. With
NiFi's data provenance you can see exactly how each message was handled, the
content and attributes at each stage, as well as replay any message at any
stage if you feel the need.
For ingesting into NiFi there are a couple of options depending on your system.
If the files already exist on the same system then there is a simple "getFile"
processor. For other set-ups you can check out the "get" processors in the
documentation [1]. For forwarding to another service again it depends on your
system. If you set up two NiFi instances you can check out "Remote Process
Group". If you're just "Put"ing it through HTTP there is a processor for that
as well. Again there are other ways for exfiltrate the data depending on your
set-up.
So for the configuration it all depends on what you want to do and how your
system is set up. If you give a bit more information on the specific services
you're using or use-case we can give even more direction.
[1] https://nifi.apache.org/docs.html
Again welcome to NiFi,Joe- - - - - - Joseph
Percivalllinkedin.com/in/Percivalle: [email protected]
On Sunday, September 20, 2015 1:15 AM, Krish <[email protected]>
wrote:
Hi,
I am a n00b at NiFi although I have worked with Storm; currently that is how we
handle data flow logic.
I am evaluating using NiFi for logging/auditing use case (might move to other
uses if this works) in a ~50 machine cluster, and am thinking if it would be a
good fit to ingest messages from various sources and spit out the graph of how
a message was handled at each stage, by each of the services.
Anyway, I was wondering how do we feed data into a NiFi cluster to get the
first stage started. Also, how does the data exit the system, if I want it to
be forwarded to another service after NiFi is done with its processing [similar
to spouts and bolts in apache storm].
Thanks.--
κρισhναν
