Hello I do not recommend viewing NiFi as something to use 'instead of' Storm or Spark. NiFi's design and the goals behind it are focused on managing the flow of information from systems which create data, to systems that process data, to systems that store data. Invariably in the process of connecting the many systems that exist throughout the enterprise there are mismatches of protocol, format, schema, priority, interest, authority, etc.. In solving these mismatches NiFi is often used to do 'data processing'. Examples of data processing that NiFi aims to be very compelling for include things like transforming from one format to another, filtering, sanitization, enrichment, aggregation, splitting, etc..
In this light there are certainly processing functions done in systems like Spark or Storm that NiFi would be perfectly fine and perhaps even better suited for. But, it is important to keep in mind the focus and intent of the systems. In my view, NiFi is oriented around dataflow - connecting systems within and between datacenters and so on. Storm and Spark are focused on processing/analysis of that data. So these systems make different design trade-offs and provide different user experiences centered around their intent. As a result I think the more compelling story is about how to leverage the strengths of these sorts of systems together. Always tough to answer this question in generic terms. Happy to talk through specific use cases. Thanks Joe On Wed, Oct 28, 2015 at 8:02 PM, Adaryl "Bob" Wakefield, MBA <[email protected]> wrote: > I've been studying NiFi and there is something I'm not quite understanding. > Can NiFi be used in place of Storm or Spark Streaming to process streaming > data? > > B.
