I agree with Bryan’s interpretation here. My understanding of that feature reference is in relation to high volume streams of data coming in and the DFM making proactive and/or realtime decisions about how to prioritize data when they don’t have the capability to capture/operate on all of it. Flowfile expiration and routing to failure (the provenance event is called “drop” for a reason) are explicit design decisions that address the scenario in legacy systems when "bad/less valuable/not currently able to be operated on" data could prevent the "better/more valuable" data from flowing.
Andy LoPresto [email protected] [email protected] PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Mar 3, 2017, at 6:15 AM, Bryan Bende <[email protected]> wrote: > > Andre, > > My understanding of that statement is that NiFi lets the user decide > how to build their dataflow, and allows them to build it in a way that > favors guaranteed delivery, or favors loss tolerance, depending on > their needs. > > I would think loss tolerance is referencing the idea of expiring data > from queues after a certain threshold, and maybe terminating failure > relationships. > > The overall point being that the user makes the decision and not the > framework, which is the same motto for a lot of other parts of NiFi. > > -Bryan > > > On Thu, Mar 2, 2017 at 11:54 PM, Andre <[email protected]> wrote: >> All, >> >> A good number of NiFi related material makes the following statement about >> NiFi: >> >> Loss tolerant vs guaranteed delivery >> >> I have not historically used NiFi in a loss tolerant environment but after >> investigating the documentation it isn't clear to me how NiFi implements >> loss tolerance? >> >> Are we referring to Clustering? Flow expiration? To the ability to discard >> flowfiles that get piped to the REL_FAILURE relationships? etc? >> >> Would anyone mind shedding some light about what are we referring to when >> presenting the "Loss tolerant vs guaranteed delivery" >> >> I thank you in advance
signature.asc
Description: Message signed with OpenPGP using GPGMail
