[
https://issues.apache.org/jira/browse/NIFI-5922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16753689#comment-16753689
]
Joseph Percivall commented on NIFI-5922:
----------------------------------------
For tracking, an initial discussion thread is here:
https://lists.apache.org/thread.html/%3c1217956179.5384158.1546477466...@mail.yahoo.com%3E
> NiFi-Fn, an alternative runtime for NiFi flows
> ----------------------------------------------
>
> Key: NIFI-5922
> URL: https://issues.apache.org/jira/browse/NIFI-5922
> Project: Apache NiFi
> Issue Type: Improvement
> Reporter: Sam Hjelmfelt
> Priority: Major
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> NiFi-Fn is a library for running NiFi flows as stateless functions. It
> provides similar delivery guarantees as NiFi without the need for on-disk
> repositories by waiting to confirm receipt ofincoming data until it has been
> written to the destination. This is similar to Storm’s acking mechanism and
> Spark’s interface for committing Kafka offsets, except that in NiFi-Fn, this
> is completely handled by the framework while still supporting all NiFi
> processors and controller services natively without change.This results in
> the ability to run NiFi flows as ephemeral, stateless functions and should be
> able to rival MirrorMaker, Distcp, and Scoop for performance,efficiency, and
> scalability while leveraging the vast library of NiFi processors and the NiFi
> UI for building custom flows.
> By leveraging container engines (e.g.YARN, Kubernetes), long-running NiFi-Fn
> flows can be deployed that take full advantage of the platform’s scale and
> multi-tenancy features. By leveraging Function as a Service engines (FaaS)
> (e.g. AWS Lambda, Apache OpenWhisk), NiFi-Fn flows can be attached to event
> sources (or just cron) for event-driven data movement where flows only run
> when triggered and pricing is measured at the 100ms granularity. By combining
> the two, large-scale batch processing could also be performed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
