I work with a medium sized Storm cluster that is used by many tenants. As the admins of the Storm cluster we must be mindful of network and CPU IO and adjust, manually, based on usage. Many of these Storm uses would be a better fit with NiFi's inbuilt capabilities and ease of use whilst leaving the high throughput work in Storm. Storm works really well out of the box with many (dozens) of separate users across hundreds of topologies. We simply add more nodes and don't have to worry much about load and users walking over each other since our failure replay is from Kafka always.
What isn't obvious to me is how local buffer storage is handled in a multi-tenant NiFi cluster and am wondering if others have patterns out there to prevent a NiFi user from eating up available disk thus downing other user's workflows. My initial thought is a management layer outside of NiFi that invokes Linux FS quotas by user account. Does NiFi have anything built in for this type of preventive measure? Thanks, Kris
