Hi All, I have been using Nifi to setup some pipelines now. Before I can absorb more use cases into this, I need to understand a few capabilities
1) Can we setup an user authentication before the web application. If yes, is there a way we can have role based access for processor groups. I would like certain teams working on only specific groups and not control all. 2) If the major use case would only involve reading from RMQ, KAFKA convert to parquet and store in S3, does it make sense to setup a cluster or just vertical scaling is good ? 3) Are the flow files in the queues (connections between processors) persisted?. Any machine failure or restart would cause a loss of data ? For instance messages are dequeued form RMQ and lost due to failure. Which would be a best way to handle this ? I think maintaining a low back pressure (threshold) can help mitigate the loss 4) Does the Kafka consumer, by default consume all partitions or is there a way to control that. 5) Can we have some of the metrics of processors pushed out as notifications or alerts (flow file count in / out or errors etc) It would be great, if someone could share resources that address these. Thanks in advance. -Dweep -- *::DISCLAIMER:: ---------------------------------------------------------------------------------------------------------------------------------------------------- The contents of this e-mail and any attachments are confidential and intended for the named recipient(s) only.E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted,lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents(with or without referred errors) shall therefore not attach any liability on the originator or redBus.com. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of redBus.com. Any form of reproduction, dissemination, copying, disclosure, modification,distribution and / or publication of this message without the prior written consent of authorized representative of redbus. <http://redbus.in/>com is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately.Before opening any email and/or attachments, please check them for viruses and other defects.*
