Hello This exchange occurred today in the #nifi IRC chatroom. Wanted to memorialize it here and respond.
Big thanks to Josh Elser for manning the booth and helping out! The question: +++++++++++++++++ 11:57 < ilesteban> Hi guys. I started to play around with NiFi recently. I'm planning to start using it for a project I'm currently working on. 11:57 < ilesteban> <ilesteban> I have some bpm background and I'm trying to see what are the things than bpm and nifi have in common and what are the things that they don't. 11:57 < ilesteban> <ilesteban> One of the first things I noticed is that, apparently, in Nifi there is no separation between the workflow definition and the workflow execution environment. 11:57 < ilesteban> <ilesteban> You define and execute your wrokflow in the same environment. 11:57 < ilesteban> <ilesteban> I like the idea, but I'm struggling to see how this fit into a distributed team environment. 11:57 < ilesteban> <ilesteban> Let's say I want to version a worflow (in git, svn, or whatever) so I can share it among different team members. I also want to keep versions from different milestones in my project. 11:57 < ilesteban> <ilesteban> How can I do that? 11:57 < ilesteban> <ilesteban> Do I have to version the whole NiFi app? 11:57 < ilesteban> <ilesteban> Is there any way to extract just the workflow definition (and maybe configuration) in a more versioning-friendly format? 11:57 < ilesteban> <ilesteban> The way I used to do this with bpm (jBPM in particular) was to export the process definition into XML and then version that XML. Another team member could then get that XML, import it into an editor and con 12:03 < elserj> Last I knew, there wasn't good versioning built into a workflow 12:03 < elserj> and it wasn't serialized in such a way that was conducive to sharing among users 12:03 < elserj> i'm not sure if that's be changed 12:03 < ilesteban> elserj, I'm thinking about templates 12:03 < elserj> you'd probably be better off asking on [email protected] though :) 12:04 < ilesteban> maybe I can export a template and version it 12:04 < ilesteban> elserj, I'll do that 12:14 < elserj> sorry I'm not of much help. I don't do much with Nifi. I just try to help out where I can :) +++++++++++++++ Response: NiFi certainly does take a different approach to the more common 'design and deploy' model. NiFI provides a real-time modification capability that affords immediate feedback. This model is really powerful in that it lets you immediately see the good or bad of changes being made. Further you can safely copy the live flow of data off for evaluation and testing without impacting the production flow. The whole idea is to foster and encourage iteration which we believe is best done in the fires of real data. But the point you bring up about being able to capture or save a design and share it with others is great too. NiFi supports templates for this exact reason. You can make real proven dataflows and then generate templates of them, version them as you wish, and share them with others. They can import them into their flows, improve them, and share them back and so on. I think there is more we can do to improve that but that will come with feedback and based on what folks are trying to accomplish. Thanks Joe
