Thanks for the clarification. I'll try to see what other options do I have. My scenario is not a simple ETL where everything gets resolved using local data. The flow interacts with a bunch of external systems to both, retrieve and send data. I was planning to use mocked services in those cases.
Thanks for your time! Regards, XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Esteban Aliverti - Blog @ http://ilesteban.wordpress.com On Tue, Feb 24, 2015 at 4:54 PM, Mark Payne <[email protected]> wrote: > Esteban, > > > > > OK, thanks for clarifying. This notion of how to test your flow has > starting coming up a lot in the last week or so on the dev list. I’ll take > this to be a good thing, meaning that people are really starting to think > about how they can use it! 😊 > > > > > We have actually intentionally avoided creating anything to allow > developers to create unit/integration tests for flows. Creating these sorts > of tests are very common for most ETL types of applications. However, NiFi > was designed specifically so that there was no need to do so. The real-time > command and control that NiFi affords you is very different than that > provided by typical ETL applications. Because of this real-time command and > control, we push operators to build a “test" flow on the production > instance and run the live data through the test flow at the same time that > it is processed by the operational flow. In the test flow, though, the > operator will likely terminate the data at the end, rather than sending to > any other system. This allows the operator to verify that everything is in > good shape with the test flow. Once it has been determined that everything > is in good shape, the operator should then enable sending the data to the > appropriate destinations. This tends to provide much faster time-to-market > of flow changes. > > > Over the past several days there has been another thread discussing the > notion of testing a flow. This thread can be viewed at > http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Great-question-on-nifi-IRC-room-today-NiFi-BPM-sharing-configuration-td787.html#a811 > > > If you have specific questions/concerns, I’d be happy to discuss in more > detail. > > > Thanks > > -Mark > > > > > > > > > > From: Esteban Aliverti > Sent: Tuesday, February 24, 2015 10:52 AM > To: [email protected] > > > > > > Thanks for the prompt response Mark. > My idea was to have a NiFi workflow defined and to be able to run some > automated tests on it. I thought that maybe there was a way to start an > embedded version of NiFi programatically and to interact with it. > I did see the TestRunner class and I even used it for some of the unit > tests of some custom processors I've implemented. TestRunner seems to be > exactly what I'm looking for, but instead of being focused in a single > processor, I need something that allows me to run multiple processors that > compose a flow. > I do agree that each individual processor must be tested using a > TestRunner, but I was planning on doing something more like an integration > test of an entire flow. > > Regards, > > > > > XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX > > Esteban Aliverti > - Blog @ http://ilesteban.wordpress.com > > On Tue, Feb 24, 2015 at 4:42 PM, Mark Payne <[email protected]> wrote: > > > Esteban, > > > > > > NiFi does not allow you to run it embedded. However, it does provide a > > “Mock Framework” that we use very heavily for doing unit testing of NiFi > > components. > > > > > > Can you explain your use case a bit further? What exactly are you trying > > to unit test? > > > > > > Thanks > > > > -Mark > > > > > > > > > > > > > > From: Esteban Aliverti > > Sent: Tuesday, February 24, 2015 8:34 AM > > To: [email protected] > > > > > > > > > > > > Hello, > > My name is Esteban Aliverti and I'm currently trying to use NiFi in one > of > > my company's projects. The idea is to use NiFi behind a java API. > > What I was wondering is if there's any way to run NiFi embedded so I can > > easily build unit tests without having to start a separate NiFi server > > before I run the tests. Another important thing is that each test should > be > > self-contained and thus I would like them to always run in a "clean" NiFi > > instance. > > What I was thinking was a way to programatically configure and start NiFi > > and maybe import a workflow from a template definition I can create > > beforehand. > > Is this currently possible? If not, any suggestion? > > > > Regards, > > > > XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX > > > > Esteban Aliverti > > - Blog @ http://ilesteban.wordpress.com > > >
