Vincent, As Aldrin mentioned there is an effort underway to help with integration testing. But I'm not sure yet if it will help with what you're looking to do from a performance baseline perspective. I totally get Mark's cautionary comments but given your response it does seem like there is something we can do to help. So let's say we have a nice way for someone to create an integration test which wires together processors to do something meaningful. This obviously verifies functionality of a chain of processors. But, you also seem to have an idea in mind for comparing performance.
I agree there could be something telling about that given the test environment could enforce single threading and such which might help it reveal interesting deviations. Not sure. How do you see that working in terms of you being able to capture the data in successive test runs to validate whether things have deviated? Would manual/eyeball processes be sufficient as a first start? Thanks Joe On Tue, Jan 5, 2016 at 5:59 PM, Vincent Russell <[email protected]> wrote: > Thanks Mark, > > I didn't mean to suggest that I would like to compare the results of my test > with actual nifi performance, but I would think that I would be able to > compare different run iterations with each other. > > I did notice that the MockFlowFile is pretty much a wrapper around a byte > array, which may or may not be a problem for my small scale testing. > > > On Tue, Jan 5, 2016 at 5:40 PM, Mark Payne <[email protected]> wrote: >> >> Vincent, >> >> I would be vary wary about trusting performance results that you obtain by >> using the Mock Framework. >> The mock framework is intended to be used only for testing correctness, >> not performance. It has very >> different threading characteristics than the "actual" NiFi framework, and >> it uses very different FlowFile, >> Content, and Provenance Repositories. Processor A may perform far better >> than Processor B in the >> mock framework, but that does not by any means guarantee that it will also >> perform better in a live >> environment. >> >> Thanks >> -Mark >> >> On Jan 5, 2016, at 5:17 PM, Vincent Russell <[email protected]> >> wrote: >> >> Hello Aldrin, >> >> Thanks for the response. >> >> My current use case is that I would like to chain several processors >> together and write a performance test against that mini flow and then be >> free to modify the processors that are in the chain and see how performance >> changes. I think I may be able to chain several TestRunners together to >> achieve my goal, although this isn't ideal. >> >> Ideally I'd be able to provide the TestRunner with multiple Processors and >> identify how the processors are chained together. >> >> Thanks, >> >> >> >> On Mon, Jan 4, 2016 at 3:26 PM, Aldrin Piri <[email protected]> wrote: >>> >>> Hello Vincent, >>> >>> This is something that does not exist and there have been a few threads >>> on this topic [1][2]. >>> >>> Summarily, these tools do not currently exist due to the preference of >>> using the interactive and real-time command and control over the flow as >>> well as the increasing difficulty of maintaining flows as they grow and >>> evolve. >>> >>> There are some good tips on how other people have tackled the problem in >>> the linked message threads. One alternative suggestion is making use of >>> NiFi's template functionality [3] to work on stubbing out flows on a >>> different instance and use that to promote an entire flow or segment to >>> another system. Templates are an area we are planning to both enhance and >>> mature as laid out in some of our feature proposals [4][5]. >>> >>> Please let us know if this is accomplishes the functionality you are >>> looking for or if we are coming up a bit short on some of what your needs >>> are for integration level testing. Your case is common and certainly one we >>> need to execute on well. Any feedback you can provide from your perspective >>> both in view of the current state of templates and the path forward as laid >>> out in the proposals would be much appreciated! >>> >>> Thanks! >>> >>> --aldrin >>> >>> [1] >>> https://mail-archives.apache.org/mod_mbox/nifi-dev/201502.mbox/%[email protected]%3E >>> [2] >>> http://apache-nifi-developer-list.39713.n7.nabble.com/Great-question-on-nifi-IRC-room-today-NiFi-BPM-sharing-configuration-td787.html#a811 >>> [3] https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#templates >>> [4] >>> https://cwiki.apache.org/confluence/display/NIFI/Extension%2C+Template%2C+Dataset+Registry >>> [5] >>> https://cwiki.apache.org/confluence/display/NIFI/Configuration+Management+of+Flows >>> >>> >>> On Mon, Jan 4, 2016 at 1:37 PM, Vincent Russell >>> <[email protected]> wrote: >>>> >>>> All, >>>> >>>> I see that there is a way to test a single processor with the TestRunner >>>> (StandardProcessorTestRunner) class, but is there a way to set up an >>>> integration test to test a complete flow or a subset of a flow? >>>> >>>> Thank you, >>>> Vincent >>> >>> >> >> >
