On Mon, Apr 15, 2013 at 10:43:25AM +0530, Prasanna Santhanam wrote: > On Fri, Apr 12, 2013 at 12:54:17PM -0400, Chip Childers wrote: > > > > > > > >Koushik, > > > > > > > >I think that it will require working with Prasanna to get a Marvin test > > > >suite for the feature ready to go. I'd personally like to see that > > > >*prior* to the merge into master. We need to "up our game" on automated > > > >testing of new features, so that we stop digging ourselves deeper and > > > >deeper into a hole. > > > > > > > >-chip > > > > > > But this approach isn't scalable either. > > > > > > Every new whiz bang network device will require infrastructure / licenses > > > / etc. > > > > > > > > > I think if there's component level tests of the code introduced (at least > > > the network elements) that should be a good first step. > > > > > > -- > > > Chiradeep > > > > Agreed that it would be a good first step, absolutely. We really do > > need to think about how we deal with all of these integrations... and > > how we do ongoing testing. It's going to be a challenge either way. > > > > The point of having a marvin test, IMO, isn't necessarily tied to the CI > > infrastructure. Instead, it allows for a regression test to be run when > > and if there is an appropriate target infrastructure. I'm sort of > > breaking the problem into two parts: having tests to run if you have the > > required setup, and figuring out how we get the required setup. > > > > In general the way we've taken care of external devices for other > network appliances, NetScaler for instance, is to assume the > deployment contains the network element present (inline, side-by-side, > thingamajig mode) and test the services it provides as a tenant would > use it. We don't test the actual deployment itself and assume the > admin has provisioned and registered the appliance appropriately. > > -- > Prasanna., >
Makes sense, and that jives with my point: having the tests is important, because then if you have the gear you know how to confirm the functionality via automated tests.
