Break down test matrix into more specific groups, this will allow to re-trigger just a smaller part of the tests vs. the entire suit today, for example:
> * API tests > * DB migrations > * UI Integration tests > * ... > > Probably a re-hash of what most developers know, but I'm going to do this anyway. Automated tests can be split into two groups -- "developer" and "everything-else" tests. The former are meant to be used often, even continuously (and are also known as unit-tests). These tests are meant to test individual pieces of functionality (in OOP -- classes) in isolation from the rest of the system (in other words without any dependencies, external or other parts of the system). The rest of automated tests, such as integration, end-to-end (including acceptance) tests can be slow and fragile, and are oftentimes executed at some intervals (e.g. once on hour). Of course, in rails environment we have almost unit-tests (model tests), almost-integration tests (controller tests), and a bunch of weird (but useful) stuff. Not sure how useful it would be to try to break up rails tests into smaller chunks (at least as the first step) -- models and controllers are rather tightly coupled, for example. I think it would be useful to be able to run *just* the rails tests however, as it will give developers and CI an ability to quickly (or at least quicker) check if the change breaks anything. The rest of the tests can be executed periodically, but possibly with different frequencies per suite of tests. One of the main and biggest drawbacks of such an approach is that the HEAD of the development branch can become unstable/broken. When it happens, it can be difficult to determine what change caused the failure (I wonder if automated git-bisect can help with this). Such an approach also requires a coordination of efforts in locating the issue, fixing it, and in abstaining from committing code while the build is broken . > Long term: > > Break down core into smaller repositories - This is probably a bigger > topic then just tests, but its worth raising in this context as well.. > Can we talk about service architecture now? -d -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
