We should start tracking test breakages in time, if we have a test that did not break for a year, lets delete it. As simple as that. It's added value is almost zero.
This is not out of my head, I've heard that on a talk somewhere. Instead of deleting, we can move it to archive - run them on a nightly basis, if we find this too much. Frankly, I prefer deletion. Also, let's ditch unit tests in favor of integration tests. We have a lot of areas covered with both and this is extra work and slowing down things. LZ On Tue, Nov 7, 2017 at 9:19 AM, Ohad Levy <[email protected]> wrote: > Hi, > > The challenge: > > As a developer, I feel under utilized waiting for tests, the current test > suite for foreman core takes over an hour, multiply it by number of really > simple code fixes based on comment and it takes up a good chunk of a day. > > I would like to suggest a short and long term solution, and would appreciate > your feedback. > > Short-term: > > Break down test matrix into more specific groups, this will allow to > re-trigger just a smaller part of the tests vs. the entire suit today, for > example: > * API tests > * DB migrations > * UI Integration tests > * ... > > Long term: > > Break down core into smaller repositories - This is probably a bigger topic > then just tests, but its worth raising in this context as well.. the > benefits i see are: > > * smaller repos - faster tests per repo (we would still need a plugin matrix > kind of tests done at some point) > * better review process - smaller repos would mean the maintainers of the > repo actually care for all/most of the pr's - today its not the case - many > PR's are handled by sub groups (e.g. anything under webpack folder is > "someone else") > * potentially better API between parts - e.g. we can create a schema > specific repo (not saying its a good idea) - which will always work with a > dummy rails app - in the long run - this will ensure our schema is resilient > to upgrades and model changes in core. > > I would even go further and enable auto merge after review + successful > tests, e.g. > 1. PR is sent > 2. repo tests are executed and pass > 3. reviewer approve the change > 4. larger test matrix is executed and pass > 5. code get auto merged. > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. -- Later, Lukas @lzap Zapletal -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
