On Tue, Nov 7, 2017 at 9:29 AM, Lukas Zapletal <[email protected]> wrote: > We should start tracking test breakages in time, if we have a test > that did not break for a year, lets delete it. As simple as that. It's > added value is almost zero. > > This is not out of my head, I've heard that on a talk somewhere. > Instead of deleting, we can move it to archive - run them on a nightly > basis, if we find this too much. Frankly, I prefer deletion.
-1: the fact that the test doesn't fail in a year can be just due to fact that we have not worked on that area for some time. It might work for some project with strictly defined use-case, but for Foreman, I don't think thats' the case. > > Also, let's ditch unit tests in favor of integration tests. We have a > lot of areas covered with both and this is extra work and slowing down > things. +1 for integration tests > unit tests: and I'm willing to talk about deleting unit-tests once we know that integration tests are covering the same. I feel unit-tests more important, when working on specific algorithm etc. We don't do it that often. We mostly integrate. > > LZ > > On Tue, Nov 7, 2017 at 9:19 AM, Ohad Levy <[email protected]> wrote: >> Hi, >> >> The challenge: >> >> As a developer, I feel under utilized waiting for tests, the current test >> suite for foreman core takes over an hour, multiply it by number of really >> simple code fixes based on comment and it takes up a good chunk of a day. >> >> I would like to suggest a short and long term solution, and would appreciate >> your feedback. >> >> Short-term: >> >> Break down test matrix into more specific groups, this will allow to >> re-trigger just a smaller part of the tests vs. the entire suit today, for >> example: >> * API tests >> * DB migrations >> * UI Integration tests >> * ... >> >> Long term: >> >> Break down core into smaller repositories - This is probably a bigger topic >> then just tests, but its worth raising in this context as well.. the >> benefits i see are: >> >> * smaller repos - faster tests per repo (we would still need a plugin matrix >> kind of tests done at some point) >> * better review process - smaller repos would mean the maintainers of the >> repo actually care for all/most of the pr's - today its not the case - many >> PR's are handled by sub groups (e.g. anything under webpack folder is >> "someone else") >> * potentially better API between parts - e.g. we can create a schema >> specific repo (not saying its a good idea) - which will always work with a >> dummy rails app - in the long run - this will ensure our schema is resilient >> to upgrades and model changes in core. >> >> I would even go further and enable auto merge after review + successful >> tests, e.g. >> 1. PR is sent >> 2. repo tests are executed and pass >> 3. reviewer approve the change >> 4. larger test matrix is executed and pass >> 5. code get auto merged. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "foreman-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. > > > > -- > Later, > Lukas @lzap Zapletal > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
