On Tue, Nov 7, 2017 at 11:17 AM, Ivan Necas <ine...@redhat.com> wrote: > On Tue, Nov 7, 2017 at 9:29 AM, Lukas Zapletal <l...@redhat.com> wrote: >> We should start tracking test breakages in time, if we have a test >> that did not break for a year, lets delete it. As simple as that. It's >> added value is almost zero. >> >> This is not out of my head, I've heard that on a talk somewhere. >> Instead of deleting, we can move it to archive - run them on a nightly >> basis, if we find this too much. Frankly, I prefer deletion. > > -1: the fact that the test doesn't fail in a year can be just due > to fact that we have not worked on that area for some time. It might > work for some project with strictly defined use-case, but for Foreman, > I don't think thats' the case. >
-1 as well for deleting the tests. They become useful once you start refactoring things even when the code wasn't touched for a long time. >> >> Also, let's ditch unit tests in favor of integration tests. We have a >> lot of areas covered with both and this is extra work and slowing down >> things. > > +1 for integration tests > unit tests: and I'm willing to talk about deleting > unit-tests once we know that integration tests are covering the same. > > I feel unit-tests more important, when working on specific algorithm etc. > We don't do it that often. We mostly integrate. +1 I see more value in integration tests in general. Unit tests for algorithms are useful too. I see less value in unit tests for attribute presence and simple validations like uniqueness. > >> >> LZ >> >> On Tue, Nov 7, 2017 at 9:19 AM, Ohad Levy <ohadl...@gmail.com> wrote: >>> Hi, >>> >>> The challenge: >>> >>> As a developer, I feel under utilized waiting for tests, the current test >>> suite for foreman core takes over an hour, multiply it by number of really >>> simple code fixes based on comment and it takes up a good chunk of a day. >>> >>> I would like to suggest a short and long term solution, and would appreciate >>> your feedback. >>> >>> Short-term: >>> >>> Break down test matrix into more specific groups, this will allow to >>> re-trigger just a smaller part of the tests vs. the entire suit today, for >>> example: >>> * API tests >>> * DB migrations >>> * UI Integration tests >>> * ... That would definitely help to improve day-to-day developer's experience. >>> >>> Long term: >>> >>> Break down core into smaller repositories - This is probably a bigger topic >>> then just tests, but its worth raising in this context as well.. the >>> benefits i see are: >>> >>> * smaller repos - faster tests per repo (we would still need a plugin matrix >>> kind of tests done at some point) >>> * better review process - smaller repos would mean the maintainers of the >>> repo actually care for all/most of the pr's - today its not the case - many >>> PR's are handled by sub groups (e.g. anything under webpack folder is >>> "someone else") >>> * potentially better API between parts - e.g. we can create a schema >>> specific repo (not saying its a good idea) - which will always work with a >>> dummy rails app - in the long run - this will ensure our schema is resilient >>> to upgrades and model changes in core. The biggest benefit of splitting the repos is imho the need for stabilizing the api between components. I believe that there would be some transfer to stability of the whole project. >>> >>> I would even go further and enable auto merge after review + successful >>> tests, e.g. >>> 1. PR is sent >>> 2. repo tests are executed and pass >>> 3. reviewer approve the change >>> 4. larger test matrix is executed and pass >>> 5. code get auto merged. >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "foreman-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to foreman-dev+unsubscr...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >> >> >> >> -- >> Later, >> Lukas @lzap Zapletal >> >> -- >> You received this message because you are subscribed to the Google Groups >> "foreman-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to foreman-dev+unsubscr...@googlegroups.com. >> For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-dev+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
