On čtvrtek 9. listopadu 2017 10:14:33 CET Tomas Strachota wrote: > On Tue, Nov 7, 2017 at 11:17 AM, Ivan Necas <[email protected]> wrote: > > On Tue, Nov 7, 2017 at 9:29 AM, Lukas Zapletal <[email protected]> wrote: > >> We should start tracking test breakages in time, if we have a test > >> that did not break for a year, lets delete it. As simple as that. It's > >> added value is almost zero. > >> > >> This is not out of my head, I've heard that on a talk somewhere. > >> Instead of deleting, we can move it to archive - run them on a nightly > >> basis, if we find this too much. Frankly, I prefer deletion. > > > > -1: the fact that the test doesn't fail in a year can be just due > > to fact that we have not worked on that area for some time. It might > > work for some project with strictly defined use-case, but for Foreman, > > I don't think thats' the case. > > -1 as well for deleting the tests. They become useful once you start > refactoring things even when the code wasn't touched for a long time.
Another -1 for deleting based on time, +1 for revisting tests and deleting tests with zero value (there are such tests). Honestly I think the biggest performance improvement would be by manually going through tests and fixing them. Minimizing DB calls where possible, remove duplicated tests etc. -- Marek > > >> Also, let's ditch unit tests in favor of integration tests. We have a > >> lot of areas covered with both and this is extra work and slowing down > >> things. > > > > +1 for integration tests > unit tests: and I'm willing to talk about > > deleting unit-tests once we know that integration tests are covering the > > same. > > > > I feel unit-tests more important, when working on specific algorithm etc. > > We don't do it that often. We mostly integrate. > > +1 I see more value in integration tests in general. Unit tests for > algorithms are useful too. > I see less value in unit tests for attribute presence and simple > validations like uniqueness. > > >> LZ > >> > >> On Tue, Nov 7, 2017 at 9:19 AM, Ohad Levy <[email protected]> wrote: > >>> Hi, > >>> > >>> The challenge: > >>> > >>> As a developer, I feel under utilized waiting for tests, the current > >>> test > >>> suite for foreman core takes over an hour, multiply it by number of > >>> really > >>> simple code fixes based on comment and it takes up a good chunk of a > >>> day. > >>> > >>> I would like to suggest a short and long term solution, and would > >>> appreciate your feedback. > >>> > >>> Short-term: > >>> > >>> Break down test matrix into more specific groups, this will allow to > >>> re-trigger just a smaller part of the tests vs. the entire suit today, > >>> for > >>> example: > >>> * API tests > >>> * DB migrations > >>> * UI Integration tests > >>> * ... > > That would definitely help to improve day-to-day developer's experience. > > >>> Long term: > >>> > >>> Break down core into smaller repositories - This is probably a bigger > >>> topic > >>> then just tests, but its worth raising in this context as well.. the > >>> benefits i see are: > >>> > >>> * smaller repos - faster tests per repo (we would still need a plugin > >>> matrix kind of tests done at some point) > >>> * better review process - smaller repos would mean the maintainers of > >>> the > >>> repo actually care for all/most of the pr's - today its not the case - > >>> many > >>> PR's are handled by sub groups (e.g. anything under webpack folder is > >>> "someone else") > >>> * potentially better API between parts - e.g. we can create a schema > >>> specific repo (not saying its a good idea) - which will always work with > >>> a > >>> dummy rails app - in the long run - this will ensure our schema is > >>> resilient to upgrades and model changes in core. > > The biggest benefit of splitting the repos is imho the need for > stabilizing the api between components. > I believe that there would be some transfer to stability of the whole > project. > >>> I would even go further and enable auto merge after review + successful > >>> tests, e.g. > >>> 1. PR is sent > >>> 2. repo tests are executed and pass > >>> 3. reviewer approve the change > >>> 4. larger test matrix is executed and pass > >>> 5. code get auto merged. > >>> > >>> -- > >>> You received this message because you are subscribed to the Google > >>> Groups > >>> "foreman-dev" group. > >>> To unsubscribe from this group and stop receiving emails from it, send > >>> an > >>> email to [email protected]. > >>> For more options, visit https://groups.google.com/d/optout. > >> > >> -- > >> Later, > >> > >> Lukas @lzap Zapletal > >> > >> -- > >> You received this message because you are subscribed to the Google Groups > >> "foreman-dev" group. To unsubscribe from this group and stop receiving > >> emails from it, send an email to > >> [email protected]. For more options, visit > >> https://groups.google.com/d/optout. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "foreman-dev" group. To unsubscribe from this group and stop receiving > > emails from it, send an email to > > [email protected]. For more options, visit > > https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
