One thing that could encourage moving certain tests from the ATH to the JTH would be JENKINS-41827 <https://issues.jenkins-ci.org/browse/JENKINS-41827>.
On Mon, Oct 2, 2017 at 1:52 PM, Oliver Gondža <[email protected]> wrote: > A lot of comments and valuable feedback has popped up by the end of the > week. Let me react here instead of on individual messages. > > - No argument on slow test run, high flakiness, fragility. Some of that is > caused by not enough attention as ATH stands a bit aside the delivery > pipeline. Bringing it to PR builds of core or plugins would point more > people to it (hopefully making it better) but it would be annoying at first > until we make it more mature/stable. The question is if we are willing to > bite the bullet here. > > - With regards to the *kinds* of tests that should be part of ATH, the > thing is this was never defined so different kinds got in. I tend to agree > that what can be written against JTH, should. Perhaps it would be > beneficial to write that down and guard no more such tests are added to > prevent further bloating. The existing ones can either be deleted, migrated > or tolerated based on how much effort does it take to maintain them. > > - Ad smoke tests: This seems like a good compromise (starting point?) for > PR integration for several reasons. No question the actual test set should > be revisited. All (sufficiently distinct) core scenarios + recommended > plugin set scenarios + some popular scenarios of non-recommended plugins? > > - Speaking of deleting flaky tests - I am willing to give this a try as > the idea of stable build sounds wonderful and almost unheard of in ATH > land. As a maintainer of downstream plugin update center certified by ATH > internally, I am less excited. Again, I would like to see some consensus on > the process. Sometimes it might be enough to check JTH covers same scenario > or it can be migrated easily. What if it does not and the scenario appears > fundamental? How to tell? > > Please let me know if there are any additional thoughts or I have not > commented on something. Later this week, I intend to summarize this in a > doc or two in ATH repo to base our future decisions on (w.r.t. removing > tests, not accepting them, etc.) and update the thread with PR link. > > -- > oliver > > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit https://groups.google.com/d/ms > gid/jenkinsci-dev/6379b1ca-a85d-9041-1695-e9cd02aa042b%40gmail.com. > > For more options, visit https://groups.google.com/d/optout. > -- Regards, Andres Rodriguez www.cloudbees.com -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAM9%3DZ%2B7bxf-Kv68T5F0VPqdB6r18K-aw6MqhfnBFrvet_8sD-g%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
