> On Oct 19, 2016, at 10:17, Dave Abrahams via swift-dev <swift-dev@swift.org> > wrote: > > > on Wed Oct 19 2016, Jordan Rose <swift-dev-AT-swift.org> wrote: > >>> On Oct 19, 2016, at 9:44, Dave Abrahams via swift-dev <swift-dev@swift.org> >>> wrote: >>> >>> >>> on Wed Oct 19 2016, Dave Abrahams <swift-dev-AT-swift.org >>> <http://swift-dev-at-swift.org/>> wrote: >>> >>>> It still seems like, for a smoke test, we're doing way too much work. >>>> This appears to be much more than what I get from build-script -t when >>>> I run tests locally. Maybe I'm misunderstanding the intended role of >>>> our smoke tests, but since nobody is correcting me, I'm betting not. >> >> Even smoke tests should run the validation tests… > > What's the point of distinguishing validation from other tests if even > the smoke tests run them?
I think the validation tests are tests that compiler developers don’t bother running locally before they move to a PR. I’d be very concerned about landing changes in master without having run the validation tests—that would get us back to the days of consistent failures because someone forgot to update compiler_crashers. > >>> Someone wrote to me privately: >>> >>> "buildbot_linux_1404" preset used in Linux smoke test contains >>> "--long-test". >>> >>> This seems wrong to me. Can we fix it? >> >> …but I could see "long tests" going either way. > > Well, what is “smoke” supposed to mean? If it requires running > validation tests and maybe even long tests, what is the “smoke test” > distinction *for*? It only builds and runs one platform per builder (no iOS/watchOS/tvOS), and doesn’t necessarily build and test all the downstream projects. I agree that if we are to have something called “smoke test”, 40 minutes is too long. Jordan _______________________________________________ swift-dev mailing list swift-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-dev