On Fri, Nov 11, 2016 at 09:22:51AM +0100, Lars Schneider wrote: > There would be an alternative way to approach the problem: > Someone (GitHub?, BitBucket?, GitLab?, ...) could setup a bunch of webservers > with popular configurations and a way to reset a clean test environment. Then > the TravisCI client tests could go against these servers. > > I realize that this idea is probably unrealistic because too much setup and > maintenance work would be required.
Yeah, it seems like it adds a lot of complexity for little gain. Plus it creates a network dependency on running the tests. I know you care mostly about Travis, but I am much more interested in all of the people (developers and not) who run "make test" on their own platforms. If you did want to have a more real-world network-based test, I think the right solution is not for GitHub to set up a bunch of mock servers, but to design client-side tests that hit the _real_ GitHub (or GitLab, or whatever) and perform some basic operations. OTOH, people running "master" (or "next", etc) are doing that implicitly every day. -Peff
