On Mon, Nov 28, 2016 at 4:50 PM, Joseph D. Darcy <joe.da...@oracle.com> wrote:
> On 11/28/2016 3:30 PM, Martin Buchholz wrote: > > As an aside, for JDK 10 I'd also like to see promoted builds on a more >> frequent schedule than once a week. >> > > People do "continuous testing and integration" these days. Set up your > integration pipeline so that it is always running. The pipeline > automatically promotes changesets to master when all tests pass. Easy! > Then every master changeset is equally stable to a "promoted build". > > > One of the internal systems mentioned above is a CI system that run > regression tests after a change is pushed. The approximate integration > model is then to promote known-good states of sources as vetted by the CI > system. > > If problems are fixed soon after they are identified, then a post-push > system gives good results with avoiding the need more complexity on the > front end to manage a series of in-flight patches. > A CI system available to every committer that would guarantee no regressions would be awesome! I fundamentally do not believe in post-submit testing. I want to be protected from my own mistakes and those of others. Google has also moved towards doing more presubmit testing. For large low-level infrastructure projects like openjdk, I envision different layers of testing. The first layer would run e.g. the jtreg and jck tests, perhaps on a distributed test farm. A second layer might run publicly available java test suites that would be too expensive to run in a first layer. Promotion to a "golden master" might be dependent on passing tests in the second layer. Of course, that would be a serious investment in testing/quality.
