Mikhail Loenko wrote:
seems like we need to refersh our agreements aiming better stability. There are questions to discuss and decision to be made. 1) We have exclude list for each paltform and for two VMs. We have discussed a separation of the common part but did not reach any agreement. To make fixing VM- and platform-specific bugs easier I suggest that we agree to separate a common part.
I thought we agreed. Either way, this is a side issue wrt regression.
2) Usually committers check the classlib patches on J9 and then commit. At this point checking the patches on DRLVM is not that convinient so at the moment we don't ask committers to do classlib pre-commit testing on DRLVM.
Yes, and we're getting near the time to switch that. People have the freedom to do what they want, but we need to get this happening regularly. I'll do something to make it easier for this to happen - I'll make a "drop-n-go" snapshot of DRLVM so it can be dropped directly into classlib's deploy/jdk/jre/bin as /default.
The only remaining issue that I can see is the infernal hythread.so problem. Would we solve that simply by renaming our hythread.so in DRLVM?
3) CC should report each time state changes (pass to fail or fail to pass) and each time failure reason changed. Failure reason may change because some committers could miss failure notifiaction and do commit. Until we are able to say that failure reason has changed we report each failure and only the first success.
Sure.
4) We have cruise controls running classlibrary tests on DRLVM. We need to decide what will we do when DRLVM+Classlib cruise control reports failure.
Stop and fix the problem. Is there really a question here? I agree with Rana and Tim - we need to find a balance, and I think that with continued social awareness like this, we'll do better. We can always revisit if things don't go well.
IMHO It's pretty obvious that before we have found a commit causing that we stop further commits. 5) If the cause of failure is DRLVM commit than it's pretty obvious that we roll it back. 6) If the cause is Classlib test commit than we check wether the test is valid (e.g. run it on RI) and not VM-specific. If the test is valid we exclude it in DRLVM exclude list. If the test is invalid or VM-specific we roll it back or fix ASAP.
I don't completely agree, because it supports the idea that our tests must run on the RI, which can't be true for all classlib tests, and results in things like "isHarmony()".
We have two VMs to work with - one, a high-quality production grade VM (j9) and the other a soon-to-be high-quality production grade VM (DRLVM) :)
I suggest that a classlib failure is tested on both and then figure out what's the right answer, using the RI if possible.
7) If the cause is a Classlib commit then this commit either reveals existing problem in DRLVM or introduces a regression. I don't have strong preferences here whether we'd better exclude the test or roll the commit back
I think that it's fine to exclude the test *if* the commit identifies the problem and someone takes on the task of solving the problem immediately, in the interest of keeping things moving.
8) We may also advise that committers don't commit changes just before going sleep or vacation. They should wait reasonable time to receive CC failure report if any. The problem is "reasonable time" is hardly predictable Comments? Thanks, Mikhail
