> Olegs seems proposed a reasonable target for the tests- 98% pass rate > of all valid tests from these test suites.
what is the % today? where can I find these numbers to make sure 98% is real?
It would be also nice to make clear what a "valid" test is. According to CC pass rate for HUT for Windows 32 is 100%. But this is only for enabled test. At the same time we have a lot of excluded tests in beans, swing and etc. And in spite of the fact that many of them seem to be "valid" IMHO it is not realistic to enable all of them (or at least 98%) till the end of the June. Regards, 06 Jun 2007 10:49:14 +0400, Egor Pasko <[EMAIL PROTECTED]>:
On the 0x2ED day of Apache Harmony Pavel Ozhdikhin wrote: > Mikhail, > > Thanks for capturing the requirements. IMO the requirements still lack > good tests to ensure stability and reliability. I propose to include > the following recently accepted tests: > > 1. VM validation test suite - http://issues.apache.org/jira/browse/HARMONY-3206 > 2. Functional test suite - http://issues.apache.org/jira/browse/HARMONY-3528 > 3. Stress tests - http://issues.apache.org/jira/browse/HARMONY-3536 > 4. Reliability test - https://issues.apache.org/jira/browse/HARMONY-2918 > > Olegs seems proposed a reasonable target for the tests- 98% pass rate > of all valid tests from these test suites. what is the % today? where can I find these numbers to make sure 98% is real? If we formulate M2 in percentage terms, then it makes sense to regularly update a wiki page with percentage of tests passed for each platform/testsuite. I think, Oleg would be glad to do that :) > Thanks, > Pavel > > On 6/6/07, Mikhail Loenko <[EMAIL PROTECTED]> wrote: > > I've captured all the proposed requirements and schedule here [1] > > (it should appear shortly, until that you may find it here [2]) > > > > The requirements do not look quite aggressive, can we do more > > by the end of month? > > > > Should I put there names of the people who proposed requirements? > > > > Sure, we can use "Due Before" field to mark JIRA issues, what is the most > > natural way to find which bugs affect some specific requirement? > > > > I personally don't like having many [sometext] in JIRA summary... > > Maybe we can add some specific text to comments? > > > > For example if requirements have names, we could add comments like > > "this one badly affects M2-REQ1" to JIRA and then use e.g. M2-REQ1 as > > a search string. > > > > The only problem I see with that is that people could drop dash or misspell > > the name in other way, and these bugs might be overlooked. Though the > > due before field won't probably allow us to overlook... > > > > Opinions? > > > > Thanks, > > Mikhail > > > > [1] http://harmony.apache.org/m2.html > > [2] http://svn.apache.org/viewvc/harmony/standard/site/docs/m2.html?view=co > > > > 2007/6/5, Rana Dasgupta <[EMAIL PROTECTED]>: > > > 2 months sounds like a reasonable interval. How about just expanding > > > the jira to mark milestone..(.in fact existing fields like "Due > > > Before" etc. could be used)...but keeping the milestone requirement > > > list on the Wiki? > > > > > > On 05 Jun 2007 18:23:22 +0400, Egor Pasko <[EMAIL PROTECTED]> wrote: > > > > On the 0x2EC day of Apache Harmony Alexei Zakharov wrote: > > > > > I also think that two months period is well balanced. As well as two > > > > > weeks for feature freeze. So I am +1. > > > > > > > > > > Egor Pasko wrote: > > > > > > > and define how we will commit between code freeze and release (each commit > > > > > > > approved by one more committer?) > > > > > > one should be enough. I think, the common process should be well > > > > > > applicable here: we will have comitter responsibility, discussions > > > > > > over dev@, etc. No reason for tight commit process, IMHO. > > > > > > > > > > I agree with Egor. The number of active committers is not so big > > > > > currently. So I think it is enough just to establish a good committing > > > > > policy and let the committer to decide for himself about each commit. > > > > > We may always find the author of bad commit and blame him publicly. > > > > > > > > > > > 2. why req tags for JIRA? Does this help committers to follow their > > > > > > areas of responsibility? > > > > > > > > > > IMO requirement tags will help to set right priorities to JIRAs. Say > > > > > you have a patch that greatly improves stability of some particular > > > > > scenario REQ1 but may affect performance of other scenarios in a bad > > > > > way. If the list of requirements is defined then you can set the > > > > > priority to critical and put [REQ1] into JIRA summary. Without this > > > > > tag it is not clear what this issue is critical for. > > > > > > > > that makes sense, sounds good, thanks > > > > > > > > > Alexey Petrenko wrote: > > > > > > About M2 marking... Can we create something like "Target milestone" > > > > > > field in our JIRA with predefined values? > > > > > > > > > > Do we have enough power to customize JIRA in this way? > > > > > > > > > > Regards, > > > > > > > > > > 05 Jun 2007 09:06:48 +0400, Egor Pasko <[EMAIL PROTECTED]>: > > > > > > On the 0x2EC day of Apache Harmony Mikhail Loenko wrote: > > > > > > > Let's have end of month (June, 30?) as a release date. Now we need to > > > > > > > define a date for code freeze (when only critical bugs are fixed) and > > > > > > > define how we will commit between code freeze and release (each commit > > > > > > > approved by one more committer?) > > > > > > > > > > > > one should be enough. I think, the common process should be well > > > > > > applicable here: we will have comitter responsibility, discussions > > > > > > over dev@, etc. No reason for tight commit process, IMHO. > > > > > > > > > > > > Tightening commit criteria requirements (that you are proposing) is good. > > > > > > > > > > > > > I think the code freeze date should depend on the longest test cycle > > > > > > > we have (I've seen somewhere about 48-hour scenarios?) and be ~2-3 > > > > > > > cycles (1 week?) prior the release. > > > > > > > > > > > > > > We also need a feature freeze date (1-2 weeks prior code freeze?) when > > > > > > > no major changes or redesigns are allowed. > > > > > > > > > > > > reasonable, thanks > > > > > > > > > > > > 2 weeks for feature freeze before M2 should be OK, IMHO > > > > > > > > > > > > > And we need to set up requirements for the release. We already see a > > > > > > > good wish-list here. The only concern I have is that its focus is > > > > > > > almost everything: stability, performance, and completeness. Though I > > > > > > > completely agree with each of these directions, I have a feeling that > > > > > > > having everything in focus means not having a focus. > > > > > > > > > > > > > > So I propose that we go this way: we have directions, we already > > > > > > > discussed them many times. Now let's create requirements based on the > > > > > > > list of directions: *each person who adds something to requirements is > > > > > > > committing to and will be responsible for meeting that requirement* > > > > > > > > > > > > > > The requirements could be to have something specific in stability, > > > > > > > have something specific in performance, completeness, java6, etc > > > > > > > > > > > > > > Once we compose a list, say 1..N of requirements, we create keys or > > > > > > > tags for JIRA, say M2-REQ1, ..., M2-REQN and mark bugs affecting > > > > > > > requirements with these key words. So each person would easily find > > > > > > > bugs affecting requirements he is responsible for. > > > > > > > > > > > > 1. why numbering? let it be descriptive requirement names. Example: > > > > > > M2-req-stable-linux-x86_64-regression-tests > > > > > > > > > > > > 2. why req tags for JIRA? Does this help committers to follow their > > > > > > areas of responsibility? If so, they could, please, please, speak > > > > > > up. I thought, all guys follow their bugs, have reasonable priorities > > > > > > regarding them, etc, etc. > > > > > > > > > > > > I do not mind such a small "commit process enhancement", but there is > > > > > > nothing but an extra burden in it for me) > > > > > > > > > > > > > Comments? Requirement proposals?
-- Alexei Zakharov, Intel ESSD
