Richard.
>
> Thanks,
> Mikhail
>
> 2006/8/30, Vladimir Ivanov <[EMAIL PROTECTED]>:
>> On 8/30/06, Richard Liang <[EMAIL PROTECTED]> wrote:
>> >
>> >
>> >
>> > Vladimir Ivanov wrote:
>> > > Also some tag for regression tests should be added.
>> > Yes. Do you think we could annotate regression test as
>> > *level.regression*? Thanks a lot.
>>
>>
>> Yes, I do. While tests can have more than one group it will enough.
>> thanks, Vladimir
>>
>>
>> Richard
>> > > thanks, Vladimir
>> > >
>> > >
>> > > On 8/28/06, Richard Liang <[EMAIL PROTECTED]> wrote:
>> > >>
>> > >>
>> > >>
>> > >> Richard Liang wrote:
>> > >> > Hello All,
>> > >> >
>> > >> > Now let's talk about the TestNG groups. I have read the
related
>> > >> > threads which posted by George, Vladimir Ivanov and Alexei
>> Zakharov.
>> > >> > All of them are good discussion about TestNG groups.
>> > >> >
>> > >> > IMHO, we may define Harmony test groups according the
following
4
>> > >> > dimensions:
>> > >> >
>> > >> > 1) [Platform] os.any, os.<platform id>
>> > >> > *os.any* - group of tests which pass on any platform. IMHO,
>> most of
>> > >> > our tests should be in this group.
>> > >> > *os.<platform id>* - group of tests which are designed for one
>> > >> > specific platform. A test may be in more than one of the
>> groups. e.g
>> > .,
>> > >> > @Test(groups={"os.win.IA32", "os.linux.IA32"})
>> > >> >
>> > >> > ** os.any and os.<platform id> are mutually exclusive, that
>> is,
>> > >> > tests in os.any group should not be in os.win.IA32.
>> > >> >
>> > >> > 2) [Test state] state.broken, state.broken.<platform id>
>> > >> > *state.broken* - group of tests which fail on every platform,
>> because
>> > >> > of bugs of tests or implementation. We need to fix the bugs of
>> tests
>> > >> > or implementation to make them pass.
>> > >> > *state.broken.<platform id>* - groups of test which only fail
>> on one
>> > >> > specific platform. A test may be in more than one of the
>> groups. e.g
>> > .,
>> > >> > @Test(groups={"state.broken.linux.IA32", "os.broken.linux.IA64
"})
>> > >> >
>> > >> > **state.broken.<platform id> group may be used as a
>> convenient
>> > way
>> > >> > to indicate that a test is platform-specific. e.g., If we
>> support 10
>> > >> > platforms, and one test are designed for 9 platforms except
for
>> > MacOS,
>> > >> > instead of list 9 os.<platform id>, we can just use
>> > state.broken.MacOS
>> > >> >
>> > >> > 3) [Test type] type.api , type.impl
>> > >> > *type.api* - group of tests which are tests for APIs in the
Java
>> > >> > Specification
>> > >> > *type.impl* - groups of tests which are tests for
>> Harmony-specific
>> > >> > implementation
>> > >> >
>> > >> > ** type.api and type.impl are also mutually exclusive.
>> > >> >
>> > >> > 4) [Test Level] level.unit, level.integration, level.system,
>> > >> > level.stress, etc. (Levels of Test refer to the increase in
>> > complexity
>> > >> > as moving through test cycle. )
>> > >> > ** A test may be in more than one of the groups.
>> > >> > ** In fact, some tests such as System tests are the
>> verification
>> > of
>> > >> > the entire system. Maybe we'll put them into a separate
project.
>> > >> > e.g., harmony/enhanced/SVT (System Verification Test).
>> > >> >
>> > >> > If we want to run all the unit test for APIs on windows, we
>> may use
>> > >> > TestNG groups to select the tests:
>> > >> > <groups>
>> > >> > <run>
>> > >> > <include name="os.any" />
>> > >> > <include name="type.api" />
>> > >> > <include name="os.win.IA32" />
>> > >> > <exclude name=" state.broken" />
>> > >> > <exclude name="state.broken.win.IA32" />
>> > >> > </run>
>> > >> > </groups>
>> > >> >
>> > >> Hello All,
>> > >>
>> > >> I'm sorry. It seems that the example does not work. I will
try to
>> > figure
>> > >> another example soon. ;-)
>> > >>
>> > >> Best regards,
>> > >> Richard
>> > >> >
>> > >> > Well, I think our most of existing tests are in the groups of
>> > >> > {"os.any", "type.api", "level.unit"}, and I have asked TestNG
>> to add
>> > a
>> > >> > new option "-groups" for its JUnitConverter which allow us to
>> specify
>> > >> > the test groups when migrate from JUnit test to TestNG test.
>> > >> >
>> > >> > Thanks for reading so far, and I will highly appreciate your
>> comments
>> > >> > or suggestion. ;-)
>> > >> >
>> > >>
>> > >> --
>> > >> Richard Liang
>> > >> China Software Development Lab, IBM
>> > >>
>> > >>
>> > >>
>> > >>
>> ---------------------------------------------------------------------
>> > >> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> > >> To unsubscribe, e-mail:
>> [EMAIL PROTECTED]
>> > >> For additional commands, e-mail:
>> [EMAIL PROTECTED]
>> > >>
>> > >>
>> > >
>> >
>> > --
>> > Richard Liang
>> > China Software Development Lab, IBM
>> >
>> >
>> >
>> >
---------------------------------------------------------------------
>> > Terms of use : http://incubator.apache.org/harmony/mailing.html
>> > To unsubscribe, e-mail:
[EMAIL PROTECTED]
>> > For additional commands, e-mail:
[EMAIL PROTECTED]
>> >
>> >
>>
>>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
--
Richard Liang
China Software Development Lab, IBM
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
