On 8/30/06, Richard Liang <[EMAIL PROTECTED]> wrote:
Vladimir Ivanov wrote:
> Also some tag for regression tests should be added.
Yes. Do you think we could annotate regression test as
*level.regression*? Thanks a lot.
Yes, I do. While tests can have more than one group it will enough.
thanks, Vladimir
Richard
> thanks, Vladimir
>
>
> On 8/28/06, Richard Liang <[EMAIL PROTECTED]> wrote:
>>
>>
>>
>> Richard Liang wrote:
>> > Hello All,
>> >
>> > Now let's talk about the TestNG groups. I have read the related
>> > threads which posted by George, Vladimir Ivanov and Alexei Zakharov.
>> > All of them are good discussion about TestNG groups.
>> >
>> > IMHO, we may define Harmony test groups according the following 4
>> > dimensions:
>> >
>> > 1) [Platform] os.any, os.<platform id>
>> > *os.any* - group of tests which pass on any platform. IMHO, most of
>> > our tests should be in this group.
>> > *os.<platform id>* - group of tests which are designed for one
>> > specific platform. A test may be in more than one of the groups. e.g
.,
>> > @Test(groups={"os.win.IA32", "os.linux.IA32"})
>> >
>> > ** os.any and os.<platform id> are mutually exclusive, that is,
>> > tests in os.any group should not be in os.win.IA32.
>> >
>> > 2) [Test state] state.broken, state.broken.<platform id>
>> > *state.broken* - group of tests which fail on every platform, because
>> > of bugs of tests or implementation. We need to fix the bugs of tests
>> > or implementation to make them pass.
>> > *state.broken.<platform id>* - groups of test which only fail on one
>> > specific platform. A test may be in more than one of the groups. e.g
.,
>> > @Test(groups={"state.broken.linux.IA32", "os.broken.linux.IA64"})
>> >
>> > **state.broken.<platform id> group may be used as a convenient
way
>> > to indicate that a test is platform-specific. e.g., If we support 10
>> > platforms, and one test are designed for 9 platforms except for
MacOS,
>> > instead of list 9 os.<platform id>, we can just use
state.broken.MacOS
>> >
>> > 3) [Test type] type.api , type.impl
>> > *type.api* - group of tests which are tests for APIs in the Java
>> > Specification
>> > *type.impl* - groups of tests which are tests for Harmony-specific
>> > implementation
>> >
>> > ** type.api and type.impl are also mutually exclusive.
>> >
>> > 4) [Test Level] level.unit, level.integration, level.system,
>> > level.stress, etc. (Levels of Test refer to the increase in
complexity
>> > as moving through test cycle. )
>> > ** A test may be in more than one of the groups.
>> > ** In fact, some tests such as System tests are the verification
of
>> > the entire system. Maybe we'll put them into a separate project.
>> > e.g., harmony/enhanced/SVT (System Verification Test).
>> >
>> > If we want to run all the unit test for APIs on windows, we may use
>> > TestNG groups to select the tests:
>> > <groups>
>> > <run>
>> > <include name="os.any" />
>> > <include name="type.api" />
>> > <include name="os.win.IA32" />
>> > <exclude name=" state.broken" />
>> > <exclude name="state.broken.win.IA32" />
>> > </run>
>> > </groups>
>> >
>> Hello All,
>>
>> I'm sorry. It seems that the example does not work. I will try to
figure
>> another example soon. ;-)
>>
>> Best regards,
>> Richard
>> >
>> > Well, I think our most of existing tests are in the groups of
>> > {"os.any", "type.api", "level.unit"}, and I have asked TestNG to add
a
>> > new option "-groups" for its JUnitConverter which allow us to specify
>> > the test groups when migrate from JUnit test to TestNG test.
>> >
>> > Thanks for reading so far, and I will highly appreciate your comments
>> > or suggestion. ;-)
>> >
>>
>> --
>> Richard Liang
>> China Software Development Lab, IBM
>>
>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>> For additional commands, e-mail: [EMAIL PROTECTED]
>>
>>
>
--
Richard Liang
China Software Development Lab, IBM
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
