I think we can treat something like *os.any*,*type.api* as default
groups.Itwill make our logic more clear and bring convenience to
testcase writing :)
for example, an os.<platformid> group indicates that it is designed for
<paltformid> only. If there is no group like os.<platformid> here, this is a
*os.any* test.
so if we want to run all *api* tests on *win32* platform which is *not
broken*, we could write the testng.xml like this:
<groups>
<run>
<include name=".*" />
<exclude name="type.impl" />
<exclude name="os\.(?!win\.IA32).*" />
<exclude name="state.broken" />
<exclude name="state.broken.win.IA32" />
</run>
</groups>
2006/8/29, Vladimir Ivanov <[EMAIL PROTECTED]>:
Also some tag for regression tests should be added.
thanks, Vladimir
On 8/28/06, Richard Liang <[EMAIL PROTECTED]> wrote:
>
>
>
> Richard Liang wrote:
> > Hello All,
> >
> > Now let's talk about the TestNG groups. I have read the related
> > threads which posted by George, Vladimir Ivanov and Alexei Zakharov.
> > All of them are good discussion about TestNG groups.
> >
> > IMHO, we may define Harmony test groups according the following 4
> > dimensions:
> >
> > 1) [Platform] os.any, os.<platform id>
> > *os.any* - group of tests which pass on any platform. IMHO, most of
> > our tests should be in this group.
> > *os.<platform id>* - group of tests which are designed for one
> > specific platform. A test may be in more than one of the groups. e.g.,
> > @Test(groups={"os.win.IA32", "os.linux.IA32"})
> >
> > ** os.any and os.<platform id> are mutually exclusive, that is,
> > tests in os.any group should not be in os.win.IA32.
> >
> > 2) [Test state] state.broken, state.broken.<platform id>
> > *state.broken* - group of tests which fail on every platform, because
> > of bugs of tests or implementation. We need to fix the bugs of tests
> > or implementation to make them pass.
> > *state.broken.<platform id>* - groups of test which only fail on one
> > specific platform. A test may be in more than one of the groups. e.g.,
> > @Test(groups={"state.broken.linux.IA32", "os.broken.linux.IA64"})
> >
> > **state.broken.<platform id> group may be used as a convenient way
> > to indicate that a test is platform-specific. e.g., If we support 10
> > platforms, and one test are designed for 9 platforms except for MacOS,
> > instead of list 9 os.<platform id>, we can just use state.broken.MacOS
> >
> > 3) [Test type] type.api , type.impl
> > *type.api* - group of tests which are tests for APIs in the Java
> > Specification
> > *type.impl* - groups of tests which are tests for Harmony-specific
> > implementation
> >
> > ** type.api and type.impl are also mutually exclusive.
> >
> > 4) [Test Level] level.unit, level.integration, level.system,
> > level.stress, etc. (Levels of Test refer to the increase in complexity
> > as moving through test cycle. )
> > ** A test may be in more than one of the groups.
> > ** In fact, some tests such as System tests are the verification of
> > the entire system. Maybe we'll put them into a separate project.
> > e.g., harmony/enhanced/SVT (System Verification Test).
> >
> > If we want to run all the unit test for APIs on windows, we may use
> > TestNG groups to select the tests:
> > <groups>
> > <run>
> > <include name="os.any" />
> > <include name="type.api" />
> > <include name="os.win.IA32" />
> > <exclude name=" state.broken" />
> > <exclude name="state.broken.win.IA32" />
> > </run>
> > </groups>
> >
> Hello All,
>
> I'm sorry. It seems that the example does not work. I will try to figure
> another example soon. ;-)
>
> Best regards,
> Richard
> >
> > Well, I think our most of existing tests are in the groups of
> > {"os.any", "type.api", "level.unit"}, and I have asked TestNG to add a
> > new option "-groups" for its JUnitConverter which allow us to specify
> > the test groups when migrate from JUnit test to TestNG test.
> >
> > Thanks for reading so far, and I will highly appreciate your comments
> > or suggestion. ;-)
> >
>
> --
> Richard Liang
> China Software Development Lab, IBM
>
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
--
Tony Wu
China Software Development Lab, IBM
