Hi, Testng provides 4 modes to run your tests in parallel[1]. Among those 4 modes, we recommend parallel="classes" mode. <suite name="My suite" parallel="classes" thread-count="X">
*parallel="methods"*: TestNG will run all your test methods in separate threads. Dependent methods will also run in separate threads but they will respect the order that you specified. If you use this mode, there is a possibility to run your test's @BeforeClass methods in different threads than the @Test methods. According to Test Framework architecture, we use variables which are initialized inside @BeforeClass methods inside our @Test methods. So executing these two methods in separate threads may cause problems. So we do not recommend to use this mode. *parallel="classes"*: TestNG will run all the methods in the same class in the same thread, but each class will be run in a separate thread. In this mode all methods inside your test class (@BeforeClass , @Test, @AfterClass) will be executed in one thread. It guarantees the values which were assigned to variables inside @BeforeClass method are available to @Test methods as expected. We recommend this mode to run your test in parallel. [1] http://testng.org/doc/documentation-main.html#parallel-running <http://testng.org/doc/documentation-main.html#parallel-running> Regards, Malintha On Thu, Nov 5, 2015 at 2:41 PM, Akalanka Pagoda Arachchi <[email protected] > wrote: > Thanks Dharshana. > > I will modify the tests as you have suggested and see if it fixes the > issue. > > Thanks, > Akalanka. > > On Thu, Nov 5, 2015 at 2:37 PM, Dharshana Warusavitharana < > [email protected]> wrote: > >> Hi Akalanka, >> >> I see some issue how you initiate parallel test run in testng.xml >> >> <test name="tests-amqpparallel" preserve-order="false" parallel="true" >> thread-count="10"> >> >> parallel= true is mainly used in test method level to run test methods >> parallel once you add this to suite level(testNG.xml) it will trigger all >> test methods in test suites run parallel. I dont think our architecture can >> support this. >> >> So what you can do is run tests parallel in class level, you can refer >> tag below to support that. >> >> <*suite* name="tests-amqp-parallel" parallel="classes" thread-count="10" >> > >> And please update testNg version of your test suite above 6.8 (latest >> is 6.9.9) where lots of improvements were introduced in those later >> versions. >> >> And other thing you can try our is make test suites run parallel. >> >> <suite name="Test-class Suite" parallel="tests" thread-count="2"> >> >> for this you can add several test tags inside testng.xml and those test >> suites will run in parallel. >> >> Thank you, >> Dharshana. >> >> On Thu, Nov 5, 2015 at 2:21 PM, Akalanka Pagoda Arachchi < >> [email protected]> wrote: >> >>> Hi Malitha, >>> >>> Thanks for taking a look at this. >>> >>> All our test cases are extended from MBIntegrationBaseTest [1]. Within >>> each test's prepare method, we invoke the 'init' method of this class. The >>> exception is occuring when invoking the line >>> backendURL = automationContext.getContextUrls().getBackEndUrl(); >>> >>> We have added a synchronise block there as well trying to rectify this >>> issue. >>> >>> [1] - >>> https://github.com/wso2/product-mb/blob/master/modules/integration/tests-common/integration-tests-utils/src/main/java/org/wso2/mb/integration/common/utils/backend/MBIntegrationBaseTest.java >>> >>> Thanks, >>> Akalanka. >>> >>> On Thu, Nov 5, 2015 at 2:12 PM, Malintha Adikari <[email protected]> >>> wrote: >>> >>>> Thank you. I will have a look. >>>> >>>> On Thu, Nov 5, 2015 at 2:10 PM, Akalanka Pagoda Arachchi < >>>> [email protected]> wrote: >>>> >>>>> Hi Malintha, >>>>> >>>>> The testng.xml file is located in [1]. >>>>> >>>>> The test cases failing are different in each run, at the moment test >>>>> [2], and [3] have failed in jenkins. >>>>> >>>>> Please note that although this is frequent in jenkins, when doing a >>>>> local run, this issue is rarely occurring. Furthermore, this issue did not >>>>> occur when the tests are run sequentially. We made it to run in parallel >>>>> since it will save more time when building. >>>>> >>>>> [1] - >>>>> https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/resources/testng.xml >>>>> [2] - >>>>> https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/java/org/wso2/mb/integration/tests/amqp/functional/AutoAcknowledgementsTestCase.java >>>>> [3] - >>>>> https://github.com/wso2/product-mb/blob/master/modules/integration/tests-integration/tests-amqp/src/test/java/org/wso2/mb/integration/tests/amqp/functional/DurableTopicSubscriptionTestCase.java >>>>> >>>>> Thanks, >>>>> Akalanka. >>>>> >>>>> >>>>> On Thu, Nov 5, 2015 at 1:51 PM, Malintha Adikari <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Akalanka, >>>>>> >>>>>> Can you point me to the test case which you are trying to run in >>>>>> parallel. >>>>>> >>>>>> Regards, >>>>>> Malintha >>>>>> >>>>>> On Thu, Nov 5, 2015 at 11:47 AM, Akalanka Pagoda Arachchi < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi All, >>>>>>> >>>>>>> When we configure to run automation tests in parallel, following >>>>>>> error is thrown intermittently. >>>>>>> >>>>>>> I"ve enabled to run 10 threads in parallel with the following config >>>>>>> in testng.xml file. >>>>>>> >>>>>>> <test name="tests-amqp-parallel" preserve-order="false" >>>>>>> parallel="true" thread-count="10"> >>>>>>> >>>>>>> Stacktrace >>>>>>> >>>>>>> java.lang.NullPointerException >>>>>>> at org.apache.xerces.dom.ParentNode.nodeListGetLength(Unknown >>>>>>> Source) >>>>>>> at org.apache.xerces.dom.ParentNode.getLength(Unknown Source) >>>>>>> at >>>>>>> org.wso2.carbon.automation.engine.context.AutomationContext.extractUser(AutomationContext.java:307) >>>>>>> at >>>>>>> org.wso2.carbon.automation.engine.context.AutomationContext.getSuperTenant(AutomationContext.java:286) >>>>>>> at >>>>>>> org.wso2.carbon.automation.engine.context.AutomationContext.getContextTenant(AutomationContext.java:336) >>>>>>> at >>>>>>> org.wso2.carbon.automation.engine.context.AutomationContext.getContextUrls(AutomationContext.java:477) >>>>>>> at >>>>>>> org.wso2.mb.integration.common.utils.backend.MBIntegrationBaseTest.init(MBIntegrationBaseTest.java:59) >>>>>>> at >>>>>>> org.wso2.mb.integration.tests.amqp.functional.AutoAcknowledgementsTestCase.prepare(AutoAcknowledgementsTestCase.java:61) >>>>>>> >>>>>>> >>>>>>> There are kind of similar issue reported regarding DOM usage with >>>>>>> threads [1]. This is happening when initialising each test with >>>>>>> following two lines. >>>>>>> >>>>>>> >>>>>>> automationContext = new AutomationContext("MB", userMode); >>>>>>> >>>>>>> backendURL = >>>>>>> automationContext.getContextUrls().getBackEndUrl(); >>>>>>> >>>>>>> >>>>>>> Any idea on this? >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Akalanka >>>>>>> >>>>>>> >>>>>>> [1] - https://issues.apache.org/jira/browse/XERCESJ-727 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Darshana Akalanka Pagoda Arachchi,* >>>>>>> *Software Engineer* >>>>>>> *078-4721791 <078-4721791>* >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Malintha Adikari* >>>>>> Software Engineer >>>>>> WSO2 Inc.; http://wso2.com >>>>>> lean.enterprise.middleware >>>>>> >>>>>> Mobile: +94 71 2312958 >>>>>> Blog: http://malinthas.blogspot.com >>>>>> Page: http://about.me/malintha >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> *Darshana Akalanka Pagoda Arachchi,* >>>>> *Software Engineer* >>>>> *078-4721791 <078-4721791>* >>>>> >>>> >>>> >>>> >>>> -- >>>> *Malintha Adikari* >>>> Software Engineer >>>> WSO2 Inc.; http://wso2.com >>>> lean.enterprise.middleware >>>> >>>> Mobile: +94 71 2312958 >>>> Blog: http://malinthas.blogspot.com >>>> Page: http://about.me/malintha >>>> >>> >>> >>> >>> -- >>> *Darshana Akalanka Pagoda Arachchi,* >>> *Software Engineer* >>> *078-4721791 <078-4721791>* >>> >>> _______________________________________________ >>> Dev mailing list >>> [email protected] >>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >>> >> >> >> -- >> >> Dharshana Warusavitharana >> Senior Software Engineer , Test Automation >> WSO2 Inc. http://wso2.com >> email : [email protected] <[email protected]> >> Tel : +94 11 214 5345 >> Fax :+94 11 2145300 >> cell : +94770342233 >> blog : http://dharshanaw.blogspot.com >> >> lean . enterprise . middleware >> > > > > -- > *Darshana Akalanka Pagoda Arachchi,* > *Software Engineer* > *078-4721791* > -- *Malintha Adikari* Software Engineer WSO2 Inc.; http://wso2.com lean.enterprise.middleware Mobile: +94 71 2312958 Blog: http://malinthas.blogspot.com Page: http://about.me/malintha
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
