kavvya97 opened a new pull request, #6252: URL: https://github.com/apache/hadoop/pull/6252
**Setup:** Java version: openjdk 11.0.20.1 Maven version: Apache Maven 3.6.3 ### **Issue**: https://issues.apache.org/jira/browse/MAPREDUCE-7461 ### Description of PR The following tests can fail due to flakiness while comparing the contents of the generated XML response. **Module**: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app `org.apache.hadoop.mapreduce.v2.app.webapp.TestAMWebServicesJobs#testJobIdXML` `org.apache.hadoop.mapreduce.v2.app.webapp.TestAMWebServicesJobs#testJobsXML` **Module**: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs `org.apache.hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesJobs#testJobIdXML` ### Steps to reproduce 1. `git clone https://github.com/apache/hadoop` 2. `mvn install -pl hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app -am -DskipTests` 3. Run the tests `mvn -pl hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app test -Dtests=org.apache.hadoop.mapreduce.v2.app.webapp.TestAMWebServicesJobs#testJobIdXML` 4. Run the test with the Nondex tool and observe the test results `mvn -pl hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.apache.hadoop.mapreduce.v2.app.webapp.TestAMWebServicesJobs#testJobsIdXML` - Test Fails when Running Nondex in ONE mode (Assumes deterministic implementation of code but shuffled once different from underlying implementation) `-DnondexMode=ONE` & FULL Mode ` -DnondexMode=FULL` (shuffles differently for each call) ### Root Cause The test attempts to send a HTTP GET request to a specific URL and expects a response in XML format. However, XML response order is not necessarily guaranteed. The contents of the XML and the tags are compared with Job contents from `appContext` based on [job Id](https://github.com/kavvya97/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobs.java#L493-L494) in [verifyAMJobXML](https://github.com/kavvya97/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobs.java#L486) / [verifyHsJobXML](https://github.com/kavvya97/hadoop/blob/9c621fcea72a988c930ef614a7c22de00d0c7d21/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobs.java#L232). When comparing t he the XML contents, The `<name>` tag occurs in multiple places inside <job> field. However, the root element within <job> is not always compared due to non-deterministic order. When the name tag is being compared, the test utilizes [WebServicesTestUtils.java getXmlString](https://github.com/kavvya97/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/webapp/WebServicesTestUtils.java#L78) method for retrieving the name from the XML content. However, It always takes the first <name> tag irrespective of whether it is nested or in root which causes the test to fail and become flaky. Since the XML contents are not ordered, The following errors occur ``` [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 8.377 s <<< FAILURE! - in org.apache.hadoop.mapreduce.v2.app.webapp.TestAMWebServicesJobs [ERROR] testJobIdXML(org.apache.hadoop.mapreduce.v2.app.webapp.TestAMWebServicesJobs) Time elapsed: 8.361 s <<< FAILURE! java.lang.AssertionError: [name] Expecting: "mapreduce.job.acl-view-job" to match pattern: "RandomWriter" ``` ### Fix Since [WebServicesTestUtils.java getXmlString](https://github.com/kavvya97/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/webapp/WebServicesTestUtils.java#L78) compares the first <name> tag only which might not necessary be the root <name> tag, The Fix uses Xpath to resolve the conflicts by identify the root <name> tag. Thus the test passes since the root tag is always retrieved irrespective of xml order. ### How was this patch tested? The fix was tested by adding a suitable fix and running the Nondex plugin again and ensuring that all the tests pass in FULL Mode and ONE Mode of the Nondex runs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
