[
https://issues.apache.org/jira/browse/MAPREDUCE-1871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12885596#action_12885596
]
Iyappan Srinivasan commented on MAPREDUCE-1871:
-----------------------------------------------
testTaskTrackerInfoAll I don't think it is a good idea to wait for arbitrary
delay, replace this with polling logic, add functionality if required using
aspects.
+ //Waiting for 20 seconds to make sure that all the completed tasks
+ //are reflected in their corresponding Tasktracker boxes.
+ Thread.sleep(20000);
The same comment holds good in testTaskTrackerInfoKilled and other places where
arbitrary delay is used.
- Replaced with Tasktracker heartbeat variable used as delay.Changed in all
places.
countLoop++ is a vestige variable has to be removed.
- removed.
FailedMapperClass still exists as part of inner class, move it to testjar, or
reuse FailedMapper already available in testjar.
- Removed. reusing FailedMapper.
+ public static TTClient getTTClientIns(MRCluster cluster, TaskInfo taskInfo)
+ throws IOException {
Apologies that my previous comment was not clear and you did move the method to
TTClient like I mentioned but my intention was different. I do not like static
method in TTClient, I would rather have a non static method in MRCluster, the
general guideline for building block, add the helper method to a class from
which it uses most of the member variables, if you have helper methods as
static method in test cases highly unlikely anyone will reuse it, please
refrain from adding static method, getTTClientIns should be moved to MRCluster
as non-static method, having more static methods gives C flavor of coding, with
less emphasis on object oriented means.
- Moved to MRCluster.
+ private int getInfoFromAllClients(String timePeriod, String taskType)
+ throws Exception {
+ List<TTClient> ttClients = cluster.getTTClients();
+ LOG.info("ttClients.size() :" + ttClients.size());
+
+ int totalTasksCount = 0;
+ int totalTasksRanForJob = 0;
+ for ( int i = 0; i< ttClients.size(); i++) { + TTClient ttClient =
(TTClient)ttClients.get(i); + TaskTrackerStatus ttStatus =
ttClient.getStatus(); + int totalTasks =
remoteJTClient.getTaskTrackerLevelStatistics( + ttStatus, timePeriod,
taskType); + totalTasksCount += totalTasks; + }
+ return totalTasksCount;
+ }
The above code can be refactored to use the new method which gets all the
information in single shot, no looping through task trackers required in client
side, will reduce the number of rpc calls.
- Refactored and made a part of TTClient, with testcase just calling it..
> Create automated test scenario for "Collect information about number of tasks
> succeeded / total per time unit for a tasktracker"
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1871
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1871
> Project: Hadoop Map/Reduce
> Issue Type: Test
> Components: test
> Reporter: Iyappan Srinivasan
> Assignee: Iyappan Srinivasan
> Attachments: 1871-ydist-security-patch.txt,
> 1871-ydist-security-patch.txt, 1871-ydist-security-patch.txt,
> 1871-ydist-security-patch.txt, MAPREDUCE-1871.patch, MAPREDUCE-1871.patch
>
>
> Create automated test scenario for "Collect information about number of tasks
> succeeded / total per time unit for a tasktracker"
> 1) Verification of all the above mentioned fields with the specified TTs.
> Total no. of tasks and successful tasks should be equal to the corresponding
> no. of tasks specified in TTs logs
> 2) Fail a task on tasktracker. Node UI should update the status of tasks on
> that TT accordingly.
> 3) Kill a task on tasktracker. Node UI should update the status of tasks on
> that TT accordingly
> 4) Positive Run simultaneous jobs and check if all the fields are populated
> with proper values of tasks. Node UI should have correct valiues for all the
> fields mentioned above.
> 5) Check the fields across one hour window Fields related to hour should be
> updated after every hour
> 6) Check the fields across one day window fields related to hour should be
> updated after every day
> 7) Restart a TT and bring it back. UI should retain the fields values.
> 8) Positive Run a bunch of jobs with 0 maps and 0 reduces simultanously.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
