[
https://issues.apache.org/jira/browse/MAPREDUCE-522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12725975#action_12725975
]
Sreekanth Ramakrishnan commented on MAPREDUCE-522:
--------------------------------------------------
Following are the tests which are executed by
{{org.apache.hadoop.mapred.TestQueueCapacities}}
* Test job submission with a single queue configured. Single queue has 100%
cluster-capacity. Submit a job which requires more than 100% of capacity.
Ensures that the job completes successfully.
* Test multiple job submissions with single queue configured. Single queue has
100% cluster-capacity. Submit a job j1 which requires the 100% of queue
capacity. Wait till all slots are occupied by the j1 and then submit j2.
Observe all tasks from J1 is completed before attending to tasks from J2.
* Test multiple job submissions with single queue configured. Single queue has
100% cluster-capacity. Submit a job j1 which requires less than queue capacity.
Wait till all tasks start running of J1 and then submit job J2. Observe that J2
tasks get scheduled. Let both jobs run to completion.
* Test single job with multiple queues configured. Submit a single job which
requires 100% oc cluster-capacity. Observe that the job fully occupies the
cluster, i.e. queue capacity expands to statisfy the job.
* Test multiple jobs with multiple queues configured. Submit jobs equal to
number of queues configured, which requires exactly the capacities of queues
configured. Observe all the jobs simultaneously starts running at same time.
Can we just combine the five tests above to a single test below to improve run
time and make it simpler?
* Configure Capacity-Scheduler with two queues each getting 50% of
cluster-capacity.
* Submit a job j1 to q1 to occupy 100% of cluster capacity.
* Wait till job j1 occupies 100% of cluster capacity, submit J2 to q2.
* Observe that when J1's task starts finishing we start scheduling tasks from
J2 to expand capacity for q2.
* Submit a job j3 in q1 and observe that when q1's capacity has been contracted
to 50% of capacity and j1's tasks gettig finished j3 is scheduled.
* Observe all the tasks from jobs complete fully.
> Rewrite TestQueueCapacities to make it simpler and avoid timeout errors
> -----------------------------------------------------------------------
>
> Key: MAPREDUCE-522
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-522
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Hemanth Yamijala
>
> We have seen TestQueueCapacities fail periodically and there have been a
> couple of times fixes partially fixed the problem, the most recent instance
> being HADOOP-5869. I found another instance of failure, while running tests
> locally while testing a different patch. This was a different symptom from
> the ones we've seen before. The core problem is that the test is too complex
> and relies on too many things working correctly to be useful. It would make
> sense to revisit the purpose of the test and see if a simpler model can serve
> it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
