[
https://issues.apache.org/jira/browse/YARN-5106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16667138#comment-16667138
]
Szilard Nemeth edited comment on YARN-5106 at 10/29/18 12:41 PM:
-----------------------------------------------------------------
About the test failures:
1. TestAMRMClient.testAMRMClientWithContainerResourceChange: seems to be
unrelated (or flaky?)
2. TestContainerAllocation: The same set of testcases are failed on trunk and
with my patch (local execution), so this is also unrelated.
3. TestAllocationFileLoaderService.testQueueNameContainingNBWhitespace: Passes
for me after 15 runs of the whole class.
Added a @Before method that deletes the allocation file, which is a good
practice on its own, regardless of the failure we experienced with the Jenkins
results.
4. TestFairScheduler.testHierarchicalQueueAllocationFileParsing: Fixed the
test, it was a trivial one as my patch did not add queueC and queueD as a
children of queueA, so it was due to a difference in the structure of the
allocation file.
5. TestFairScheduler.testFairShareAndWeightsInNestedUserQueueRule: This also
generated changes in the allocation.xml file compared to the original code, so
I fixed it.
However, the config my patch was generated with the testcases' code was
invalid in a way that it procuded an NPE.
I created YARN-8951 for this to investigate further.
Notes for patch001:
I just did a simple find for all tests that generates allocation files (i.e.
searched for "<allocations>") and replaced all of queue file creations with
{{PrintWriter}}s with the queue config file builder introduced in YARN-7841.
There's one exception, though: TestFairSchedulerPreemption#writeAllocFile,
this is because work is in progress on YARN-8059, so I wanted to avoid huge
merge conflicts.
If YARN-8059 goes in first, I will rebase the patch and apply the changes to
preemption test, otherwise I will create a separate jira to apply the changes
to {{TestFairSchedulerPreemption}} as well.
was (Author: snemeth):
About the test failures:
1. TestAMRMClient.testAMRMClientWithContainerResourceChange: seems to be
unrelated (or flaky?)
2. TestContainerAllocation: The same set of testcases are failed on trunk and
with my patch (local execution)
3. TestAllocationFileLoaderService.testQueueNameContainingNBWhitespace: Passes
for me after 15 runs of the whole class.
Added a @Before method that deletes the allocation file, which is a good
practice on its own, regardless of the failure we experienced from the Jenkins
run.
4. TestFairScheduler.testHierarchicalQueueAllocationFileParsing: Fixed the
test, it was a trivial one as my patch did not add queueC and queueD as a
children of queueA, so it was due to a difference in the structure of the
allocation file.
5. TestFairScheduler.testFairShareAndWeightsInNestedUserQueueRule: This also
generated changes in the allocation.xml file compared to the original code, so
I fixed it.
However, the config my patch was generated with the testcases' code was invalid
in a way that it procuded an NPE.
I created YARN-8951 for this to investigate further.
Notes for patch001:
I just did a simple find for all tests that generates allocation files (i.e.
searched for "<allocations>") and replaced all of queue file creations with
printwriters with the queue config file builder introduced in YARN-7841.
There's one exception, though: TestFairSchedulerPreemption#writeAllocFile, this
is because work is in progress on YARN-8059, so I wanted to avoid huge merge
conflicts.
If YARN-8059 goes in first, I will rebase the patch and apply the changes to
preemption test, otherwise I will create a separate jira to apply the changes
to \{{TestFairSchedulerPreemption}} as well.
> Provide a builder interface for FairScheduler allocations for use in tests
> --------------------------------------------------------------------------
>
> Key: YARN-5106
> URL: https://issues.apache.org/jira/browse/YARN-5106
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler
> Affects Versions: 2.8.0
> Reporter: Karthik Kambatla
> Assignee: Szilard Nemeth
> Priority: Major
> Labels: newbie++
> Attachments: YARN-5106.001.patch
>
>
> Most, if not all, fair scheduler tests create an allocations XML file. Having
> a helper class that potentially uses a builder would make the tests cleaner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
