[
https://issues.apache.org/jira/browse/YARN-6124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16254641#comment-16254641
]
Zian Chen commented on YARN-6124:
---------------------------------
Hi [~eepayne]
Thank you for your previous comments. Actually, I got the same observation with
you. And my previous comments about the bug fix is very similar to what you
suggested.
Another issue you mentioned in your previous comment is that the preemption
cannot be disabled when setting yarn.resourcemanager.scheduler.monitor.enable
from true to false. I have the same issue too. The reason it was not working is
that when we call refreshQueues, it's actually loaded capacity-scheduler.xml
rather than yarn-site.xml, so if we set
yarn.resourcemanager.scheduler.monitor.enable inside yarn-site.xml, this won't
work when we doing the refershQueues, we could set this param inside
capacity-scheduler.xml then do the refershQueues and it will work. The code
below proves what I think,
{code:java}
public void refreshQueues() throws IOException,
YarnException {
rm.getRMContext().getScheduler().reinitialize(getConfig(),
this.rm.getRMContext());
// refresh the reservation system
ReservationSystem rSystem = rm.getRMContext().getReservationSystem();
if (rSystem != null) {
rSystem.reinitialize(getConfig(), rm.getRMContext());
}
}
{code}
the reinitialize method inside CapacityScheduler loads
CapacitySchedulerConfiguration as the conf.
{code:java}
writeLock.lock();
Configuration configuration = new Configuration(newConf);
CapacitySchedulerConfiguration oldConf = this.conf;
this.conf = csConfProvider.loadConfiguration(configuration);
validateConf(this.conf);
try {
LOG.info("Re-initializing queues...");
refreshMaximumAllocation(this.conf.getMaximumAllocation());
reinitializeQueues(this.conf);
} catch(Throwable t) {
this.conf = oldConf;
refreshMaximumAllocation(this.conf.getMaximumAllocation());
throw new IOException("Failed to re-init queues : " + t.getMessage(),
t);
}
{code}
[~leftnoteasy] Do you think this makes sense to you? Thanks!
> Make SchedulingEditPolicy can be enabled / disabled / updated with RMAdmin
> -refreshQueues
> -----------------------------------------------------------------------------------------
>
> Key: YARN-6124
> URL: https://issues.apache.org/jira/browse/YARN-6124
> Project: Hadoop YARN
> Issue Type: Task
> Reporter: Wangda Tan
> Assignee: Zian Chen
> Attachments: YARN-6124.wip.1.patch, YARN-6124.wip.2.patch
>
>
> Now enabled / disable / update SchedulingEditPolicy config requires restart
> RM. This is inconvenient when admin wants to make changes to
> SchedulingEditPolicies.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
