[
https://issues.apache.org/jira/browse/YARN-11781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17931730#comment-17931730
]
ASF GitHub Bot commented on YARN-11781:
---------------------------------------
TaoYang526 opened a new pull request, #7448:
URL: https://github.com/apache/hadoop/pull/7448
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines:
https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g.,
'HADOOP-17799. Your PR title ...'.
-->
### Description of PR
Details:
* Dynamic Request Handling: The RequestsHandler class has been added to
handle and modify resource and scheduling requests at the beginning of the
CapacityScheduler#allocate method.
* Configuration-Driven Updates: The behavior of the request handler can be
configured dynamically via capacity-scheduler.xml configuration, allowing for
updates to be applied without restarting the ResourceManager.
* Scriptable Matching: Requests can be matched and modified based on
JavaScript expressions, providing flexibility in defining which requests should
be updated and how.
* Support for ResourceRequest to SchedulingRequest Conversion: The handler
supports converting ResourceRequest objects to SchedulingRequest objects,
enabling more advanced scheduling capabilities.
Usage: To enable and configure the request handler, administrators can set
the following properties in capacity-scheduler.xml:
yarn.scheduler.capacity.request-handler.enabled: Enables or disables the
request handler.
yarn.scheduler.capacity.request-handler.updates: Specifies the JSON
configuration for the request updates.
Configuration example:
```xml
<property>
<name>yarn.scheduler.capacity.request-handler.enabled</name>
<value>true</value>
</property>
<property>
<name>yarn.scheduler.capacity.request-handler.updates</name>
<value>
{
"items": [
{
"appMatchExpr": "queue == 'default'",
"requestMatchExpr": "priority > 10",
"isRRToSR": true,
"partition": "partitionA",
"executionType": "GUARANTEED",
"allocationTags": ["tag1", "tag2"],
"placementConstraint": "and(in,rack,hbase:notin,node,zk)"
}
]
}
</value>
</property>
```
### How was this patch tested?
### For code changes:
- [x] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
> Implement Dynamic Requests Handling in CapacityScheduler
> --------------------------------------------------------
>
> Key: YARN-11781
> URL: https://issues.apache.org/jira/browse/YARN-11781
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: capacityscheduler
> Reporter: Tao Yang
> Assignee: Tao Yang
> Priority: Major
>
> This feature introduces a dynamic request handling mechanism in the
> CapacityScheduler, allowing for the modification of resource and scheduling
> requests based on configurable rules. This enables more flexible and adaptive
> resource management without requiring a restart of the ResourceManager.
> Benefits:
> * Increased Flexibility: Administrators can define custom rules for
> modifying requests, allowing for more tailored resource allocation strategies.
> * Enhanced Scheduling: The ability to convert ResourceRequest to
> SchedulingRequest opens up new possibilities for advanced scheduling and
> placement constraints.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
