-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47815/
-----------------------------------------------------------
Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.
Bugs: AMBARI-16888
https://issues.apache.org/jira/browse/AMBARI-16888
Repository: ambari
Description
-------
- The 1st call to SA gets capacity-scheduler configs as dictionary compared to
"/n" separated single string of all the configs in subsequent calls.
* 1st invocation, passed-in capacity-scheduler looks like :
------
"capacity-scheduler" : {
"properties" : {
"capacity-scheduler" : "null",
"yarn.scheduler.capacity.root.accessible-node-labels" : "*",
"yarn.scheduler.capacity.maximum-am-resource-percent" : "1",
"yarn.scheduler.capacity.root.acl_administer_queue" : "*",
"yarn.scheduler.capacity.queue-mappings-override.enable" : 'false',
"yarn.scheduler.capacity.root.default.capacity" : "100",
"yarn.scheduler.capacity.root.default.user-limit-factor" : "1",
"yarn.scheduler.capacity.root.queues" : "default",
"yarn.scheduler.capacity.root.capacity" : "100",
"yarn.scheduler.capacity.root.default.acl_submit_applications" :
"*",
"yarn.scheduler.capacity.root.default.maximum-capacity" : "100",
"yarn.scheduler.capacity.node-locality-delay" : "40",
"yarn.scheduler.capacity.maximum-applications" : "10000",
"yarn.scheduler.capacity.root.default.state" : "RUNNING"
}
},
------
* subsequent invocations gets capacity-schdeuler as:
------
"capacity-scheduler": {
"properties": {
"capacity-scheduler":
"yarn.scheduler.capacity.root.queues=default\n"
"yarn.scheduler.capacity.root.default.user-limit-factor=1\n"
"yarn.scheduler.capacity.root.default.state=RUNNING\n"
"yarn.scheduler.capacity.root.default.maximum-capacity=100\n"
"yarn.scheduler.capacity.root.default.capacity=100\n"
"yarn.scheduler.capacity.root.default.acl_submit_applications=*\n"
"yarn.scheduler.capacity.root.capacity=100\n"
"yarn.scheduler.capacity.root.acl_administer_queue=*\n"
"yarn.scheduler.capacity.root.accessible-node-labels=*\n"
"yarn.scheduler.capacity.node-locality-delay=40\n"
"yarn.scheduler.capacity.maximum-applications=10000\n"
"yarn.scheduler.capacity.maximum-am-resource-percent=1\n"
"yarn.scheduler.capacity.queue-mappings-override.enable=false\n"
}
},
------
- Therefore, SA fails to create the 'llap' queue for Hive Server Interactive,
as SA knows to handle only the 2nd case. Further, llap configuration related
caluclations also bail out.
- This specific issue was seen with while deploying cluster with Blueprints.
RESOLUTION:
==========
Added support for dealing with the capacity-schdeuler scenario when it's
configs have been passed-in as dictionary.
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py
af812fe
ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 51c0d0c
Diff: https://reviews.apache.org/r/47815/diff/
Testing
-------
- Tested Blueprint deploy with strategy
"ALWAYS_APPLY_DONT_OVERRIDE_CUSTOM_VALUES " : Works.
- Tested UI deploy : Works.
-
- Added Python UT's to test the behavior when 'capacity-scheduler' is passed in
as a dictionary.
- Python UT passes.
Thanks,
Swapan Shridhar
