[ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417590#comment-16417590 ]
Sunil G commented on YARN-7494: ------------------------------- Thanks [~cheersyang] and [~leftnoteasy] bq.Maintaining a separate cache will require to update them which not seem to be necessary to me. This will help us in future as well. We depend on labelManager a lot and its always to get node per partition. Such a cache, will help for metrics and for other apis which we can do in a clean up jira. bq.You'll get a lot of UT failures in FIFO path. A way to fix this is to do this in {{CapacityScheduler#addApplicationAttempt}}, such as Makes sense. I ll update in next patch. bq.Suggest to load it from configuration file, such as This will force such a config to be only at CS level. I dont see any pblm but this will add to a complex per-policy config. I thought of making this even more simpler. we are now configuring policy like {{yarn.scheduler.capacity.multi-node-sorting.policy}} to a value like "resource-based" or "org....customPolicy.class". Now such a complex name which is fully qualified will make your proposal more complex. Hence we can opt for [name=resource-based, timeout=1200] or something similar in nature. Thoughts? bq.CS#getCandidateNodeSet, the get_node_nodes_from_give_partition operation is very expensive Yes. I though i used that in CS api as well. Infact i used only in MultiNodeManager. I ll change that in next patch. Thank you. I will wait for config comment from [~cheersyang] before uploading next patch. > Add muti node lookup support for better placement > ------------------------------------------------- > > Key: YARN-7494 > URL: https://issues.apache.org/jira/browse/YARN-7494 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler > Reporter: Sunil G > Assignee: Sunil G > Priority: Major > Attachments: YARN-7494.001.patch, YARN-7494.002.patch, > YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch, > YARN-7494.v0.patch, YARN-7494.v1.patch, multi-node-designProposal.png > > > Instead of single node, for effectiveness we can consider a multi node lookup > based on partition to start with. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org