lakshmi-manasa-g commented on pull request #1480: URL: https://github.com/apache/samza/pull/1480#issuecomment-816892407
**Problem:** this first patch does not work: throws NPE as appCtx.getAMContainerResourceRequest() returns null and is null till the end of the ClientHelper.submitApplication method. As in, AMContainerResourceRequest will always remain null within ApplicationSubmissionContext while in ClientHelper. **NPE Reason:** Peeking into yarn code shows that AMContainerResourceRequest is created, if null within [RMAppManager.validateAndCreateResourceRequest](https://github.com/apache/hadoop/blob/release-2.7.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java#L372) by using the mem,cpu set in the ApplicationSubmissionContext.setResource. **YARN code into samza?**: The only way to set node label only for AM during submit application is to use [setAMContainerResourceRequest.setNodeLabelExpression in ApplicationSubmissionContext](https://hadoop.apache.org/docs/r2.7.3/hadoop-yarn/hadoop-yarn-site/NodeLabel.html). However, creating AMContianerResourceRequest within samza's ClientHelper will require AM_CONTAINER_PRIORITY constant which is a part of yarn-server package NOT api, common or client which samza uses. Pulling yarn-server package into samza is seems plain WRONG! Additionally, it also means in some sense replicating what RMAppManager.validateAndCreateResourceRequest does to create the AMContainerRequest. **Workaround (tested and works)** - is in the samza-job, use yarn.am.container.label=desired-label AND yarn.container.label="" -- relying on this javadoc in yarn for[ SubmissionApplicationContext](https://github.com/apache/hadoop/blob/release-2.7.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationSubmissionContext.java#L437) and this comment from [YARN-2493](https://issues.apache.org/jira/browse/YARN-2493?focusedCommentId=14125163&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14125163). This workaround places AM on desired-label node while containers go to general nodes without a label. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
