[jira] [Commented] (YARN-226) Log aggregation should not assume an AppMaster will have containerId 1
[ https://issues.apache.org/jira/browse/YARN-226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13603527#comment-13603527 ] Eli Reisman commented on YARN-226: -- Hmm. It would not be impossible to ask for one container, launch it, then the rest of the graph tasks so that first one can be set up to know its first. I think the whole task 0 thing is old in Giraph and could be handled another way by now. I will look into it. It didn't make a lot of sense when that approach worked to start with, but it worked right away so I went with it ;) Log aggregation should not assume an AppMaster will have containerId 1 -- Key: YARN-226 URL: https://issues.apache.org/jira/browse/YARN-226 Project: Hadoop YARN Issue Type: Sub-task Reporter: Siddharth Seth In case of reservcations, etc - AppMasters may not get container id 1. We likely need additional info in the CLC / tokens indicating whether a container is an AM or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-226) Log aggregation should not assume an AppMaster will have containerId 1
[ https://issues.apache.org/jira/browse/YARN-226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13602614#comment-13602614 ] Eli Reisman commented on YARN-226: -- Hmm, that stirs up some trouble. Giraph tasks may or may not need to be contiguous ID's but will need a task 0 for at least one of the reserved containers (so container 2 right now is the one) in order to bootstrap our master election process. I am using container Id's to translate into giraph task Id's rigth now by just subtracting two from container Id! It works in all my tests, but the reservation thing could kick in on big asks (1000 container ask etc.) is that what you're saying? How big can the ask be? Perhaps I can move this bootstrap stuff from Giraph into my app master if this is a big problem. Good to know, thanks! Log aggregation should not assume an AppMaster will have containerId 1 -- Key: YARN-226 URL: https://issues.apache.org/jira/browse/YARN-226 Project: Hadoop YARN Issue Type: Sub-task Reporter: Siddharth Seth In case of reservcations, etc - AppMasters may not get container id 1. We likely need additional info in the CLC / tokens indicating whether a container is an AM or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-226) Log aggregation should not assume an AppMaster will have containerId 1
[ https://issues.apache.org/jira/browse/YARN-226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13602661#comment-13602661 ] Robert Joseph Evans commented on YARN-226: -- Big means amount of memory/CPU relative to the minimum allocation size. For example you ask for a 4 GB container with a min allocation size of 500MB. Log aggregation should not assume an AppMaster will have containerId 1 -- Key: YARN-226 URL: https://issues.apache.org/jira/browse/YARN-226 Project: Hadoop YARN Issue Type: Sub-task Reporter: Siddharth Seth In case of reservcations, etc - AppMasters may not get container id 1. We likely need additional info in the CLC / tokens indicating whether a container is an AM or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-226) Log aggregation should not assume an AppMaster will have containerId 1
[ https://issues.apache.org/jira/browse/YARN-226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13602667#comment-13602667 ] Hitesh Shah commented on YARN-226: -- @Sid, nevermind - reservations could effectively increment the id and never assign 1 to anything. @Eli, this will occur on clusters when the AM resource ask is greater than a single slot and it requires multiple scheduling cycles before a large free slot is available to launch the AM. Log aggregation should not assume an AppMaster will have containerId 1 -- Key: YARN-226 URL: https://issues.apache.org/jira/browse/YARN-226 Project: Hadoop YARN Issue Type: Sub-task Reporter: Siddharth Seth In case of reservcations, etc - AppMasters may not get container id 1. We likely need additional info in the CLC / tokens indicating whether a container is an AM or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-226) Log aggregation should not assume an AppMaster will have containerId 1
[ https://issues.apache.org/jira/browse/YARN-226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13601952#comment-13601952 ] Siddharth Seth commented on YARN-226: - bq. The Giraph to YARN port assumes giraph tasks start at 2 and up as far as container #'s go. Is this unsafe for the future? One scenario in which the AM does not get container id 1 is when it requires more resources than the minimum allocation - in which case reservations come in to play. Depending on whether the reservation is the final allocation or whether it happens elsewhere - the container id may not be one. Similarly, assuming the container IDs are contiguous is not valid. IDs can be skipped. Log aggregation should not assume an AppMaster will have containerId 1 -- Key: YARN-226 URL: https://issues.apache.org/jira/browse/YARN-226 Project: Hadoop YARN Issue Type: Sub-task Reporter: Siddharth Seth In case of reservcations, etc - AppMasters may not get container id 1. We likely need additional info in the CLC / tokens indicating whether a container is an AM or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
