[jira] [Commented] (YARN-5724) [Umbrella] Better Queue Management in YARN
[ https://issues.apache.org/jira/browse/YARN-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16873445#comment-16873445 ] Erkin Alp Güney commented on YARN-5724: --- [~Naganarasimha] FS and other schedulers can also benefit. Currently, each scheduler implements its own queue management and it's a mess. > [Umbrella] Better Queue Management in YARN > -- > > Key: YARN-5724 > URL: https://issues.apache.org/jira/browse/YARN-5724 > Project: Hadoop YARN > Issue Type: Task > Components: capacity scheduler >Reporter: Xuan Gong >Assignee: Xuan Gong >Priority: Major > Attachments: > Designdocv1-Configuration-basedQueueManagementinYARN.pdf, > Designdocv2-Configuration-basedQueueManagementinYARN.pdf > > > This serves as an umbrella ticket for tasks related to better queue > management in YARN. > Today's the only way to manage the queue is through admins editing > configuration files and then issuing a refresh command. This will bring many > inconveniences. For example, the users can not create / delete /modify their > own queues without talking to site level admins. > Even in today's approach (configuration-based), we still have several places > needed to improve: > * It is possible today to add or modify queues without restarting the RM, > via a CS refresh. But for deleting queue, we have to restart the > resourcemanager. > * When a queue is STOPPED, resources allocated to the queue can be handled > better. Currently, they'll only be used if the other queues are setup to go > over their capacity. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5724) [Umbrella] Better Queue Management in YARN
[ https://issues.apache.org/jira/browse/YARN-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802882#comment-15802882 ] Xuan Gong commented on YARN-5724: - update the design doc based on the discussion on YARN-5556 > [Umbrella] Better Queue Management in YARN > -- > > Key: YARN-5724 > URL: https://issues.apache.org/jira/browse/YARN-5724 > Project: Hadoop YARN > Issue Type: Task > Components: capacity scheduler >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: > Designdocv1-Configuration-basedQueueManagementinYARN.pdf, > Designdocv2-Configuration-basedQueueManagementinYARN.pdf > > > This serves as an umbrella ticket for tasks related to better queue > management in YARN. > Today's the only way to manage the queue is through admins editing > configuration files and then issuing a refresh command. This will bring many > inconveniences. For example, the users can not create / delete /modify their > own queues without talking to site level admins. > Even in today's approach (configuration-based), we still have several places > needed to improve: > * It is possible today to add or modify queues without restarting the RM, > via a CS refresh. But for deleting queue, we have to restart the > resourcemanager. > * When a queue is STOPPED, resources allocated to the queue can be handled > better. Currently, they'll only be used if the other queues are setup to go > over their capacity. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5724) [Umbrella] Better Queue Management in YARN
[ https://issues.apache.org/jira/browse/YARN-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15799431#comment-15799431 ] Naganarasimha G R commented on YARN-5724: - I have added some [comments|https://issues.apache.org/jira/browse/YARN-5556?focusedCommentId=15799398=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15799398] for the doc in YARN-5556 related to deletion of the queue, we can discuss more there . > [Umbrella] Better Queue Management in YARN > -- > > Key: YARN-5724 > URL: https://issues.apache.org/jira/browse/YARN-5724 > Project: Hadoop YARN > Issue Type: Task > Components: capacity scheduler >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: Designdocv1-Configuration-basedQueueManagementinYARN.pdf > > > This serves as an umbrella ticket for tasks related to better queue > management in YARN. > Today's the only way to manage the queue is through admins editing > configuration files and then issuing a refresh command. This will bring many > inconveniences. For example, the users can not create / delete /modify their > own queues without talking to site level admins. > Even in today's approach (configuration-based), we still have several places > needed to improve: > * It is possible today to add or modify queues without restarting the RM, > via a CS refresh. But for deleting queue, we have to restart the > resourcemanager. > * When a queue is STOPPED, resources allocated to the queue can be handled > better. Currently, they'll only be used if the other queues are setup to go > over their capacity. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5724) [Umbrella] Better Queue Management in YARN
[ https://issues.apache.org/jira/browse/YARN-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15593753#comment-15593753 ] Karthik Kambatla commented on YARN-5724: [~xgong] - is this proposal specific to CapacityScheduler? Or, are you suggesting common changes that all schedulers could benefit? > [Umbrella] Better Queue Management in YARN > -- > > Key: YARN-5724 > URL: https://issues.apache.org/jira/browse/YARN-5724 > Project: Hadoop YARN > Issue Type: Task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: Designdocv1-Configuration-basedQueueManagementinYARN.pdf > > > This serves as an umbrella ticket for tasks related to better queue > management in YARN. > Today's the only way to manage the queue is through admins editing > configuration files and then issuing a refresh command. This will bring many > inconveniences. For example, the users can not create / delete /modify their > own queues without talking to site level admins. > Even in today's approach (configuration-based), we still have several places > needed to improve: > * It is possible today to add or modify queues without restarting the RM, > via a CS refresh. But for deleting queue, we have to restart the > resourcemanager. > * When a queue is STOPPED, resources allocated to the queue can be handled > better. Currently, they'll only be used if the other queues are setup to go > over their capacity. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5724) [Umbrella] Better Queue Management in YARN
[ https://issues.apache.org/jira/browse/YARN-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15573601#comment-15573601 ] Jonathan Hung commented on YARN-5724: - Hi [~xgong], we have a ticket open here: YARN-5734 for a similar functionality. Please take a look and let us know what you think! > [Umbrella] Better Queue Management in YARN > -- > > Key: YARN-5724 > URL: https://issues.apache.org/jira/browse/YARN-5724 > Project: Hadoop YARN > Issue Type: Task >Reporter: Xuan Gong >Assignee: Xuan Gong > > This serves as an umbrella ticket for tasks related to better queue > management in YARN. > Today's the only way to manage the queue is through admins editing > configuration files and then issuing a refresh command. This will bring many > inconveniences. For example, the users can not create / delete /modify their > own queues without talking to site level admins. > Even in today's approach (configuration-based), we still have several places > needed to improve: > * It is possible today to add or modify queues without restarting the RM, > via a CS refresh. But for deleting queue, we have to restart the > resourcemanager. > * When a queue is STOPPED, resources allocated to the queue can be handled > better. Currently, they'll only be used if the other queues are setup to go > over their capacity. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5724) [Umbrella] Better Queue Management in YARN
[ https://issues.apache.org/jira/browse/YARN-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569933#comment-15569933 ] Naganarasimha G R commented on YARN-5724: - It's more specific to CS right? > [Umbrella] Better Queue Management in YARN > -- > > Key: YARN-5724 > URL: https://issues.apache.org/jira/browse/YARN-5724 > Project: Hadoop YARN > Issue Type: Task >Reporter: Xuan Gong >Assignee: Xuan Gong > > This serves as an umbrella ticket for tasks related to better queue > management in YARN. > Today's the only way to manage the queue is through admins editing > configuration files and then issuing a refresh command. This will bring many > inconveniences. For example, the users can not create / delete /modify their > own queues without talking to site level admins. > Even in today's approach (configuration-based), we still have several places > needed to improve: > * It is possible today to add or modify queues without restarting the RM, > via a CS refresh. But for deleting queue, we have to restart the > resourcemanager. > * When a queue is STOPPED, resources allocated to the queue can be handled > better. Currently, they'll only be used if the other queues are setup to go > over their capacity. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5724) [Umbrella] Better Queue Management in YARN
[ https://issues.apache.org/jira/browse/YARN-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15569710#comment-15569710 ] Xuan Gong commented on YARN-5724: - Will add a proposal soon. > [Umbrella] Better Queue Management in YARN > -- > > Key: YARN-5724 > URL: https://issues.apache.org/jira/browse/YARN-5724 > Project: Hadoop YARN > Issue Type: Task >Reporter: Xuan Gong >Assignee: Xuan Gong > > This serves as an umbrella ticket for tasks related to better queue > management in YARN. > Today's the only way to manage the queue is through admins editing > configuration files and then issuing a refresh command. This will bring many > inconveniences. For example, the users can not create / delete /modify their > own queues without talking to site level admins. > Even in today's approach (configuration-based), we still have several places > needed to improve: > * It is possible today to add or modify queues without restarting the RM, > via a CS refresh. But for deleting queue, we have to restart the > resourcemanager. > * When a queue is STOPPED, resources allocated to the queue can be handled > better. Currently, they'll only be used if the other queues are setup to go > over their capacity. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
