[jira] [Updated] (YARN-6831) Miscellaneous refactoring changes of ContainScheduler
[ https://issues.apache.org/jira/browse/YARN-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shilun Fan updated YARN-6831: - Target Version/s: 3.5.0 (was: 3.4.0) > Miscellaneous refactoring changes of ContainScheduler > -- > > Key: YARN-6831 > URL: https://issues.apache.org/jira/browse/YARN-6831 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > > While reviewing YARN-6706, Karthik pointed out a few issues for improvment in > ContainerScheduler > *Make ResourceUtilizationTracker pluggable. That way, we could use a > different tracker when oversubscription is enabled. > *ContainerScheduler > ##Why do we need maxOppQueueLength given queuingLimit? > ##Is there value in splitting runningContainers into runningGuaranteed and > runningOpportunistic? > ##getOpportunisticContainersStatus method implementation feels awkward. How > about capturing the state in the field here, and have metrics etc. pull from > here? > ##startContainersFromQueue: Local variable resourcesAvailable is unnecessary > *OpportunisticContainersStatus > ##Let us clearly differentiate between allocated, used and utilized. Maybe, > we should rename current Used methods to Allocated? > ##I prefer either full name Opportunistic (in method) or Opp (shortest name > that makes sense). Opport is neither short nor fully descriptive. > ##Have we considered folding ContainerQueuingLimit class into this? > We decided to move the issues into this follow up jira to keep YARN-6706 > moving forward to unblock oversubscription work. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6831) Miscellaneous refactoring changes of ContainScheduler
[ https://issues.apache.org/jira/browse/YARN-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brahma Reddy Battula updated YARN-6831: --- Target Version/s: 3.4.0 (was: 3.3.0) Bulk update: moved all 3.3.0 non-blocker issues, please move back if it is a blocker. > Miscellaneous refactoring changes of ContainScheduler > -- > > Key: YARN-6831 > URL: https://issues.apache.org/jira/browse/YARN-6831 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > > While reviewing YARN-6706, Karthik pointed out a few issues for improvment in > ContainerScheduler > *Make ResourceUtilizationTracker pluggable. That way, we could use a > different tracker when oversubscription is enabled. > *ContainerScheduler > ##Why do we need maxOppQueueLength given queuingLimit? > ##Is there value in splitting runningContainers into runningGuaranteed and > runningOpportunistic? > ##getOpportunisticContainersStatus method implementation feels awkward. How > about capturing the state in the field here, and have metrics etc. pull from > here? > ##startContainersFromQueue: Local variable resourcesAvailable is unnecessary > *OpportunisticContainersStatus > ##Let us clearly differentiate between allocated, used and utilized. Maybe, > we should rename current Used methods to Allocated? > ##I prefer either full name Opportunistic (in method) or Opp (shortest name > that makes sense). Opport is neither short nor fully descriptive. > ##Have we considered folding ContainerQueuingLimit class into this? > We decided to move the issues into this follow up jira to keep YARN-6706 > moving forward to unblock oversubscription work. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6831) Miscellaneous refactoring changes of ContainScheduler
[ https://issues.apache.org/jira/browse/YARN-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-6831: - Target Version/s: 3.3.0 (was: 3.2.0) Bulk update: moved all 3.2.0 non-blocker issues, please move back if it is a blocker. > Miscellaneous refactoring changes of ContainScheduler > -- > > Key: YARN-6831 > URL: https://issues.apache.org/jira/browse/YARN-6831 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Major > > While reviewing YARN-6706, Karthik pointed out a few issues for improvment in > ContainerScheduler > *Make ResourceUtilizationTracker pluggable. That way, we could use a > different tracker when oversubscription is enabled. > *ContainerScheduler > ##Why do we need maxOppQueueLength given queuingLimit? > ##Is there value in splitting runningContainers into runningGuaranteed and > runningOpportunistic? > ##getOpportunisticContainersStatus method implementation feels awkward. How > about capturing the state in the field here, and have metrics etc. pull from > here? > ##startContainersFromQueue: Local variable resourcesAvailable is unnecessary > *OpportunisticContainersStatus > ##Let us clearly differentiate between allocated, used and utilized. Maybe, > we should rename current Used methods to Allocated? > ##I prefer either full name Opportunistic (in method) or Opp (shortest name > that makes sense). Opport is neither short nor fully descriptive. > ##Have we considered folding ContainerQueuingLimit class into this? > We decided to move the issues into this follow up jira to keep YARN-6706 > moving forward to unblock oversubscription work. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6831) Miscellaneous refactoring changes of ContainScheduler
[ https://issues.apache.org/jira/browse/YARN-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6831: - Description: While reviewing YARN-6706, Karthik pointed out a few issues for improvment in ContainerScheduler *Make ResourceUtilizationTracker pluggable. That way, we could use a different tracker when oversubscription is enabled. *ContainerScheduler ##Why do we need maxOppQueueLength given queuingLimit? ##Is there value in splitting runningContainers into runningGuaranteed and runningOpportunistic? ##getOpportunisticContainersStatus method implementation feels awkward. How about capturing the state in the field here, and have metrics etc. pull from here? ##startContainersFromQueue: Local variable resourcesAvailable is unnecessary #OpportunisticContainersStatus ##Let us clearly differentiate between allocated, used and utilized. Maybe, we should rename current Used methods to Allocated? ##I prefer either full name Opportunistic (in method) or Opp (shortest name that makes sense). Opport is neither short nor fully descriptive. ##Have we considered folding ContainerQueuingLimit class into this? We decided to move the issues into this follow up jira to keep YARN-6706 moving forward to unblock oversubscription work. was: While reviewing YARN-6706, Karthik pointed out a few issues for improvment in ContainerScheduler #Make ResourceUtilizationTracker pluggable. That way, we could use a different tracker when oversubscription is enabled. #ContainerScheduler ##Why do we need maxOppQueueLength given queuingLimit? ##Is there value in splitting runningContainers into runningGuaranteed and runningOpportunistic? ##getOpportunisticContainersStatus method implementation feels awkward. How about capturing the state in the field here, and have metrics etc. pull from here? ##startContainersFromQueue: Local variable resourcesAvailable is unnecessary #OpportunisticContainersStatus ##Let us clearly differentiate between allocated, used and utilized. Maybe, we should rename current Used methods to Allocated? ##I prefer either full name Opportunistic (in method) or Opp (shortest name that makes sense). Opport is neither short nor fully descriptive. ##Have we considered folding ContainerQueuingLimit class into this? We decided to move the issues into this follow up jira to keep YARN-6706 moving forward to unblock oversubscription work. > Miscellaneous refactoring changes of ContainScheduler > -- > > Key: YARN-6831 > URL: https://issues.apache.org/jira/browse/YARN-6831 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Haibo Chen >Assignee: Haibo Chen > > While reviewing YARN-6706, Karthik pointed out a few issues for improvment in > ContainerScheduler > *Make ResourceUtilizationTracker pluggable. That way, we could use a > different tracker when oversubscription is enabled. > *ContainerScheduler > ##Why do we need maxOppQueueLength given queuingLimit? > ##Is there value in splitting runningContainers into runningGuaranteed and > runningOpportunistic? > ##getOpportunisticContainersStatus method implementation feels awkward. How > about capturing the state in the field here, and have metrics etc. pull from > here? > ##startContainersFromQueue: Local variable resourcesAvailable is unnecessary > #OpportunisticContainersStatus > ##Let us clearly differentiate between allocated, used and utilized. Maybe, > we should rename current Used methods to Allocated? > ##I prefer either full name Opportunistic (in method) or Opp (shortest name > that makes sense). Opport is neither short nor fully descriptive. > ##Have we considered folding ContainerQueuingLimit class into this? > We decided to move the issues into this follow up jira to keep YARN-6706 > moving forward to unblock oversubscription work. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6831) Miscellaneous refactoring changes of ContainScheduler
[ https://issues.apache.org/jira/browse/YARN-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6831: - Description: While reviewing YARN-6706, Karthik pointed out a few issues for improvment in ContainerScheduler *Make ResourceUtilizationTracker pluggable. That way, we could use a different tracker when oversubscription is enabled. *ContainerScheduler ##Why do we need maxOppQueueLength given queuingLimit? ##Is there value in splitting runningContainers into runningGuaranteed and runningOpportunistic? ##getOpportunisticContainersStatus method implementation feels awkward. How about capturing the state in the field here, and have metrics etc. pull from here? ##startContainersFromQueue: Local variable resourcesAvailable is unnecessary *OpportunisticContainersStatus ##Let us clearly differentiate between allocated, used and utilized. Maybe, we should rename current Used methods to Allocated? ##I prefer either full name Opportunistic (in method) or Opp (shortest name that makes sense). Opport is neither short nor fully descriptive. ##Have we considered folding ContainerQueuingLimit class into this? We decided to move the issues into this follow up jira to keep YARN-6706 moving forward to unblock oversubscription work. was: While reviewing YARN-6706, Karthik pointed out a few issues for improvment in ContainerScheduler *Make ResourceUtilizationTracker pluggable. That way, we could use a different tracker when oversubscription is enabled. *ContainerScheduler ##Why do we need maxOppQueueLength given queuingLimit? ##Is there value in splitting runningContainers into runningGuaranteed and runningOpportunistic? ##getOpportunisticContainersStatus method implementation feels awkward. How about capturing the state in the field here, and have metrics etc. pull from here? ##startContainersFromQueue: Local variable resourcesAvailable is unnecessary #OpportunisticContainersStatus ##Let us clearly differentiate between allocated, used and utilized. Maybe, we should rename current Used methods to Allocated? ##I prefer either full name Opportunistic (in method) or Opp (shortest name that makes sense). Opport is neither short nor fully descriptive. ##Have we considered folding ContainerQueuingLimit class into this? We decided to move the issues into this follow up jira to keep YARN-6706 moving forward to unblock oversubscription work. > Miscellaneous refactoring changes of ContainScheduler > -- > > Key: YARN-6831 > URL: https://issues.apache.org/jira/browse/YARN-6831 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Haibo Chen >Assignee: Haibo Chen > > While reviewing YARN-6706, Karthik pointed out a few issues for improvment in > ContainerScheduler > *Make ResourceUtilizationTracker pluggable. That way, we could use a > different tracker when oversubscription is enabled. > *ContainerScheduler > ##Why do we need maxOppQueueLength given queuingLimit? > ##Is there value in splitting runningContainers into runningGuaranteed and > runningOpportunistic? > ##getOpportunisticContainersStatus method implementation feels awkward. How > about capturing the state in the field here, and have metrics etc. pull from > here? > ##startContainersFromQueue: Local variable resourcesAvailable is unnecessary > *OpportunisticContainersStatus > ##Let us clearly differentiate between allocated, used and utilized. Maybe, > we should rename current Used methods to Allocated? > ##I prefer either full name Opportunistic (in method) or Opp (shortest name > that makes sense). Opport is neither short nor fully descriptive. > ##Have we considered folding ContainerQueuingLimit class into this? > We decided to move the issues into this follow up jira to keep YARN-6706 > moving forward to unblock oversubscription work. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6831) Miscellaneous refactoring changes of ContainScheduler
[ https://issues.apache.org/jira/browse/YARN-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6831: - Description: While reviewing YARN-6706, Karthik pointed out a few issues for improvment in ContainerScheduler #Make ResourceUtilizationTracker pluggable. That way, we could use a different tracker when oversubscription is enabled. #ContainerScheduler ##Why do we need maxOppQueueLength given queuingLimit? ##Is there value in splitting runningContainers into runningGuaranteed and runningOpportunistic? ##getOpportunisticContainersStatus method implementation feels awkward. How about capturing the state in the field here, and have metrics etc. pull from here? ##startContainersFromQueue: Local variable resourcesAvailable is unnecessary #OpportunisticContainersStatus ##Let us clearly differentiate between allocated, used and utilized. Maybe, we should rename current Used methods to Allocated? ##I prefer either full name Opportunistic (in method) or Opp (shortest name that makes sense). Opport is neither short nor fully descriptive. ##Have we considered folding ContainerQueuingLimit class into this? We decided to move the issues into this follow up jira to keep YARN-6706 moving forward to unblock oversubscription work. was: While reviewing YARN-6706, Karthik pointed out a few issues for improvment in ContainerScheduler #Make ResourceUtilizationTracker pluggable. That way, we could use a different tracker when oversubscription is enabled. #ContainerScheduler ##Why do we need maxOppQueueLength given queuingLimit? ##Is there value in splitting runningContainers into runningGuaranteed and runningOpportunistic? ##getOpportunisticContainersStatus method implementation feels awkward. How about capturing the state in the field here, and have metrics etc. pull from here? ##startContainersFromQueue: Local variable resourcesAvailable is unnecessary #OpportunisticContainersStatus ##Let us clearly differentiate between allocated, used and utilized. Maybe, we should rename current Used methods to Allocated? ##I prefer either full name Opportunistic (in method) or Opp (shortest name that makes sense). Opport is neither short nor fully descriptive. ##Have we considered folding ContainerQueuingLimit class into this? We decided to move the issues into this follow up jira to keep YARN-6706 moving forward to unblock oversubscription work. > Miscellaneous refactoring changes of ContainScheduler > -- > > Key: YARN-6831 > URL: https://issues.apache.org/jira/browse/YARN-6831 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Haibo Chen >Assignee: Haibo Chen > > While reviewing YARN-6706, Karthik pointed out a few issues for improvment in > ContainerScheduler > #Make ResourceUtilizationTracker pluggable. That way, we could use a > different tracker when oversubscription is enabled. > #ContainerScheduler > ##Why do we need maxOppQueueLength given queuingLimit? > ##Is there value in splitting runningContainers into runningGuaranteed and > runningOpportunistic? > ##getOpportunisticContainersStatus method implementation feels awkward. How > about capturing the state in the field here, and have metrics etc. pull from > here? > ##startContainersFromQueue: Local variable resourcesAvailable is unnecessary > #OpportunisticContainersStatus > ##Let us clearly differentiate between allocated, used and utilized. Maybe, > we should rename current Used methods to Allocated? > ##I prefer either full name Opportunistic (in method) or Opp (shortest name > that makes sense). Opport is neither short nor fully descriptive. > ##Have we considered folding ContainerQueuingLimit class into this? > We decided to move the issues into this follow up jira to keep YARN-6706 > moving forward to unblock oversubscription work. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6831) Miscellaneous refactoring changes of ContainScheduler
[ https://issues.apache.org/jira/browse/YARN-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6831: - Description: While reviewing YARN-6706, Karthik pointed out a few issues for improvment in ContainerScheduler #Make ResourceUtilizationTracker pluggable. That way, we could use a different tracker when oversubscription is enabled. #ContainerScheduler ##Why do we need maxOppQueueLength given queuingLimit? ##Is there value in splitting runningContainers into runningGuaranteed and runningOpportunistic? ##getOpportunisticContainersStatus method implementation feels awkward. How about capturing the state in the field here, and have metrics etc. pull from here? ##startContainersFromQueue: Local variable resourcesAvailable is unnecessary #OpportunisticContainersStatus ##Let us clearly differentiate between allocated, used and utilized. Maybe, we should rename current Used methods to Allocated? ##I prefer either full name Opportunistic (in method) or Opp (shortest name that makes sense). Opport is neither short nor fully descriptive. ##Have we considered folding ContainerQueuingLimit class into this? We decided to move the issues into this follow up jira to keep YARN-6706 moving forward to unblock oversubscription work. was: While reviewing YARN-6706, Karthik pointed out a few issues for improvment in ContainerScheduler " #Make ResourceUtilizationTracker pluggable. That way, we could use a different tracker when oversubscription is enabled. #ContainerScheduler ##Why do we need maxOppQueueLength given queuingLimit? ##Is there value in splitting runningContainers into runningGuaranteed and runningOpportunistic? ##getOpportunisticContainersStatus method implementation feels awkward. How about capturing the state in the field here, and have metrics etc. pull from here? ##startContainersFromQueue: Local variable resourcesAvailable is unnecessary #OpportunisticContainersStatus ##Let us clearly differentiate between allocated, used and utilized. Maybe, we should rename current Used methods to Allocated? ##I prefer either full name Opportunistic (in method) or Opp (shortest name that makes sense). Opport is neither short nor fully descriptive. ##Have we considered folding ContainerQueuingLimit class into this? " We decided to move the issues into this follow up jira to keep YARN-6706 moving forward to unblock oversubscription work. > Miscellaneous refactoring changes of ContainScheduler > -- > > Key: YARN-6831 > URL: https://issues.apache.org/jira/browse/YARN-6831 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Haibo Chen >Assignee: Haibo Chen > > While reviewing YARN-6706, Karthik pointed out a few issues for improvment in > ContainerScheduler > #Make ResourceUtilizationTracker pluggable. That way, we could use a > different tracker when oversubscription is enabled. > #ContainerScheduler > ##Why do we need maxOppQueueLength given queuingLimit? > ##Is there value in splitting runningContainers into runningGuaranteed and > runningOpportunistic? > ##getOpportunisticContainersStatus method implementation feels awkward. How > about capturing the state in the field here, and have metrics etc. pull from > here? > ##startContainersFromQueue: Local variable resourcesAvailable is unnecessary > #OpportunisticContainersStatus > ##Let us clearly differentiate between allocated, used and utilized. Maybe, > we should rename current Used methods to Allocated? > ##I prefer either full name Opportunistic (in method) or Opp (shortest name > that makes sense). Opport is neither short nor fully descriptive. > ##Have we considered folding ContainerQueuingLimit class into this? > We decided to move the issues into this follow up jira to keep YARN-6706 > moving forward to unblock oversubscription work. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6831) Miscellaneous refactoring changes of ContainScheduler
[ https://issues.apache.org/jira/browse/YARN-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-6831: - Description: While reviewing YARN-6706, Karthik pointed out a few issues for improvment in ContainerScheduler " #Make ResourceUtilizationTracker pluggable. That way, we could use a different tracker when oversubscription is enabled. #ContainerScheduler ##Why do we need maxOppQueueLength given queuingLimit? ##Is there value in splitting runningContainers into runningGuaranteed and runningOpportunistic? ##getOpportunisticContainersStatus method implementation feels awkward. How about capturing the state in the field here, and have metrics etc. pull from here? ##startContainersFromQueue: Local variable resourcesAvailable is unnecessary #OpportunisticContainersStatus ##Let us clearly differentiate between allocated, used and utilized. Maybe, we should rename current Used methods to Allocated? ##I prefer either full name Opportunistic (in method) or Opp (shortest name that makes sense). Opport is neither short nor fully descriptive. ##Have we considered folding ContainerQueuingLimit class into this? " We decided to move the issues into this follow up jira to keep YARN-6706 moving forward to unblock oversubscription work. > Miscellaneous refactoring changes of ContainScheduler > -- > > Key: YARN-6831 > URL: https://issues.apache.org/jira/browse/YARN-6831 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Haibo Chen >Assignee: Haibo Chen > > While reviewing YARN-6706, Karthik pointed out a few issues for improvment in > ContainerScheduler > " > #Make ResourceUtilizationTracker pluggable. That way, we could use a > different tracker when oversubscription is enabled. > #ContainerScheduler > ##Why do we need maxOppQueueLength given queuingLimit? > ##Is there value in splitting runningContainers into runningGuaranteed and > runningOpportunistic? > ##getOpportunisticContainersStatus method implementation feels awkward. How > about capturing the state in the field here, and have metrics etc. pull from > here? > ##startContainersFromQueue: Local variable resourcesAvailable is unnecessary > #OpportunisticContainersStatus > ##Let us clearly differentiate between allocated, used and utilized. Maybe, > we should rename current Used methods to Allocated? > ##I prefer either full name Opportunistic (in method) or Opp (shortest name > that makes sense). Opport is neither short nor fully descriptive. > ##Have we considered folding ContainerQueuingLimit class into this? > " > We decided to move the issues into this follow up jira to keep YARN-6706 > moving forward to unblock oversubscription work. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
