[jira] [Assigned] (YARN-10580) Fix some issues in TestRMWebServicesCapacitySchedDynamicConfig
[
https://issues.apache.org/jira/browse/YARN-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ankit Kumar reassigned YARN-10580:
--
Assignee: Ankit Kumar
> Fix some issues in TestRMWebServicesCapacitySchedDynamicConfig
> --
>
> Key: YARN-10580
> URL: https://issues.apache.org/jira/browse/YARN-10580
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Szilard Nemeth
>Assignee: Ankit Kumar
>Priority: Minor
> Labels: newbie, newbie++
>
> YARN-10512 introduced some changes that could be fixed, [~pbacsko]
> highlighted those issues with this comment.
> Pasting the contents of the comment as a reference
> #1 In TestRMWebServicesCapacitySchedDynamicConfig:
> {noformat}
> config.set(YarnConfiguration.SCHEDULER_CONFIGURATION_STORE_CLASS,
> YarnConfiguration.MEMORY_CONFIGURATION_STORE);
> {noformat}
> This call is repeated multiple times, this could be set somewhere else.
>
> #2 In TestRMWebServicesCapacitySchedDynamicConfig:
> {noformat}
> validateSchedulerInfo(json, "weight", "root.default", "root.test1",
> "root.test2");
> {noformat}
> "root.default", "root.test1" and "root.test2" are the same in all cases, you
> might want to drop them
>
> #3 In TestRMWebServicesCapacitySchedDynamicConfig
> {noformat}
> @Before
> @Override
> public void setUp() throws Exception {
> super.setUp();
> }
> {noformat}
> This method does nothing, can be removed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10537) Change type of LogAggregationService threadPool
[ https://issues.apache.org/jira/browse/YARN-10537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Kumar reassigned YARN-10537: -- Assignee: Ankit Kumar > Change type of LogAggregationService threadPool > --- > > Key: YARN-10537 > URL: https://issues.apache.org/jira/browse/YARN-10537 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Xie YiFan >Assignee: Ankit Kumar >Priority: Minor > > Now, LogAggregationService threadPool is FixedThreadPool which of default > threadPoolSize is 100. LogAggregationService will construct AppLogAggregator > for new come application and submit to threadPool. AppLogAggregator do while > loop unitl application finished. Some application may run very long time due > to reason such as no enough resource or other. As result, it occupy one > thread of threadPool. When this application size greater than threadPoolSize, > the later short-live application can't upload logs until previous long-live > application finished. So, i think we should replace FixedThreadPool to > CachedThreadPool. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10522) Document for Flexible Auto Queue Creation in Capacity Scheduler.
[ https://issues.apache.org/jira/browse/YARN-10522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Kumar reassigned YARN-10522: -- Assignee: Ankit Kumar > Document for Flexible Auto Queue Creation in Capacity Scheduler. > > > Key: YARN-10522 > URL: https://issues.apache.org/jira/browse/YARN-10522 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: zhuqi >Assignee: Ankit Kumar >Priority: Major > > We should update document to support this feature. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10491) Fix deprecation warnings in SLSWebApp.java
[
https://issues.apache.org/jira/browse/YARN-10491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ankit Kumar reassigned YARN-10491:
--
Assignee: Ankit Kumar
> Fix deprecation warnings in SLSWebApp.java
> --
>
> Key: YARN-10491
> URL: https://issues.apache.org/jira/browse/YARN-10491
> Project: Hadoop YARN
> Issue Type: Bug
> Components: build
>Reporter: Akira Ajisaka
>Assignee: Ankit Kumar
>Priority: Minor
> Labels: newbie
>
> {code}
> simulateInfoTemplate = IOUtils.toString(
> cl.getResourceAsStream("html/simulate.info.html.template"));
> simulateTemplate = IOUtils.toString(
> cl.getResourceAsStream("html/simulate.html.template"));
> trackTemplate = IOUtils.toString(
> cl.getResourceAsStream("html/track.html.template"));
> {code}
> {{IOUtils.toString(InputStream, Charset)}} should be used.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10511) Update yarn.nodemanager.env-whitelist value in Hadoop Cluster Setup
[
https://issues.apache.org/jira/browse/YARN-10511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ankit Kumar reassigned YARN-10511:
--
Assignee: (was: Ankit Kumar)
> Update yarn.nodemanager.env-whitelist value in Hadoop Cluster Setup
> ---
>
> Key: YARN-10511
> URL: https://issues.apache.org/jira/browse/YARN-10511
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Andrea Scarpino
>Priority: Trivial
> Labels: documentation, pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The [Hadoop Cluster
> Setup|https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/ClusterSetup.html]
> documentation page references an old value for the
> {{yarn.nodemanager.env-whitelist}} variable because the value referenced in
> that page has been changed since YARN-6694.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10511) Update yarn.nodemanager.env-whitelist value in Hadoop Cluster Setup
[
https://issues.apache.org/jira/browse/YARN-10511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ankit Kumar reassigned YARN-10511:
--
Assignee: Ankit Kumar
> Update yarn.nodemanager.env-whitelist value in Hadoop Cluster Setup
> ---
>
> Key: YARN-10511
> URL: https://issues.apache.org/jira/browse/YARN-10511
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Andrea Scarpino
>Assignee: Ankit Kumar
>Priority: Trivial
> Labels: documentation
>
> The [Hadoop Cluster
> Setup|https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/ClusterSetup.html]
> documentation page references an old value for the
> {{yarn.nodemanager.env-whitelist}} variable because the value referenced in
> that page has been changed since YARN-6694.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10509) Backport HADOOP-12760 to branch-2.7
[ https://issues.apache.org/jira/browse/YARN-10509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Kumar reassigned YARN-10509: -- Assignee: Ankit Kumar > Backport HADOOP-12760 to branch-2.7 > --- > > Key: YARN-10509 > URL: https://issues.apache.org/jira/browse/YARN-10509 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Yuming Wang >Assignee: Ankit Kumar >Priority: Major > > Backport HADOOP-12760 to branch-2.7 to support JDK 11. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10326) ApplicationHistory Web UI should show the queue where the task is actually running
[ https://issues.apache.org/jira/browse/YARN-10326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Kumar reassigned YARN-10326: -- Assignee: Ankit Kumar > ApplicationHistory Web UI should show the queue where the task is actually > running > -- > > Key: YARN-10326 > URL: https://issues.apache.org/jira/browse/YARN-10326 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 2.7.2, 3.2.1 >Reporter: yehuanhuan >Assignee: Ankit Kumar >Priority: Critical > > ApplicationHistory Web UI should show the queue where the task is actually > running instead of the queue specified by the client. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10488) Several typos in package: org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair
[ https://issues.apache.org/jira/browse/YARN-10488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Kumar reassigned YARN-10488: -- Assignee: Ankit Kumar > Several typos in package: > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair > -- > > Key: YARN-10488 > URL: https://issues.apache.org/jira/browse/YARN-10488 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Ankit Kumar >Priority: Minor > Labels: newbie, newbie++ > > 1. Typo in field name: > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.VisitedResourceRequestTracker.TrackerPerPriorityResource#racksVisted > 2. Typo in method: > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager#setChildResourceLimits > There's a comment: "... max reource ...", typo in the word 'resource'. > 3. Typo in javadoc of method: > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt#reserve > "bookeeping" -> "bookkeeping" > 4. There's a local variable in this method: > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt#updateAMDiagnosticMsg, > > called diagnosticMessageBldr. It's an abbreviation, but could be changed to > something more meaningful. > 5. Typo in javadoc of method: > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.MaxRunningAppsEnforcer#updateRunnabilityOnReload > "reinitilized" --> "reinitialized" > 6. And last but not least, a funny typo in the method name of: > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.policies.DominantResourceFairnessPolicy.DominantResourceFairnessComparator#compareAttribrutes -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10443) Document options of logs CLI
[
https://issues.apache.org/jira/browse/YARN-10443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ankit Kumar updated YARN-10443:
---
Attachment: YARN-10443.002.patch
> Document options of logs CLI
>
>
> Key: YARN-10443
> URL: https://issues.apache.org/jira/browse/YARN-10443
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.3.0
>Reporter: Adam Antal
>Assignee: Ankit Kumar
>Priority: Major
> Attachments: YARN-10443.001.patch, YARN-10443.002.patch
>
>
> It's bugging me a lot that the YARN logs CLI is poorly documented. I always
> have to type {{yarn logs -help}} to see the full list of supported commands.
> It would be nice to have it nicely documented in our website.
> Current
> [documentation|https://hadoop.apache.org/docs/r3.3.0/hadoop-yarn/hadoop-yarn-site/YarnCommands.html#logs]
> in the website shows only 5 supported options.
> The output of the help command however shows more:
> {noformat}
> Retrieve logs for YARN applications.
> usage: yarn logs -applicationId [OPTIONS]
> general options are:
> -am Prints the AM Container logs
> for this application.
> Specify comma-separated
> value to get logs for
> related AM Container. For
> example, If we specify -am
> 1,2, we will get the logs
> for the first AM Container
> as well as the second AM
> Container. To get logs for
> all AM Containers, use -am
> ALL. To get logs for the
> latest AM Container, use -am
> -1. By default, it will
> print all available logs.
> Work with -log_files to get
> only specific logs.
> -appOwner AppOwner (assumed to be
> current user if not
> specified)
> -client_max_retries Set max retry number for a
> retry client to get the
> container logs for the
> running applications. Use a
> negative value to make retry
> forever. The default value
> is 30.
> -client_retry_interval_msWork with
> --client_max_retries to
> create a retry client. The
> default value is 1000.
> -clusterId ClusterId. By default, it
> will take default cluster id
> from the RM
> -containerId ContainerId. By default, it
> will print all available
> logs. Work with -log_files
> to get only specific logs.
> If specified, the
> applicationId can be omitted
> -helpDisplays help for all
> commands.
> -list_nodes Show the list of nodes that
> successfully aggregated
> logs. This option can only
> be used with finished
> applications.
> -log_filesSpecify comma-separated
> value to get exact matched
> log files. Use "ALL" or "*"
> to fetch all the log files
> for the container.
> -log_files_pattern Specify comma-separated
> value to get matched
[jira] [Commented] (YARN-10443) Document options of logs CLI
[
https://issues.apache.org/jira/browse/YARN-10443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199824#comment-17199824
]
Ankit Kumar commented on YARN-10443:
I have created a PR for this:
[https://github.com/apache/hadoop/pull/2325]
> Document options of logs CLI
>
>
> Key: YARN-10443
> URL: https://issues.apache.org/jira/browse/YARN-10443
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.3.0
>Reporter: Adam Antal
>Assignee: Ankit Kumar
>Priority: Major
> Attachments: YARN-10443.001.patch
>
>
> It's bugging me a lot that the YARN logs CLI is poorly documented. I always
> have to type {{yarn logs -help}} to see the full list of supported commands.
> It would be nice to have it nicely documented in our website.
> Current
> [documentation|https://hadoop.apache.org/docs/r3.3.0/hadoop-yarn/hadoop-yarn-site/YarnCommands.html#logs]
> in the website shows only 5 supported options.
> The output of the help command however shows more:
> {noformat}
> Retrieve logs for YARN applications.
> usage: yarn logs -applicationId [OPTIONS]
> general options are:
> -am Prints the AM Container logs
> for this application.
> Specify comma-separated
> value to get logs for
> related AM Container. For
> example, If we specify -am
> 1,2, we will get the logs
> for the first AM Container
> as well as the second AM
> Container. To get logs for
> all AM Containers, use -am
> ALL. To get logs for the
> latest AM Container, use -am
> -1. By default, it will
> print all available logs.
> Work with -log_files to get
> only specific logs.
> -appOwner AppOwner (assumed to be
> current user if not
> specified)
> -client_max_retries Set max retry number for a
> retry client to get the
> container logs for the
> running applications. Use a
> negative value to make retry
> forever. The default value
> is 30.
> -client_retry_interval_msWork with
> --client_max_retries to
> create a retry client. The
> default value is 1000.
> -clusterId ClusterId. By default, it
> will take default cluster id
> from the RM
> -containerId ContainerId. By default, it
> will print all available
> logs. Work with -log_files
> to get only specific logs.
> If specified, the
> applicationId can be omitted
> -helpDisplays help for all
> commands.
> -list_nodes Show the list of nodes that
> successfully aggregated
> logs. This option can only
> be used with finished
> applications.
> -log_filesSpecify comma-separated
> value to get exact matched
> log files. Use "ALL" or "*"
> to fetch all the log files
> for the container.
> -log_files_pattern Specify comma-separated
>
[jira] [Commented] (YARN-10443) Document options of logs CLI
[
https://issues.apache.org/jira/browse/YARN-10443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17198916#comment-17198916
]
Ankit Kumar commented on YARN-10443:
Hey [~adam.antal]
Are you working on this? If not, I will be happy to pick it up.
> Document options of logs CLI
>
>
> Key: YARN-10443
> URL: https://issues.apache.org/jira/browse/YARN-10443
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.3.0
>Reporter: Adam Antal
>Priority: Major
>
> It's bugging me a lot that the YARN logs CLI is poorly documented. I always
> have to type {{yarn logs -help}} to see the full list of supported commands.
> It would be nice to have it nicely documented in our website.
> Current
> [documentation|https://hadoop.apache.org/docs/r3.3.0/hadoop-yarn/hadoop-yarn-site/YarnCommands.html#logs]
> in the website shows only 5 supported options.
> The output of the help command however shows more:
> {noformat}
> Retrieve logs for YARN applications.
> usage: yarn logs -applicationId [OPTIONS]
> general options are:
> -am Prints the AM Container logs
> for this application.
> Specify comma-separated
> value to get logs for
> related AM Container. For
> example, If we specify -am
> 1,2, we will get the logs
> for the first AM Container
> as well as the second AM
> Container. To get logs for
> all AM Containers, use -am
> ALL. To get logs for the
> latest AM Container, use -am
> -1. By default, it will
> print all available logs.
> Work with -log_files to get
> only specific logs.
> -appOwner AppOwner (assumed to be
> current user if not
> specified)
> -client_max_retries Set max retry number for a
> retry client to get the
> container logs for the
> running applications. Use a
> negative value to make retry
> forever. The default value
> is 30.
> -client_retry_interval_msWork with
> --client_max_retries to
> create a retry client. The
> default value is 1000.
> -clusterId ClusterId. By default, it
> will take default cluster id
> from the RM
> -containerId ContainerId. By default, it
> will print all available
> logs. Work with -log_files
> to get only specific logs.
> If specified, the
> applicationId can be omitted
> -helpDisplays help for all
> commands.
> -list_nodes Show the list of nodes that
> successfully aggregated
> logs. This option can only
> be used with finished
> applications.
> -log_filesSpecify comma-separated
> value to get exact matched
> log files. Use "ALL" or "*"
> to fetch all the log files
> for the container.
> -log_files_pattern Specify comma-separated
> value to get matched log
>
