[jira] [Created] (YARN-8957) Add Serializable interface to ComponentContainers
Zhankun Tang created YARN-8957:
--
Summary: Add Serializable interface to ComponentContainers
Key: YARN-8957
URL: https://issues.apache.org/jira/browse/YARN-8957
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Zhankun Tang
Assignee: Zhankun Tang
In YARN service API:
public class ComponentContainers {
private static final long serialVersionUID = -1456748479118874991L;
...
}
seems should be
public class ComponentContainers {color:#d04437}implements Serializable{color} {
private static final long serialVersionUID = -1456748479118874991L;
...
}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8956) Fix findbugs warnings in RegistryDNS.java
Akira Ajisaka created YARN-8956:
---
Summary: Fix findbugs warnings in RegistryDNS.java
Key: YARN-8956
URL: https://issues.apache.org/jira/browse/YARN-8956
Project: Hadoop YARN
Issue Type: Bug
Reporter: Akira Ajisaka
{noformat}
FindBugs :
module:hadoop-common-project/hadoop-registry
Exceptional return value of
java.util.concurrent.ExecutorService.submit(Callable) ignored in
org.apache.hadoop.registry.server.dns.RegistryDNS.addNIOTCP(InetAddress, int)
At RegistryDNS.java:ignored in
org.apache.hadoop.registry.server.dns.RegistryDNS.addNIOTCP(InetAddress, int)
At RegistryDNS.java:[line 900]
Exceptional return value of
java.util.concurrent.ExecutorService.submit(Callable) ignored in
org.apache.hadoop.registry.server.dns.RegistryDNS.addNIOUDP(InetAddress, int)
At RegistryDNS.java:ignored in
org.apache.hadoop.registry.server.dns.RegistryDNS.addNIOUDP(InetAddress, int)
At RegistryDNS.java:[line 926]
Exceptional return value of
java.util.concurrent.ExecutorService.submit(Callable) ignored in
org.apache.hadoop.registry.server.dns.RegistryDNS.serveNIOTCP(ServerSocketChannel,
InetAddress, int) At RegistryDNS.java:ignored in
org.apache.hadoop.registry.server.dns.RegistryDNS.serveNIOTCP(ServerSocketChannel,
InetAddress, int) At RegistryDNS.java:[line 850]
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8955) Add a flag to use local docker image instead of getting latest from registry
Eric Yang created YARN-8955: --- Summary: Add a flag to use local docker image instead of getting latest from registry Key: YARN-8955 URL: https://issues.apache.org/jira/browse/YARN-8955 Project: Hadoop YARN Issue Type: Sub-task Reporter: Eric Yang Some companies have security policy to use local docker images instead of getting latest images from internet. When docker image is pulled in localization phase, there are two possible out comes. The image is getting the latest from trusted registries, or the image is a static local copy. This task is to add a configuration flag to give priority to local image over trusted registry image. If a image already exist locally, node manager does not trigger docker pull to get the latest image from trusted registries. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8954) Reservations list field in ReservationListInfo is not accessible
Oleksandr Shevchenko created YARN-8954: -- Summary: Reservations list field in ReservationListInfo is not accessible Key: YARN-8954 URL: https://issues.apache.org/jira/browse/YARN-8954 Project: Hadoop YARN Issue Type: Improvement Components: resourcemanager, restapi Reporter: Oleksandr Shevchenko We need to add the getter for Reservations list field since the field cannot be accessible after the unmarshal. The similar problem described in YARN-2280. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/ [Oct 29, 2018 12:04:59 AM] (weichiu) Revert "HADOOP-15864. Job submitter / executor fail when SBN domain name -1 overall The following subsystems voted -1: asflicense findbugs hadolint pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/public/crossdomain.xml FindBugs : module:hadoop-common-project/hadoop-registry Exceptional return value of java.util.concurrent.ExecutorService.submit(Callable) ignored in org.apache.hadoop.registry.server.dns.RegistryDNS.addNIOTCP(InetAddress, int) At RegistryDNS.java:ignored in org.apache.hadoop.registry.server.dns.RegistryDNS.addNIOTCP(InetAddress, int) At RegistryDNS.java:[line 900] Exceptional return value of java.util.concurrent.ExecutorService.submit(Callable) ignored in org.apache.hadoop.registry.server.dns.RegistryDNS.addNIOUDP(InetAddress, int) At RegistryDNS.java:ignored in org.apache.hadoop.registry.server.dns.RegistryDNS.addNIOUDP(InetAddress, int) At RegistryDNS.java:[line 926] Exceptional return value of java.util.concurrent.ExecutorService.submit(Callable) ignored in org.apache.hadoop.registry.server.dns.RegistryDNS.serveNIOTCP(ServerSocketChannel, InetAddress, int) At RegistryDNS.java:ignored in org.apache.hadoop.registry.server.dns.RegistryDNS.serveNIOTCP(ServerSocketChannel, InetAddress, int) At RegistryDNS.java:[line 850] Failed CTEST tests : test_test_libhdfs_threaded_hdfs_static test_libhdfs_threaded_hdfspp_test_shim_static Failed junit tests : hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped hadoop.yarn.server.nodemanager.containermanager.TestContainerManager hadoop.yarn.server.resourcemanager.TestApplicationMasterServiceCapacity hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation hadoop.yarn.client.api.impl.TestAMRMClient hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage hadoop.yarn.sls.appmaster.TestAMSimulator cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/diff-compile-javac-root.txt [324K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/diff-checkstyle-root.txt [17M] hadolint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/diff-patch-hadolint.txt [4.0K] pathlen: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/pathlen.txt [12K] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/diff-patch-pylint.txt [40K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/diff-patch-shellcheck.txt [68K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/diff-patch-shelldocs.txt [12K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/whitespace-eol.txt [9.3M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/whitespace-tabs.txt [1.1M] xml: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/xml.txt [4.0K] findbugs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/branch-findbugs-hadoop-common-project_hadoop-registry-warnings.html [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/branch-findbugs-hadoop-hdds_client.txt [24K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/branch-findbugs-hadoop-hdds_container-service.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/branch-findbugs-hadoop-hdds_framework.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/branch-findbugs-hadoop-hdds_server-scm.txt [12K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/branch-findbugs-hadoop-hdds_tools.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/branch-findbugs-hadoop-ozone_client.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/
[jira] [Created] (YARN-8953) Add CSI adaptor module
Weiwei Yang created YARN-8953: - Summary: Add CSI adaptor module Key: YARN-8953 URL: https://issues.apache.org/jira/browse/YARN-8953 Project: Hadoop YARN Issue Type: Sub-task Reporter: Weiwei Yang Assignee: Weiwei Yang CSI adaptor is a layer between YARN and CSI driver, it transforms YARN internal concepts and boxes them according to CSI protocol. Then forward the call to a CSI driver. The adaptor should support both controller/node/identity services. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8952) [UI2] Improvements in YARN Tools page
Akhil PB created YARN-8952: -- Summary: [UI2] Improvements in YARN Tools page Key: YARN-8952 URL: https://issues.apache.org/jira/browse/YARN-8952 Project: Hadoop YARN Issue Type: Bug Components: yarn-ui-v2 Reporter: Akhil PB Assignee: Akhil PB This JIRA is to improve YARN UI2 tools page layout. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8951) Defining default queue placement rule in allocations file with create="false" throws an NPE
Szilard Nemeth created YARN-8951:
Summary: Defining default queue placement rule in allocations file
with create="false" throws an NPE
Key: YARN-8951
URL: https://issues.apache.org/jira/browse/YARN-8951
Project: Hadoop YARN
Issue Type: Bug
Reporter: Szilard Nemeth
Assignee: Szilard Nemeth
Attachments: default-placement-rule-with-create-false.patch
If the default queue placement rule is defined with {{create="false"}} and a
scheduling request is created for queue {{"root.default"}}, then
{{FairScheduler#assignToQueue}} throws an NPE, while trying to construct an
error message in the catch block of {{IllegalStateException}}, relying on the
fact that the {{rmApp}} is not null but it is.
Example of such a config file:
{code:java}
1024mb,0vcores
{code}
This is suspicious, as there are some null checks for {{rmApp}} in the same
method.
Not sure if this is a special case for the tests or it is reproducable in a
cluster, this needs further investigation.
In any case, it's not good that we try to dereference the {{rmApp}} that is
null.
On the other hand, I'm not sure if the default queue placement rule with
{{create="false"}} makes sense at all. Looking at the documentation
([https://hadoop.apache.org/docs/r3.1.0/hadoop-yarn/hadoop-yarn-site/FairScheduler.html):]
{quote}default: the app is placed into the queue specified in the ‘queue’
attribute of the default rule. *If ‘queue’ attribute is not specified, the app
is placed into ‘root.default’ queue.*
A queuePlacementPolicy element: which contains a list of rule elements that
tell the scheduler how to place incoming apps into queues. Rules are applied in
the order that they are listed. Rules may take arguments. *All rules accept the
“create” argument, which indicates whether the rule can create a new queue.
“Create” defaults to true; if set to false and the rule would place the app in
a queue that is not configured in the allocations file, we continue on to the
next rule.* The last rule must be one that can never issue a continue
{quote}
In this case, the rule has the queue property suppressed so the apps should be
placed to the {{root.default}} queue (which is an undefined queue according to
the config file), and create is false, meaning that the queue {{root.default}}
cannot be created at all.
*This seems to be a case of an invalid queue configuration file for me.*
[~jlowe], [~leftnoteasy]: What is your take on this?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Apache Hadoop 2.9.2 release plan
Hi all, We have released Apache Hadoop 2.9.1 on May 5, 2018. To further improve the quality of the release, I'm planning to release 2.9.2. The focus of 2.9.2 will be fixing blocker/critical bugs and other enhancements. So far there are 189 JIRAs [1] have fixed in branch-2.9. There are no blocker/critical bugs targeted for 2.9.2 now [2], so I plan to cut branch-2.9.2 and create RC by the end of this week. If someone want to be a release manager for 2.9.2. You can do it instead of me, and I'll help you. Please feel free to share your thoughts. Thanks, Akira Ajisaka [1] https://s.apache.org/2.9.2-fixed-jiras [2] https://s.apache.org/2.9.2-targeted-jiras - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
