[jira] [Created] (YARN-6836) [ATS1/1.5] IllegalStateException while posting entity into TimelineServer
Rohith Sharma K S created YARN-6836:
---
Summary: [ATS1/1.5] IllegalStateException while posting entity
into TimelineServer
Key: YARN-6836
URL: https://issues.apache.org/jira/browse/YARN-6836
Project: Hadoop YARN
Issue Type: Bug
Components: timelineclient
Reporter: Rohith Sharma K S
It is observed that timelinecilent unable to post entities to timelineserver
and throw IllegalStateException.
{noformat}
2017-07-13 06:42:15,376 ERROR metrics.SystemMetricsPublisher
(SystemMetricsPublisher.java:putEntity(549)) - Error when publishing entity
[YARN_APPLICATION,application_1499926197597_0002]
com.sun.jersey.api.client.ClientHandlerException:
java.lang.IllegalStateException: connect in progress
at
com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
at
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$TimelineJerseyRetryFilter$1.run(TimelineClientImpl.java:237)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-6835) Remove runningContainers from ContainerScheduler
Arun Suresh created YARN-6835: - Summary: Remove runningContainers from ContainerScheduler Key: YARN-6835 URL: https://issues.apache.org/jira/browse/YARN-6835 Project: Hadoop YARN Issue Type: Sub-task Reporter: Arun Suresh Assignee: Arun Suresh The *runningContainers* collection contains both running containers as well as container that are scheduled but not yet started. We can remove this data structure completely by introducing a *LAUNCHING* container state. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/467/ [Jul 16, 2017 10:59:34 AM] (aengineer) HDFS-11786. Add support to make copyFromLocal multi threaded. [Jul 17, 2017 1:05:15 AM] (sunilg) MAPREDUCE-6889. Add Job#close API to shutdown MR client services. [Jul 17, 2017 2:27:55 AM] (jitendra) HADOOP-14640. Azure: Support affinity for service running on localhost -1 overall The following subsystems voted -1: findbugs unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: FindBugs : module:hadoop-hdfs-project/hadoop-hdfs-client Possible exposure of partially initialized object in org.apache.hadoop.hdfs.DFSClient.initThreadsNumForStripedReads(int) At DFSClient.java:object in org.apache.hadoop.hdfs.DFSClient.initThreadsNumForStripedReads(int) At DFSClient.java:[line 2888] org.apache.hadoop.hdfs.server.protocol.SlowDiskReports.equals(Object) makes inefficient use of keySet iterator instead of entrySet iterator At SlowDiskReports.java:keySet iterator instead of entrySet iterator At SlowDiskReports.java:[line 105] FindBugs : module:hadoop-hdfs-project/hadoop-hdfs Possible null pointer dereference in org.apache.hadoop.hdfs.qjournal.server.JournalNode.getJournalsStatus() due to return value of called method Dereferenced at JournalNode.java:org.apache.hadoop.hdfs.qjournal.server.JournalNode.getJournalsStatus() due to return value of called method Dereferenced at JournalNode.java:[line 302] org.apache.hadoop.hdfs.server.common.HdfsServerConstants$StartupOption.setClusterId(String) unconditionally sets the field clusterId At HdfsServerConstants.java:clusterId At HdfsServerConstants.java:[line 193] org.apache.hadoop.hdfs.server.common.HdfsServerConstants$StartupOption.setForce(int) unconditionally sets the field force At HdfsServerConstants.java:force At HdfsServerConstants.java:[line 217] org.apache.hadoop.hdfs.server.common.HdfsServerConstants$StartupOption.setForceFormat(boolean) unconditionally sets the field isForceFormat At HdfsServerConstants.java:isForceFormat At HdfsServerConstants.java:[line 229] org.apache.hadoop.hdfs.server.common.HdfsServerConstants$StartupOption.setInteractiveFormat(boolean) unconditionally sets the field isInteractiveFormat At HdfsServerConstants.java:isInteractiveFormat At HdfsServerConstants.java:[line 237] Possible null pointer dereference in org.apache.hadoop.hdfs.server.datanode.DataStorage.linkBlocksHelper(File, File, int, HardLink, boolean, File, List) due to return value of called method Dereferenced at DataStorage.java:org.apache.hadoop.hdfs.server.datanode.DataStorage.linkBlocksHelper(File, File, int, HardLink, boolean, File, List) due to return value of called method Dereferenced at DataStorage.java:[line 1339] Possible null pointer dereference in org.apache.hadoop.hdfs.server.namenode.NNStorageRetentionManager.purgeOldLegacyOIVImages(String, long) due to return value of called method Dereferenced at NNStorageRetentionManager.java:org.apache.hadoop.hdfs.server.namenode.NNStorageRetentionManager.purgeOldLegacyOIVImages(String, long) due to return value of called method Dereferenced at NNStorageRetentionManager.java:[line 258] Possible null pointer dereference in org.apache.hadoop.hdfs.server.namenode.NNUpgradeUtil$1.visitFile(Path, BasicFileAttributes) due to return value of called method Dereferenced at NNUpgradeUtil.java:org.apache.hadoop.hdfs.server.namenode.NNUpgradeUtil$1.visitFile(Path, BasicFileAttributes) due to return value of called method Dereferenced at NNUpgradeUtil.java:[line 133] Useless condition:argv.length >= 1 at this point At DFSAdmin.java:[line 2085] Useless condition:numBlocks == -1 at this point At ImageLoaderCurrent.java:[line 727] FindBugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Useless object stored in variable removedNullContainers of method org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) At NodeStatusUpdaterImpl.java:removedNullContainers of method org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) At NodeStatusUpdaterImpl.java:[line 642] org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache() makes inefficient use of keySet iterator instead of entrySet iterator At NodeStatusUpdaterImpl.java:keySet iterator instead of entrySet iterator At NodeStatusUpdaterImpl.java:[line 719] Hard coded reference to an absolute pathname in org.apache.hadoop.yarn.server
[jira] [Created] (YARN-6834) A container request with only racks specified and relax locality set to false is never honoured
Muhammad Samir Khan created YARN-6834: - Summary: A container request with only racks specified and relax locality set to false is never honoured Key: YARN-6834 URL: https://issues.apache.org/jira/browse/YARN-6834 Project: Hadoop YARN Issue Type: Bug Components: capacity scheduler Reporter: Muhammad Samir Khan A patch for a unit test is attached to reproduce the issue. It creates a container request with only racks specified (nodes=null) and relax locality set to false. With the node-locality-delay conf set appropriately, we wait indefinitely for a container allocation and the test will timeout. My understanding of what causes this issue is as follows. The RegularContainerAllocator delays a rack local allocation based on the node-locality-delay parameter. This delay is based on missed opportunities. However, the corresponding off-switch request is skipped but does not count towards a missed opportunity (because relax locality is set to false). So the allocator waits indefinitely. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-6832) Tests use assertTrue(....equals(...)) instead of assertEquals()
Daniel Templeton created YARN-6832: -- Summary: Tests use assertTrue(equals(...)) instead of assertEquals() Key: YARN-6832 URL: https://issues.apache.org/jira/browse/YARN-6832 Project: Hadoop YARN Issue Type: Improvement Components: test Affects Versions: 3.0.0-alpha4, 2.8.1 Reporter: Daniel Templeton Assignee: Daniel Templeton Priority: Minor -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-6831) Miscellaneous refactoring changes of ContainScheduler
Haibo Chen created YARN-6831: Summary: Miscellaneous refactoring changes of ContainScheduler Key: YARN-6831 URL: https://issues.apache.org/jira/browse/YARN-6831 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Reporter: Haibo Chen Assignee: Haibo Chen -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-6830) Support quoted strings for environment variables
Shane Kumpf created YARN-6830:
-
Summary: Support quoted strings for environment variables
Key: YARN-6830
URL: https://issues.apache.org/jira/browse/YARN-6830
Project: Hadoop YARN
Issue Type: Bug
Reporter: Shane Kumpf
There are cases where it is necessary to allow for quoted string literals
within environment variables values when passed via the yarn command line
interface.
For example, consider the follow environment variables for a MR map task.
{{MODE=bar}}
{{IMAGE_NAME=foo}}
{{MOUNTS=/tmp/foo,/tmp/bar}}
When running the MR job, these environment variables are supplied as a comma
delimited string.
{{-Dmapreduce.map.env="MODE=bar,IMAGE_NAME=foo,MOUNTS=/tmp/foo,/tmp/bar"}}
In this case, {{MOUNTS}} will be parsed and added to the task environment as
{{MOUNTS=/tmp/foo}}. Any attempts to quote the embedded comma separated value
results in quote characters becoming part of the value, and parsing still
breaks down at the comma.
This issue is to allow for quoting the comma separated value (escaped double or
single quote). This was mentioned on YARN-4595 and will impact YARN-5534 as
well.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
