[jira] [Created] (YARN-6214) NullPointer Exception while querying timeline server API
Ravi Teja Ch N V created YARN-6214: -- Summary: NullPointer Exception while querying timeline server API Key: YARN-6214 URL: https://issues.apache.org/jira/browse/YARN-6214 Project: Hadoop YARN Issue Type: Bug Components: timelineserver Affects Versions: 2.7.1 Reporter: Ravi Teja Ch N V The apps API works fine and give all applications, including Mapreduce and Tez http://:8188/ws/v1/applicationhistory/apps But when queried with application types with these APIs, it fails with NullpointerException. http://:8188/ws/v1/applicationhistory/apps?applicationTypes=TEZ http://:8188/ws/v1/applicationhistory/apps?applicationTypes=MAPREDUCE NullPointerExceptionjava.lang.NullPointerException Blocked on this issue as we are not able to run analytics on the tez job counters on the prod jobs. Timeline Logs: |2017-02-22 11:47:57,183 WARN webapp.GenericExceptionHandler (GenericExceptionHandler.java:toResponse(98)) - INTERNAL_SERVER_ERROR java.lang.NullPointerException at org.apache.hadoop.yarn.server.webapp.WebServices.getApps(WebServices.java:195) at org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices.getApps(AHSWebServices.java:96) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288) Complete stacktrace: http://pastebin.com/bRgxVabf -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-6213) Failure handling and retry for performFailover in RetryInvocationHandler
Botong Huang created YARN-6213:
--
Summary: Failure handling and retry for performFailover in
RetryInvocationHandler
Key: YARN-6213
URL: https://issues.apache.org/jira/browse/YARN-6213
Project: Hadoop YARN
Issue Type: Bug
Reporter: Botong Huang
Assignee: Botong Huang
Priority: Minor
In {{RetryInvocationHandler}}, when the method invocation fails, we reply on
{{FailoverProxyProvider}} to performFailover and get a new proxy, so that we
can retry the method invocation.
However, the performFailover and get new proxy itself might fail (throw
exception or return null proxy). This is not handled properly currently, we end
up throwing the exception out of the while loop. Instead, we should catch the
exception (or check for null proxy) and retry performFailover again, until the
max fail over count reaches the maximum.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-6212) NodeManager metrics returning wrong negative values
Abhishek Shivanna created YARN-6212:
---
Summary: NodeManager metrics returning wrong negative values
Key: YARN-6212
URL: https://issues.apache.org/jira/browse/YARN-6212
Project: Hadoop YARN
Issue Type: Bug
Components: metrics
Affects Versions: 2.7.3
Reporter: Abhishek Shivanna
It looks like the metrics returned by the NodeManager have negative values for
metrics that never should be negative. Here is an output form NM endpoint
{noformat}
/jmx?qry=Hadoop:service=NodeManager,name=NodeManagerMetrics
{noformat}
{noformat}
{
"beans" : [ {
"name" : "Hadoop:service=NodeManager,name=NodeManagerMetrics",
"modelerType" : "NodeManagerMetrics",
"tag.Context" : "yarn",
"tag.Hostname" : "",
"ContainersLaunched" : 707,
"ContainersCompleted" : 9,
"ContainersFailed" : 124,
"ContainersKilled" : 579,
"ContainersIniting" : 0,
"ContainersRunning" : 19,
"AllocatedGB" : -26,
"AllocatedContainers" : -5,
"AvailableGB" : 252,
"AllocatedVCores" : -5,
"AvailableVCores" : 101,
"ContainerLaunchDurationNumOps" : 718,
"ContainerLaunchDurationAvgTime" : 18.0
} ]
}
{noformat}
Is there any circumstance under which the value for AllocatedGB,
AllocatedContainers and AllocatedVCores go below 0?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/ppc64le
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/236/ [Feb 20, 2017 4:21:46 PM] (stevel) HADOOP-14081. S3A: Consider avoiding array copy in S3ABlockOutputStream [Feb 21, 2017 6:55:37 AM] (varunsaxena) YARN-6159. Documentation changes for TimelineV2Client (Naganarasimha G R [Feb 21, 2017 12:43:19 PM] (vinayakumarb) HDFS-11177. 'storagepolicies -getStoragePolicy' command should accept -1 overall The following subsystems voted -1: compile unit The following subsystems voted -1 but were configured to be filtered/ignored: cc javac The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.hdfs.TestWriteReadStripedFile hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting hadoop.hdfs.web.TestWebHdfsTimeouts hadoop.yarn.server.timeline.TestRollingLevelDB hadoop.yarn.server.timeline.TestTimelineDataManager hadoop.yarn.server.timeline.TestLeveldbTimelineStore hadoop.yarn.server.timeline.webapp.TestTimelineWebServices hadoop.yarn.server.timeline.recovery.TestLeveldbTimelineStateStore hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryServer hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppStarvation hadoop.yarn.server.resourcemanager.TestRMRestart hadoop.yarn.server.TestMiniYarnClusterNodeUtilization hadoop.yarn.server.TestContainerManagerSecurity hadoop.yarn.server.timeline.TestLevelDBCacheTimelineStore hadoop.yarn.server.timeline.TestOverrideTimelineStoreYarnClient hadoop.yarn.server.timeline.TestEntityGroupFSTimelineStore hadoop.yarn.applications.distributedshell.TestDistributedShell hadoop.mapred.TestShuffleHandler hadoop.mapreduce.v2.hs.TestHistoryServerLeveldbStateStoreService Timed out junit tests : org.apache.hadoop.hdfs.TestLeaseRecovery2 org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCache compile: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/236/artifact/out/patch-compile-root.txt [124K] cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/236/artifact/out/patch-compile-root.txt [124K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/236/artifact/out/patch-compile-root.txt [124K] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/236/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [272K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/236/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt [16K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/236/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt [52K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/236/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt [72K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/236/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt [324K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/236/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timeline-pluginstorage.txt [28K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/236/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-applications-distributedshell.txt [260K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/236/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-ui.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/236/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-shuffle.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/236/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-hs.txt [16K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/236/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-nativetask.txt [48K] Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/324/ [Feb 20, 2017 4:21:46 PM] (stevel) HADOOP-14081. S3A: Consider avoiding array copy in S3ABlockOutputStream -1 overall The following subsystems voted -1: asflicense unit The following subsystems voted -1 but were configured to be filtered/ignored: cc javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure hadoop.hdfs.server.datanode.checker.TestThrottledAsyncChecker hadoop.yarn.server.timeline.webapp.TestTimelineWebServices hadoop.yarn.server.TestContainerManagerSecurity hadoop.yarn.server.TestMiniYarnClusterNodeUtilization cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/324/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/324/artifact/out/diff-compile-javac-root.txt [168K] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/324/artifact/out/diff-patch-pylint.txt [20K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/324/artifact/out/diff-patch-shellcheck.txt [24K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/324/artifact/out/diff-patch-shelldocs.txt [12K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/324/artifact/out/whitespace-eol.txt [11M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/324/artifact/out/whitespace-tabs.txt [1.3M] javadoc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/324/artifact/out/diff-javadoc-javadoc-root.txt [2.2M] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/324/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [264K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/324/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt [12K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/324/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt [324K] asflicense: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/324/artifact/out/patch-asflicense-problems.txt [4.0K] Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-6211) Synchronization improvement in move and priority
Bibin A Chundatt created YARN-6211: -- Summary: Synchronization improvement in move and priority Key: YARN-6211 URL: https://issues.apache.org/jira/browse/YARN-6211 Project: Hadoop YARN Issue Type: Bug Reporter: Bibin A Chundatt Assignee: Bibin A Chundatt -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
