[jira] [Commented] (YARN-3767) Yarn Scheduler Load Simulator does not work

2017-09-26 Thread sandeep (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181758#comment-16181758
 ] 

sandeep commented on YARN-3767:
---

Simulator UI throws 404 when we click on simulate or track and throws below 
exception. I have created rumen from my this file and ran sls. it creates some 
metrics in output directory but nothing shows in UI for realtimemetrics metrics 
or any other. Is this a bug ? 

[root@xyz~]# /usr/lib/hadoop/share/hadoop/tools/sls/bin/slsrun.sh 
--input-rumen=/tmp/rumen/jobs-rumen-jh.json --output-dir=/tmp/rumen_output
log4j:WARN No appenders could be found for logger 
(org.apache.hadoop.util.Shell).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more 
info.
Sep 26, 2017 4:39:30 PM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering 
org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver as a 
provider class
Sep 26, 2017 4:39:30 PM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering 
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices as a root 
resource class
Sep 26, 2017 4:39:30 PM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a 
provider class
Sep 26, 2017 4:39:30 PM 
com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM'
Sep 26, 2017 4:39:30 PM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory 
getComponentProvider
INFO: Binding 
org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver to 
GuiceManagedComponentProvider with the scope "Singleton"
Sep 26, 2017 4:39:30 PM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory 
getComponentProvider
INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to 
GuiceManagedComponentProvider with the scope "Singleton"
Sep 26, 2017 4:39:30 PM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory 
getComponentProvider
INFO: Binding 
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices to 
GuiceManagedComponentProvider with the scope "Singleton"
Exception in thread "main" java.lang.NullPointerException
at 
org.apache.hadoop.yarn.sls.utils.SLSUtils.parseNodesFromRumenTrace(SLSUtils.java:70)
at org.apache.hadoop.yarn.sls.SLSRunner.startNM(SLSRunner.java:189)
at org.apache.hadoop.yarn.sls.SLSRunner.start(SLSRunner.java:143)
at org.apache.hadoop.yarn.sls.SLSRunner.main(SLSRunner.java:529)
java.lang.NullPointerException
at 
org.apache.hadoop.yarn.sls.web.SLSWebApp.printPageSimulate(SLSWebApp.java:228)
at 
org.apache.hadoop.yarn.sls.web.SLSWebApp.access$100(SLSWebApp.java:54)
at org.apache.hadoop.yarn.sls.web.SLSWebApp$1.handle(SLSWebApp.java:140)
at 
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:322)
at 
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at 
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at 
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at 
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)



> Yarn Scheduler Load Simulator does not work
> ---
>
> Key: YARN-3767
> URL: https://issues.apache.org/jira/browse/YARN-3767
> Project: Hadoop YARN
>  Issue Type: Bug
>Affects Versions: 2.7.0
> Environment: OS X 10.10.  JDK 1.7
>Reporter: David Kjerrumgaard
>
> Running the SLS, as per the instructions on the web results in a 
> NullPointerException being thrown.
> Steps followed to create error:
> 1) Download Apache Hadoop 2.7.0 tarball from Apache site
> 2) Untar 2.7.0 tarball into /opt directory
> 3) Execute the following command: 
> /opt/hadoop-2.7.0/share/hadoop/tools/sls//bin/slsrun.sh 
> --input-rumen=/opt/hadoop-2.7.0/share/hadoop/tools/sls/sample-data/2jobs2min-rumen-jh.json
>  --output-dir=/tmp
> Results in the following error:
> 15/06/04 10:25:41 INFO rmnode.RMNodeImpl: a2118.smile.com:2 Node Transitioned 
> from NEW to RUNNING
> 15/06/04 10:25:41 INFO capacity.CapacityScheduler: Added node 
> a2118.smile.com:2 clusterResource: 
> 15/06/04 10:25:41 INFO util.RackResolver: Resolved a2115.smile.com to 
> /default-rack
> 15/06/04

[jira] [Commented] (YARN-3767) Yarn Scheduler Load Simulator does not work

2017-03-17 Thread Yufei Gu (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930352#comment-15930352
 ] 

Yufei Gu commented on YARN-3767:


Yes, agree.

> Yarn Scheduler Load Simulator does not work
> ---
>
> Key: YARN-3767
> URL: https://issues.apache.org/jira/browse/YARN-3767
> Project: Hadoop YARN
>  Issue Type: Bug
>Affects Versions: 2.7.0
> Environment: OS X 10.10.  JDK 1.7
>Reporter: David Kjerrumgaard
>
> Running the SLS, as per the instructions on the web results in a 
> NullPointerException being thrown.
> Steps followed to create error:
> 1) Download Apache Hadoop 2.7.0 tarball from Apache site
> 2) Untar 2.7.0 tarball into /opt directory
> 3) Execute the following command: 
> /opt/hadoop-2.7.0/share/hadoop/tools/sls//bin/slsrun.sh 
> --input-rumen=/opt/hadoop-2.7.0/share/hadoop/tools/sls/sample-data/2jobs2min-rumen-jh.json
>  --output-dir=/tmp
> Results in the following error:
> 15/06/04 10:25:41 INFO rmnode.RMNodeImpl: a2118.smile.com:2 Node Transitioned 
> from NEW to RUNNING
> 15/06/04 10:25:41 INFO capacity.CapacityScheduler: Added node 
> a2118.smile.com:2 clusterResource: 
> 15/06/04 10:25:41 INFO util.RackResolver: Resolved a2115.smile.com to 
> /default-rack
> 15/06/04 10:25:41 INFO resourcemanager.ResourceTrackerService: NodeManager 
> from node a2115.smile.com(cmPort: 3 httpPort: 80) registered with capability: 
> , assigned nodeId a2115.smile.com:3
> 15/06/04 10:25:41 INFO rmnode.RMNodeImpl: a2115.smile.com:3 Node Transitioned 
> from NEW to RUNNING
> 15/06/04 10:25:41 INFO capacity.CapacityScheduler: Added node 
> a2115.smile.com:3 clusterResource: 
> Exception in thread "main" java.lang.RuntimeException: 
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:134)
>   at 
> org.apache.hadoop.yarn.sls.SLSRunner.startAMFromRumenTraces(SLSRunner.java:398)
>   at org.apache.hadoop.yarn.sls.SLSRunner.startAM(SLSRunner.java:250)
>   at org.apache.hadoop.yarn.sls.SLSRunner.start(SLSRunner.java:145)
>   at org.apache.hadoop.yarn.sls.SLSRunner.main(SLSRunner.java:528)
> Caused by: java.lang.NullPointerException
>   at 
> java.util.concurrent.ConcurrentHashMap.hash(ConcurrentHashMap.java:333)
>   at 
> java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:988)
>   at 
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:126)
>   ... 4 more



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-3767) Yarn Scheduler Load Simulator does not work

2017-03-17 Thread Carlo Curino (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930347#comment-15930347
 ] 

Carlo Curino commented on YARN-3767:


Makes sense. Is it ok to leave this JIRA closed, since we are following up in 
YARN-5065? (Just trying to spring clean a bit :-))

> Yarn Scheduler Load Simulator does not work
> ---
>
> Key: YARN-3767
> URL: https://issues.apache.org/jira/browse/YARN-3767
> Project: Hadoop YARN
>  Issue Type: Bug
>Affects Versions: 2.7.0
> Environment: OS X 10.10.  JDK 1.7
>Reporter: David Kjerrumgaard
>
> Running the SLS, as per the instructions on the web results in a 
> NullPointerException being thrown.
> Steps followed to create error:
> 1) Download Apache Hadoop 2.7.0 tarball from Apache site
> 2) Untar 2.7.0 tarball into /opt directory
> 3) Execute the following command: 
> /opt/hadoop-2.7.0/share/hadoop/tools/sls//bin/slsrun.sh 
> --input-rumen=/opt/hadoop-2.7.0/share/hadoop/tools/sls/sample-data/2jobs2min-rumen-jh.json
>  --output-dir=/tmp
> Results in the following error:
> 15/06/04 10:25:41 INFO rmnode.RMNodeImpl: a2118.smile.com:2 Node Transitioned 
> from NEW to RUNNING
> 15/06/04 10:25:41 INFO capacity.CapacityScheduler: Added node 
> a2118.smile.com:2 clusterResource: 
> 15/06/04 10:25:41 INFO util.RackResolver: Resolved a2115.smile.com to 
> /default-rack
> 15/06/04 10:25:41 INFO resourcemanager.ResourceTrackerService: NodeManager 
> from node a2115.smile.com(cmPort: 3 httpPort: 80) registered with capability: 
> , assigned nodeId a2115.smile.com:3
> 15/06/04 10:25:41 INFO rmnode.RMNodeImpl: a2115.smile.com:3 Node Transitioned 
> from NEW to RUNNING
> 15/06/04 10:25:41 INFO capacity.CapacityScheduler: Added node 
> a2115.smile.com:3 clusterResource: 
> Exception in thread "main" java.lang.RuntimeException: 
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:134)
>   at 
> org.apache.hadoop.yarn.sls.SLSRunner.startAMFromRumenTraces(SLSRunner.java:398)
>   at org.apache.hadoop.yarn.sls.SLSRunner.startAM(SLSRunner.java:250)
>   at org.apache.hadoop.yarn.sls.SLSRunner.start(SLSRunner.java:145)
>   at org.apache.hadoop.yarn.sls.SLSRunner.main(SLSRunner.java:528)
> Caused by: java.lang.NullPointerException
>   at 
> java.util.concurrent.ConcurrentHashMap.hash(ConcurrentHashMap.java:333)
>   at 
> java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:988)
>   at 
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:126)
>   ... 4 more



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-3767) Yarn Scheduler Load Simulator does not work

2017-03-16 Thread Yufei Gu (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15928579#comment-15928579
 ] 

Yufei Gu commented on YARN-3767:


SLS is broken in some ways. We are trying to fix it in Umbrella JIRA YARN-5065.

> Yarn Scheduler Load Simulator does not work
> ---
>
> Key: YARN-3767
> URL: https://issues.apache.org/jira/browse/YARN-3767
> Project: Hadoop YARN
>  Issue Type: Bug
>Affects Versions: 2.7.0
> Environment: OS X 10.10.  JDK 1.7
>Reporter: David Kjerrumgaard
>
> Running the SLS, as per the instructions on the web results in a 
> NullPointerException being thrown.
> Steps followed to create error:
> 1) Download Apache Hadoop 2.7.0 tarball from Apache site
> 2) Untar 2.7.0 tarball into /opt directory
> 3) Execute the following command: 
> /opt/hadoop-2.7.0/share/hadoop/tools/sls//bin/slsrun.sh 
> --input-rumen=/opt/hadoop-2.7.0/share/hadoop/tools/sls/sample-data/2jobs2min-rumen-jh.json
>  --output-dir=/tmp
> Results in the following error:
> 15/06/04 10:25:41 INFO rmnode.RMNodeImpl: a2118.smile.com:2 Node Transitioned 
> from NEW to RUNNING
> 15/06/04 10:25:41 INFO capacity.CapacityScheduler: Added node 
> a2118.smile.com:2 clusterResource: 
> 15/06/04 10:25:41 INFO util.RackResolver: Resolved a2115.smile.com to 
> /default-rack
> 15/06/04 10:25:41 INFO resourcemanager.ResourceTrackerService: NodeManager 
> from node a2115.smile.com(cmPort: 3 httpPort: 80) registered with capability: 
> , assigned nodeId a2115.smile.com:3
> 15/06/04 10:25:41 INFO rmnode.RMNodeImpl: a2115.smile.com:3 Node Transitioned 
> from NEW to RUNNING
> 15/06/04 10:25:41 INFO capacity.CapacityScheduler: Added node 
> a2115.smile.com:3 clusterResource: 
> Exception in thread "main" java.lang.RuntimeException: 
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:134)
>   at 
> org.apache.hadoop.yarn.sls.SLSRunner.startAMFromRumenTraces(SLSRunner.java:398)
>   at org.apache.hadoop.yarn.sls.SLSRunner.startAM(SLSRunner.java:250)
>   at org.apache.hadoop.yarn.sls.SLSRunner.start(SLSRunner.java:145)
>   at org.apache.hadoop.yarn.sls.SLSRunner.main(SLSRunner.java:528)
> Caused by: java.lang.NullPointerException
>   at 
> java.util.concurrent.ConcurrentHashMap.hash(ConcurrentHashMap.java:333)
>   at 
> java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:988)
>   at 
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:126)
>   ... 4 more



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-3767) Yarn Scheduler Load Simulator does not work

2015-06-04 Thread Varun Saxena (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573269#comment-14573269
 ] 

Varun Saxena commented on YARN-3767:


This belongs to YARN.

 Yarn Scheduler Load Simulator does not work
 ---

 Key: YARN-3767
 URL: https://issues.apache.org/jira/browse/YARN-3767
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 2.7.0
 Environment: OS X 10.10.  JDK 1.7
Reporter: David Kjerrumgaard
Assignee: Varun Saxena

 Running the SLS, as per the instructions on the web results in a 
 NullPointerException being thrown.
 Steps followed to create error:
 1) Download Apache Hadoop 2.7.0 tarball from Apache site
 2) Untar 2.7.0 tarball into /opt directory
 3) Execute the following command: 
 /opt/hadoop-2.7.0/share/hadoop/tools/sls//bin/slsrun.sh 
 --input-rumen=/opt/hadoop-2.7.0/share/hadoop/tools/sls/sample-data/2jobs2min-rumen-jh.json
  --output-dir=/tmp
 Results in the following error:
 15/06/04 10:25:41 INFO rmnode.RMNodeImpl: a2118.smile.com:2 Node Transitioned 
 from NEW to RUNNING
 15/06/04 10:25:41 INFO capacity.CapacityScheduler: Added node 
 a2118.smile.com:2 clusterResource: memory:30720, vCores:30
 15/06/04 10:25:41 INFO util.RackResolver: Resolved a2115.smile.com to 
 /default-rack
 15/06/04 10:25:41 INFO resourcemanager.ResourceTrackerService: NodeManager 
 from node a2115.smile.com(cmPort: 3 httpPort: 80) registered with capability: 
 memory:10240, vCores:10, assigned nodeId a2115.smile.com:3
 15/06/04 10:25:41 INFO rmnode.RMNodeImpl: a2115.smile.com:3 Node Transitioned 
 from NEW to RUNNING
 15/06/04 10:25:41 INFO capacity.CapacityScheduler: Added node 
 a2115.smile.com:3 clusterResource: memory:40960, vCores:40
 Exception in thread main java.lang.RuntimeException: 
 java.lang.NullPointerException
   at 
 org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:134)
   at 
 org.apache.hadoop.yarn.sls.SLSRunner.startAMFromRumenTraces(SLSRunner.java:398)
   at org.apache.hadoop.yarn.sls.SLSRunner.startAM(SLSRunner.java:250)
   at org.apache.hadoop.yarn.sls.SLSRunner.start(SLSRunner.java:145)
   at org.apache.hadoop.yarn.sls.SLSRunner.main(SLSRunner.java:528)
 Caused by: java.lang.NullPointerException
   at 
 java.util.concurrent.ConcurrentHashMap.hash(ConcurrentHashMap.java:333)
   at 
 java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:988)
   at 
 org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:126)
   ... 4 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (YARN-3767) Yarn Scheduler Load Simulator does not work

2015-06-04 Thread Varun Saxena (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573273#comment-14573273
 ] 

Varun Saxena commented on YARN-3767:


Yes it will work if you copy {{sls-runner.xml}} to {{etc/hadoop}}. This is 
mentioned in documentation as well. Refer to : 
http://hadoop.apache.org/docs/r2.4.1/hadoop-sls/SchedulerLoadSimulator.html#Step_1:_Configure_Hadoop_and_the_simulator

It mentions Before we start, make sure Hadoop and the simulator are configured 
well. All configuration files for Hadoop and the simulator should be placed in 
directory $HADOOP_ROOT/etc/hadoop, where the ResourceManager and Yarn scheduler 
load their configurations. Directory 
$HADOOP_ROOT/share/hadoop/tools/sls/sample-conf/ provides several example 
configurations, that can be used to start a demo.

 Yarn Scheduler Load Simulator does not work
 ---

 Key: YARN-3767
 URL: https://issues.apache.org/jira/browse/YARN-3767
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 2.7.0
 Environment: OS X 10.10.  JDK 1.7
Reporter: David Kjerrumgaard
Assignee: Varun Saxena

 Running the SLS, as per the instructions on the web results in a 
 NullPointerException being thrown.
 Steps followed to create error:
 1) Download Apache Hadoop 2.7.0 tarball from Apache site
 2) Untar 2.7.0 tarball into /opt directory
 3) Execute the following command: 
 /opt/hadoop-2.7.0/share/hadoop/tools/sls//bin/slsrun.sh 
 --input-rumen=/opt/hadoop-2.7.0/share/hadoop/tools/sls/sample-data/2jobs2min-rumen-jh.json
  --output-dir=/tmp
 Results in the following error:
 15/06/04 10:25:41 INFO rmnode.RMNodeImpl: a2118.smile.com:2 Node Transitioned 
 from NEW to RUNNING
 15/06/04 10:25:41 INFO capacity.CapacityScheduler: Added node 
 a2118.smile.com:2 clusterResource: memory:30720, vCores:30
 15/06/04 10:25:41 INFO util.RackResolver: Resolved a2115.smile.com to 
 /default-rack
 15/06/04 10:25:41 INFO resourcemanager.ResourceTrackerService: NodeManager 
 from node a2115.smile.com(cmPort: 3 httpPort: 80) registered with capability: 
 memory:10240, vCores:10, assigned nodeId a2115.smile.com:3
 15/06/04 10:25:41 INFO rmnode.RMNodeImpl: a2115.smile.com:3 Node Transitioned 
 from NEW to RUNNING
 15/06/04 10:25:41 INFO capacity.CapacityScheduler: Added node 
 a2115.smile.com:3 clusterResource: memory:40960, vCores:40
 Exception in thread main java.lang.RuntimeException: 
 java.lang.NullPointerException
   at 
 org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:134)
   at 
 org.apache.hadoop.yarn.sls.SLSRunner.startAMFromRumenTraces(SLSRunner.java:398)
   at org.apache.hadoop.yarn.sls.SLSRunner.startAM(SLSRunner.java:250)
   at org.apache.hadoop.yarn.sls.SLSRunner.start(SLSRunner.java:145)
   at org.apache.hadoop.yarn.sls.SLSRunner.main(SLSRunner.java:528)
 Caused by: java.lang.NullPointerException
   at 
 java.util.concurrent.ConcurrentHashMap.hash(ConcurrentHashMap.java:333)
   at 
 java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:988)
   at 
 org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:126)
   ... 4 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)