[jira] [Updated] (MYRIAD-125) Myriad /config API issue for YARN_NODEMANAGER_OPTS param when RM's hostname is passed.
[
https://issues.apache.org/jira/browse/MYRIAD-125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarjeet Singh updated MYRIAD-125:
-
Description:
The issue is with YARN_NODEMANAGER_OPTS param that has
-Dyarn.resourcemanager.hostname=host-name keep appending on each offer
created by myriad to yarnEnvironment.
Found this piece of code that has the issue.
In TaskFactory.java:
code
String rmHostName = System.getProperty(YARN_RESOURCEMANAGER_HOSTNAME);
if (rmHostName != null !rmHostName.isEmpty()) {
String nmOpts =
nmTaskConfig.getYarnEnvironment().get(YARN_NODEMANAGER_OPTS_KEY);
if (nmOpts == null) {
nmOpts = ;
}
nmOpts += + -D + YARN_RESOURCEMANAGER_HOSTNAME + = +
rmHostName;
nmTaskConfig.getYarnEnvironment().put(YARN_NODEMANAGER_OPTS_KEY, nmOpts);
LOGGER.info(YARN_RESOURCEMANAGER_HOSTNAME + is set to +
rmHostName +
via YARN_RESOURCEMANAGER_OPTS. Passing it into
YARN_NODEMANAGER_OPTS.);
}
/code
Observed this when I tried to start RM from marathon, and HTTP GET the /config
API and checked YARN_NODEMANAGER_OPTS between 10-15 min interval .
Here is what I observed:
yarnEnvironment: {
YARN_HOME: /opt/mapr/hadoop/hadoop-2.7.0/,
YARN_NODEMANAGER_OPTS: -Dnodemanager.resource.io-spindles=4.0
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
},
After 10-15 min of being idle:
yarnEnvironment: {
YARN_HOME: /opt/mapr/hadoop/hadoop-2.7.0/,
YARN_NODEMANAGER_OPTS: -Dnodemanager.resource.io-spindles=4.0
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
},
Let me know if need any addiotinal detail about the issue? Thanks.
FYI, this should be easy to reproducible by just starting Myriad and check
/config API output between 10-15 min interval.
was:
The issue is with YARN_NODEMANAGER_OPTS param that has
-Dyarn.resourcemanager.hostname=host-name keep appending on each offer
created by myriad to yarnEnvironment.
Found this piece of code that has the issue.
In TaskFactory.java:
code
String rmHostName = System.getProperty(YARN_RESOURCEMANAGER_HOSTNAME);
if (rmHostName != null !rmHostName.isEmpty()) {
String nmOpts =
nmTaskConfig.getYarnEnvironment().get(YARN_NODEMANAGER_OPTS_KEY);
if (nmOpts == null) {
nmOpts = ;
}
nmOpts += + -D + YARN_RESOURCEMANAGER_HOSTNAME + = +
rmHostName;
nmTaskConfig.getYarnEnvironment().put(YARN_NODEMANAGER_OPTS_KEY, nmOpts);
LOGGER.info(YARN_RESOURCEMANAGER_HOSTNAME + is set to +
rmHostName +
via YARN_RESOURCEMANAGER_OPTS. Passing it into
YARN_NODEMANAGER_OPTS.);
}
Observed this when I tried to start RM from marathon, and HTTP GET the /config
API and checked YARN_NODEMANAGER_OPTS between 10-15 min interval .
/code
Here is what I observed:
yarnEnvironment: {
YARN_HOME: /opt/mapr/hadoop/hadoop-2.7.0/,
YARN_NODEMANAGER_OPTS: -Dnodemanager.resource.io-spindles=4.0
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
},
After 10-15 min of being idle:
yarnEnvironment: {
YARN_HOME: /opt/mapr/hadoop/hadoop-2.7.0/,
YARN_NODEMANAGER_OPTS: -Dnodemanager.resource.io-spindles=4.0
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
[jira] [Created] (MYRIAD-125) Myriad /config API issue for YARN_NODEMANAGER_OPTS param when RM's hostname is passed.
Sarjeet Singh created MYRIAD-125:
Summary: Myriad /config API issue for YARN_NODEMANAGER_OPTS
param when RM's hostname is passed.
Key: MYRIAD-125
URL: https://issues.apache.org/jira/browse/MYRIAD-125
Project: Myriad
Issue Type: Bug
Components: Scheduler
Affects Versions: Myriad 0.1.0
Reporter: Sarjeet Singh
The issue is with YARN_NODEMANAGER_OPTS param that has
-Dyarn.resourcemanager.hostname=host-name keep appending on each offer
created by myriad to yarnEnvironment.
Found this piece of code that has the issue.
In TaskFactory.java:
code
String rmHostName = System.getProperty(YARN_RESOURCEMANAGER_HOSTNAME);
if (rmHostName != null !rmHostName.isEmpty()) {
String nmOpts =
nmTaskConfig.getYarnEnvironment().get(YARN_NODEMANAGER_OPTS_KEY);
if (nmOpts == null) {
nmOpts = ;
}
nmOpts += + -D + YARN_RESOURCEMANAGER_HOSTNAME + = +
rmHostName;
nmTaskConfig.getYarnEnvironment().put(YARN_NODEMANAGER_OPTS_KEY, nmOpts);
LOGGER.info(YARN_RESOURCEMANAGER_HOSTNAME + is set to +
rmHostName +
via YARN_RESOURCEMANAGER_OPTS. Passing it into
YARN_NODEMANAGER_OPTS.);
}
Observed this when I tried to start RM from marathon, and HTTP GET the /config
API and checked YARN_NODEMANAGER_OPTS between 10-15 min interval .
/code
Here is what I observed:
yarnEnvironment: {
YARN_HOME: /opt/mapr/hadoop/hadoop-2.7.0/,
YARN_NODEMANAGER_OPTS: -Dnodemanager.resource.io-spindles=4.0
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
},
After 10-15 min of being idle:
yarnEnvironment: {
YARN_HOME: /opt/mapr/hadoop/hadoop-2.7.0/,
YARN_NODEMANAGER_OPTS: -Dnodemanager.resource.io-spindles=4.0
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
-Dyarn.resourcemanager.hostname=rm.marathon.mesos
},
Let me know if need any addiotinal detail about the issue? Thanks.
FYI, this should be easy to reproducible by just starting Myriad and check
/config API output between 10-15 min interval.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Myriad 0.1 release scope
Ok, I was going off https://github.com/mesos/myriad/blob/phase1/docs/myriad-configuration.md I will try it. John On Tue, Aug 18, 2015 at 3:40 PM, yuliya Feldman yufeld...@yahoo.com.invalid wrote: You actually do not need to rebuild even today - just keep this file in hadoop config directory that is on the classpath: like .../etc/hadoop From: John Omernik j...@omernik.com To: dev@myriad.incubator.apache.org Sent: Tuesday, August 18, 2015 1:35 PM Subject: Re: Myriad 0.1 release scope On the release scope, will having the myriad configuration file exist outside the jar (i.e. you can change configuration without rebuilding) be part of the .1 release scope? On Mon, Aug 10, 2015 at 10:01 PM, Santosh Marella smare...@maprtech.com wrote: Hello All, I've merged the FGS changes into phase1. Built and tested both coarse grained scaling and fine grained scaling, UI on a 4 node cluster. If anyone finds things are not working as expected, please let me know. Thanks, Santosh On Fri, Aug 7, 2015 at 10:46 AM, Santosh Marella smare...@maprtech.com wrote: Hello guys, I propose merging FGS into phase1. As I said before, I think it's at a point where the functionality works reasonably well. Any future improvements/fixes/UI changes can be done via separate JIRAs. Unless there are any major concerns, I'd like to merge FGS into phase1 *EOD Monday* (PDT). Thanks, Santosh On Wed, Aug 5, 2015 at 8:16 PM, Santosh Marella smare...@maprtech.com wrote: I feel FGS is very close to making it into 0.1. PR 116 addresses moving to hadoop 2.7 and making FGS and CGS coexist. This PR was recently reviewed by Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and 2.7.x clusters and it seemed to have worked as expected. Unless there are more reviews/feedback, it can be merged into issue_14. Once PR 116 is merged into issue_14, issue_14 can be merged into phase1. Thanks, Santosh On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon a...@mesosphere.io wrote: We do have a JIRA 0.1.0 fix version field, but none of our issues use it yet. I think the goal was just to take what we have and make it work under Apache infrastructure, then vote on that for 0.1.0. Although other features like HA or FGS would be great, let's try to get our first Apache release out ASAP. We can create 0.1.1 or 0.2.0 fix versions for subsequent releases with other issues/features. Roadmap would be great. (I'm just summarizing what we discussed a month or two ago. Feel free to correct me or disagree with this approach.) On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade swapnil.daing...@gmail.com wrote: Hi all, Was wondering what would be the scope for the Myriad 0.1 release. It would be nice to have a roadmap page somewhere and target features to releases (JIRA 'fix version' field perhaps) Regards Swapnil
[jira] [Created] (MYRIAD-126) Myriad initialization fails when 'nmInstances' are not specified in config .yml
Santosh Marella created MYRIAD-126: -- Summary: Myriad initialization fails when 'nmInstances' are not specified in config .yml Key: MYRIAD-126 URL: https://issues.apache.org/jira/browse/MYRIAD-126 Project: Myriad Issue Type: Bug Components: Scheduler Reporter: Santosh Marella Priority: Minor 15/08/18 17:00:28 INFO mortbay.log: Started SelectChannelConnector@0.0.0.0:8192 15/08/18 17:00:28 INFO myriad.Main: Initializing HealthChecks 15/08/18 17:00:28 INFO myriad.Main: Initializing Profiles 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile tiny with CPU: 1 and Memory: 4096 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile small with CPU: 2 and Memory: 8192 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile medium with CPU: 4 and Memory: 16384 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile large with CPU: 8 and Memory: 32768 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile huge with CPU: 12 and Memory: 49152 15/08/18 17:00:28 INFO myriad.Main: Validating nmInstances.. 15/08/18 17:00:28 INFO service.AbstractService: Service org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler failed in state INITED; cause: java.lang.RuntimeException: Failed to initialize myriad java.lang.RuntimeException: Failed to initialize myriad at com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor.init(MyriadInitializationInterceptor.java:35) at com.ebay.myriad.scheduler.yarn.interceptor.CompositeInterceptor.init(CompositeInterceptor.java:76) at com.ebay.myriad.scheduler.yarn.MyriadFairScheduler.serviceInit(MyriadFairScheduler.java:50) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:570) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:997) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:262) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1220) Caused by: java.lang.NullPointerException at com.ebay.myriad.Main.validateNMInstances(Main.java:166) at com.ebay.myriad.Main.run(Main.java:98) at com.ebay.myriad.Main.initialize(Main.java:80) at com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor.init(MyriadInitializationInterceptor.java:32) ... 10 more 15/08/18 17:00:28 INFO service.AbstractService: Service RMActiveServices failed in state INITED; cause: java.lang.RuntimeException: Failed to initialize myriad java.lang.RuntimeException: Failed to initialize myriad at com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor.init(MyriadInitializationInterceptor.java:35) at com.ebay.myriad.scheduler.yarn.interceptor.CompositeInterceptor.init(CompositeInterceptor.java:76) at com.ebay.myriad.scheduler.yarn.MyriadFairScheduler.serviceInit(MyriadFairScheduler.java:50) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:570) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:997) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:262) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1220) Caused by: java.lang.NullPointerException at com.ebay.myriad.Main.validateNMInstances(Main.java:166) at com.ebay.myriad.Main.run(Main.java:98) at com.ebay.myriad.Main.initialize(Main.java:80) at com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor.init(MyriadInitializationInterceptor.java:32) ... 10 more 15/08/18 17:00:28 INFO impl.MetricsSystemImpl: Stopping ResourceManager metrics system... -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Myriad 0.1 release scope
Ok, so I tried the remote distribution of the Myriad per the docs, I guess,it could probably use some information related to how to run resource manager if it's in the tar.gz. Perhaps an example marathon json. I am playing with it now to figure it out. On Tue, Aug 18, 2015 at 3:48 PM, yuliya Feldman yufeld...@yahoo.com.invalid wrote: mesos/myriad is the right one so far From: John Omernik j...@omernik.com To: dev@myriad.incubator.apache.org; yuliya Feldman yufeld...@yahoo.com Sent: Tuesday, August 18, 2015 1:44 PM Subject: Re: Myriad 0.1 release scope (So if I clone that repo, am I cloning the right one?) On Tue, Aug 18, 2015 at 3:43 PM, John Omernik j...@omernik.com wrote: Ok, I was going off https://github.com/mesos/myriad/blob/phase1/docs/myriad-configuration.md I will try it. John On Tue, Aug 18, 2015 at 3:40 PM, yuliya Feldman yufeld...@yahoo.com.invalid wrote: You actually do not need to rebuild even today - just keep this file in hadoop config directory that is on the classpath: like .../etc/hadoop From: John Omernik j...@omernik.com To: dev@myriad.incubator.apache.org Sent: Tuesday, August 18, 2015 1:35 PM Subject: Re: Myriad 0.1 release scope On the release scope, will having the myriad configuration file exist outside the jar (i.e. you can change configuration without rebuilding) be part of the .1 release scope? On Mon, Aug 10, 2015 at 10:01 PM, Santosh Marella smare...@maprtech.com wrote: Hello All, I've merged the FGS changes into phase1. Built and tested both coarse grained scaling and fine grained scaling, UI on a 4 node cluster. If anyone finds things are not working as expected, please let me know. Thanks, Santosh On Fri, Aug 7, 2015 at 10:46 AM, Santosh Marella smare...@maprtech.com wrote: Hello guys, I propose merging FGS into phase1. As I said before, I think it's at a point where the functionality works reasonably well. Any future improvements/fixes/UI changes can be done via separate JIRAs. Unless there are any major concerns, I'd like to merge FGS into phase1 *EOD Monday* (PDT). Thanks, Santosh On Wed, Aug 5, 2015 at 8:16 PM, Santosh Marella smare...@maprtech.com wrote: I feel FGS is very close to making it into 0.1. PR 116 addresses moving to hadoop 2.7 and making FGS and CGS coexist. This PR was recently reviewed by Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and 2.7.x clusters and it seemed to have worked as expected. Unless there are more reviews/feedback, it can be merged into issue_14. Once PR 116 is merged into issue_14, issue_14 can be merged into phase1. Thanks, Santosh On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon a...@mesosphere.io wrote: We do have a JIRA 0.1.0 fix version field, but none of our issues use it yet. I think the goal was just to take what we have and make it work under Apache infrastructure, then vote on that for 0.1.0. Although other features like HA or FGS would be great, let's try to get our first Apache release out ASAP. We can create 0.1.1 or 0.2.0 fix versions for subsequent releases with other issues/features. Roadmap would be great. (I'm just summarizing what we discussed a month or two ago. Feel free to correct me or disagree with this approach.) On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade swapnil.daing...@gmail.com wrote: Hi all, Was wondering what would be the scope for the Myriad 0.1 release. It would be nice to have a roadmap page somewhere and target features to releases (JIRA 'fix version' field perhaps) Regards Swapnil
Re: Odd Errors
I think your myriad-config-default.yml is missing a nmInstances section. Are you running it with a older .yml file? Santosh On Tue, Aug 18, 2015 at 3:06 PM, John Omernik j...@omernik.com wrote: I am working to stumble through this as Santosh helped me get a pre incubator version of Myriad running, and now I upgraded a bunch of stuff and wanted to try some of the more recent features. I setup the remote distribution, created what I think would be a good a json for marathon and then I am getting the dreaded Null Pointer Exception without much help... Based on the logs, it appears to be pulling my URI down with the proper pathing and trying to execute the resource manager from the tar ball, perhaps this will get me kicked off the dev list but my dev foo is weak, thus I am not sure how to troubleshoot this. :) Any help would be appreciated. 15/08/18 17:00:28 INFO mortbay.log: Started SelectChannelConnector@0.0.0.0:8192 15/08/18 17:00:28 INFO myriad.Main: Initializing HealthChecks 15/08/18 17:00:28 INFO myriad.Main: Initializing Profiles 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile tiny with CPU: 1 and Memory: 4096 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile small with CPU: 2 and Memory: 8192 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile medium with CPU: 4 and Memory: 16384 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile large with CPU: 8 and Memory: 32768 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile huge with CPU: 12 and Memory: 49152 15/08/18 17:00:28 INFO myriad.Main: Validating nmInstances.. 15/08/18 17:00:28 INFO service.AbstractService: Service org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler failed in state INITED; cause: java.lang.RuntimeException: Failed to initialize myriad java.lang.RuntimeException: Failed to initialize myriad at com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor.init(MyriadInitializationInterceptor.java:35) at com.ebay.myriad.scheduler.yarn.interceptor.CompositeInterceptor.init(CompositeInterceptor.java:76) at com.ebay.myriad.scheduler.yarn.MyriadFairScheduler.serviceInit(MyriadFairScheduler.java:50) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:570) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:997) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:262) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1220) Caused by: java.lang.NullPointerException at com.ebay.myriad.Main.validateNMInstances(Main.java:166) at com.ebay.myriad.Main.run(Main.java:98) at com.ebay.myriad.Main.initialize(Main.java:80) at com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor.init(MyriadInitializationInterceptor.java:32) ... 10 more 15/08/18 17:00:28 INFO service.AbstractService: Service RMActiveServices failed in state INITED; cause: java.lang.RuntimeException: Failed to initialize myriad java.lang.RuntimeException: Failed to initialize myriad at com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor.init(MyriadInitializationInterceptor.java:35) at com.ebay.myriad.scheduler.yarn.interceptor.CompositeInterceptor.init(CompositeInterceptor.java:76) at com.ebay.myriad.scheduler.yarn.MyriadFairScheduler.serviceInit(MyriadFairScheduler.java:50) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:570) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:997) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:262) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1220) Caused by: java.lang.NullPointerException at
Re: Myriad 0.1 release scope
On the release scope, will having the myriad configuration file exist outside the jar (i.e. you can change configuration without rebuilding) be part of the .1 release scope? On Mon, Aug 10, 2015 at 10:01 PM, Santosh Marella smare...@maprtech.com wrote: Hello All, I've merged the FGS changes into phase1. Built and tested both coarse grained scaling and fine grained scaling, UI on a 4 node cluster. If anyone finds things are not working as expected, please let me know. Thanks, Santosh On Fri, Aug 7, 2015 at 10:46 AM, Santosh Marella smare...@maprtech.com wrote: Hello guys, I propose merging FGS into phase1. As I said before, I think it's at a point where the functionality works reasonably well. Any future improvements/fixes/UI changes can be done via separate JIRAs. Unless there are any major concerns, I'd like to merge FGS into phase1 *EOD Monday* (PDT). Thanks, Santosh On Wed, Aug 5, 2015 at 8:16 PM, Santosh Marella smare...@maprtech.com wrote: I feel FGS is very close to making it into 0.1. PR 116 addresses moving to hadoop 2.7 and making FGS and CGS coexist. This PR was recently reviewed by Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and 2.7.x clusters and it seemed to have worked as expected. Unless there are more reviews/feedback, it can be merged into issue_14. Once PR 116 is merged into issue_14, issue_14 can be merged into phase1. Thanks, Santosh On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon a...@mesosphere.io wrote: We do have a JIRA 0.1.0 fix version field, but none of our issues use it yet. I think the goal was just to take what we have and make it work under Apache infrastructure, then vote on that for 0.1.0. Although other features like HA or FGS would be great, let's try to get our first Apache release out ASAP. We can create 0.1.1 or 0.2.0 fix versions for subsequent releases with other issues/features. Roadmap would be great. (I'm just summarizing what we discussed a month or two ago. Feel free to correct me or disagree with this approach.) On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade swapnil.daing...@gmail.com wrote: Hi all, Was wondering what would be the scope for the Myriad 0.1 release. It would be nice to have a roadmap page somewhere and target features to releases (JIRA 'fix version' field perhaps) Regards Swapnil
Re: Myriad 0.1 release scope
mesos/myriad is the right one so far From: John Omernik j...@omernik.com To: dev@myriad.incubator.apache.org; yuliya Feldman yufeld...@yahoo.com Sent: Tuesday, August 18, 2015 1:44 PM Subject: Re: Myriad 0.1 release scope (So if I clone that repo, am I cloning the right one?) On Tue, Aug 18, 2015 at 3:43 PM, John Omernik j...@omernik.com wrote: Ok, I was going off https://github.com/mesos/myriad/blob/phase1/docs/myriad-configuration.md I will try it. John On Tue, Aug 18, 2015 at 3:40 PM, yuliya Feldman yufeld...@yahoo.com.invalid wrote: You actually do not need to rebuild even today - just keep this file in hadoop config directory that is on the classpath: like .../etc/hadoop From: John Omernik j...@omernik.com To: dev@myriad.incubator.apache.org Sent: Tuesday, August 18, 2015 1:35 PM Subject: Re: Myriad 0.1 release scope On the release scope, will having the myriad configuration file exist outside the jar (i.e. you can change configuration without rebuilding) be part of the .1 release scope? On Mon, Aug 10, 2015 at 10:01 PM, Santosh Marella smare...@maprtech.com wrote: Hello All, I've merged the FGS changes into phase1. Built and tested both coarse grained scaling and fine grained scaling, UI on a 4 node cluster. If anyone finds things are not working as expected, please let me know. Thanks, Santosh On Fri, Aug 7, 2015 at 10:46 AM, Santosh Marella smare...@maprtech.com wrote: Hello guys, I propose merging FGS into phase1. As I said before, I think it's at a point where the functionality works reasonably well. Any future improvements/fixes/UI changes can be done via separate JIRAs. Unless there are any major concerns, I'd like to merge FGS into phase1 *EOD Monday* (PDT). Thanks, Santosh On Wed, Aug 5, 2015 at 8:16 PM, Santosh Marella smare...@maprtech.com wrote: I feel FGS is very close to making it into 0.1. PR 116 addresses moving to hadoop 2.7 and making FGS and CGS coexist. This PR was recently reviewed by Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and 2.7.x clusters and it seemed to have worked as expected. Unless there are more reviews/feedback, it can be merged into issue_14. Once PR 116 is merged into issue_14, issue_14 can be merged into phase1. Thanks, Santosh On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon a...@mesosphere.io wrote: We do have a JIRA 0.1.0 fix version field, but none of our issues use it yet. I think the goal was just to take what we have and make it work under Apache infrastructure, then vote on that for 0.1.0. Although other features like HA or FGS would be great, let's try to get our first Apache release out ASAP. We can create 0.1.1 or 0.2.0 fix versions for subsequent releases with other issues/features. Roadmap would be great. (I'm just summarizing what we discussed a month or two ago. Feel free to correct me or disagree with this approach.) On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade swapnil.daing...@gmail.com wrote: Hi all, Was wondering what would be the scope for the Myriad 0.1 release. It would be nice to have a roadmap page somewhere and target features to releases (JIRA 'fix version' field perhaps) Regards Swapnil
Re: Odd Errors
Could you paste the stderr/stdout of the executor as well? I think this may be a bug so you won't get booted:). Darin On Aug 18, 2015 3:07 PM, John Omernik j...@omernik.com wrote: I am working to stumble through this as Santosh helped me get a pre incubator version of Myriad running, and now I upgraded a bunch of stuff and wanted to try some of the more recent features. I setup the remote distribution, created what I think would be a good a json for marathon and then I am getting the dreaded Null Pointer Exception without much help... Based on the logs, it appears to be pulling my URI down with the proper pathing and trying to execute the resource manager from the tar ball, perhaps this will get me kicked off the dev list but my dev foo is weak, thus I am not sure how to troubleshoot this. :) Any help would be appreciated. 15/08/18 17:00:28 INFO mortbay.log: Started SelectChannelConnector@0.0.0.0:8192 15/08/18 17:00:28 INFO myriad.Main: Initializing HealthChecks 15/08/18 17:00:28 INFO myriad.Main: Initializing Profiles 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile tiny with CPU: 1 and Memory: 4096 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile small with CPU: 2 and Memory: 8192 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile medium with CPU: 4 and Memory: 16384 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile large with CPU: 8 and Memory: 32768 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile huge with CPU: 12 and Memory: 49152 15/08/18 17:00:28 INFO myriad.Main: Validating nmInstances.. 15/08/18 17:00:28 INFO service.AbstractService: Service org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler failed in state INITED; cause: java.lang.RuntimeException: Failed to initialize myriad java.lang.RuntimeException: Failed to initialize myriad at com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor.init(MyriadInitializationInterceptor.java:35) at com.ebay.myriad.scheduler.yarn.interceptor.CompositeInterceptor.init(CompositeInterceptor.java:76) at com.ebay.myriad.scheduler.yarn.MyriadFairScheduler.serviceInit(MyriadFairScheduler.java:50) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:570) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:997) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:262) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1220) Caused by: java.lang.NullPointerException at com.ebay.myriad.Main.validateNMInstances(Main.java:166) at com.ebay.myriad.Main.run(Main.java:98) at com.ebay.myriad.Main.initialize(Main.java:80) at com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor.init(MyriadInitializationInterceptor.java:32) ... 10 more 15/08/18 17:00:28 INFO service.AbstractService: Service RMActiveServices failed in state INITED; cause: java.lang.RuntimeException: Failed to initialize myriad java.lang.RuntimeException: Failed to initialize myriad at com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor.init(MyriadInitializationInterceptor.java:35) at com.ebay.myriad.scheduler.yarn.interceptor.CompositeInterceptor.init(CompositeInterceptor.java:76) at com.ebay.myriad.scheduler.yarn.MyriadFairScheduler.serviceInit(MyriadFairScheduler.java:50) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:570) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:997) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:262) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1220) Caused by: java.lang.NullPointerException at com.ebay.myriad.Main.validateNMInstances(Main.java:166) at
Re: Odd Errors
That's the issue (the yml) I'll merge tomorrow and try it. Thanks Santosh On Aug 18, 2015 5:19 PM, Santosh Marella smare...@maprtech.com wrote: I think your myriad-config-default.yml is missing a nmInstances section. Are you running it with a older .yml file? Santosh On Tue, Aug 18, 2015 at 3:06 PM, John Omernik j...@omernik.com wrote: I am working to stumble through this as Santosh helped me get a pre incubator version of Myriad running, and now I upgraded a bunch of stuff and wanted to try some of the more recent features. I setup the remote distribution, created what I think would be a good a json for marathon and then I am getting the dreaded Null Pointer Exception without much help... Based on the logs, it appears to be pulling my URI down with the proper pathing and trying to execute the resource manager from the tar ball, perhaps this will get me kicked off the dev list but my dev foo is weak, thus I am not sure how to troubleshoot this. :) Any help would be appreciated. 15/08/18 17:00:28 INFO mortbay.log: Started SelectChannelConnector@0.0.0.0:8192 15/08/18 17:00:28 INFO myriad.Main: Initializing HealthChecks 15/08/18 17:00:28 INFO myriad.Main: Initializing Profiles 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile tiny with CPU: 1 and Memory: 4096 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile small with CPU: 2 and Memory: 8192 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile medium with CPU: 4 and Memory: 16384 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile large with CPU: 8 and Memory: 32768 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile huge with CPU: 12 and Memory: 49152 15/08/18 17:00:28 INFO myriad.Main: Validating nmInstances.. 15/08/18 17:00:28 INFO service.AbstractService: Service org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler failed in state INITED; cause: java.lang.RuntimeException: Failed to initialize myriad java.lang.RuntimeException: Failed to initialize myriad at com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor.init(MyriadInitializationInterceptor.java:35) at com.ebay.myriad.scheduler.yarn.interceptor.CompositeInterceptor.init(CompositeInterceptor.java:76) at com.ebay.myriad.scheduler.yarn.MyriadFairScheduler.serviceInit(MyriadFairScheduler.java:50) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:570) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:997) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:262) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1220) Caused by: java.lang.NullPointerException at com.ebay.myriad.Main.validateNMInstances(Main.java:166) at com.ebay.myriad.Main.run(Main.java:98) at com.ebay.myriad.Main.initialize(Main.java:80) at com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor.init(MyriadInitializationInterceptor.java:32) ... 10 more 15/08/18 17:00:28 INFO service.AbstractService: Service RMActiveServices failed in state INITED; cause: java.lang.RuntimeException: Failed to initialize myriad java.lang.RuntimeException: Failed to initialize myriad at com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor.init(MyriadInitializationInterceptor.java:35) at com.ebay.myriad.scheduler.yarn.interceptor.CompositeInterceptor.init(CompositeInterceptor.java:76) at com.ebay.myriad.scheduler.yarn.MyriadFairScheduler.serviceInit(MyriadFairScheduler.java:50) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:570) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:997) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:262) at
Re: Myriad 0.1 release scope
John, The remote distribution doesn't require the nm to be run from marathon though it's possible. Essentially, it's the same configuration for the rm you'd do for the non remote version + adding a uri for the tarball. I've got jsons for running the rm in marathon, I'll try to get them and some documentation up soon. Currently at a conference though which means probably next week. Darin Darin On Aug 18, 2015 2:49 PM, John Omernik j...@omernik.com wrote: Ok, so I tried the remote distribution of the Myriad per the docs, I guess,it could probably use some information related to how to run resource manager if it's in the tar.gz. Perhaps an example marathon json. I am playing with it now to figure it out. On Tue, Aug 18, 2015 at 3:48 PM, yuliya Feldman yufeld...@yahoo.com.invalid wrote: mesos/myriad is the right one so far From: John Omernik j...@omernik.com To: dev@myriad.incubator.apache.org; yuliya Feldman yufeld...@yahoo.com Sent: Tuesday, August 18, 2015 1:44 PM Subject: Re: Myriad 0.1 release scope (So if I clone that repo, am I cloning the right one?) On Tue, Aug 18, 2015 at 3:43 PM, John Omernik j...@omernik.com wrote: Ok, I was going off https://github.com/mesos/myriad/blob/phase1/docs/myriad-configuration.md I will try it. John On Tue, Aug 18, 2015 at 3:40 PM, yuliya Feldman yufeld...@yahoo.com.invalid wrote: You actually do not need to rebuild even today - just keep this file in hadoop config directory that is on the classpath: like .../etc/hadoop From: John Omernik j...@omernik.com To: dev@myriad.incubator.apache.org Sent: Tuesday, August 18, 2015 1:35 PM Subject: Re: Myriad 0.1 release scope On the release scope, will having the myriad configuration file exist outside the jar (i.e. you can change configuration without rebuilding) be part of the .1 release scope? On Mon, Aug 10, 2015 at 10:01 PM, Santosh Marella smare...@maprtech.com wrote: Hello All, I've merged the FGS changes into phase1. Built and tested both coarse grained scaling and fine grained scaling, UI on a 4 node cluster. If anyone finds things are not working as expected, please let me know. Thanks, Santosh On Fri, Aug 7, 2015 at 10:46 AM, Santosh Marella smare...@maprtech.com wrote: Hello guys, I propose merging FGS into phase1. As I said before, I think it's at a point where the functionality works reasonably well. Any future improvements/fixes/UI changes can be done via separate JIRAs. Unless there are any major concerns, I'd like to merge FGS into phase1 *EOD Monday* (PDT). Thanks, Santosh On Wed, Aug 5, 2015 at 8:16 PM, Santosh Marella smare...@maprtech.com wrote: I feel FGS is very close to making it into 0.1. PR 116 addresses moving to hadoop 2.7 and making FGS and CGS coexist. This PR was recently reviewed by Yulia and Darin. Darin had also tried out FGS on hadoop 2.6.x and 2.7.x clusters and it seemed to have worked as expected. Unless there are more reviews/feedback, it can be merged into issue_14. Once PR 116 is merged into issue_14, issue_14 can be merged into phase1. Thanks, Santosh On Tue, Aug 4, 2015 at 4:54 PM, Adam Bordelon a...@mesosphere.io wrote: We do have a JIRA 0.1.0 fix version field, but none of our issues use it yet. I think the goal was just to take what we have and make it work under Apache infrastructure, then vote on that for 0.1.0. Although other features like HA or FGS would be great, let's try to get our first Apache release out ASAP. We can create 0.1.1 or 0.2.0 fix versions for subsequent releases with other issues/features. Roadmap would be great. (I'm just summarizing what we discussed a month or two ago. Feel free to correct me or disagree with this approach.) On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade swapnil.daing...@gmail.com wrote: Hi all, Was wondering what would be the scope for the Myriad 0.1 release. It would be nice to have a roadmap page somewhere and target features to releases (JIRA 'fix version' field perhaps) Regards Swapnil
Odd Errors
I am working to stumble through this as Santosh helped me get a pre incubator version of Myriad running, and now I upgraded a bunch of stuff and wanted to try some of the more recent features. I setup the remote distribution, created what I think would be a good a json for marathon and then I am getting the dreaded Null Pointer Exception without much help... Based on the logs, it appears to be pulling my URI down with the proper pathing and trying to execute the resource manager from the tar ball, perhaps this will get me kicked off the dev list but my dev foo is weak, thus I am not sure how to troubleshoot this. :) Any help would be appreciated. 15/08/18 17:00:28 INFO mortbay.log: Started SelectChannelConnector@0.0.0.0:8192 15/08/18 17:00:28 INFO myriad.Main: Initializing HealthChecks 15/08/18 17:00:28 INFO myriad.Main: Initializing Profiles 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile tiny with CPU: 1 and Memory: 4096 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile small with CPU: 2 and Memory: 8192 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile medium with CPU: 4 and Memory: 16384 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile large with CPU: 8 and Memory: 32768 15/08/18 17:00:28 INFO scheduler.NMProfileManager: Adding profile huge with CPU: 12 and Memory: 49152 15/08/18 17:00:28 INFO myriad.Main: Validating nmInstances.. 15/08/18 17:00:28 INFO service.AbstractService: Service org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler failed in state INITED; cause: java.lang.RuntimeException: Failed to initialize myriad java.lang.RuntimeException: Failed to initialize myriad at com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor.init(MyriadInitializationInterceptor.java:35) at com.ebay.myriad.scheduler.yarn.interceptor.CompositeInterceptor.init(CompositeInterceptor.java:76) at com.ebay.myriad.scheduler.yarn.MyriadFairScheduler.serviceInit(MyriadFairScheduler.java:50) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:570) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:997) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:262) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1220) Caused by: java.lang.NullPointerException at com.ebay.myriad.Main.validateNMInstances(Main.java:166) at com.ebay.myriad.Main.run(Main.java:98) at com.ebay.myriad.Main.initialize(Main.java:80) at com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor.init(MyriadInitializationInterceptor.java:32) ... 10 more 15/08/18 17:00:28 INFO service.AbstractService: Service RMActiveServices failed in state INITED; cause: java.lang.RuntimeException: Failed to initialize myriad java.lang.RuntimeException: Failed to initialize myriad at com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor.init(MyriadInitializationInterceptor.java:35) at com.ebay.myriad.scheduler.yarn.interceptor.CompositeInterceptor.init(CompositeInterceptor.java:76) at com.ebay.myriad.scheduler.yarn.MyriadFairScheduler.serviceInit(MyriadFairScheduler.java:50) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:570) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:997) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:262) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1220) Caused by: java.lang.NullPointerException at com.ebay.myriad.Main.validateNMInstances(Main.java:166) at com.ebay.myriad.Main.run(Main.java:98) at com.ebay.myriad.Main.initialize(Main.java:80) at com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor.init(MyriadInitializationInterceptor.java:32) ... 10 more
