Re: Not able to launch Node Managers via Myriad

[Aashreya Shankar]

Please check the white listed users in the following properties file :

HADOOP_HOME/etc/hadoop/container-executor.cfg

Need to set either of these 2 properties.
min.user.id=500 (make it < 99)
allowed.system.users=mapr (or add user here)

Need to give permissions for the user to run the job.

Thank you
Aashreya


On Fri, Feb 26, 2016 at 12:22 AM, Björn Hagemeier  wrote:

> Dear Santosh,
>
> thanks a lot for having me check the myriad-config-default.yml. I did
> not have my own configuration and due to a lack of warnings in any of
> the logs, I had assumed everything to be ok to some extent.
>
> So, now I have the myriad-config-default.yml with much of the needed
> configuration. On the way I ran into an issue also mentioned in [1], so
> I have set JAVA_HOME and YARN_NODEMANAGER_OPTS now as well as
> HADOOP_CONF_DIR and HADOOP_LIBEXEC_DIR, which are used by the NM startup
> scripts.
>
> I can now see NMs being started and torn down as a result of flexing
> up/down from the Myriad UI. However, when sending Yarn jobs, e.g.
> terasort, I am having issues with the user mapping.
>
> I get the following message:
>
> ==
> Diagnostics: Application application_1456472909573_0002 initialization
> failed (exitCode=255) with output: main : command provided 0
> main : run as user is nobody
> main : requested yarn user is bjoernh
> Requested user nobody is not whitelisted and has id 99,which is below
> the minimum allowed 1000
> ==
>
> When I still used my Yarn installation w/o Myriad, this worked fine.
> Could this be related to the framework user? I set it to "yarn" as this
> is that the package installation expects and has been working to the
> best of my knowledge.
>
>
> Cheers,
> Björn
>
>
> [1]
>
> https://mail-archives.apache.org/mod_mbox/myriad-dev/201512.mbox/%3cca+kw_myt4wpr-b54kjyo0hxkwsrthez2x41fftcuomvmqrq...@mail.gmail.com%3E
>
> Am 23.02.2016 um 20:00 schrieb Santosh Marella:
> >> What makes me wonder is that I do have a running Myriad, i.e. I can
> >> access the Myriad UI on port 8192, but it does not show up in Mesos.
> >
> > It's possible you have a wrong Myriad configuration. Can you please check
> > the mesosMaster address in your myriad-config-default.yml file?
> >
> > e.g.:
> >
> https://github.com/apache/incubator-myriad/blob/master/myriad-scheduler/src/main/resources/myriad-config-default.yml#L19
> >
> > Santosh
> >
> > On Tue, Feb 23, 2016 at 3:55 AM, Björn Hagemeier <
> b.hageme...@fz-juelich.de>
> > wrote:
> >
> >> Hi Santosh,
> >>
> >> Am 23.02.2016 um 01:21 schrieb Santosh Marella:
> >>> Hey guys.. the flow of things is as below:
> >>>
> >>> 1. Myriad registers itself as a Mesos framework. (Frameworks view on
> >> Mesos
> >>> UI should show this)
> >> I think this is where it is already failing for me. I do not see the
> >> Myriad framework in the UI.
> >>> 2. Myriad by default attempts to launch 1 medium profile NM as soon as
> RM
> >>> process is up.
> >>> This is queued up until Myriad receives an offer from Mesos that is
> >> big
> >>> enough to launch a medium profle NM.
> >> Yes, I do see the medium profile NM in the Myriad UI.
> >>> (Mesos' UI should show a task such as "nm.medium." if Myriad
> >>> attempted to launch a NM).
> >> Nope, unfortunately it does not.
> >>
> >> What makes me wonder is that I do have a running Myriad, i.e. I can
> >> access the Myriad UI on port 8192, but it does not show up in Mesos.
> >>
> >> I guess my other problem, not seeing any offers, is a follow-on of this.
> >>
> >>>
> >>> Mayank - in your cluster, does Mesos UI show Myriad as a registered
> >>> framework? Does it show if Myriad launched NM tasks? Perhaps a good
> >>> starting point is to check what are the sizes of the offers Myriad is
> >>> receiving (Mesos' master log should show this). You can also enable
> DEBUG
> >>> logging in RM for myriad by setting "org.apache.myriad" to DEBUG in
> RM's
> >>> log4j.properties.
> >> I will try and increase the log level to DEBUG.
> >>
> >>
> >> Cheers,
> >> Björn
> >>>
> >>>
> >>>
> >>> Santosh
> >>>
> >>> On Mon, Feb 22, 2016 at 10:39 AM, Björn Hagemeier <
> >> b.hageme...@fz-juelich.de
>  wrote:
> >>>
>  Hi Mayank,
> 
>  it seems you and I are having similar issues. Let's see whether they
> can
>  be traced to the same causes.
> 
> 
>  Best regards,
>  Björn
> 
>  Am 22.02.2016 um 19:33 schrieb Mayank Bansal:
> > Hi,
> >
> > I am trying to use Hadoop 2.6.3 with Myriad origin/0.1.x
> >
> >
> > I have followed all the instructions in the wiki
> >
> > So the resource manager is up and i am able to launch the web UI of
>  myriad
> > as well however whenever I try to use flex up it launches some tasks
> >> and
> > put them in to pending queue and after that nothing launches.
> >
> > I am attaching below the resourcemanager.out and 

Re: Not able to launch Node Managers via Myriad

[Björn Hagemeier]

Dear Santosh,

thanks a lot for having me check the myriad-config-default.yml. I did
not have my own configuration and due to a lack of warnings in any of
the logs, I had assumed everything to be ok to some extent.

So, now I have the myriad-config-default.yml with much of the needed
configuration. On the way I ran into an issue also mentioned in [1], so
I have set JAVA_HOME and YARN_NODEMANAGER_OPTS now as well as
HADOOP_CONF_DIR and HADOOP_LIBEXEC_DIR, which are used by the NM startup
scripts.

I can now see NMs being started and torn down as a result of flexing
up/down from the Myriad UI. However, when sending Yarn jobs, e.g.
terasort, I am having issues with the user mapping.

I get the following message:

==
Diagnostics: Application application_1456472909573_0002 initialization
failed (exitCode=255) with output: main : command provided 0
main : run as user is nobody
main : requested yarn user is bjoernh
Requested user nobody is not whitelisted and has id 99,which is below
the minimum allowed 1000
==

When I still used my Yarn installation w/o Myriad, this worked fine.
Could this be related to the framework user? I set it to "yarn" as this
is that the package installation expects and has been working to the
best of my knowledge.


Cheers,
Björn


[1]
https://mail-archives.apache.org/mod_mbox/myriad-dev/201512.mbox/%3cca+kw_myt4wpr-b54kjyo0hxkwsrthez2x41fftcuomvmqrq...@mail.gmail.com%3E

Am 23.02.2016 um 20:00 schrieb Santosh Marella:
>> What makes me wonder is that I do have a running Myriad, i.e. I can
>> access the Myriad UI on port 8192, but it does not show up in Mesos.
> 
> It's possible you have a wrong Myriad configuration. Can you please check
> the mesosMaster address in your myriad-config-default.yml file?
> 
> e.g.:
> https://github.com/apache/incubator-myriad/blob/master/myriad-scheduler/src/main/resources/myriad-config-default.yml#L19
> 
> Santosh
> 
> On Tue, Feb 23, 2016 at 3:55 AM, Björn Hagemeier 
> wrote:
> 
>> Hi Santosh,
>>
>> Am 23.02.2016 um 01:21 schrieb Santosh Marella:
>>> Hey guys.. the flow of things is as below:
>>>
>>> 1. Myriad registers itself as a Mesos framework. (Frameworks view on
>> Mesos
>>> UI should show this)
>> I think this is where it is already failing for me. I do not see the
>> Myriad framework in the UI.
>>> 2. Myriad by default attempts to launch 1 medium profile NM as soon as RM
>>> process is up.
>>> This is queued up until Myriad receives an offer from Mesos that is
>> big
>>> enough to launch a medium profle NM.
>> Yes, I do see the medium profile NM in the Myriad UI.
>>> (Mesos' UI should show a task such as "nm.medium." if Myriad
>>> attempted to launch a NM).
>> Nope, unfortunately it does not.
>>
>> What makes me wonder is that I do have a running Myriad, i.e. I can
>> access the Myriad UI on port 8192, but it does not show up in Mesos.
>>
>> I guess my other problem, not seeing any offers, is a follow-on of this.
>>
>>>
>>> Mayank - in your cluster, does Mesos UI show Myriad as a registered
>>> framework? Does it show if Myriad launched NM tasks? Perhaps a good
>>> starting point is to check what are the sizes of the offers Myriad is
>>> receiving (Mesos' master log should show this). You can also enable DEBUG
>>> logging in RM for myriad by setting "org.apache.myriad" to DEBUG in RM's
>>> log4j.properties.
>> I will try and increase the log level to DEBUG.
>>
>>
>> Cheers,
>> Björn
>>>
>>>
>>>
>>> Santosh
>>>
>>> On Mon, Feb 22, 2016 at 10:39 AM, Björn Hagemeier <
>> b.hageme...@fz-juelich.de
 wrote:
>>>
 Hi Mayank,

 it seems you and I are having similar issues. Let's see whether they can
 be traced to the same causes.


 Best regards,
 Björn

 Am 22.02.2016 um 19:33 schrieb Mayank Bansal:
> Hi,
>
> I am trying to use Hadoop 2.6.3 with Myriad origin/0.1.x
>
>
> I have followed all the instructions in the wiki
>
> So the resource manager is up and i am able to launch the web UI of
 myriad
> as well however whenever I try to use flex up it launches some tasks
>> and
> put them in to pending queue and after that nothing launches.
>
> I am attaching below the resourcemanager.out and log snippets.
>
> Please let me know if I am missing something.
>
> Thanks,
>
>
> *I see some exceptions in resourcemanager.out files those are as
>> follows*
>
>
> Feb 22, 2016 6:21:42 PM
>
>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8
> resolve
>
> SEVERE: null
>
> java.lang.IllegalAccessException: Class
>
>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8
> can not access a member of class javax.ws.rs.core.Response with
>> modifiers
> "protected"
>
> at
>> 

Re: Not able to launch Node Managers via Myriad

[Santosh Marella]

This probably requires a code change in Myriad.

Currently Myriad uses "$YARN_HOME/bin/yarn nodemanager" as the command to
launch the NM. This makes the NM's log4j output to be streamed to stdout. (
https://github.com/apache/incubator-myriad/blob/master/myriad-scheduler/src/main/java/org/apache/myriad/scheduler/NMExecutorCLGenImpl.java#L65
)

I think we should use "$YARN_HOME/sbin/yarn-daemon.sh start nodemanager" so
that the log4j output is written to $YARN_HOME/logs/xxx-nodemanger-yyy.log.
Do you happen to have a Myriad git clone to try it out and see if it helps?

Thanks,
Santosh

On Tue, Feb 23, 2016 at 1:49 PM, Mayank Bansal  wrote:

> Thanks Santosh for the prompt reply and all the great help.
>
> Is there a way I can redirect mesos task logs to NodeManager logs.
> Users/Admins/tools are used to look at the logs in specific location. Its
> very hard to change all of it. Is there any way we can achieve that?
>
> Thanks,
> Mayank
>
> On Tue, Feb 23, 2016 at 11:21 AM, Santosh Marella 
> wrote:
>
>> Great to hear your cluster is up.
>>
>> 1. I am not seeing Nodemanager logs, I see YARN LOG DIR is set however
>> logs are not going there
>> NM logs are accessible via the Mesos UI (click on the sandbox for
>> nm.medium.xxx task and then on stderr link).
>> Since NMs are launched via Mesos, the NM logs currently do not go to
>> YARN_HOME/logs dir.
>>
>> 2. Log aggregation is also not working in YARN , don't know why , as I
>> don't have logs
>> This should work, as Myriad is not really touching any of the traditional
>> YARN behavior/features.
>>
>> 3. Other issue is when i do first time flex up then it launches 2 node
>> managers instead of one.
>> This is expected. Once Myriad/RM starts, one medium profile NM is
>> launched automatically without requiring an explicit flexup.
>>
>> Thanks,
>> Santosh
>>
>> On Tue, Feb 23, 2016 at 11:06 AM, Mayank Bansal 
>> wrote:
>>
>>> Hi Santosh,
>>>
>>> Thanks for looking into it.
>>>
>>> I got it working and now I am able to run Node Managers through Myriad.
>>>
>>> There are couple of issues which i am still facing , I am sure you guys
>>> would have faced those and already have the solutions for those.
>>>
>>> 1. I am not seeing Nodemanager logs, I see YARN LOG DIR is set however
>>> logs are not going there
>>> 2. Log aggregation is also not working in YARN , don't know why , as I
>>> don't have logs
>>> 3. Other issue is when i do first time flex up then it launches 2 node
>>> managers instead of one.
>>>
>>> Please let me know if u know about those issues.
>>>
>>> Thanks,
>>> Mayank
>>>
>>>
>>> On Mon, Feb 22, 2016 at 4:21 PM, Santosh Marella 
>>> wrote:
>>>
 Hey guys.. the flow of things is as below:

 1. Myriad registers itself as a Mesos framework. (Frameworks view on
 Mesos
 UI should show this)
 2. Myriad by default attempts to launch 1 medium profile NM as soon as
 RM
 process is up.
 This is queued up until Myriad receives an offer from Mesos that is
 big
 enough to launch a medium profle NM.
 (Mesos' UI should show a task such as "nm.medium." if Myriad
 attempted to launch a NM).

 Mayank - in your cluster, does Mesos UI show Myriad as a registered
 framework? Does it show if Myriad launched NM tasks? Perhaps a good
 starting point is to check what are the sizes of the offers Myriad is
 receiving (Mesos' master log should show this). You can also enable
 DEBUG
 logging in RM for myriad by setting "org.apache.myriad" to DEBUG in RM's
 log4j.properties.



 Santosh

 On Mon, Feb 22, 2016 at 10:39 AM, Björn Hagemeier <
 b.hageme...@fz-juelich.de
 > wrote:

 > Hi Mayank,
 >
 > it seems you and I are having similar issues. Let's see whether they
 can
 > be traced to the same causes.
 >
 >
 > Best regards,
 > Björn
 >
 > Am 22.02.2016 um 19:33 schrieb Mayank Bansal:
 > > Hi,
 > >
 > > I am trying to use Hadoop 2.6.3 with Myriad origin/0.1.x
 > >
 > >
 > > I have followed all the instructions in the wiki
 > >
 > > So the resource manager is up and i am able to launch the web UI of
 > myriad
 > > as well however whenever I try to use flex up it launches some
 tasks and
 > > put them in to pending queue and after that nothing launches.
 > >
 > > I am attaching below the resourcemanager.out and log snippets.
 > >
 > > Please let me know if I am missing something.
 > >
 > > Thanks,
 > >
 > >
 > > *I see some exceptions in resourcemanager.out files those are as
 follows*
 > >
 > >
 > > Feb 22, 2016 6:21:42 PM
 > >
 com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8
 > > resolve
 > >
 > > SEVERE: null
 > >
 > > java.lang.IllegalAccessException: Class
 > >
 

Re: Not able to launch Node Managers via Myriad

[Mayank Bansal]

Thanks Santosh for the prompt reply and all the great help.

Is there a way I can redirect mesos task logs to NodeManager logs.
Users/Admins/tools are used to look at the logs in specific location. Its
very hard to change all of it. Is there any way we can achieve that?

Thanks,
Mayank

On Tue, Feb 23, 2016 at 11:21 AM, Santosh Marella 
wrote:

> Great to hear your cluster is up.
>
> 1. I am not seeing Nodemanager logs, I see YARN LOG DIR is set however
> logs are not going there
> NM logs are accessible via the Mesos UI (click on the sandbox for
> nm.medium.xxx task and then on stderr link).
> Since NMs are launched via Mesos, the NM logs currently do not go to
> YARN_HOME/logs dir.
>
> 2. Log aggregation is also not working in YARN , don't know why , as I
> don't have logs
> This should work, as Myriad is not really touching any of the traditional
> YARN behavior/features.
>
> 3. Other issue is when i do first time flex up then it launches 2 node
> managers instead of one.
> This is expected. Once Myriad/RM starts, one medium profile NM is launched
> automatically without requiring an explicit flexup.
>
> Thanks,
> Santosh
>
> On Tue, Feb 23, 2016 at 11:06 AM, Mayank Bansal  wrote:
>
>> Hi Santosh,
>>
>> Thanks for looking into it.
>>
>> I got it working and now I am able to run Node Managers through Myriad.
>>
>> There are couple of issues which i am still facing , I am sure you guys
>> would have faced those and already have the solutions for those.
>>
>> 1. I am not seeing Nodemanager logs, I see YARN LOG DIR is set however
>> logs are not going there
>> 2. Log aggregation is also not working in YARN , don't know why , as I
>> don't have logs
>> 3. Other issue is when i do first time flex up then it launches 2 node
>> managers instead of one.
>>
>> Please let me know if u know about those issues.
>>
>> Thanks,
>> Mayank
>>
>>
>> On Mon, Feb 22, 2016 at 4:21 PM, Santosh Marella 
>> wrote:
>>
>>> Hey guys.. the flow of things is as below:
>>>
>>> 1. Myriad registers itself as a Mesos framework. (Frameworks view on
>>> Mesos
>>> UI should show this)
>>> 2. Myriad by default attempts to launch 1 medium profile NM as soon as RM
>>> process is up.
>>> This is queued up until Myriad receives an offer from Mesos that is
>>> big
>>> enough to launch a medium profle NM.
>>> (Mesos' UI should show a task such as "nm.medium." if Myriad
>>> attempted to launch a NM).
>>>
>>> Mayank - in your cluster, does Mesos UI show Myriad as a registered
>>> framework? Does it show if Myriad launched NM tasks? Perhaps a good
>>> starting point is to check what are the sizes of the offers Myriad is
>>> receiving (Mesos' master log should show this). You can also enable DEBUG
>>> logging in RM for myriad by setting "org.apache.myriad" to DEBUG in RM's
>>> log4j.properties.
>>>
>>>
>>>
>>> Santosh
>>>
>>> On Mon, Feb 22, 2016 at 10:39 AM, Björn Hagemeier <
>>> b.hageme...@fz-juelich.de
>>> > wrote:
>>>
>>> > Hi Mayank,
>>> >
>>> > it seems you and I are having similar issues. Let's see whether they
>>> can
>>> > be traced to the same causes.
>>> >
>>> >
>>> > Best regards,
>>> > Björn
>>> >
>>> > Am 22.02.2016 um 19:33 schrieb Mayank Bansal:
>>> > > Hi,
>>> > >
>>> > > I am trying to use Hadoop 2.6.3 with Myriad origin/0.1.x
>>> > >
>>> > >
>>> > > I have followed all the instructions in the wiki
>>> > >
>>> > > So the resource manager is up and i am able to launch the web UI of
>>> > myriad
>>> > > as well however whenever I try to use flex up it launches some tasks
>>> and
>>> > > put them in to pending queue and after that nothing launches.
>>> > >
>>> > > I am attaching below the resourcemanager.out and log snippets.
>>> > >
>>> > > Please let me know if I am missing something.
>>> > >
>>> > > Thanks,
>>> > >
>>> > >
>>> > > *I see some exceptions in resourcemanager.out files those are as
>>> follows*
>>> > >
>>> > >
>>> > > Feb 22, 2016 6:21:42 PM
>>> > >
>>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8
>>> > > resolve
>>> > >
>>> > > SEVERE: null
>>> > >
>>> > > java.lang.IllegalAccessException: Class
>>> > >
>>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8
>>> > > can not access a member of class javax.ws.rs.core.Response with
>>> modifiers
>>> > > "protected"
>>> > >
>>> > > at
>>> sun.reflect.Reflection.ensureMemberAccess(Reflection.java:102)
>>> > >
>>> > > at java.lang.Class.newInstance(Class.java:436)
>>> > >
>>> > > at
>>> > >
>>> >
>>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8.resolve(WadlGeneratorJAXBGrammarGenerator.java:467)
>>> > >
>>> > > at
>>> > >
>>> >
>>> com.sun.jersey.server.wadl.WadlGenerator$ExternalGrammarDefinition.resolve(WadlGenerator.java:181)
>>> > >
>>> > > at
>>> > >
>>> >
>>> com.sun.jersey.server.wadl.ApplicationDescription.resolve(ApplicationDescription.java:81)
>>> > >
>>> > > at

Re: Not able to launch Node Managers via Myriad

[Santosh Marella]

Great to hear your cluster is up.

1. I am not seeing Nodemanager logs, I see YARN LOG DIR is set however logs
are not going there
NM logs are accessible via the Mesos UI (click on the sandbox for
nm.medium.xxx task and then on stderr link).
Since NMs are launched via Mesos, the NM logs currently do not go to
YARN_HOME/logs dir.

2. Log aggregation is also not working in YARN , don't know why , as I
don't have logs
This should work, as Myriad is not really touching any of the traditional
YARN behavior/features.

3. Other issue is when i do first time flex up then it launches 2 node
managers instead of one.
This is expected. Once Myriad/RM starts, one medium profile NM is launched
automatically without requiring an explicit flexup.

Thanks,
Santosh

On Tue, Feb 23, 2016 at 11:06 AM, Mayank Bansal  wrote:

> Hi Santosh,
>
> Thanks for looking into it.
>
> I got it working and now I am able to run Node Managers through Myriad.
>
> There are couple of issues which i am still facing , I am sure you guys
> would have faced those and already have the solutions for those.
>
> 1. I am not seeing Nodemanager logs, I see YARN LOG DIR is set however
> logs are not going there
> 2. Log aggregation is also not working in YARN , don't know why , as I
> don't have logs
> 3. Other issue is when i do first time flex up then it launches 2 node
> managers instead of one.
>
> Please let me know if u know about those issues.
>
> Thanks,
> Mayank
>
>
> On Mon, Feb 22, 2016 at 4:21 PM, Santosh Marella 
> wrote:
>
>> Hey guys.. the flow of things is as below:
>>
>> 1. Myriad registers itself as a Mesos framework. (Frameworks view on Mesos
>> UI should show this)
>> 2. Myriad by default attempts to launch 1 medium profile NM as soon as RM
>> process is up.
>> This is queued up until Myriad receives an offer from Mesos that is
>> big
>> enough to launch a medium profle NM.
>> (Mesos' UI should show a task such as "nm.medium." if Myriad
>> attempted to launch a NM).
>>
>> Mayank - in your cluster, does Mesos UI show Myriad as a registered
>> framework? Does it show if Myriad launched NM tasks? Perhaps a good
>> starting point is to check what are the sizes of the offers Myriad is
>> receiving (Mesos' master log should show this). You can also enable DEBUG
>> logging in RM for myriad by setting "org.apache.myriad" to DEBUG in RM's
>> log4j.properties.
>>
>>
>>
>> Santosh
>>
>> On Mon, Feb 22, 2016 at 10:39 AM, Björn Hagemeier <
>> b.hageme...@fz-juelich.de
>> > wrote:
>>
>> > Hi Mayank,
>> >
>> > it seems you and I are having similar issues. Let's see whether they can
>> > be traced to the same causes.
>> >
>> >
>> > Best regards,
>> > Björn
>> >
>> > Am 22.02.2016 um 19:33 schrieb Mayank Bansal:
>> > > Hi,
>> > >
>> > > I am trying to use Hadoop 2.6.3 with Myriad origin/0.1.x
>> > >
>> > >
>> > > I have followed all the instructions in the wiki
>> > >
>> > > So the resource manager is up and i am able to launch the web UI of
>> > myriad
>> > > as well however whenever I try to use flex up it launches some tasks
>> and
>> > > put them in to pending queue and after that nothing launches.
>> > >
>> > > I am attaching below the resourcemanager.out and log snippets.
>> > >
>> > > Please let me know if I am missing something.
>> > >
>> > > Thanks,
>> > >
>> > >
>> > > *I see some exceptions in resourcemanager.out files those are as
>> follows*
>> > >
>> > >
>> > > Feb 22, 2016 6:21:42 PM
>> > >
>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8
>> > > resolve
>> > >
>> > > SEVERE: null
>> > >
>> > > java.lang.IllegalAccessException: Class
>> > >
>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8
>> > > can not access a member of class javax.ws.rs.core.Response with
>> modifiers
>> > > "protected"
>> > >
>> > > at
>> sun.reflect.Reflection.ensureMemberAccess(Reflection.java:102)
>> > >
>> > > at java.lang.Class.newInstance(Class.java:436)
>> > >
>> > > at
>> > >
>> >
>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8.resolve(WadlGeneratorJAXBGrammarGenerator.java:467)
>> > >
>> > > at
>> > >
>> >
>> com.sun.jersey.server.wadl.WadlGenerator$ExternalGrammarDefinition.resolve(WadlGenerator.java:181)
>> > >
>> > > at
>> > >
>> >
>> com.sun.jersey.server.wadl.ApplicationDescription.resolve(ApplicationDescription.java:81)
>> > >
>> > > at
>> > >
>> >
>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator.attachTypes(WadlGeneratorJAXBGrammarGenerator.java:518)
>> > >
>> > > at
>> > > com.sun.jersey.server.wadl.WadlBuilder.generate(WadlBuilder.java:124)
>> > >
>> > > at
>> > >
>> >
>> com.sun.jersey.server.impl.wadl.WadlApplicationContextImpl.getApplication(WadlApplicationContextImpl.java:104)
>> > >
>> > > at
>> > >
>> >
>> com.sun.jersey.server.impl.wadl.WadlResource.getWadl(WadlResource.java:89)
>> > >
>> > > at 

Re: Not able to launch Node Managers via Myriad

[Mayank Bansal]

Hi Santosh,

Thanks for looking into it.

I got it working and now I am able to run Node Managers through Myriad.

There are couple of issues which i am still facing , I am sure you guys
would have faced those and already have the solutions for those.

1. I am not seeing Nodemanager logs, I see YARN LOG DIR is set however logs
are not going there
2. Log aggregation is also not working in YARN , don't know why , as I
don't have logs
3. Other issue is when i do first time flex up then it launches 2 node
managers instead of one.

Please let me know if u know about those issues.

Thanks,
Mayank


On Mon, Feb 22, 2016 at 4:21 PM, Santosh Marella 
wrote:

> Hey guys.. the flow of things is as below:
>
> 1. Myriad registers itself as a Mesos framework. (Frameworks view on Mesos
> UI should show this)
> 2. Myriad by default attempts to launch 1 medium profile NM as soon as RM
> process is up.
> This is queued up until Myriad receives an offer from Mesos that is big
> enough to launch a medium profle NM.
> (Mesos' UI should show a task such as "nm.medium." if Myriad
> attempted to launch a NM).
>
> Mayank - in your cluster, does Mesos UI show Myriad as a registered
> framework? Does it show if Myriad launched NM tasks? Perhaps a good
> starting point is to check what are the sizes of the offers Myriad is
> receiving (Mesos' master log should show this). You can also enable DEBUG
> logging in RM for myriad by setting "org.apache.myriad" to DEBUG in RM's
> log4j.properties.
>
>
>
> Santosh
>
> On Mon, Feb 22, 2016 at 10:39 AM, Björn Hagemeier <
> b.hageme...@fz-juelich.de
> > wrote:
>
> > Hi Mayank,
> >
> > it seems you and I are having similar issues. Let's see whether they can
> > be traced to the same causes.
> >
> >
> > Best regards,
> > Björn
> >
> > Am 22.02.2016 um 19:33 schrieb Mayank Bansal:
> > > Hi,
> > >
> > > I am trying to use Hadoop 2.6.3 with Myriad origin/0.1.x
> > >
> > >
> > > I have followed all the instructions in the wiki
> > >
> > > So the resource manager is up and i am able to launch the web UI of
> > myriad
> > > as well however whenever I try to use flex up it launches some tasks
> and
> > > put them in to pending queue and after that nothing launches.
> > >
> > > I am attaching below the resourcemanager.out and log snippets.
> > >
> > > Please let me know if I am missing something.
> > >
> > > Thanks,
> > >
> > >
> > > *I see some exceptions in resourcemanager.out files those are as
> follows*
> > >
> > >
> > > Feb 22, 2016 6:21:42 PM
> > >
> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8
> > > resolve
> > >
> > > SEVERE: null
> > >
> > > java.lang.IllegalAccessException: Class
> > >
> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8
> > > can not access a member of class javax.ws.rs.core.Response with
> modifiers
> > > "protected"
> > >
> > > at
> sun.reflect.Reflection.ensureMemberAccess(Reflection.java:102)
> > >
> > > at java.lang.Class.newInstance(Class.java:436)
> > >
> > > at
> > >
> >
> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8.resolve(WadlGeneratorJAXBGrammarGenerator.java:467)
> > >
> > > at
> > >
> >
> com.sun.jersey.server.wadl.WadlGenerator$ExternalGrammarDefinition.resolve(WadlGenerator.java:181)
> > >
> > > at
> > >
> >
> com.sun.jersey.server.wadl.ApplicationDescription.resolve(ApplicationDescription.java:81)
> > >
> > > at
> > >
> >
> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator.attachTypes(WadlGeneratorJAXBGrammarGenerator.java:518)
> > >
> > > at
> > > com.sun.jersey.server.wadl.WadlBuilder.generate(WadlBuilder.java:124)
> > >
> > > at
> > >
> >
> com.sun.jersey.server.impl.wadl.WadlApplicationContextImpl.getApplication(WadlApplicationContextImpl.java:104)
> > >
> > > at
> > >
> >
> com.sun.jersey.server.impl.wadl.WadlResource.getWadl(WadlResource.java:89)
> > >
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >
> > > at
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > >
> > > at
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > >
> > > at java.lang.reflect.Method.invoke(Method.java:498)
> > >
> > > at
> > >
> >
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> > >
> > > at
> > >
> >
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
> > >
> > > at
> > >
> >
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> > >
> > > at
> > >
> >
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
> > >
> > >

Re: Not able to launch Node Managers via Myriad

[Santosh Marella]

>What makes me wonder is that I do have a running Myriad, i.e. I can
>access the Myriad UI on port 8192, but it does not show up in Mesos.

It's possible you have a wrong Myriad configuration. Can you please check
the mesosMaster address in your myriad-config-default.yml file?

e.g.:
https://github.com/apache/incubator-myriad/blob/master/myriad-scheduler/src/main/resources/myriad-config-default.yml#L19

Santosh

On Tue, Feb 23, 2016 at 3:55 AM, Björn Hagemeier 
wrote:

> Hi Santosh,
>
> Am 23.02.2016 um 01:21 schrieb Santosh Marella:
> > Hey guys.. the flow of things is as below:
> >
> > 1. Myriad registers itself as a Mesos framework. (Frameworks view on
> Mesos
> > UI should show this)
> I think this is where it is already failing for me. I do not see the
> Myriad framework in the UI.
> > 2. Myriad by default attempts to launch 1 medium profile NM as soon as RM
> > process is up.
> > This is queued up until Myriad receives an offer from Mesos that is
> big
> > enough to launch a medium profle NM.
> Yes, I do see the medium profile NM in the Myriad UI.
> > (Mesos' UI should show a task such as "nm.medium." if Myriad
> > attempted to launch a NM).
> Nope, unfortunately it does not.
>
> What makes me wonder is that I do have a running Myriad, i.e. I can
> access the Myriad UI on port 8192, but it does not show up in Mesos.
>
> I guess my other problem, not seeing any offers, is a follow-on of this.
>
> >
> > Mayank - in your cluster, does Mesos UI show Myriad as a registered
> > framework? Does it show if Myriad launched NM tasks? Perhaps a good
> > starting point is to check what are the sizes of the offers Myriad is
> > receiving (Mesos' master log should show this). You can also enable DEBUG
> > logging in RM for myriad by setting "org.apache.myriad" to DEBUG in RM's
> > log4j.properties.
> I will try and increase the log level to DEBUG.
>
>
> Cheers,
> Björn
> >
> >
> >
> > Santosh
> >
> > On Mon, Feb 22, 2016 at 10:39 AM, Björn Hagemeier <
> b.hageme...@fz-juelich.de
> >> wrote:
> >
> >> Hi Mayank,
> >>
> >> it seems you and I are having similar issues. Let's see whether they can
> >> be traced to the same causes.
> >>
> >>
> >> Best regards,
> >> Björn
> >>
> >> Am 22.02.2016 um 19:33 schrieb Mayank Bansal:
> >>> Hi,
> >>>
> >>> I am trying to use Hadoop 2.6.3 with Myriad origin/0.1.x
> >>>
> >>>
> >>> I have followed all the instructions in the wiki
> >>>
> >>> So the resource manager is up and i am able to launch the web UI of
> >> myriad
> >>> as well however whenever I try to use flex up it launches some tasks
> and
> >>> put them in to pending queue and after that nothing launches.
> >>>
> >>> I am attaching below the resourcemanager.out and log snippets.
> >>>
> >>> Please let me know if I am missing something.
> >>>
> >>> Thanks,
> >>>
> >>>
> >>> *I see some exceptions in resourcemanager.out files those are as
> follows*
> >>>
> >>>
> >>> Feb 22, 2016 6:21:42 PM
> >>>
> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8
> >>> resolve
> >>>
> >>> SEVERE: null
> >>>
> >>> java.lang.IllegalAccessException: Class
> >>>
> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8
> >>> can not access a member of class javax.ws.rs.core.Response with
> modifiers
> >>> "protected"
> >>>
> >>> at
> sun.reflect.Reflection.ensureMemberAccess(Reflection.java:102)
> >>>
> >>> at java.lang.Class.newInstance(Class.java:436)
> >>>
> >>> at
> >>>
> >>
> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8.resolve(WadlGeneratorJAXBGrammarGenerator.java:467)
> >>>
> >>> at
> >>>
> >>
> com.sun.jersey.server.wadl.WadlGenerator$ExternalGrammarDefinition.resolve(WadlGenerator.java:181)
> >>>
> >>> at
> >>>
> >>
> com.sun.jersey.server.wadl.ApplicationDescription.resolve(ApplicationDescription.java:81)
> >>>
> >>> at
> >>>
> >>
> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator.attachTypes(WadlGeneratorJAXBGrammarGenerator.java:518)
> >>>
> >>> at
> >>> com.sun.jersey.server.wadl.WadlBuilder.generate(WadlBuilder.java:124)
> >>>
> >>> at
> >>>
> >>
> com.sun.jersey.server.impl.wadl.WadlApplicationContextImpl.getApplication(WadlApplicationContextImpl.java:104)
> >>>
> >>> at
> >>>
> >>
> com.sun.jersey.server.impl.wadl.WadlResource.getWadl(WadlResource.java:89)
> >>>
> >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >>>
> >>> at
> >>>
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >>>
> >>> at
> >>>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >>>
> >>> at java.lang.reflect.Method.invoke(Method.java:498)
> >>>
> >>> at
> >>>
> >>
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> >>>
> >>> at
> >>>
> >>
> 

Re: Not able to launch Node Managers via Myriad

[yuliya Feldman]

Could you provide your yarn-site.xml and myriad-config-default.yml ?
Let's make sure you have following in your yarn-site.xml:
      yarn.resourcemanager.scheduler.class    
com.ebay.myriad.scheduler.yarn.MyriadFairScheduler    
One can configure other scehdulers as well from following list: 
com.ebay.myriad.scheduler.yarn.MyriadCapacityScheduler, 
com.ebay.myriad.scheduler.yarn.MyriadFifoScheduler    
    yarn.nodemanager.resource.cpu-vcores    
${nodemanager.resource.cpu-vcores}        
yarn.nodemanager.resource.memory-mb    
${nodemanager.resource.memory-mb}        
yarn.nodemanager.address    
${myriad.yarn.nodemanager.address}        
yarn.nodemanager.webapp.address    
${myriad.yarn.nodemanager.webapp.address}    
    yarn.nodemanager.webapp.https.address    
${myriad.yarn.nodemanager.webapp.address}    
    yarn.nodemanager.localizer.address    
${myriad.yarn.nodemanager.localizer.address}    
   mapreduce.shuffle.port   
${myriad.mapreduce.shuffle.port}        
yarn.nodemanager.aux-services    
mapreduce_shuffle,myriad_executor        
yarn.nodemanager.aux-services.myriad_executor.class    
com.ebay.myriad.executor.MyriadExecutorAuxService  
And in your yml file (should be at least linked to */etc/hadoop place)You have 
following line defined correctly:
mesosMaster: zk:///mesos
Thanks,Yuliya

  From: Björn Hagemeier <b.hageme...@fz-juelich.de>
 To: dev@myriad.incubator.apache.org 
 Sent: Tuesday, February 23, 2016 6:21 AM
 Subject: Re: Not able to launch Node Managers via Myriad
   
Hi all,

Am 23.02.2016 um 12:55 schrieb Björn Hagemeier:
> Hi Santosh,
> 
> Am 23.02.2016 um 01:21 schrieb Santosh Marella:
>> Hey guys.. the flow of things is as below:
>>
>> 1. Myriad registers itself as a Mesos framework. (Frameworks view on Mesos
>> UI should show this)
does Myriad register itself as a Mesos framework automatically, just by
starting the RM as one would do it without Mesos? Or do I need to start
the RM as a Mesos framework to make the magic happen, i.e. by "mesos
execute"?

In the latter case, I think I found out why my Myriad does not seem to
do anything. I did not start the resource manager as a Mesos framework.
I somehow reckoned, the myriad integration would take care of that by
itself. When I do this, I do see the framework, but with classpath
issues (missing myriad for some reason).

I'll keep digging in that direction.


Cheers,
Björn

> I think this is where it is already failing for me. I do not see the
> Myriad framework in the UI.
>> 2. Myriad by default attempts to launch 1 medium profile NM as soon as RM
>> process is up.
>>    This is queued up until Myriad receives an offer from Mesos that is big
>> enough to launch a medium profle NM.
> Yes, I do see the medium profile NM in the Myriad UI.
>>    (Mesos' UI should show a task such as "nm.medium." if Myriad
>> attempted to launch a NM).
> Nope, unfortunately it does not.
> 
> What makes me wonder is that I do have a running Myriad, i.e. I can
> access the Myriad UI on port 8192, but it does not show up in Mesos.
> 
> I guess my other problem, not seeing any offers, is a follow-on of this.
> 
>>
>> Mayank - in your cluster, does Mesos UI show Myriad as a registered
>> framework? Does it show if Myriad launched NM tasks? Perhaps a good
>> starting point is to check what are the sizes of the offers Myriad is
>> receiving (Mesos' master log should show this). You can also enable DEBUG
>> logging in RM for myriad by setting "org.apache.myriad" to DEBUG in RM's
>> log4j.properties.
> I will try and increase the log level to DEBUG.
> 
> 
> Cheers,
> Björn
>>
>>
>>
>> Santosh
>>
>> On Mon, Feb 22, 2016 at 10:39 AM, Björn Hagemeier <b.hageme...@fz-juelich.de
>>> wrote:
>>
>>> Hi Mayank,
>>>
>>> it seems you and I are having similar issues. Let's see whether they can
>>> be traced to the same causes.
>>>
>>>
>>> Best regards,
>>> Björn
>>>
>>> Am 22.02.2016 um 19:33 schrieb Mayank Bansal:
>>>> Hi,
>>>>
>>>> I am trying to use Hadoop 2.6.3 with Myriad origin/0.1.x
>>>>
>>>>
>>>> I have followed all the instructions in the wiki
>>>>
>>>> So the resource manager is up and i am able to launch the web UI of
>>> myriad
>>>> as well however whenever I try to use flex up it launches some tasks and
>>>> put them in to pending queue and after that nothing launches.
>>>>
>>>> I am attaching below the resourcemanager.out and log snippets.
>>>>
>>>> Please let me know if I am missing something.
>>>>
>>>> Thanks,
>>

Re: Not able to launch Node Managers via Myriad

[Björn Hagemeier]

Hi all,

Am 23.02.2016 um 12:55 schrieb Björn Hagemeier:
> Hi Santosh,
> 
> Am 23.02.2016 um 01:21 schrieb Santosh Marella:
>> Hey guys.. the flow of things is as below:
>>
>> 1. Myriad registers itself as a Mesos framework. (Frameworks view on Mesos
>> UI should show this)
does Myriad register itself as a Mesos framework automatically, just by
starting the RM as one would do it without Mesos? Or do I need to start
the RM as a Mesos framework to make the magic happen, i.e. by "mesos
execute"?

In the latter case, I think I found out why my Myriad does not seem to
do anything. I did not start the resource manager as a Mesos framework.
I somehow reckoned, the myriad integration would take care of that by
itself. When I do this, I do see the framework, but with classpath
issues (missing myriad for some reason).

I'll keep digging in that direction.


Cheers,
Björn

> I think this is where it is already failing for me. I do not see the
> Myriad framework in the UI.
>> 2. Myriad by default attempts to launch 1 medium profile NM as soon as RM
>> process is up.
>> This is queued up until Myriad receives an offer from Mesos that is big
>> enough to launch a medium profle NM.
> Yes, I do see the medium profile NM in the Myriad UI.
>> (Mesos' UI should show a task such as "nm.medium." if Myriad
>> attempted to launch a NM).
> Nope, unfortunately it does not.
> 
> What makes me wonder is that I do have a running Myriad, i.e. I can
> access the Myriad UI on port 8192, but it does not show up in Mesos.
> 
> I guess my other problem, not seeing any offers, is a follow-on of this.
> 
>>
>> Mayank - in your cluster, does Mesos UI show Myriad as a registered
>> framework? Does it show if Myriad launched NM tasks? Perhaps a good
>> starting point is to check what are the sizes of the offers Myriad is
>> receiving (Mesos' master log should show this). You can also enable DEBUG
>> logging in RM for myriad by setting "org.apache.myriad" to DEBUG in RM's
>> log4j.properties.
> I will try and increase the log level to DEBUG.
> 
> 
> Cheers,
> Björn
>>
>>
>>
>> Santosh
>>
>> On Mon, Feb 22, 2016 at 10:39 AM, Björn Hagemeier >> wrote:
>>
>>> Hi Mayank,
>>>
>>> it seems you and I are having similar issues. Let's see whether they can
>>> be traced to the same causes.
>>>
>>>
>>> Best regards,
>>> Björn
>>>
>>> Am 22.02.2016 um 19:33 schrieb Mayank Bansal:
 Hi,

 I am trying to use Hadoop 2.6.3 with Myriad origin/0.1.x


 I have followed all the instructions in the wiki

 So the resource manager is up and i am able to launch the web UI of
>>> myriad
 as well however whenever I try to use flex up it launches some tasks and
 put them in to pending queue and after that nothing launches.

 I am attaching below the resourcemanager.out and log snippets.

 Please let me know if I am missing something.

 Thanks,


 *I see some exceptions in resourcemanager.out files those are as follows*


 Feb 22, 2016 6:21:42 PM
 com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8
 resolve

 SEVERE: null

 java.lang.IllegalAccessException: Class
 com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8
 can not access a member of class javax.ws.rs.core.Response with modifiers
 "protected"

 at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:102)

 at java.lang.Class.newInstance(Class.java:436)

 at

>>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8.resolve(WadlGeneratorJAXBGrammarGenerator.java:467)

 at

>>> com.sun.jersey.server.wadl.WadlGenerator$ExternalGrammarDefinition.resolve(WadlGenerator.java:181)

 at

>>> com.sun.jersey.server.wadl.ApplicationDescription.resolve(ApplicationDescription.java:81)

 at

>>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator.attachTypes(WadlGeneratorJAXBGrammarGenerator.java:518)

 at
 com.sun.jersey.server.wadl.WadlBuilder.generate(WadlBuilder.java:124)

 at

>>> com.sun.jersey.server.impl.wadl.WadlApplicationContextImpl.getApplication(WadlApplicationContextImpl.java:104)

 at

>>> com.sun.jersey.server.impl.wadl.WadlResource.getWadl(WadlResource.java:89)

 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

 at

>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

 at

>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

 at java.lang.reflect.Method.invoke(Method.java:498)

 at

>>> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)

 at

>>> 

Re: Not able to launch Node Managers via Myriad

[Björn Hagemeier]

Hi Santosh,

Am 23.02.2016 um 01:21 schrieb Santosh Marella:
> Hey guys.. the flow of things is as below:
> 
> 1. Myriad registers itself as a Mesos framework. (Frameworks view on Mesos
> UI should show this)
I think this is where it is already failing for me. I do not see the
Myriad framework in the UI.
> 2. Myriad by default attempts to launch 1 medium profile NM as soon as RM
> process is up.
> This is queued up until Myriad receives an offer from Mesos that is big
> enough to launch a medium profle NM.
Yes, I do see the medium profile NM in the Myriad UI.
> (Mesos' UI should show a task such as "nm.medium." if Myriad
> attempted to launch a NM).
Nope, unfortunately it does not.

What makes me wonder is that I do have a running Myriad, i.e. I can
access the Myriad UI on port 8192, but it does not show up in Mesos.

I guess my other problem, not seeing any offers, is a follow-on of this.

> 
> Mayank - in your cluster, does Mesos UI show Myriad as a registered
> framework? Does it show if Myriad launched NM tasks? Perhaps a good
> starting point is to check what are the sizes of the offers Myriad is
> receiving (Mesos' master log should show this). You can also enable DEBUG
> logging in RM for myriad by setting "org.apache.myriad" to DEBUG in RM's
> log4j.properties.
I will try and increase the log level to DEBUG.


Cheers,
Björn
> 
> 
> 
> Santosh
> 
> On Mon, Feb 22, 2016 at 10:39 AM, Björn Hagemeier > wrote:
> 
>> Hi Mayank,
>>
>> it seems you and I are having similar issues. Let's see whether they can
>> be traced to the same causes.
>>
>>
>> Best regards,
>> Björn
>>
>> Am 22.02.2016 um 19:33 schrieb Mayank Bansal:
>>> Hi,
>>>
>>> I am trying to use Hadoop 2.6.3 with Myriad origin/0.1.x
>>>
>>>
>>> I have followed all the instructions in the wiki
>>>
>>> So the resource manager is up and i am able to launch the web UI of
>> myriad
>>> as well however whenever I try to use flex up it launches some tasks and
>>> put them in to pending queue and after that nothing launches.
>>>
>>> I am attaching below the resourcemanager.out and log snippets.
>>>
>>> Please let me know if I am missing something.
>>>
>>> Thanks,
>>>
>>>
>>> *I see some exceptions in resourcemanager.out files those are as follows*
>>>
>>>
>>> Feb 22, 2016 6:21:42 PM
>>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8
>>> resolve
>>>
>>> SEVERE: null
>>>
>>> java.lang.IllegalAccessException: Class
>>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8
>>> can not access a member of class javax.ws.rs.core.Response with modifiers
>>> "protected"
>>>
>>> at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:102)
>>>
>>> at java.lang.Class.newInstance(Class.java:436)
>>>
>>> at
>>>
>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8.resolve(WadlGeneratorJAXBGrammarGenerator.java:467)
>>>
>>> at
>>>
>> com.sun.jersey.server.wadl.WadlGenerator$ExternalGrammarDefinition.resolve(WadlGenerator.java:181)
>>>
>>> at
>>>
>> com.sun.jersey.server.wadl.ApplicationDescription.resolve(ApplicationDescription.java:81)
>>>
>>> at
>>>
>> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator.attachTypes(WadlGeneratorJAXBGrammarGenerator.java:518)
>>>
>>> at
>>> com.sun.jersey.server.wadl.WadlBuilder.generate(WadlBuilder.java:124)
>>>
>>> at
>>>
>> com.sun.jersey.server.impl.wadl.WadlApplicationContextImpl.getApplication(WadlApplicationContextImpl.java:104)
>>>
>>> at
>>>
>> com.sun.jersey.server.impl.wadl.WadlResource.getWadl(WadlResource.java:89)
>>>
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>
>>> at
>>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>
>>> at
>>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>
>>> at
>>>
>> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
>>>
>>> at
>>>
>> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
>>>
>>> at
>>>
>> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
>>>
>>> at
>>>
>> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
>>>
>>> at
>>>
>> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
>>>
>>> at
>>>
>> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
>>>
>>> at
>>>
>> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
>>>
>>> at
>>>
>> 

Re: Not able to launch Node Managers via Myriad

[Santosh Marella]

Hey guys.. the flow of things is as below:

1. Myriad registers itself as a Mesos framework. (Frameworks view on Mesos
UI should show this)
2. Myriad by default attempts to launch 1 medium profile NM as soon as RM
process is up.
This is queued up until Myriad receives an offer from Mesos that is big
enough to launch a medium profle NM.
(Mesos' UI should show a task such as "nm.medium." if Myriad
attempted to launch a NM).

Mayank - in your cluster, does Mesos UI show Myriad as a registered
framework? Does it show if Myriad launched NM tasks? Perhaps a good
starting point is to check what are the sizes of the offers Myriad is
receiving (Mesos' master log should show this). You can also enable DEBUG
logging in RM for myriad by setting "org.apache.myriad" to DEBUG in RM's
log4j.properties.



Santosh

On Mon, Feb 22, 2016 at 10:39 AM, Björn Hagemeier  wrote:

> Hi Mayank,
>
> it seems you and I are having similar issues. Let's see whether they can
> be traced to the same causes.
>
>
> Best regards,
> Björn
>
> Am 22.02.2016 um 19:33 schrieb Mayank Bansal:
> > Hi,
> >
> > I am trying to use Hadoop 2.6.3 with Myriad origin/0.1.x
> >
> >
> > I have followed all the instructions in the wiki
> >
> > So the resource manager is up and i am able to launch the web UI of
> myriad
> > as well however whenever I try to use flex up it launches some tasks and
> > put them in to pending queue and after that nothing launches.
> >
> > I am attaching below the resourcemanager.out and log snippets.
> >
> > Please let me know if I am missing something.
> >
> > Thanks,
> >
> >
> > *I see some exceptions in resourcemanager.out files those are as follows*
> >
> >
> > Feb 22, 2016 6:21:42 PM
> > com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8
> > resolve
> >
> > SEVERE: null
> >
> > java.lang.IllegalAccessException: Class
> > com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8
> > can not access a member of class javax.ws.rs.core.Response with modifiers
> > "protected"
> >
> > at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:102)
> >
> > at java.lang.Class.newInstance(Class.java:436)
> >
> > at
> >
> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8.resolve(WadlGeneratorJAXBGrammarGenerator.java:467)
> >
> > at
> >
> com.sun.jersey.server.wadl.WadlGenerator$ExternalGrammarDefinition.resolve(WadlGenerator.java:181)
> >
> > at
> >
> com.sun.jersey.server.wadl.ApplicationDescription.resolve(ApplicationDescription.java:81)
> >
> > at
> >
> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator.attachTypes(WadlGeneratorJAXBGrammarGenerator.java:518)
> >
> > at
> > com.sun.jersey.server.wadl.WadlBuilder.generate(WadlBuilder.java:124)
> >
> > at
> >
> com.sun.jersey.server.impl.wadl.WadlApplicationContextImpl.getApplication(WadlApplicationContextImpl.java:104)
> >
> > at
> >
> com.sun.jersey.server.impl.wadl.WadlResource.getWadl(WadlResource.java:89)
> >
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >
> > at java.lang.reflect.Method.invoke(Method.java:498)
> >
> > at
> >
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> >
> > at
> >
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
> >
> > at
> >
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> >
> > at
> >
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
> >
> > at
> >
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> >
> > at
> >
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> >
> > at
> >
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> >
> > at
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
> >
> > at
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
> >
> > at
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
> >
> > at
> >
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
> >
> > at
> >
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
> >
> >