Re: Not able to launch Node Managers via Myriad

2016-02-26 Thread Aashreya Shankar
Please check the white listed users in the following properties file : HADOOP_HOME/etc/hadoop/container-executor.cfg Need to set either of these 2 properties. min.user.id=500 (make it < 99) allowed.system.users=mapr (or add user here) Need to give permissions for the user to run the job. Thank

Re: Not able to launch Node Managers via Myriad

2016-02-26 Thread Björn Hagemeier
Dear Santosh, thanks a lot for having me check the myriad-config-default.yml. I did not have my own configuration and due to a lack of warnings in any of the logs, I had assumed everything to be ok to some extent. So, now I have the myriad-config-default.yml with much of the needed

Re: Not able to launch Node Managers via Myriad

2016-02-23 Thread Santosh Marella
This probably requires a code change in Myriad. Currently Myriad uses "$YARN_HOME/bin/yarn nodemanager" as the command to launch the NM. This makes the NM's log4j output to be streamed to stdout. (

Re: Not able to launch Node Managers via Myriad

2016-02-23 Thread Mayank Bansal
Thanks Santosh for the prompt reply and all the great help. Is there a way I can redirect mesos task logs to NodeManager logs. Users/Admins/tools are used to look at the logs in specific location. Its very hard to change all of it. Is there any way we can achieve that? Thanks, Mayank On Tue,

Re: Not able to launch Node Managers via Myriad

2016-02-23 Thread Santosh Marella
Great to hear your cluster is up. 1. I am not seeing Nodemanager logs, I see YARN LOG DIR is set however logs are not going there NM logs are accessible via the Mesos UI (click on the sandbox for nm.medium.xxx task and then on stderr link). Since NMs are launched via Mesos, the NM logs currently

Re: Not able to launch Node Managers via Myriad

2016-02-23 Thread Mayank Bansal
Hi Santosh, Thanks for looking into it. I got it working and now I am able to run Node Managers through Myriad. There are couple of issues which i am still facing , I am sure you guys would have faced those and already have the solutions for those. 1. I am not seeing Nodemanager logs, I see

Re: Not able to launch Node Managers via Myriad

2016-02-23 Thread Santosh Marella
>What makes me wonder is that I do have a running Myriad, i.e. I can >access the Myriad UI on port 8192, but it does not show up in Mesos. It's possible you have a wrong Myriad configuration. Can you please check the mesosMaster address in your myriad-config-default.yml file? e.g.:

Re: Not able to launch Node Managers via Myriad

2016-02-23 Thread yuliya Feldman
be at least linked to */etc/hadoop place)You have following line defined correctly: mesosMaster: zk:///mesos Thanks,Yuliya From: Björn Hagemeier <b.hageme...@fz-juelich.de> To: dev@myriad.incubator.apache.org Sent: Tuesday, February 23, 2016 6:21 AM Subject: Re: Not able to launc

Re: Not able to launch Node Managers via Myriad

2016-02-23 Thread Björn Hagemeier
Hi all, Am 23.02.2016 um 12:55 schrieb Björn Hagemeier: > Hi Santosh, > > Am 23.02.2016 um 01:21 schrieb Santosh Marella: >> Hey guys.. the flow of things is as below: >> >> 1. Myriad registers itself as a Mesos framework. (Frameworks view on Mesos >> UI should show this) does Myriad register

Re: Not able to launch Node Managers via Myriad

2016-02-23 Thread Björn Hagemeier
Hi Santosh, Am 23.02.2016 um 01:21 schrieb Santosh Marella: > Hey guys.. the flow of things is as below: > > 1. Myriad registers itself as a Mesos framework. (Frameworks view on Mesos > UI should show this) I think this is where it is already failing for me. I do not see the Myriad framework in

Re: Not able to launch Node Managers via Myriad

2016-02-22 Thread Santosh Marella
Hey guys.. the flow of things is as below: 1. Myriad registers itself as a Mesos framework. (Frameworks view on Mesos UI should show this) 2. Myriad by default attempts to launch 1 medium profile NM as soon as RM process is up. This is queued up until Myriad receives an offer from Mesos that