Yuliya,
In regards to 1. I believe "chown ${frameworkUser} ." was needed when I set
this up because the capsule plugin expanded some jars into the ./capsule
directory. Now the we're using the nodemanager directly that may not be an
issue.
As for 2) it's common in frameworks for schedulers to pass configurations
to the executors and I consider it a feature (maybe one that should be
cleaned up). I also view config changes as something that happens more
often than binary changes, so I actually leave the configs outside the
tarball and specifty an additional uri for the configs so I'm not
constantly making new tarballs. I'd hate to see it go away, especially
since I'm cleaning up my script that wgets the hadoop.tgz and does all the
surgery for you to do a super quick deploy.
I'm also not sure what it gets you as the config would essentially be
static in the tarball.
Open to a compromise of a config boolean.
Thanks for looking at remote disto.
On Tue, Sep 22, 2015 at 9:49 PM, yuliya Feldman <[email protected]> wrote:
> Hello Darrin,
>
> I have couple of questions regarding binary distro:
>
> 1. Just wonder if we still need to do "chown" on distro directory after
> untarring? It is working fine for me w/o changing permissions.
> 2. configuration you are getting from RM (config URL) to put into
> yarn-site.xml for NMs - why is this important?
> I assume you would use the same tar.gz to deploy RM and NMs (unless
> not true), so configuration would be the same. Unless I am missing
> something.
> My problem is that in Mapr Hadoop distro we use property to identify
> current physical host for shuffle purposes (Mapr is using maprfs for
> shuffle and not local disk), so I can not blindly copy configuration from
> RM to NM.
>
> Thanks,
> Yuliya
>
