Yongyu, Sounds like you've made some progress! I'm not sure you'll be able to get the cluster completely under CM management and the node managers launched by Myriad aren't running the Cloudera agent (I don't think we'll ever officially support that, but you could tweak the source to run it in the background, if you could make it a general process it might be a feature we'd add). Though if you're launching the Resource Manager via CM I'd expect it to show up. I haven't attempted to put my resource manager under the CM though.
Darin On Tue, Apr 26, 2016 at 4:45 AM, 陈泳宇 <yc...@linkernetworks.com> wrote: > Hi Darin, This is Yongyu from Linkernetworks, I have proved that myriad > can work well with CDH. Currently, I am also trying to myriad our hadoop > cluster under the management of CM. > > I noticed that you have already complete the mission, but I still got some > problem here: The status shown on CM dashboard will not change… > > Here are my steps: > > 1. Add YARN service on CM dashboard.. > > 2. Stop the YARN service > > 3. Cd to /opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p0.45 > > 4. Copy native libs 、myriad jars as well we the config files. > > 5. Tar this package and upload it to hdfs. > > 6. Start the resourcemanager. > > Myriad service will be launched successfully, but nothing changes on the > dashboard, which means the cluster is not under the management of CM.. > > > > > > Regards, > > Yongyu > > > Begin forwarded message: > > *From:* Darin Johnson <dbjohnson1...@gmail.com> > *Date:* April 21, 2016 at 8:50:22 PM GMT+8 > *To:* Dev <dev@myriad.incubator.apache.org> > *Subject:* *Re: Hello Guys* > *Reply-To:* dev@myriad.incubator.apache.org > > Hey Sam, > > I'm already myriad alongside a CM managed hadoop cluster. It's a little > hacky right now, I'm working on stream lining this, it may involve CM > and/or some docker integration. Here are my current steps: > > 0. I strongly recommend pulling of master and building from source - it has > some really useful patches. We're working on another release now (couple > weeks out though). > > 1. Let Cloudera Manager configure hdfs - it does a good job of this. > > 2. Grab a cloudera tar ball from here: > > http://www.cloudera.com/documentation/enterprise/release-notes/topics/cdh_vd_cdh_package_tarball_eom.html#topic_3 > (I've also just used apache-hadoop tarballs). > > 3. Extract the tar ball, copy the native libraries cdh install on your > system in hadoop-*/lib/native > > 4 cp myriad/myriad-*/build/libs/* hadoop-*/share/hadoop/yarn/lib > > 5 copy your hadoop configs to hadoop-*/etc/hadoop/ > > 6 create a myriad-default-config.yml in hadoop-*/etc/hadoop follow > instructions on wiki for remote distribution for edits to > myriad-default-config.yml. NB: don't enable yarn cgroups yet - I'm fixing > a bug. > > 7 chown -R root:root hadoop-*y > > 8 chown root:yarn hadoop-*/bin/container-executor ; chmod g+s > hadoop*/bin/container-executor > > 9 mv hadoop-<..> hadoop-myriad > > 10 tar -zxvf hadoop-myriad.tgz hadoop-myriad ; hadoop fs -put > hadoop-myriad.tgz /dist/ > > 11 cd hadoop-myriad/ && sudo -u yarn bin/yarn resource manager > > 12 hit the web-ui at host:8192 and flexup some node managers > > This has been pretty stable. > > Alternatively, if you're running mesos and docker you could look at PR > https://github.com/apache/incubator-myriad/pull/64, it's still WIP but > avoids a lot of setup - would be happy to work with you as I document and > harden this feature for the 0.2.0 release. Currently that runs off generic > hadoop put we could certainly create distribution specific dockerfiles. > > Let me know if you need help. Keep in mind this is still an alpha project, > so expect some issues. Would love to get feedback, use cases and feature > ideas. > > Also a mesos tip: if your running services like hdfs outside of mesos you > should adjust you're mesos resources appropriately or you'll end up over > subscribed and processes will slow or die. > > Thanks for trying myriad! > > Darin > > > On Apr 21, 2016 4:22 AM, "rchen" <rc...@linkernetworks.com> wrote: > > Hi Guys, > Currrently, we are working on Myriad with Cloudera distribution. And we > have proved that Mesos and work with Yarn. > However talking about CM, how to integrate ? Any comments is welcome. > appreciated. > > > Regards > Sam > >
