Hi Michael I am using the latest Cloudera release and it's working fine. You can use any Linux distro you are comfortable with. Centos is mostly used for server deployments and it's quite stable.
Thanks Divjot On 15-Aug-2017 2:09 AM, "Michael Chen" <yiningchen2...@u.northwestern.edu> wrote: Hi Divjot, Thanks for the information! I was wondering if there is a specific version of cloudera manager and CDH that works best with Nutch 2.x (HBase 1.2.3, Hadoop 2.5.2)? Also, is there a specific reason to use Centos 7 instead of Amazon Linux or Red Hat? I’ll try to get started with the setup. Thanks! Michael From: Divjot Singh Sent: Tuesday, August 8, 2017 04:06 To: user@nutch.apache.org Subject: Re: Best practice for Nutch 2.x on AWS? Hi We have a setup of Hbase on an AWS cluster with centos 7. The setup was done using cloudera-manager. Nutch can be then run in standalone mode or over yarn by running the deployment jar in deploy folder. I have not tested with S3 directly but your can always backup the hbase data daily to S3. Hope this helps.Let me know if you have further queries. Divjot On Sun, Aug 6, 2017 at 5:59 AM, Michael Chen < yiningchen2...@u.northwestern.edu> wrote: > Hi, > > I'm trying to set up Nutch 2.x on AWS EC2 clusters, and I was wondering if > anyone know of a "best set up" for it. The hadoop and hbase version in > current EMR releases doesn't seem to work with Nutch 2.x. Does it sound > like a good idea to manually set up Hadoop clusters and then run Nutch on > it? Will I be able to use S3 as data storage so that I can keep the data > when EC2 instance stops? > > Any suggestions would be very much helpful! > > Thanks in advance, > > Michael > >
