One more question, would it matter what version of Hadoop that I have?
On Thu, May 30, 2013 at 6:57 PM, Lewis John Mcgibbney < [email protected]> wrote: > In all honesty I would make sure that you have a local and up-to-date > nutch-$version.job file generated and try it out in runtime/local before > using the job in /runtime/deploy on your cluster. > You will know if it is good to go or not. > When you are ready to deploy it to your cluster (e.g. once your satisfied > that it works on a test/sub set crawl) setup then just make it available to > your Hadoop Job tracker classpath. > > > On Thu, May 30, 2013 at 3:48 PM, Yves S. Garret > <[email protected]>wrote: > > > I have $HADOOP_INSTALL in my path, would this be enough Lewis? Or > > would I need to copy around some jar files? > > > > > > On Thu, May 30, 2013 at 6:35 PM, Lewis John Mcgibbney < > > [email protected]> wrote: > > > > > Make sure that everything is compiled and you are running from runtime > or > > > with the Jar in hadoop > > > > > > > > > On Thu, May 30, 2013 at 3:00 PM, Yves S. Garret > > > <[email protected]>wrote: > > > > > > > Here is my hbase-site.xml: > > > > http://bin.cakephp.org/view/2054577438 > > > > > > > > I've set this property as well. > > > > > > > > > > > > On Thu, May 30, 2013 at 5:57 PM, Shah, Nishant <[email protected]> > > > wrote: > > > > > > > > > What about your storage.data.store.class property in nutch-site.xml > > ? I > > > > > think you have to change the value to use hbase. For me it is > > > > > org.apache.gora.hbase.store.HBasetore. > > > > > > > > > > -----Original Message----- > > > > > From: Yves S. Garret [mailto:[email protected]] > > > > > Sent: Thursday, May 30, 2013 2:52 PM > > > > > To: [email protected] > > > > > Subject: Re: How to setup HBase as backend > > > > > > > > > > Yes. For the moment, for simplicity sake, I have everything going > to > > > > /tmp. > > > > > > > > > > hbase(main):004:0> scan 'test' > > > > > ROW > > > > > COLUMN+CELL > > > > > > > > > > 0 row(s) in 0.2370 seconds > > > > > > > > > > I _should_ have a table "webpage being created when I run Nutch. > > > > > > > > > > > > > > > On Thu, May 30, 2013 at 5:23 PM, Shah, Nishant <[email protected] > > > > > > wrote: > > > > > > > > > > > Is your hbase running ? > > > > > > > > > > > > -----Original Message----- > > > > > > From: Yves S. Garret [mailto:[email protected]] > > > > > > Sent: Thursday, May 30, 2013 2:18 PM > > > > > > To: [email protected] > > > > > > Subject: Re: How to setup HBase as backend > > > > > > > > > > > > Even when I do bin/nutch generate, this is what I get: > > > > > > http://bin.cakephp.org/view/1815127825 > > > > > > > > > > > > > > > > > > On Thu, May 30, 2013 at 5:14 PM, Yves S. Garret > > > > > > <[email protected]>wrote: > > > > > > > > > > > > > Ok, similar issue: > > > > > > > http://bin.cakephp.org/view/180499048 > > > > > > > > > > > > > > I've left the defaults for config as they were, except this is > in > > > > > > > gora.properties in apache nutch. > > > > > > > gora.datastore.default=org.apache.gora.hbase.store.HBaseStore > > > > > > > > > > > > > > > > > > > > > On Wed, May 29, 2013 at 7:40 PM, Lewis John Mcgibbney < > > > > > > > [email protected]> wrote: > > > > > > > > > > > > > >> Yes, as Tejas mentioned, He runs fine with 0.90.6 API changes > > make > > > > > > >> more recent HBase versions incompatible. > > > > > > >> We will be upgrading HBase API usage in Gora within the > current > > > > > > >> development drive. > > > > > > >> Lewis > > > > > > >> > > > > > > >> > > > > > > >> On Wed, May 29, 2013 at 4:36 PM, Yves S. Garret > > > > > > >> <[email protected]>wrote: > > > > > > >> > > > > > > >> > Would HBase 0.90.X and Nutch 2.1 work? > > > > > > >> > > > > > > > >> > > > > > > > >> > On Wed, May 29, 2013 at 5:05 PM, Lewis John Mcgibbney < > > > > > > >> > [email protected]> wrote: > > > > > > >> > > > > > > > >> > > This is incompatible. > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > On Wed, May 29, 2013 at 1:59 PM, Yves S. Garret > > > > > > >> > > <[email protected]>wrote: > > > > > > >> > > > > > > > > >> > > > Hi all, I'm using HBase 0.94.7 and Nutch 2.1. > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > On Wed, May 29, 2013 at 4:55 PM, Adriana Farina > > > > > > >> > > > <[email protected]>wrote: > > > > > > >> > > > > > > > > > >> > > > > Hi Yves, > > > > > > >> > > > > > > > > > > >> > > > > as Tejas said, your issue is almost certainly due to a > > > > > > >> compatibility > > > > > > >> > > > > problem between the version of Nutch and the one of > > HBase. > > > > > > >> > > > > > > > > > > >> > > > > I had the same problem and in my case it was due to > the > > > > > > >> > > > > HBase > > > > > > >> > version. > > > > > > >> > > > > > > > > > > >> > > > > I use Nutch 2.1 with HBase 0.90.4 and it works fine. > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > 2013/5/29 Yves S. Garret <[email protected]> > > > > > > >> > > > > > > > > > > >> > > > > > Hi, I'm trying to run Nutch this time around with > > HBase > > > > > > >> > > > > > in the > > > > > > >> > > > > background, > > > > > > >> > > > > > as > > > > > > >> > > > > > opposed to having MySQL instead. > > > > > > >> > > > > > > > > > > > >> > > > > > In the past, I followed this tutorial: > > > > > > >> > > > > > http://nlp.solutions.asia/?p=180 > > > > > > >> > > > > > > > > > > > >> > > > > > This was all in good, but now that I have my HBase, > > I'd > > > > > > >> > > > > > like to > > > > > > >> use > > > > > > >> > > > that. > > > > > > >> > > > > > I left the configuration of Nutch as it was and > > > proceeded > > > > > > >> > > > > > to > > > > > > >> crawl > > > > > > >> > > > > > nutch.apache.org. I got this error: > > > > > > >> > > > > > http://bin.cakephp.org/view/1301117746 > > > > > > >> > > > > > > > > > > > >> > > > > > What am I doing wrong? > > > > > > >> > > > > > > > > > > > >> > > > > > At the moment, I'm reading through this, trying to > get > > > my > > > > > > >> > > > > > stack > > > > > > >> to > > > > > > >> > > > work, > > > > > > >> > > > > > will write back if I make any progress: > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > http://sujitpal.blogspot.com/2011/01/exploring-nutch-20-hbase-stora > > > > > > >> ge > > > > > > >> .html > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > -- > > > > > > >> > > > > Adriana Farina > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > -- > > > > > > >> > > *Lewis* > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> -- > > > > > > >> *Lewis* > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > *Lewis* > > > > > > > > > -- > *Lewis* >
