Thanks to all! Now I can run nutch and hbase. just wonder if there are good information abouthow to move further with nutch, e.g. how to configure regular expression for crawling etc. Regards/Dan
________________________________________ Från: Divjot Singh [[email protected]] Skickat: den 15 februari 2016 12:56 Till: [email protected] Ämne: Re: no respond after inject you must have messed up with the directory permissions. Login with "hddan" user and try to install hbase from scatch and run it again. Follow this tutorial http://www.tutorialspoint.com/hbase/hbase_installation.htm Thanks On Fri, Feb 12, 2016 at 5:34 PM, <[email protected]> wrote: > Hi Divjot, > I followed the tutorial, installed the hadoop under a new user "hddan" > then I move the "hbase-0.98.8-hadoop2" (installed before) folder to the > new owner "hddan" > > ???Hadoop is running as described, excepted that accessing to browser > failed??? > tried http://localhost:50070 > http://localhost:9000 > > ???when I run start-hase.sh, the terminal shows??? > Error: Could not find or load main class > org.apache.hadoop.hbase.util.HBaseConfTool > Error: Could not find or load main class > org.apache.hadoop.hbase.zookeeper.ZKServerTool > mkdir: cannot create directory ‘/home/dan/hbase-0.98.8-hadoop2’: > Permission denied > starting master, logging to > /home/dan/hbase-0.98.8-hadoop2/logs/hbase-hddan-master-dan-inlab.out > /usr/local/hbase-0.98.8-hadoop2/bin/hbase-daemon.sh: line 185: > /home/dan/hbase-0.98.8-hadoop2/logs/hbase-hddan-master-dan-inlab.out: No > such file or directory > head: cannot open > ‘/home/dan/hbase-0.98.8-hadoop2/logs/hbase-hddan-master-dan-inlab.out’ for > reading: No such file or directory > cat: /home/dan/hbase-0.98.8-hadoop2/conf/regionservers: No such file or > directory > > I did installed hbase earlier under /home/dan..., now I have changed them > under /usr/local/ and with the new owner as hddan > > I have changed the variable in .bashrc for HADOOP_HOME, JAVA_HOME AND > HBASE_HOME accordingly. > > in the conf/hbase-site.xml I configured as followings > > <property> > <name>hbase.rootdir</name> > <value>file:///usr/local/hbase-0.98.8-hadoop2/</value> > </property> > <property> > <name>hbase.cluster.distributed</name> > <value>false</value> > </property> > <property> > <name>hbase.zookeeper.property.dataDir</name> > <value>/home/hddan/hadoop/zookeeper</value> > </property> > > I am not sure if rootdir is a path variable for hbase configuration, or a > directory for saving files. > Many thanks for any opinions! > ________________________________________ > Från: Divjot Singh [[email protected]] > Skickat: den 11 februari 2016 18:45 > Till: [email protected] > Ämne: Re: no respond after inject > > This error is because hbase is not up. That's why nutch is not able to > connect. Please start Hbase in standalone mode and try to create a table > using Hbase shell. If this works then nutch would work. Remember to run > Hbase at localhost because that is the IP nutch is trying to connect to. > If you are confused with Hbase check a basic tutorial for setting up Hbase > at http://www.tutorialspoint.com/hbase/hbase_installation.htm > > When you run Hbase is standalone mode it starts zookeeper on its own.Later > you can change it to distributed mode. > > Hope this helps > > Thanks > Divjot > > On Thu, Feb 11, 2016 at 10:14 PM, <[email protected]> wrote: > > > Errors found in logs > > > > Unable to load native-hadoop library for your platform... using > > builtin-java classes where applicable > > 2016-02-09 09:21:30,012 WARN zookeeper.ClientCnxn - Session 0x0 for > > server null, unexpected error, closing socket connection and attempting > > reconnect > > java.net.ConnectException: Connection refused > > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > > at > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) > > at > > > org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) > > at > > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068) > > 2016-02-09 09:21:30,128 WARN zookeeper.RecoverableZooKeeper - Possibly > > transient ZooKeeper, quorum=localhost:2181, > > exception=org.apache.zookeeper.KeeperException$ConnectionLossException: > > KeeperErrorCode = ConnectionLoss for /hbase/hbaseid > > java.net.ConnectException: Connection refused > > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > > at > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) > > at > > > org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) > > at > > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068) > > 2016-02-09 09:21:30,128 WARN zookeeper.RecoverableZooKeeper - Possibly > > transient ZooKeeper, quorum=localhost:2181, > > exception=org.apache.zookeeper.KeeperException$ConnectionLossException: > > KeeperErrorCode = ConnectionLoss for /hbase/hbaseid > > > > I downloaded hbase-0.98.8-hadoop2, and set the following in > hbase-site,xml > > > > <property> > > <name>hbase.cluster.distributed</name> > > <value>false</value> > > </property> > > > > The question is Do I need to download zookeeper, which I did not do. > > > > In the hbase/lib, I have jar files like > > haddop-client-2.2.0.jar > > hadoop-common-2.2.0.jar > > hadoop-hdfs-2.2.0.jar > > .... > > zookeeper-3.4.6.jar > > > > MANY THANKS FOR HELPING! > > ________________________________________ > > Från: Divjot Singh [[email protected]] > > Skickat: den 11 februari 2016 12:38 > > Till: [email protected] > > Ämne: Re: no respond after inject > > > > Hi > > > > Can you please check *runtime/local/logs/hadoop.log* file. It would > contain > > the exact error. Also make sure that Hbase is running fine and you can > > create/read table from hbase shell. > > > > Also check that you have set this property to true in gora.properties. > > > > *gora.datastore.autocreateschema=true* > > > > > > Thanks > > Divjot > > > > On Thu, Feb 11, 2016 at 2:47 PM, <[email protected]> wrote: > > > > > Yes, I was in the directory runtime/local, tried > > > bin/nutch inject seed/urls.txt > > > and ./bin/nutch inject seed/urls.txt > > > > > > the system reacts the same. > > > two lines come first > > > > InjectorJob: starting at 2016-02-09 09:21:29 > > > > InjectorJob: Injecting urlDir: seed/urls.txt > > > after a long time, around 15 minutes, comes the third line > > > > InjectorJob: Using class org.apache.gora.hbase.store.HBaseStore as > the > > > > Gora storage class. > > > ________________________________________ > > > Från: Lewis John Mcgibbney [[email protected]] > > > Skickat: den 10 februari 2016 16:00 > > > Till: [email protected] > > > Ämne: Re: no respond after inject > > > > > > Hi Dan, > > > > > > On Wed, Feb 10, 2016 at 5:06 AM, <[email protected]> > > > wrote: > > > > > > > > > > > Sorry for another beginner question. After I installed nutch-2.3.1. > > > > hbase-0.98.9 and elasticsearch-2.1.0 > > > > I start to test to crawl one website with 'nutch inject urls.txt' > > > > > > > > The terminal window shows only this, which seems the crawling work is > > > > going on for ever. > > > > > > > > InjectorJob: starting at 2016-02-09 09:21:29 > > > > InjectorJob: Injecting urlDir: seed/urls.txt > > > > InjectorJob: Using class org.apache.gora.hbase.store.HBaseStore as > the > > > > Gora storage class. > > > > > > > > what can be the possible error for this? Many thanks. > > > > > > > > > > > I think the issue may be here that you are meant to pass a directory as > > the > > > argument for the InjectorJob as oppose to a flat file. Can you please > try > > > > > > ./bin/nutch inject urls/urls.txt > > > > > > Thanks > > > Lewis > > > > > >
