Thanks Pavan! I've fixed a few spots of bugs where you're re-instatiating HBaseConfiguration objects within a Task JVM run code, which would have lead to you losing the config you actually send with the job (therefore, localhost).
You can view the new revision here: http://pastebin.com/9AXcXkCF. A diff is available at http://pastebin.com/diff.php?i=9AXcXkCF On Sun, Aug 25, 2013 at 10:31 AM, Pavan Sudheendra <[email protected]> wrote: > @Harsh, here it is http://pastebin.com/Qx0SFD3X > > Don't ask why i'm doing a join of hbase tables here.. Its probably > inefficient too.. I should optimize it.. > > > On Sun, Aug 25, 2013 at 10:16 AM, Harsh J <[email protected]> wrote: > >> Hello Pavan, >> >> Can you please share your entire Job Driver class source? >> >> On Thu, Aug 22, 2013 at 6:30 PM, Pavan Sudheendra <[email protected]> >> wrote: >> > All the zookeeper warnings are coming from takstracker logs. >> > >> > >> > On Thu, Aug 22, 2013 at 5:10 PM, Jean-Marc Spaggiari < >> > [email protected]> wrote: >> > >> >> The log file where you have extracted that from: >> >> 2013-08-21 13:38:55,815 INFO org.apache.zookeeper. >> >> ClientCnxn: Opening >> >> socket connection to server localhost/127.0.0.1:2181. Will not attempt >> to >> >> authenticate using SASL (Unable to locate a login configuration) >> >> java.net.ConnectException: Connection refused >> >> >> >> >> >> 2013/8/21 Pavan Sudheendra <[email protected]> >> >> >> >> > Sorry what files are you talking about? >> >> > >> >> > Regards, >> >> > Pavan >> >> > On Aug 22, 2013 12:04 AM, "Jean-Marc Spaggiari" < >> [email protected] >> >> > >> >> > wrote: >> >> > >> >> > > TaskTrackers and Job trackers are MR nodes. You also have HDFS nodes >> >> and >> >> > > HBase nodes. >> >> > > >> >> > > What's the file name where you got that from? >> >> > > >> >> > > JM >> >> > > >> >> > > 2013/8/21 Pavan Sudheendra <[email protected]> >> >> > > >> >> > > > We have 5 tasktrackers and 1 job tracker.. >> >> > > > I don't know why we have this.. This architecture was already >> >> existing >> >> > > when >> >> > > > i started working on this.. >> >> > > > >> >> > > > I just tried pinging the datanode from a namenode using >> >> > ip-10-34-187-170 >> >> > > > and it is able to ping it.. >> >> > > > Not sure its not able to pick it up and show it in the namenode >> >> logs... >> >> > > > Frankly i thought the namenode logs should be more useful and >> >> specific >> >> > > > instead of just giving out warning messages like this.. Let me >> know >> >> if >> >> > > you >> >> > > > need any more files to look at .. >> >> > > > >> >> > > > thanks for all the help :) >> >> > > > >> >> > > > >> >> > > > On Wed, Aug 21, 2013 at 7:13 PM, Jean-Marc Spaggiari < >> >> > > > [email protected]> wrote: >> >> > > > >> >> > > > > Hi Pavan, >> >> > > > > >> >> > > > > How mane namenodes do you have? Are you running HA and that's >> why >> >> you >> >> > > > have >> >> > > > > more than 1 namenode? >> >> > > > > >> >> > > > > Also, yes, if you don't have DNS, you need to have hosts file >> >> > > configured. >> >> > > > > >> >> > > > > First try to ping from one server to the other one using the >> name >> >> > only, >> >> > > > if >> >> > > > > it works, they you don't need to update the hosts file. >> >> > > > > >> >> > > > > JM >> >> > > > > >> >> > > > > 2013/8/21 Pavan Sudheendra <[email protected]> >> >> > > > > >> >> > > > > > I should update the /etc/hosts file on every namenode correct? >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > On Wed, Aug 21, 2013 at 7:09 PM, Pavan Sudheendra < >> >> > > [email protected] >> >> > > > > > >wrote: >> >> > > > > > >> >> > > > > > > But Jean all my namenodes log the same thing.. >> >> > > > > > > 2013-08-21 13:38:55,815 INFO >> org.apache.zookeeper.ClientCnxn: >> >> > > Opening >> >> > > > > > > socket connection to server localhost/127.0.0.1:2181. Will >> not >> >> > > > attempt >> >> > > > > > to >> >> > > > > > > authenticate using SASL (Unable to locate a login >> >> configuration) >> >> > > > > > > java.net.ConnectException: Connection refused >> >> > > > > > > >> >> > > > > > > Although i believe you, i'm starting to worry a bit.. Its >> >> > taking a >> >> > > > lot >> >> > > > > > of >> >> > > > > > > time for hbase to process 1 million entries in the cluster, >> let >> >> > > alone >> >> > > > > 19 >> >> > > > > > > million.. So, huge performance blow there.. >> >> > > > > > > >> >> > > > > > > >> >> > > > > > > >> >> > > > > > > >> >> > > > > > > On Wed, Aug 21, 2013 at 6:56 PM, Jean-Marc Spaggiari < >> >> > > > > > > [email protected]> wrote: >> >> > > > > > > >> >> > > > > > >> I'm running with this INFO for more than a year now ;) So >> no, >> >> I >> >> > > > don't >> >> > > > > > >> think >> >> > > > > > >> this is going to pose any real threats. You have everything >> >> > > > configured >> >> > > > > > >> correctly and everything seems to be working fine. >> >> > > > > > >> >> >> > > > > > >> JM >> >> > > > > > >> >> >> > > > > > >> 2013/8/21 Pavan Sudheendra <[email protected]> >> >> > > > > > >> >> >> > > > > > >> > it doesn't pose any real threats? >> >> > > > > > >> > >> >> > > > > > >> > >> >> > > > > > >> > On Wed, Aug 21, 2013 at 6:30 PM, Jean-Marc Spaggiari < >> >> > > > > > >> > [email protected]> wrote: >> >> > > > > > >> > >> >> > > > > > >> > > All fine on those logs too. >> >> > > > > > >> > > >> >> > > > > > >> > > So everything is working fine, ZK, HBase, the job is >> >> working >> >> > > > fine >> >> > > > > > too. >> >> > > > > > >> > The >> >> > > > > > >> > > only issue is this INFO regarding SAS, correct? >> >> > > > > > >> > > >> >> > > > > > >> > > I think you should simply ignore it. >> >> > > > > > >> > > >> >> > > > > > >> > > If it's annoying you, just turn >> >> > > org.apache.zookeeper.ClientCnxn >> >> > > > > > >> loglevel >> >> > > > > > >> > to >> >> > > > > > >> > > WARN on log4j.properties. (It's the setting I have on >> my >> >> own >> >> > > > > > cluster). >> >> > > > > > >> > > >> >> > > > > > >> > > JM >> >> > > > > > >> > > >> >> > > > > > >> > > 2013/8/21 Pavan Sudheendra <[email protected]> >> >> > > > > > >> > > >> >> > > > > > >> > > > @Jean the log which i got at the start of running the >> >> > hadoop >> >> > > > > jar: >> >> > > > > > >> Maybe >> >> > > > > > >> > > you >> >> > > > > > >> > > > can spot something >> >> > > > > > >> > > > >> >> > > > > > >> > > > 11:51:44,431 INFO ZooKeeper:100 - Client >> >> > > > > > >> > > > >> environment:java.library.path=/usr/lib/hadoop/lib/native >> >> > > > > > >> > > > 11:51:44,432 INFO ZooKeeper:100 - Client >> >> > > > > > >> > environment:java.io.tmpdir=/tmp >> >> > > > > > >> > > > 11:51:44,432 INFO ZooKeeper:100 - Client >> >> > > > > > >> > environment:java.compiler=<NA> >> >> > > > > > >> > > > 11:51:44,432 INFO ZooKeeper:100 - Client >> environment: >> >> > > os.name >> >> > > > > > =Linux >> >> > > > > > >> > > > 11:51:44,432 INFO ZooKeeper:100 - Client >> >> > > > > > environment:os.arch=amd64 >> >> > > > > > >> > > > 11:51:44,432 INFO ZooKeeper:100 - Client >> >> > > > > > >> > > > environment:os.version=3.2.0-23-virtual >> >> > > > > > >> > > > 11:51:44,432 INFO ZooKeeper:100 - Client >> environment: >> >> > > > user.name >> >> > > > > > >> =root >> >> > > > > > >> > > > 11:51:44,433 INFO ZooKeeper:100 - Client >> >> > > > > > >> environment:user.home=/root >> >> > > > > > >> > > > 11:51:44,433 INFO ZooKeeper:100 - Client >> >> > > > > > >> > > > >> >> > > environment:user.dir=/home/ubuntu/pasudhee/ActionDataInterpret >> >> > > > > > >> > > > 11:51:44,437 INFO ZooKeeper:438 - Initiating client >> >> > > > connection, >> >> > > > > > >> > > > connectString=localhost:2181 sessionTimeout=180000 >> >> > > > > > >> watcher=hconnection >> >> > > > > > >> > > > 11:51:44,493 INFO ClientCnxn:966 - Opening socket >> >> > > connection >> >> > > > to >> >> > > > > > >> server >> >> > > > > > >> > > > localhost/127.0.0.1:2181. Will not attempt to >> >> > authenticate >> >> > > > > using >> >> > > > > > >> SASL >> >> > > > > > >> > > > (Unable to locate a login configuration) >> >> > > > > > >> > > > 11:51:44,500 INFO RecoverableZooKeeper:104 - The >> >> > identifier >> >> > > > of >> >> > > > > > this >> >> > > > > > >> > > > process is >> >> > [email protected] >> >> > > > > > >> > > > 11:51:44,513 INFO ClientCnxn:849 - Socket connection >> >> > > > > established >> >> > > > > > to >> >> > > > > > >> > > > localhost/127.0.0.1:2181, initiating session >> >> > > > > > >> > > > 11:51:44,532 INFO ClientCnxn:1207 - Session >> >> establishment >> >> > > > > > complete >> >> > > > > > >> on >> >> > > > > > >> > > > server localhost/127.0.0.1:2181, sessionid = >> >> > > > 0x13ff1cff71bb167, >> >> > > > > > >> > > negotiated >> >> > > > > > >> > > > timeout = 60000 >> >> > > > > > >> > > > 11:51:44,743 INFO ZooKeeper:438 - Initiating client >> >> > > > connection, >> >> > > > > > >> > > > connectString=localhost:2181 sessionTimeout=180000 >> >> > > > > > >> watcher=hconnection >> >> > > > > > >> > > > 11:51:44,747 INFO ClientCnxn:966 - Opening socket >> >> > > connection >> >> > > > to >> >> > > > > > >> server >> >> > > > > > >> > > > localhost/127.0.0.1:2181. Will not attempt to >> >> > authenticate >> >> > > > > using >> >> > > > > > >> SASL >> >> > > > > > >> > > > (Unable to locate a login configuration) >> >> > > > > > >> > > > 11:51:44,747 INFO ClientCnxn:849 - Socket connection >> >> > > > > established >> >> > > > > > to >> >> > > > > > >> > > > localhost/127.0.0.1:2181, initiating session >> >> > > > > > >> > > > 11:51:44,747 INFO RecoverableZooKeeper:104 - The >> >> > identifier >> >> > > > of >> >> > > > > > this >> >> > > > > > >> > > > process is >> >> > [email protected] >> >> > > > > > >> > > > 11:51:44,749 INFO ClientCnxn:1207 - Session >> >> establishment >> >> > > > > > complete >> >> > > > > > >> on >> >> > > > > > >> > > > server localhost/127.0.0.1:2181, sessionid = >> >> > > > 0x13ff1cff71bb168, >> >> > > > > > >> > > negotiated >> >> > > > > > >> > > > timeout = 60000 >> >> > > > > > >> > > > 11:51:44,803 WARN Configuration:824 - >> hadoop.native.lib >> >> > is >> >> > > > > > >> deprecated. >> >> > > > > > >> > > > Instead, use io.native.lib.available >> >> > > > > > >> > > > 11:51:45,051 INFO >> >> > > > > > >> HConnectionManager$HConnectionImplementation:1789 - >> >> > > > > > >> > > > Closed zookeeper sessionid=0x13ff1cff71bb168 >> >> > > > > > >> > > > 11:51:45,054 INFO ZooKeeper:684 - Session: >> >> > > 0x13ff1cff71bb168 >> >> > > > > > closed >> >> > > > > > >> > > > 11:51:45,054 INFO ClientCnxn:509 - EventThread shut >> >> down >> >> > > > > > >> > > > 11:51:45,057 INFO ZooKeeper:438 - Initiating client >> >> > > > connection, >> >> > > > > > >> > > > connectString=localhost:2181 sessionTimeout=180000 >> >> > > > > > >> watcher=hconnection >> >> > > > > > >> > > > 11:51:45,059 INFO ClientCnxn:966 - Opening socket >> >> > > connection >> >> > > > to >> >> > > > > > >> server >> >> > > > > > >> > > > localhost/127.0.0.1:2181. Will not attempt to >> >> > authenticate >> >> > > > > using >> >> > > > > > >> SASL >> >> > > > > > >> > > > (Unable to locate a login configuration) >> >> > > > > > >> > > > 11:51:45,060 INFO ClientCnxn:849 - Socket connection >> >> > > > > established >> >> > > > > > to >> >> > > > > > >> > > > localhost/127.0.0.1:2181, initiating session >> >> > > > > > >> > > > 11:51:45,061 INFO ClientCnxn:1207 - Session >> >> establishment >> >> > > > > > complete >> >> > > > > > >> on >> >> > > > > > >> > > > server localhost/127.0.0.1:2181, sessionid = >> >> > > > 0x13ff1cff71bb169, >> >> > > > > > >> > > negotiated >> >> > > > > > >> > > > timeout = 60000 >> >> > > > > > >> > > > 11:51:45,065 INFO RecoverableZooKeeper:104 - The >> >> > identifier >> >> > > > of >> >> > > > > > this >> >> > > > > > >> > > > process is >> >> > [email protected] >> >> > > > > > >> > > > 11:51:45,135 INFO ZooKeeper:438 - Initiating client >> >> > > > connection, >> >> > > > > > >> > > > connectString=10.34.187.170:2181sessionTimeout=180000 >> >> > > > > > >> > > watcher=hconnection >> >> > > > > > >> > > > 11:51:45,137 INFO ClientCnxn:966 - Opening socket >> >> > > connection >> >> > > > to >> >> > > > > > >> server >> >> > > > > > >> > > > ip-10-34-187-170.eu-west-1.compute.internal/ >> >> > > > 10.34.187.170:2181. >> >> > > > > > >> Will >> >> > > > > > >> > not >> >> > > > > > >> > > > attempt to authenticate using SASL (Unable to locate >> a >> >> > login >> >> > > > > > >> > > configuration) >> >> > > > > > >> > > > 11:51:45,138 INFO ClientCnxn:849 - Socket connection >> >> > > > > established >> >> > > > > > to >> >> > > > > > >> > > > ip-10-34-187-170.eu-west-1.compute.internal/ >> >> > > > 10.34.187.170:2181, >> >> > > > > > >> > > initiating >> >> > > > > > >> > > > session >> >> > > > > > >> > > > 11:51:45,138 INFO RecoverableZooKeeper:104 - The >> >> > identifier >> >> > > > of >> >> > > > > > this >> >> > > > > > >> > > > process is >> >> > [email protected] >> >> > > > > > >> > > > 11:51:45,140 INFO ClientCnxn:1207 - Session >> >> establishment >> >> > > > > > complete >> >> > > > > > >> on >> >> > > > > > >> > > > server ip-10-34-187-170.eu-west-1.compute.internal/ >> >> > > > > > >> 10.34.187.170:2181, >> >> > > > > > >> > > > sessionid = 0x13ff1cff71bb16a, negotiated timeout = >> >> 60000 >> >> > > > > > >> > > > 11:51:45,173 INFO ZooKeeper:438 - Initiating client >> >> > > > connection, >> >> > > > > > >> > > > connectString=10.34.187.170:2181sessionTimeout=180000 >> >> > > > > > >> > > > >> >> > > > > > >> > > > >> >> > > > > > >> > > >> >> > > > > > >> > >> >> > > > > > >> >> >> > > > > > >> >> > > > > >> >> > > > >> >> > > >> >> > >> >> >> watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7444f787 >> >> > > > > > >> > > > 11:51:45,176 INFO ClientCnxn:966 - Opening socket >> >> > > connection >> >> > > > to >> >> > > > > > >> server >> >> > > > > > >> > > > ip-10-34-187-170.eu-west-1.compute.internal/ >> >> > > > 10.34.187.170:2181. >> >> > > > > > >> Will >> >> > > > > > >> > not >> >> > > > > > >> > > > attempt to authenticate using SASL (Unable to locate >> a >> >> > login >> >> > > > > > >> > > configuration) >> >> > > > > > >> > > > 11:51:45,176 INFO ClientCnxn:849 - Socket connection >> >> > > > > established >> >> > > > > > to >> >> > > > > > >> > > > ip-10-34-187-170.eu-west-1.compute.internal/ >> >> > > > 10.34.187.170:2181, >> >> > > > > > >> > > initiating >> >> > > > > > >> > > > session >> >> > > > > > >> > > > 11:51:45,178 INFO ClientCnxn:1207 - Session >> >> establishment >> >> > > > > > complete >> >> > > > > > >> on >> >> > > > > > >> > > > server ip-10-34-187-170.eu-west-1.compute.internal/ >> >> > > > > > >> 10.34.187.170:2181, >> >> > > > > > >> > > > sessionid = 0x13ff1cff71bb16b, negotiated timeout = >> >> 60000 >> >> > > > > > >> > > > 11:51:45,180 INFO RecoverableZooKeeper:104 - The >> >> > identifier >> >> > > > of >> >> > > > > > this >> >> > > > > > >> > > > process is >> >> > [email protected] >> >> > > > > > >> > > > 11:51:45,211 INFO ZooKeeper:684 - Session: >> >> > > 0x13ff1cff71bb16b >> >> > > > > > closed >> >> > > > > > >> > > > 11:51:45,211 INFO ClientCnxn:509 - EventThread shut >> >> down >> >> > > > > > >> > > > 11:51:45,218 INFO ZooKeeper:438 - Initiating client >> >> > > > connection, >> >> > > > > > >> > > > connectString=10.34.187.170:2181sessionTimeout=180000 >> >> > > > > > >> > > > >> >> > > > > > >> > > > >> >> > > > > > >> > > >> >> > > > > > >> > >> >> > > > > > >> >> >> > > > > > >> >> > > > > >> >> > > > >> >> > > >> >> > >> >> >> watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@7444f787 >> >> > > > > > >> > > > 11:51:45,220 INFO ClientCnxn:966 - Opening socket >> >> > > connection >> >> > > > to >> >> > > > > > >> server >> >> > > > > > >> > > > ip-10-34-187-170.eu-west-1.compute.internal/ >> >> > > > 10.34.187.170:2181. >> >> > > > > > >> Will >> >> > > > > > >> > not >> >> > > > > > >> > > > attempt to authenticate using SASL (Unable to locate >> a >> >> > login >> >> > > > > > >> > > configuration) >> >> > > > > > >> > > > >> >> > > > > > >> > > > >> >> > > > > > >> > > > >> >> > > > > > >> > > > On Wed, Aug 21, 2013 at 6:23 PM, Pavan Sudheendra < >> >> > > > > > >> [email protected] >> >> > > > > > >> > > > >wrote: >> >> > > > > > >> > > > >> >> > > > > > >> > > > > Yes .. The zookeeper server is also 10.34.187.170 >> .. >> >> > > > > > >> > > > > >> >> > > > > > >> > > > > >> >> > > > > > >> > > > > On Wed, Aug 21, 2013 at 6:21 PM, Jean-Marc >> Spaggiari < >> >> > > > > > >> > > > > [email protected]> wrote: >> >> > > > > > >> > > > > >> >> > > > > > >> > > > >> Are you able to connect to your ZK server shell >> and >> >> > list >> >> > > > the >> >> > > > > > >> nodes? >> >> > > > > > >> > > > >> >> >> > > > > > >> > > > >> 2013/8/21 Pavan Sudheendra <[email protected]> >> >> > > > > > >> > > > >> >> >> > > > > > >> > > > >> > Yes.. I can do everything.. But i do not want my >> >> > Hadoop >> >> > > > > > >> Namenode >> >> > > > > > >> > to >> >> > > > > > >> > > > >> report >> >> > > > > > >> > > > >> > logs like this.. Also, it says >> >> > > > > > >> > > > >> > >> >> > > > > > >> > > > >> > KeeperException, re-throwing exception >> >> > > > > > >> > > > >> > >> >> > > > > org.apache.zookeeper.KeeperException$ConnectionLossException: >> >> > > > > > >> > > > >> > KeeperErrorCode = ConnectionLoss for /hbase/ >> >> > > > > > >> > > > >> > >> >> > > > > > >> > > > >> > >> >> > > > > > >> > > > >> > >> >> > > > > > >> > > > >> > >> >> > > > > > >> > > > >> > >> >> > > > > > >> > > > >> > On Wed, Aug 21, 2013 at 6:14 PM, Jean-Marc >> >> Spaggiari >> >> > < >> >> > > > > > >> > > > >> > [email protected]> wrote: >> >> > > > > > >> > > > >> > >> >> > > > > > >> > > > >> > > Sound correct. You are able to start the shell >> >> and >> >> > > scan >> >> > > > > the >> >> > > > > > >> few >> >> > > > > > >> > > > first >> >> > > > > > >> > > > >> > line >> >> > > > > > >> > > > >> > > of the tables, right? >> >> > > > > > >> > > > >> > > >> >> > > > > > >> > > > >> > > 2013/8/21 Pavan Sudheendra < >> [email protected]> >> >> > > > > > >> > > > >> > > >> >> > > > > > >> > > > >> > > > This is my hbase-site.xml file if it helps: >> >> > > > > > >> > > > >> > > > >> >> > > > > > >> > > > >> > > > <?xml version="1.0" encoding="UTF-8"?> >> >> > > > > > >> > > > >> > > > >> >> > > > > > >> > > > >> > > > <!--Autogenerated by Cloudera CM on >> >> > > > > > >> > 2013-07-09T09:26:49.841Z--> >> >> > > > > > >> > > > >> > > > <configuration> >> >> > > > > > >> > > > >> > > > <property> >> >> > > > > > >> > > > >> > > > <name>hbase.rootdir</name> >> >> > > > > > >> > > > >> > > > >> >> > > > > > >> > > > >> > > > >> >> > > > > > >> > > > >> > > > >> >> > > > > > >> > > > >> > > >> >> > > > > > >> > > > >> > >> >> > > > > > >> > > > >> >> >> > > > > > >> > > > >> >> > > > > > >> > > >> >> > > > > > >> > >> >> > > > > > >> >> >> > > > > > >> >> > > > > >> >> > > > >> >> > > >> >> > >> >> >> <value>hdfs://ip-10-34-187-170.eu-west-1.compute.internal:8020/hbase</value> >> >> > > > > > >> > > > >> > > > </property> >> >> > > > > > >> > > > >> > > > <property> >> >> > > > > > >> > > > >> > > > <name>hbase.client.write.buffer</name> >> >> > > > > > >> > > > >> > > > <value>2097152</value> >> >> > > > > > >> > > > >> > > > </property> >> >> > > > > > >> > > > >> > > > <property> >> >> > > > > > >> > > > >> > > > <name>hbase.client.pause</name> >> >> > > > > > >> > > > >> > > > <value>1000</value> >> >> > > > > > >> > > > >> > > > </property> >> >> > > > > > >> > > > >> > > > <property> >> >> > > > > > >> > > > >> > > > <name>hbase.client.retries.number</name> >> >> > > > > > >> > > > >> > > > <value>10</value> >> >> > > > > > >> > > > >> > > > </property> >> >> > > > > > >> > > > >> > > > <property> >> >> > > > > > >> > > > >> > > > >> <name>hbase.client.scanner.caching</name> >> >> > > > > > >> > > > >> > > > <value>1</value> >> >> > > > > > >> > > > >> > > > </property> >> >> > > > > > >> > > > >> > > > <property> >> >> > > > > > >> > > > >> > > > >> <name>hbase.client.keyvalue.maxsize</name> >> >> > > > > > >> > > > >> > > > <value>10485760</value> >> >> > > > > > >> > > > >> > > > </property> >> >> > > > > > >> > > > >> > > > <property> >> >> > > > > > >> > > > >> > > > <name>hbase.rpc.timeout</name> >> >> > > > > > >> > > > >> > > > <value>60000</value> >> >> > > > > > >> > > > >> > > > </property> >> >> > > > > > >> > > > >> > > > <property> >> >> > > > > > >> > > > >> > > > >> <name>hbase.security.authentication</name> >> >> > > > > > >> > > > >> > > > <value>simple</value> >> >> > > > > > >> > > > >> > > > </property> >> >> > > > > > >> > > > >> > > > <property> >> >> > > > > > >> > > > >> > > > <name>zookeeper.session.timeout</name> >> >> > > > > > >> > > > >> > > > <value>60000</value> >> >> > > > > > >> > > > >> > > > </property> >> >> > > > > > >> > > > >> > > > <property> >> >> > > > > > >> > > > >> > > > <name>zookeeper.znode.parent</name> >> >> > > > > > >> > > > >> > > > <value>/hbase</value> >> >> > > > > > >> > > > >> > > > </property> >> >> > > > > > >> > > > >> > > > <property> >> >> > > > > > >> > > > >> > > > <name>zookeeper.znode.rootserver</name> >> >> > > > > > >> > > > >> > > > <value>root-region-server</value> >> >> > > > > > >> > > > >> > > > </property> >> >> > > > > > >> > > > >> > > > <property> >> >> > > > > > >> > > > >> > > > <name>hbase.zookeeper.quorum</name> >> >> > > > > > >> > > > >> > > > >> >> > > > > > >> <value>ip-10-34-187-170.eu-west-1.compute.internal</value> >> >> > > > > > >> > > > >> > > > </property> >> >> > > > > > >> > > > >> > > > <property> >> >> > > > > > >> > > > >> > > > >> >> > > <name>hbase.zookeeper.property.clientPort</name> >> >> > > > > > >> > > > >> > > > <value>2181</value> >> >> > > > > > >> > > > >> > > > </property> >> >> > > > > > >> > > > >> > > > </configuration> >> >> > > > > > >> > > > >> > > > >> >> > > > > > >> > > > >> > > > >> >> > > > > > >> > > > >> > > > >> >> > > > > > >> > > > >> > > > On Wed, Aug 21, 2013 at 6:09 PM, Jean-Marc >> >> > > Spaggiari >> >> > > > < >> >> > > > > > >> > > > >> > > > [email protected]> wrote: >> >> > > > > > >> > > > >> > > > >> >> > > > > > >> > > > >> > > > > Hum. >> >> > > > > > >> > > > >> > > > > >> >> > > > > > >> > > > >> > > > > Things seems to be correct there. >> >> > > > > > >> > > > >> > > > > >> >> > > > > > >> > > > >> > > > > Can you try something simple like: >> >> > > > > > >> > > > >> > > > > >> >> > > > > > >> > > > >> > > > > Configuration config = >> >> > > > > > >> > > HBaseConfiguration.create(); >> >> > > > > > >> > > > >> > > > > >> >> > config.set("hbase.zookeeper.quorum", >> >> > > > > > >> > > > >> "ip-10-34-187-170"); >> >> > > > > > >> > > > >> > > > > HTable table = new HTable >> >> (config, >> >> > > > > > >> > > > >> > > > > Bytes.toBytes("TABLE_NAME")); >> >> > > > > > >> > > > >> > > > > >> >> > > > > > >> > > > >> > > > > And see if it works? >> >> > > > > > >> > > > >> > > > > >> >> > > > > > >> > > > >> > > > > JM >> >> > > > > > >> > > > >> > > > > >> >> > > > > > >> > > > >> > > > > 2013/8/21 Pavan Sudheendra < >> >> > [email protected]> >> >> > > > > > >> > > > >> > > > > >> >> > > > > > >> > > > >> > > > > > Is this a zookeeper specific error or >> >> > > something? >> >> > > > > > >> > > > >> > > > > > >> >> > > > > > >> > > > >> > > > > > >> >> > > > > > >> > > > >> > > > > > On Wed, Aug 21, 2013 at 6:06 PM, Pavan >> >> > > > Sudheendra < >> >> > > > > > >> > > > >> > > [email protected] >> >> > > > > > >> > > > >> > > > > > >wrote: >> >> > > > > > >> > > > >> > > > > > >> >> > > > > > >> > > > >> > > > > > > Hi Jean, >> >> > > > > > >> > > > >> > > > > > > >> >> > > > > > >> > > > >> > > > > > > ubuntu@ip-10-34-187-170:~$ cat >> >> > /etc/hostname >> >> > > > > > >> > > > >> > > > > > > ip-10-34-187-170 >> >> > > > > > >> > > > >> > > > > > > ubuntu@ip-10-34-187-170:~$ hostname >> >> > > > > > >> > > > >> > > > > > > ip-10-34-187-170 >> >> > > > > > >> > > > >> > > > > > > >> >> > > > > > >> > > > >> > > > > > > >> >> > > > > > >> > > > >> > > > > > > >> >> > > > > > >> > > > >> > > > > > > On Wed, Aug 21, 2013 at 6:01 PM, >> >> Jean-Marc >> >> > > > > > Spaggiari >> >> > > > > > >> < >> >> > > > > > >> > > > >> > > > > > > [email protected]> wrote: >> >> > > > > > >> > > > >> > > > > > > >> >> > > > > > >> > > > >> > > > > > >> And what about: >> >> > > > > > >> > > > >> > > > > > >> # cat /etc/hostname >> >> > > > > > >> > > > >> > > > > > >> >> >> > > > > > >> > > > >> > > > > > >> and >> >> > > > > > >> > > > >> > > > > > >> # hostname >> >> > > > > > >> > > > >> > > > > > >> >> >> > > > > > >> > > > >> > > > > > >> ? >> >> > > > > > >> > > > >> > > > > > >> >> >> > > > > > >> > > > >> > > > > > >> 2013/8/21 Pavan Sudheendra < >> >> > > > [email protected] >> >> > > > > > >> >> > > > > > >> > > > >> > > > > > >> >> >> > > > > > >> > > > >> > > > > > >> > Sure.. >> >> > > > > > >> > > > >> > > > > > >> > /etc/hosts file: >> >> > > > > > >> > > > >> > > > > > >> > >> >> > > > > > >> > > > >> > > > > > >> > 127.0.0.1 localhost >> >> > > > > > >> > > > >> > > > > > >> > 10.34.187.170 ip-10-34-187-170 >> >> > > > > > >> > > > >> > > > > > >> > # The following lines are desirable >> >> for >> >> > > IPv6 >> >> > > > > > >> capable >> >> > > > > > >> > > > hosts >> >> > > > > > >> > > > >> > > > > > >> > ::1 ip6-localhost ip6-loopback >> >> > > > > > >> > > > >> > > > > > >> > fe00::0 ip6-localnet >> >> > > > > > >> > > > >> > > > > > >> > ff00::0 ip6-mcastprefix >> >> > > > > > >> > > > >> > > > > > >> > ff02::1 ip6-allnodes >> >> > > > > > >> > > > >> > > > > > >> > ff02::2 ip6-allrouters >> >> > > > > > >> > > > >> > > > > > >> > ff02::3 ip6-allhosts >> >> > > > > > >> > > > >> > > > > > >> > >> >> > > > > > >> > > > >> > > > > > >> > Configuration conf = >> >> > > > > > HBaseConfiguration.create(); >> >> > > > > > >> > > > >> > > > > > >> > conf.set("hbase.zookeeper.quorum", >> >> > > > > > >> "10.34.187.170"); >> >> > > > > > >> > > > >> > > > > > >> > >> >> > > > > > >> > > >> conf.set("hbase.zookeeper.property.clientPort","2181"); >> >> > > > > > >> > > > >> > > > > > >> > >> >> > > conf.set("hbase.master","10.34.187.170"); >> >> > > > > > >> > > > >> > > > > > >> > Job job = new Job(conf, >> >> > ViewersTable); >> >> > > > > > >> > > > >> > > > > > >> > >> >> > > > > > >> > > > >> > > > > > >> > I'm trying to process table data >> which >> >> > has >> >> > > > 19 >> >> > > > > > >> million >> >> > > > > > >> > > > >> rows..It >> >> > > > > > >> > > > >> > > > runs >> >> > > > > > >> > > > >> > > > > > fine >> >> > > > > > >> > > > >> > > > > > >> > for a while although i don't see >> the >> >> map >> >> > > > > percent >> >> > > > > > >> > > > completion >> >> > > > > > >> > > > >> > > change >> >> > > > > > >> > > > >> > > > > > from >> >> > > > > > >> > > > >> > > > > > >> 0% >> >> > > > > > >> > > > >> > > > > > >> > .. After a while it says >> >> > > > > > >> > > > >> > > > > > >> > >> >> > > > > > >> > > > >> > > > > > >> > Task >> >> > attempt_201304161625_0028_m_000000_0 >> >> > > > > failed >> >> > > > > > >> to >> >> > > > > > >> > > > report >> >> > > > > > >> > > > >> > > status >> >> > > > > > >> > > > >> > > > > for >> >> > > > > > >> > > > >> > > > > > >> > 600 seconds. Killing! >> >> > > > > > >> > > > >> > > > > > >> > >> >> > > > > > >> > > > >> > > > > > >> > >> >> > > > > > >> > > > >> > > > > > >> > >> >> > > > > > >> > > > >> > > > > > >> > >> >> > > > > > >> > > > >> > > > > > >> > >> >> > > > > > >> > > > >> > > > > > >> > On Wed, Aug 21, 2013 at 5:52 PM, >> >> > Jean-Marc >> >> > > > > > >> Spaggiari >> >> > > > > > >> > < >> >> > > > > > >> > > > >> > > > > > >> > [email protected]> wrote: >> >> > > > > > >> > > > >> > > > > > >> > >> >> > > > > > >> > > > >> > > > > > >> > > Can you past you host file here >> >> again >> >> > > with >> >> > > > > the >> >> > > > > > >> > > > >> modification >> >> > > > > > >> > > > >> > > you >> >> > > > > > >> > > > >> > > > > have >> >> > > > > > >> > > > >> > > > > > >> > done? >> >> > > > > > >> > > > >> > > > > > >> > > >> >> > > > > > >> > > > >> > > > > > >> > > Also, can you share a big more of >> >> you >> >> > > > code? >> >> > > > > > What >> >> > > > > > >> > are >> >> > > > > > >> > > > you >> >> > > > > > >> > > > >> > doing >> >> > > > > > >> > > > >> > > > > with >> >> > > > > > >> > > > >> > > > > > >> the >> >> > > > > > >> > > > >> > > > > > >> > > config object after, how do you >> >> create >> >> > > > your >> >> > > > > > >> table >> >> > > > > > >> > > > object, >> >> > > > > > >> > > > >> > > etc.? >> >> > > > > > >> > > > >> > > > > > >> > > >> >> > > > > > >> > > > >> > > > > > >> > > Thanks, >> >> > > > > > >> > > > >> > > > > > >> > > >> >> > > > > > >> > > > >> > > > > > >> > > JM >> >> > > > > > >> > > > >> > > > > > >> > > >> >> > > > > > >> > > > >> > > > > > >> > > 2013/8/21 Pavan Sudheendra < >> >> > > > > > [email protected] >> >> > > > > > >> > >> >> > > > > > >> > > > >> > > > > > >> > > >> >> > > > > > >> > > > >> > > > > > >> > > > @Jean tried your method didn't >> >> > work.. >> >> > > > > > >> > > > >> > > > > > >> > > > >> >> > > > > > >> > > > >> > > > > > >> > > > 2013-08-21 12:17:10,908 INFO >> >> > > > > > >> > > > >> > > org.apache.zookeeper.ClientCnxn: >> >> > > > > > >> > > > >> > > > > > >> Opening >> >> > > > > > >> > > > >> > > > > > >> > > > socket connection to server >> >> > localhost/ >> >> > > > > > >> > > 127.0.0.1:2181 >> >> > > > > > >> > > > . >> >> > > > > > >> > > > >> > Will >> >> > > > > > >> > > > >> > > > not >> >> > > > > > >> > > > >> > > > > > >> attempt >> >> > > > > > >> > > > >> > > > > > >> > > to >> >> > > > > > >> > > > >> > > > > > >> > > > authenticate using SASL >> (Unable to >> >> > > > locate >> >> > > > > a >> >> > > > > > >> login >> >> > > > > > >> > > > >> > > > configuration) >> >> > > > > > >> > > > >> > > > > > >> > > > 2013-08-21 12:17:10,908 WARN >> >> > > > > > >> > > > >> > > org.apache.zookeeper.ClientCnxn: >> >> > > > > > >> > > > >> > > > > > >> Session >> >> > > > > > >> > > > >> > > > > > >> > 0x0 >> >> > > > > > >> > > > >> > > > > > >> > > > for server null, unexpected >> error, >> >> > > > closing >> >> > > > > > >> socket >> >> > > > > > >> > > > >> > connection >> >> > > > > > >> > > > >> > > > and >> >> > > > > > >> > > > >> > > > > > >> > > attempting >> >> > > > > > >> > > > >> > > > > > >> > > > reconnect >> >> > > > > > >> > > > >> > > > > > >> > > > java.net.ConnectException: >> >> > Connection >> >> > > > > > refused >> >> > > > > > >> > > > >> > > > > > >> > > > at >> >> > > > > > >> > > > sun.nio.ch.SocketChannelImpl.checkConnect(Native >> >> > > > > > >> > > > >> > > > Method) >> >> > > > > > >> > > > >> > > > > > >> > > > at >> >> > > > > > >> > > > >> > > > > > >> > > > >> >> > > > > > >> > > > >> > > > > > >> >> >> > > > > > >> > > > >> > > > >> >> > > > > > >> > > > >> >> >> > > > > > >> > >> >> > > > > >> >> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567) >> >> > > > > > >> > > > >> > > > > > >> > > > at >> >> > > > > > >> > > > >> > > > > > >> > > > >> >> > > > > > >> > > > >> > > > > > >> > > > >> >> > > > > > >> > > > >> > > > > > >> > > >> >> > > > > > >> > > > >> > > > > > >> > >> >> > > > > > >> > > > >> > > > > > >> >> >> > > > > > >> > > > >> > > > > > >> >> > > > > > >> > > > >> > > > > >> >> > > > > > >> > > > >> > > > >> >> > > > > > >> > > > >> > > >> >> > > > > > >> > > > >> > >> >> > > > > > >> > > > >> >> >> > > > > > >> > > > >> >> > > > > > >> > > >> >> > > > > > >> > >> >> > > > > > >> >> >> > > > > > >> >> > > > > >> >> > > > >> >> > > >> >> > >> >> >> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) >> >> > > > > > >> > > > >> > > > > > >> > > > at >> >> > > > > > >> > > > >> > > > > > >> > > >> >> > > > > > >> > > > >> > > > > >> >> > > > > > >> > > > >> >> >> > > > > > >> >> >> > > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068) >> >> > > > > > >> > > > >> > > > > > >> > > > 2013-08-21 12:17:11,009 WARN >> >> > > > > > >> > > > >> > > > > > >> > > > >> >> > > > > > >> > > > >> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: >> >> > > > > > >> > > > >> > > > Possibly >> >> > > > > > >> > > > >> > > > > > >> > > transient >> >> > > > > > >> > > > >> > > > > > >> > > > ZooKeeper exception: >> >> > > > > > >> > > > >> > > > > > >> > > > >> >> > > > > > >> > > > >> > > >> >> > > > > > org.apache.zookeeper.KeeperException$ConnectionLossException: >> >> > > > > > >> > > > >> > > > > > >> > > > KeeperErrorCode = >> ConnectionLoss >> >> for >> >> > > > > /hbase >> >> > > > > > >> > > > >> > > > > > >> > > > 2013-08-21 12:17:11,009 INFO >> >> > > > > > >> > > > >> > > > > > >> >> >> org.apache.hadoop.hbase.util.RetryCounter: >> >> > > > > > >> > > > >> > > > > > >> > > > Sleeping 8000ms before retry >> >> #3...\ >> >> > > > > > >> > > > >> > > > > > >> > > > >> >> > > > > > >> > > > >> > > > > > >> > > > Any tips? >> >> > > > > > >> > > > >> > > > > > >> > > > >> >> > > > > > >> > > > >> > > > > > >> > > > >> >> > > > > > >> > > > >> > > > > > >> > > > >> >> > > > > > >> > > > >> > > > > > >> > > > On Wed, Aug 21, 2013 at 5:15 >> PM, >> >> > > > Jean-Marc >> >> > > > > > >> > > Spaggiari >> >> > > > > > >> > > > < >> >> > > > > > >> > > > >> > > > > > >> > > > [email protected]> >> wrote: >> >> > > > > > >> > > > >> > > > > > >> > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > Hi Pavan, >> >> > > > > > >> > > > >> > > > > > >> > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > I don't think Cloudera >> Manager >> >> > > assign >> >> > > > > the >> >> > > > > > >> > address >> >> > > > > > >> > > > to >> >> > > > > > >> > > > >> > your >> >> > > > > > >> > > > >> > > > > > >> computer. >> >> > > > > > >> > > > >> > > > > > >> > > When >> >> > > > > > >> > > > >> > > > > > >> > > > CM >> >> > > > > > >> > > > >> > > > > > >> > > > > is down, your computer still >> >> have >> >> > an >> >> > > > IP, >> >> > > > > > and >> >> > > > > > >> > even >> >> > > > > > >> > > > if >> >> > > > > > >> > > > >> you >> >> > > > > > >> > > > >> > > > > > >> un-install >> >> > > > > > >> > > > >> > > > > > >> > CM, >> >> > > > > > >> > > > >> > > > > > >> > > > you >> >> > > > > > >> > > > >> > > > > > >> > > > > will still have an IP >> assigned >> >> to >> >> > > your >> >> > > > > > >> > computer. >> >> > > > > > >> > > > >> > > > > > >> > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > If you have not configured >> >> > anything >> >> > > > > there, >> >> > > > > > >> then >> >> > > > > > >> > > you >> >> > > > > > >> > > > >> most >> >> > > > > > >> > > > >> > > > > > probably >> >> > > > > > >> > > > >> > > > > > >> > have >> >> > > > > > >> > > > >> > > > > > >> > > a >> >> > > > > > >> > > > >> > > > > > >> > > > > DHCP. Just give a try to >> what I >> >> > told >> >> > > > you >> >> > > > > > on >> >> > > > > > >> the >> >> > > > > > >> > > > other >> >> > > > > > >> > > > >> > > > message. >> >> > > > > > >> > > > >> > > > > > >> > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > JM >> >> > > > > > >> > > > >> > > > > > >> > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > 2013/8/21 Pavan Sudheendra < >> >> > > > > > >> > [email protected]> >> >> > > > > > >> > > > >> > > > > > >> > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > @Manoj i have set >> >> > > > > hbase.zookeeper.quorum >> >> > > > > > >> in >> >> > > > > > >> > my >> >> > > > > > >> > > > M-R >> >> > > > > > >> > > > >> > > > > > application.. >> >> > > > > > >> > > > >> > > > > > >> > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > @Jean The cloudera manager >> >> picks >> >> > > up >> >> > > > > the >> >> > > > > > ip >> >> > > > > > >> > > > address >> >> > > > > > >> > > > >> > > > > > >> automatically.. >> >> > > > > > >> > > > >> > > > > > >> > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > On Wed, Aug 21, 2013 at >> 5:07 >> >> PM, >> >> > > > manoj >> >> > > > > > p < >> >> > > > > > >> > > > >> > > > [email protected] >> >> > > > > > >> > > > >> > > > > > >> >> > > > > > >> > > > >> > > > > > >> > wrote: >> >> > > > > > >> > > > >> > > > > > >> > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > Can you try passing the >> >> > argument >> >> > > > > > >> > > > >> > > > > > >> > > > >> >> > -Dhbase.zookeeper.quorum=10.34.187.170 >> >> > > > > > >> > > > >> > > > > > >> > > > > > > while running the program >> >> > > > > > >> > > > >> > > > > > >> > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > If this does'nt work >> either >> >> > > please >> >> > > > > > >> check if >> >> > > > > > >> > > > >> > HBASE_HOME >> >> > > > > > >> > > > >> > > > and >> >> > > > > > >> > > > >> > > > > > >> > > > > HBASE_CONF_DIR >> >> > > > > > >> > > > >> > > > > > >> > > > > > > is set correctly. >> >> > > > > > >> > > > >> > > > > > >> > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > BR/Manoj >> >> > > > > > >> > > > >> > > > > > >> > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > On Wed, Aug 21, 2013 at >> 4:48 >> >> > PM, >> >> > > > > Pavan >> >> > > > > > >> > > > >> Sudheendra < >> >> > > > > > >> > > > >> > > > > > >> > > > [email protected] >> >> > > > > > >> > > > >> > > > > > >> > > > > > > >wrote: >> >> > > > > > >> > > > >> > > > > > >> > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > Yes. My /etc/hosts have >> >> the >> >> > > > > correct >> >> > > > > > >> > mapping >> >> > > > > > >> > > > to >> >> > > > > > >> > > > >> > > > localhost >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > 127.0.0.1 localhost >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > # The following lines >> are >> >> > > > > desirable >> >> > > > > > >> for >> >> > > > > > >> > > IPv6 >> >> > > > > > >> > > > >> > capable >> >> > > > > > >> > > > >> > > > > hosts >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > ::1 ip6-localhost >> >> > > > ip6-loopback >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > fe00::0 ip6-localnet >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > ff00::0 ip6-mcastprefix >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > ff02::1 ip6-allnodes >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > ff02::2 ip6-allrouters >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > I've added the HBase >> jars >> >> to >> >> > > the >> >> > > > > > >> Hadoop >> >> > > > > > >> > > > >> Classpath >> >> > > > > > >> > > > >> > as >> >> > > > > > >> > > > >> > > > > well. >> >> > > > > > >> > > > >> > > > > > >> Not >> >> > > > > > >> > > > >> > > > > > >> > > sure >> >> > > > > > >> > > > >> > > > > > >> > > > > > why.. >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > I'm running this on a 6 >> >> node >> >> > > > > > cloudera >> >> > > > > > >> > > cluster >> >> > > > > > >> > > > >> > which >> >> > > > > > >> > > > >> > > > > > consist >> >> > > > > > >> > > > >> > > > > > >> of >> >> > > > > > >> > > > >> > > > > > >> > 1 >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > jobtrackers and 5 >> >> > > tasktrackers.. >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > After a while all my >> map >> >> > jobs >> >> > > > > fail.. >> >> > > > > > >> > > > Completely >> >> > > > > > >> > > > >> > > > baffled >> >> > > > > > >> > > > >> > > > > > >> because >> >> > > > > > >> > > > >> > > > > > >> > > the >> >> > > > > > >> > > > >> > > > > > >> > > > > map >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > tasks were doing the >> >> > required >> >> > > > > > tasks.. >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > On Wed, Aug 21, 2013 at >> >> 4:45 >> >> > > PM, >> >> > > > > > >> manoj p >> >> > > > > > >> > < >> >> > > > > > >> > > > >> > > > > > [email protected] >> >> > > > > > >> > > > >> > > > > > >> > >> >> > > > > > >> > > > >> > > > > > >> > > > wrote: >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > For your code to run, >> >> > please >> >> > > > > > ensure >> >> > > > > > >> if >> >> > > > > > >> > > you >> >> > > > > > >> > > > >> use >> >> > > > > > >> > > > >> > the >> >> > > > > > >> > > > >> > > > > > correct >> >> > > > > > >> > > > >> > > > > > >> > > > > > HBase/Hadoop >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > jar >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > versions while >> compiling >> >> > > your >> >> > > > > > >> program. >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > BR/Manoj >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > On Wed, Aug 21, 2013 >> at >> >> > 4:38 >> >> > > > PM, >> >> > > > > > >> manoj >> >> > > > > > >> > p >> >> > > > > > >> > > < >> >> > > > > > >> > > > >> > > > > > >> [email protected]> >> >> > > > > > >> > > > >> > > > > > >> > > > > wrote: >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > > Check your >> /etc/hosts >> >> > file >> >> > > > if >> >> > > > > > you >> >> > > > > > >> > have >> >> > > > > > >> > > > the >> >> > > > > > >> > > > >> > > correct >> >> > > > > > >> > > > >> > > > > > >> mapping >> >> > > > > > >> > > > >> > > > > > >> > to >> >> > > > > > >> > > > >> > > > > > >> > > > > > > localhost >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > > for 127.0.0.1. Also >> >> > ensure >> >> > > > > that >> >> > > > > > if >> >> > > > > > >> > you >> >> > > > > > >> > > > have >> >> > > > > > >> > > > >> > > > > > >> > > > > hbase.zookeeper.quorum >> >> > > > > > >> > > > >> > > > > > >> > > > > > in >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > your >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > > configuration and >> also >> >> > > check >> >> > > > > if >> >> > > > > > >> HBase >> >> > > > > > >> > > > >> > classpath >> >> > > > > > >> > > > >> > > is >> >> > > > > > >> > > > >> > > > > > >> appended >> >> > > > > > >> > > > >> > > > > > >> > > to >> >> > > > > > >> > > > >> > > > > > >> > > > > > Hadoop >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > > classpath. >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > > BR/Manoj >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > > On Wed, Aug 21, >> 2013 >> >> at >> >> > > 4:10 >> >> > > > > PM, >> >> > > > > > >> > Pavan >> >> > > > > > >> > > > >> > > Sudheendra >> >> > > > > > >> > > > >> > > > < >> >> > > > > > >> > > > >> > > > > > >> > > > > > > [email protected] >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >wrote: >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> Hadoop Namenode >> >> reports >> >> > > the >> >> > > > > > >> > following >> >> > > > > > >> > > > >> error >> >> > > > > > >> > > > >> > > which >> >> > > > > > >> > > > >> > > > > is >> >> > > > > > >> > > > >> > > > > > >> > > unusual : >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> 013-08-21 >> >> 09:21:12,328 >> >> > > INFO >> >> > > > > > >> > > > >> > > > > > >> > org.apache.zookeeper.ClientCnxn: >> >> > > > > > >> > > > >> > > > > > >> > > > > > Opening >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> socket >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> connection to >> server >> >> > > > > localhost/ >> >> > > > > > >> > > > >> > 127.0.0.1:2181. >> >> > > > > > >> > > > >> > > > > Will >> >> > > > > > >> > > > >> > > > > > >> not >> >> > > > > > >> > > > >> > > > > > >> > > > attempt >> >> > > > > > >> > > > >> > > > > > >> > > > > > to >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> authenticate using >> >> SASL >> >> > > > > (Unable >> >> > > > > > >> to >> >> > > > > > >> > > > locate >> >> > > > > > >> > > > >> a >> >> > > > > > >> > > > >> > > login >> >> > > > > > >> > > > >> > > > > > >> > > > configuration) >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> >> > > java.net.ConnectException: >> >> > > > > > >> > Connection >> >> > > > > > >> > > > >> refused >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> at >> >> > > > > > >> > > > >> > > > > > >> >> > > sun.nio.ch.SocketChannelImpl.checkConnect(Native >> >> > > > > > >> > > > >> > > > > > >> > > > Method) >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> at >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> >> > > > > > >> > > > >> > > > > > >> > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > >> >> > > > > > >> > > > >> > > > > > >> >> >> > > > > > >> > > > >> > > > >> >> > > > > > >> > > > >> >> >> > > > > > >> > >> >> > > > > >> >> > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567) >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> at >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > >> >> > > > > > >> > > > >> > > > > > >> > > >> >> > > > > > >> > > > >> > > > > > >> > >> >> > > > > > >> > > > >> > > > > > >> >> >> > > > > > >> > > > >> > > > > > >> >> > > > > > >> > > > >> > > > > >> >> > > > > > >> > > > >> > > > >> >> > > > > > >> > > > >> > > >> >> > > > > > >> > > > >> > >> >> > > > > > >> > > > >> >> >> > > > > > >> > > > >> >> > > > > > >> > > >> >> > > > > > >> > >> >> > > > > > >> >> >> > > > > > >> >> > > > > >> >> > > > >> >> > > >> >> > >> >> >> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> at >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> >> > > > > > >> > > > >> > > > > > >> > > > > > >> >> > > > > > >> > > > >> > > > > > >> > >> >> > > > > > >> > > > >> > > > >> >> > > > > > >> > > > >> >> > > > > > >> >> > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068) >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> 2013-08-21 >> >> 09:33:11,033 >> >> > > > WARN >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> >> > > > > > >> > > > >> > > > > > >> >> > > > > > org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: >> >> > > > > > >> > > > >> > > > > > >> > > > Possibly >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > transient >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> ZooKeeper >> exception: >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> >> > > > > > >> > > > >> > > > > > >> > > >> >> > > > > > >> > > > >> > >> >> > > > > org.apache.zookeeper.KeeperException$ConnectionLossException: >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> KeeperErrorCode = >> >> > > > > > ConnectionLoss >> >> > > > > > >> for >> >> > > > > > >> > > > >> /hbase >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> 2013-08-21 >> >> 09:33:11,033 >> >> > > > INFO >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > >> >> > > > > > >> > org.apache.hadoop.hbase.util.RetryCounter: >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> Sleeping 8000ms >> >> before >> >> > > > retry >> >> > > > > > >> #3... >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> 2013-08-21 >> >> 09:33:11,043 >> >> > > > WARN >> >> > > > > > >> > > > >> > > > > > >> > org.apache.hadoop.mapred.Task: >> >> > > > > > >> > > > >> > > > > > >> > > > > Parent >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > died. >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> Exiting >> >> > > > > > >> > > > >> attempt_201307181246_0548_m_000022_2 >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> Because i have >> >> > specified >> >> > > > the >> >> > > > > > >> address >> >> > > > > > >> > > in >> >> > > > > > >> > > > >> the >> >> > > > > > >> > > > >> > > java >> >> > > > > > >> > > > >> > > > > file >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> Configuration >> >> conf >> >> > = >> >> > > > > > >> > > > >> > > > > HBaseConfiguration.create(); >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> >> > > > > > >> > conf.set("hbase.zookeeper.quorum", >> >> > > > > > >> > > > >> > > > > > >> "10.34.187.170"); >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> >> > > > > > >> > > > >> > > > > > >> > >> >> > > > > > >> > conf.set("hbase.zookeeper.property.clientPort","2181"); >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> >> > > > > > >> > > > >> conf.set("hbase.master","10.34.187.170"); >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> All my map tasks >> fail >> >> > > like >> >> > > > > > this! >> >> > > > > > >> > > Please >> >> > > > > > >> > > > >> > help.. >> >> > > > > > >> > > > >> > > > I'm >> >> > > > > > >> > > > >> > > > > > on a >> >> > > > > > >> > > > >> > > > > > >> > > > timebomb >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> -- >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> Regards- >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> Pavan >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > -- >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > Regards- >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > Pavan >> >> > > > > > >> > > > >> > > > > > >> > > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > > -- >> >> > > > > > >> > > > >> > > > > > >> > > > > > Regards- >> >> > > > > > >> > > > >> > > > > > >> > > > > > Pavan >> >> > > > > > >> > > > >> > > > > > >> > > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > > >> >> > > > > > >> > > > >> > > > > > >> > > > >> >> > > > > > >> > > > >> > > > > > >> > > > >> >> > > > > > >> > > > >> > > > > > >> > > > >> >> > > > > > >> > > > >> > > > > > >> > > > -- >> >> > > > > > >> > > > >> > > > > > >> > > > Regards- >> >> > > > > > >> > > > >> > > > > > >> > > > Pavan >> >> > > > > > >> > > > >> > > > > > >> > > > >> >> > > > > > >> > > > >> > > > > > >> > > >> >> > > > > > >> > > > >> > > > > > >> > >> >> > > > > > >> > > > >> > > > > > >> > >> >> > > > > > >> > > > >> > > > > > >> > >> >> > > > > > >> > > > >> > > > > > >> > -- >> >> > > > > > >> > > > >> > > > > > >> > Regards- >> >> > > > > > >> > > > >> > > > > > >> > Pavan >> >> > > > > > >> > > > >> > > > > > >> > >> >> > > > > > >> > > > >> > > > > > >> >> >> > > > > > >> > > > >> > > > > > > >> >> > > > > > >> > > > >> > > > > > > >> >> > > > > > >> > > > >> > > > > > > >> >> > > > > > >> > > > >> > > > > > > -- >> >> > > > > > >> > > > >> > > > > > > Regards- >> >> > > > > > >> > > > >> > > > > > > Pavan >> >> > > > > > >> > > > >> > > > > > > >> >> > > > > > >> > > > >> > > > > > >> >> > > > > > >> > > > >> > > > > > >> >> > > > > > >> > > > >> > > > > > >> >> > > > > > >> > > > >> > > > > > -- >> >> > > > > > >> > > > >> > > > > > Regards- >> >> > > > > > >> > > > >> > > > > > Pavan >> >> > > > > > >> > > > >> > > > > > >> >> > > > > > >> > > > >> > > > > >> >> > > > > > >> > > > >> > > > >> >> > > > > > >> > > > >> > > > >> >> > > > > > >> > > > >> > > > >> >> > > > > > >> > > > >> > > > -- >> >> > > > > > >> > > > >> > > > Regards- >> >> > > > > > >> > > > >> > > > Pavan >> >> > > > > > >> > > > >> > > > >> >> > > > > > >> > > > >> > > >> >> > > > > > >> > > > >> > >> >> > > > > > >> > > > >> > >> >> > > > > > >> > > > >> > >> >> > > > > > >> > > > >> > -- >> >> > > > > > >> > > > >> > Regards- >> >> > > > > > >> > > > >> > Pavan >> >> > > > > > >> > > > >> > >> >> > > > > > >> > > > >> >> >> > > > > > >> > > > > >> >> > > > > > >> > > > > >> >> > > > > > >> > > > > >> >> > > > > > >> > > > > -- >> >> > > > > > >> > > > > Regards- >> >> > > > > > >> > > > > Pavan >> >> > > > > > >> > > > > >> >> > > > > > >> > > > >> >> > > > > > >> > > > >> >> > > > > > >> > > > >> >> > > > > > >> > > > -- >> >> > > > > > >> > > > Regards- >> >> > > > > > >> > > > Pavan >> >> > > > > > >> > > > >> >> > > > > > >> > > >> >> > > > > > >> > >> >> > > > > > >> > >> >> > > > > > >> > >> >> > > > > > >> > -- >> >> > > > > > >> > Regards- >> >> > > > > > >> > Pavan >> >> > > > > > >> > >> >> > > > > > >> >> >> > > > > > > >> >> > > > > > > >> >> > > > > > > >> >> > > > > > > -- >> >> > > > > > > Regards- >> >> > > > > > > Pavan >> >> > > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > -- >> >> > > > > > Regards- >> >> > > > > > Pavan >> >> > > > > > >> >> > > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > -- >> >> > > > Regards- >> >> > > > Pavan >> >> > > > >> >> > > >> >> > >> >> >> > >> > >> > >> > -- >> > Regards- >> > Pavan >> >> >> >> -- >> Harsh J >> > > > > -- > Regards- > Pavan -- Harsh J
