Also, in looking at the master, it clearly thinks it's ip address is 127.0.0.1, although it is accepting remote connections on port 60000 now.
On Thu, May 1, 2008 at 12:47 PM, Daniel Leffel <[EMAIL PROTECTED]> wrote: > Switching to all IPs (both in hbase-site.xml and regionservers file) causes > the local region server and master to start up fine. The remote ones start > throwing the following exception (debug enabled): > > 2008-05-01 14:45:32,284 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: > Initializing RPC Metrics with hostName=60020, port=60020 > 2008-05-01 14:45:32,326 DEBUG org.apache.hadoop.hbase.HRegionServer: > Telling master at 10.10.1.114:60000 that we are up > 2008-05-01 14:45:32,376 WARN org.apache.hadoop.hbase.HRegionServer: error > telling master we are up > org.apache.hadoop.ipc.RemoteException: > org.apache.hadoop.hbase.Leases$LeaseStillHeldException > at org.apache.hadoop.hbase.Leases.createLease(Leases.java:147) > at > org.apache.hadoop.hbase.HMaster.regionServerStartup(HMaster.java:1318) > at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:585) > at > org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:901) > > at org.apache.hadoop.ipc.Client.call(Client.java:512) > at > org.apache.hadoop.hbase.ipc.HbaseRPC$Invoker.invoke(HbaseRPC.java:210) > at $Proxy0.regionServerStartup(Unknown Source) > at > org.apache.hadoop.hbase.HRegionServer.reportForDuty(HRegionServer.java:1112) > at > org.apache.hadoop.hbase.HRegionServer.run(HRegionServer.java:712) > at java.lang.Thread.run(Thread.java:595) > > > > > On Thu, May 1, 2008 at 10:13 AM, stack <[EMAIL PROTECTED]> wrote: > >> Daniel: >> >> The line out of RpcMetrics reporting port for hostname looks like a little >> bug we should fix up in hadoop core (I took a quick look; looks like the >> right args are being passed -- problem must be further up the stack or >> perhaps your hadoop is older?). >> >> Regards the next line reporting localhost, thats odd. Is your networking >> setup properly? What does the host report for IP when you query using the >> hostname you've inserted as the value for hbase.master? What if you use IP >> instead? >> >> Here is what my logs looks like: >> >> 2008-05-01 04:15:34,234 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: >> Initializing RPC Metrics with hostName=60000, port=60000 >> 2008-05-01 04:15:34,361 INFO org.apache.hadoop.hbase.HMaster: HMaster >> initialized on XXX.XXX.44.139:60000 >> >> Here is my hbase.master: >> >> aa0-XXX-XXX.at.our.domain:60000 >> >> St.Ack >> >> P.S. And on offlining of tables, we're still working on it..... should >> have a candidate up in the next day or so w/ a fix included. >> >> >> >> Daniel Leffel wrote: >> >>> Actually, I did find something curious in the master log with debug >>> turned >>> on: >>> >>> 2008-05-01 11:05:56,407 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: >>> Initializing RPC Metrics with hostName=60000, port=60000 >>> 2008-05-01 11:05:56,482 INFO org.apache.hadoop.hbase.HMaster: HMaster >>> initialized on 127.0.0.1:60000 >>> >>> hostname 60000? but then the next line shows localhost? >>> >>> On Thu, May 1, 2008 at 9:32 AM, Daniel Leffel <[EMAIL PROTECTED]> >>> wrote: >>> >>> >>> >>>> Not sure what's going on with this and thought I'd ask. I can't find any >>>> rationale for this on the wiki or in the logs. >>>> >>>> I've begun to set up our production environment which is a multi-node >>>> setup. (To-date our development environment was a single node in >>>> psudeo-distributed mode). >>>> >>>> The HBase master is listening on port 60000 only for connections that >>>> are >>>> local and is refusing remote connections. All the other services on the >>>> box >>>> (60010,600020 and 600030) listen both locally and remotely. There's no >>>> firewall on the box. >>>> >>>> In the hbase-site.xml, I've specificed hbase.master to be >>>> [hostname]:60000 >>>> >>>> Any ideas? >>>> >>>> >>>> >>> >>> >>> >> >> >
