Others will have to chime in for details, but typically this means you are having DNS issues. That is the hostname is resolving to an ip and not resolving back to the same name or vice versa or any other combo of non-roundtripping involving ip and dns names.
-ryan On Fri, Jul 9, 2010 at 6:41 PM, Arun Ramakrishnan <[email protected]> wrote: > I shutdown hbase. Added some new nodes to hdfs, rebalanced. Also added those > nodes to hbase regionservers. > Then started hbase. > > I am having this strange problem where the new nodes let's say host1 thru > host4 gets repeatedly reported/added to the regionservers list. > > Initially when I did a "report 'simple'" from the shell, it showed me 10 > unique hosts. Then within a matter of minutes it grew to 17 ( with the newly > added hosts repeating multiple times). > > Also, the web UI failed with the following error. > > ############## > HTTP ERROR: 500 > Trying to contact region server 192.168.130.63:60020 for region .META.,,1, > row '', but failed after 3 attempts. > Exceptions: > org.apache.hadoop.hbase.NotServingRegionException: > org.apache.hadoop.hbase.NotServingRegionException: .META.,,1 > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2266) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1845) > at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > ############### > > > Any insight into why the regions get repeated multiple times. I did a hadoop > fsck / and it reports that all the blocks have been replicated 3 times ( the > configured value ). > > > Thanks > Arun >
