And nothing after that? Because this would be the first assignment we see in the region server log.
J-D On Fri, Jun 25, 2010 at 2:49 PM, Ted Yu <[email protected]> wrote: > 10.32.56.159 and sjc9-flash-grid05 are the same server: > > 2010-06-25 09:56:52,308 INFO org.apache.hadoop.hbase.master.ServerManager: 2 > region servers, 0 dead, average load 22.52010-06-25 09:56:52,490 INFO > org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner > scanning meta region {server: 10.32.56.159:60020, regionname: -ROOT-,,0, > startKey: <>} > 2010-06-25 09:56:52,498 DEBUG org.apache.hadoop.hbase.master.BaseScanner: > Current assignment of .META.,,1 is not valid; serverAddress= > 10.32.56.160:60020, startCode=1277484838693 unknown. > 2010-06-25 09:56:52,499 INFO org.apache.hadoop.hbase.master.BaseScanner: > RegionManager.rootScanner scan of 1 row(s) of meta region {server: > 10.32.56.159:60020, regionname: -ROOT-,,0, startKey: <>} complete > 2010-06-25 09:56:52,847 DEBUG org.apache.hadoop.hbase.master.RegionManager: > Assigning for address: 10.32.56.159:60020, startcode: 1277484792423, load: > (requests=5, regions=23, usedHeap=30, maxHeap=3991): total nregions to > assign=0, regions to give other servers than this=1, isMetaAssign=true > 2010-06-25 09:56:52,847 DEBUG org.apache.hadoop.hbase.master.RegionManager: > Assigning address: 10.32.56.159:60020, startcode: 1277484792423, load: > (requests=5, regions=23, usedHeap=30, maxHeap=3991) 0 regions > 2010-06-25 09:56:52,847 INFO org.apache.hadoop.hbase.master.RegionManager: > Assigning region .META.,,1 to sjc9-flash-grid05.carrieriq.com > ,60020,1277484792423 > 2010-06-25 09:56:52,849 INFO org.apache.hadoop.hbase.master.ServerManager: > Processing MSG_REPORT_OPEN: .META.,,1 from > sjc9-flash-grid05.carrieriq.com,60020,1277484792423; > 1 of 1 > 2010-06-25 09:56:52,850 DEBUG org.apache.hadoop.hbase.master.HMaster: > Processing todo: PendingOpenOperation from sjc9-flash-grid05.carrieriq.com > ,60020,1277484792423 > 2010-06-25 09:56:52,850 INFO > org.apache.hadoop.hbase.master.RegionServerOperation: .META.,,1 open on > 10.32.56.159:60020 > 2010-06-25 09:56:52,852 INFO > org.apache.hadoop.hbase.master.RegionServerOperation: Updated row .META.,,1 > in region -ROOT-,,0 with startcode=1277484792423, server=10.32.56.159:60020 > 2010-06-25 09:56:52,852 DEBUG > org.apache.hadoop.hbase.master.RegionServerOperation: Adding to > onlineMetaRegions: {server: 10.32.56.159:60020, regionname: .META.,,1, > startKey: <>} > 2010-06-25 09:56:52,853 INFO org.apache.hadoop.hbase.master.BaseScanner: > RegionManager.metaScanner scanning meta region {server: 10.32.56.159:60020, > regionname: .META.,,1, startKey: <>} > > On Fri, Jun 25, 2010 at 2:33 PM, Jean-Daniel Cryans > <[email protected]>wrote: > >> While taking a second look, I see that actually your region server is >> the one who is opening it over and over again: >> >> 2010-06-25 09:56:03,918 INFO >> org.apache.hadoop.hbase.regionserver.HRegionServer: Worker: >> MSG_REGION_OPEN: .META.,,1 >> 2010-06-25 09:56:03,919 DEBUG >> org.apache.hadoop.hbase.regionserver.HRegion: Creating region >> .META.,,1, encoded=1028785192 >> 2010-06-25 09:56:03,933 DEBUG >> org.apache.hadoop.hbase.regionserver.Store: loaded >> /hbase/.META./1028785192/info/7628076653206028272, isReference=false, >> sequence id=2991, length=51941, majorCompaction=false >> ... >> 2010-06-25 10:00:03,447 INFO >> org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN: >> .META.,,1 >> 2010-06-25 10:00:03,448 INFO >> org.apache.hadoop.hbase.regionserver.HRegionServer: Worker: >> MSG_REGION_OPEN: .META.,,1 >> ... >> 2010-06-25 10:02:03,130 INFO >> org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN: >> .META.,,1 >> 2010-06-25 10:02:03,130 INFO >> org.apache.hadoop.hbase.regionserver.HRegionServer: Worker: >> MSG_REGION_OPEN: .META.,,1 >> >> These messages come from the master, look for "Assigning region >> .META.,,1" not the encoded name. >> >> J-D >> >> >> On Fri, Jun 25, 2010 at 2:22 PM, Ted Yu <[email protected]> wrote: >> > Hi, >> > 2010-06-25 09:56:03,936 INFO >> org.apache.hadoop.hbase.regionserver.HRegion: >> > region .META.,,1/1028785192 available; sequence id is 2992 >> > >> > I searched for 1028785192 in master log but didn't find any occurrence. >> > The only log about compaction I see in master log is something like this: >> > >> > 2010-06-25 09:59:39,898 DEBUG >> org.apache.hadoop.hbase.regionserver.HRegion: >> > Closing TED-PACKAGESUMMARY-1277485167424-0,,1277485173774: disabling >> > compactions & flushes >> > >> > Any insight ? >> > >> > Thanks >> > >> > On Fri, Jun 25, 2010 at 11:31 AM, Jean-Daniel Cryans < >> [email protected]> >> > wrote: >> >> >> >> At first glance it looks like a double assignment of .META., and the >> >> file it's trying to get was probably already compacted by another >> >> region server. Check the master log to see why/how it happened. >> >> >> >> J-D >> >> >> >> On Fri, Jun 25, 2010 at 11:13 AM, Ted Yu <[email protected]> wrote: >> >> > Hi, >> >> > I upgraded a 3 node (1 master and 2 RS) cluster to 0.20.5 this >> morning. >> >> > hadoop version is Cloudera 0.20.2+228 >> >> > I saw the following in >> >> > hbase-hadoop-regionserver-sjc9-flash-grid05.ciq.com.log >> >> > (/hbase/.META./1028785192/info/7628076653206028272 was not on hdfs): >> >> > >> >> > http://pastebin.com/0rH56Vzb >> >> > >> >> > This is from hadoop-hadoop-datanode-sjc9-flash-grid05.ciq.com.log: >> >> > http://pastebin.com/eYvq6FEV >> >> > >> >> > Has anyone seen similar issue ? >> >> > >> >> > Thanks >> >> > >> > >> > >> >
