Hi Vidhya, Indeed it looks like your table must have a "hole" in it - ie that you have no region for this key range. That means your table somehow got corrupted, since a table should always cover the entirety of key ranges.
Any idea what happened to the region that should include this HFile? We can probably fix up the hfile loader bug so it doesn't infinite loop, but the root cause is that your table is corrupt. -Todd On Fri, Sep 10, 2010 at 3:19 PM, Vidhyashankar Venkataraman < [email protected]> wrote: > JG > What I ran into was not an issue with region splits: the created Hfile > does not fit in the range of any existing region which means a new region > needs to be created. And in this case, the system goes to the mode of > repeatedly splitting this Hfile... > Shall I report a bug and follow up on it? > Vidhya > > On 9/10/10 1:42 PM, "Jonathan Gray" <[email protected]> wrote: > > I ran into something like this as well but were in a rush to get the import > done so didn't look into it. I forgot about it so didn't follow up. > > We ended up ensuring regions would not be split during the job (configuring > the split size way up) and reran the MR job. > > JG > > > -----Original Message----- > > From: Vidhyashankar Venkataraman [mailto:[email protected]] > > Sent: Friday, September 10, 2010 11:43 AM > > To: [email protected]; [email protected] > > Subject: Problem with bulk incremental loads.. > > > > I was trying to bulk increment some files into a HBAse (0.89) table and > > found this problem.. > > > > If a file does not fit into any of the regions in the existing table, > > then the tool gets into an infinite loop of splitting the files.. I > > have attached a sample output.. Todd, is this a known issue? > > > > Vidhya > > > > 10/09/07 01:57:29 INFO mapreduce.LoadIncrementalHFiles: Trying to load > > hfile=hdfs://b3130080.yst.yahoo.net:4600/user/crawler/docd_inc_v1/bigCo > > lumn/8375781572558986795 first=0000003511885973 last=0000003511999994 > > 10/09/07 01:57:29 INFO mapreduce.LoadIncrementalHFiles: HFile at > > hdfs://b3130080.yst.yahoo.net:4600/user/crawler/docd_inc_v1/bigColumn/8 > > 375781572558986795 no longer fits inside a single region. Splitting... > > 10/09/07 01:57:37 INFO mapreduce.LoadIncrementalHFiles: Successfully > > split into new HFiles > > hdfs://b3130080.yst.yahoo.net:4600/user/crawler/docd_inc_v1/bigColumn/_ > > tmp/fae6ef95297635e32e24c572bec9056e.bottom and > > hdfs://b3130080.yst.yahoo.net:4600/user/crawler/docd_inc_v1/bigColumn/_ > > tmp/fae6ef95297635e32e24c572bec9056e.top > > 10/09/07 01:57:37 INFO mapreduce.LoadIncrementalHFiles: Trying to load > > hfile=hdfs://b3130080.yst.yahoo.net:4600/user/crawler/docd_inc_v1/bigCo > > lumn/_tmp/fae6ef95297635e32e24c572bec9056e.top first=0000003511885973 > > last=0000003511999994 > > 10/09/07 01:57:37 INFO mapreduce.LoadIncrementalHFiles: HFile at > > hdfs://b3130080.yst.yahoo.net:4600/user/crawler/docd_inc_v1/bigColumn/_ > > tmp/fae6ef95297635e32e24c572bec9056e.top no longer fits inside a single > > region. Splitting... > > 10/09/07 01:57:44 INFO mapreduce.LoadIncrementalHFiles: Successfully > > split into new HFiles > > hdfs://b3130080.yst.yahoo.net:4600/user/crawler/docd_inc_v1/bigColumn/_ > > tmp/_tmp/fae6ef95297635e32e24c572bec9056e.bottom and > > hdfs://b3130080.yst.yahoo.net:4600/user/crawler/docd_inc_v1/bigColumn/_ > > tmp/_tmp/fae6ef95297635e32e24c572bec9056e.top > > 10/09/07 01:57:44 INFO mapreduce.LoadIncrementalHFiles: Trying to load > > hfile=hdfs://b3130080.yst.yahoo.net:4600/user/crawler/docd_inc_v1/bigCo > > lumn/_tmp/_tmp/fae6ef95297635e32e24c572bec9056e.top > > first=0000003511885973 last=0000003511999994 > > 10/09/07 01:57:44 INFO mapreduce.LoadIncrementalHFiles: HFile at > > hdfs://b3130080.yst.yahoo.net:4600/user/crawler/docd_inc_v1/bigColumn/_ > > tmp/_tmp/fae6ef95297635e32e24c572bec9056e.top no longer fits inside a > > single region. Splitting... > > 10/09/07 01:57:51 INFO mapreduce.LoadIncrementalHFiles: Successfully > > split into new HFiles > > hdfs://b3130080.yst.yahoo.net:4600/user/crawler/docd_inc_v1/bigColumn/_ > > tmp/_tmp/_tmp/fae6ef95297635e32e24c572bec9056e.bottom and > > hdfs://b3130080.yst.yahoo.net:4600/user/crawler/docd_inc_v1/bigColumn/_ > > tmp/_tmp/_tmp/fae6ef95297635e32e24c572bec9056e.top > > 10/09/07 01:57:51 INFO mapreduce.LoadIncrementalHFiles: Trying to load > > hfile=hdfs://b3130080.yst.yahoo.net:4600/user/crawler/docd_inc_v1/bigCo > > lumn/_tmp/_tmp/_tmp/fae6ef95297635e32e24c572bec9056e.top > > first=0000003511885973 last=0000003511999994 > > 10/09/07 01:57:51 INFO mapreduce.LoadIncrementalHFiles: HFile at > > hdfs://b3130080.yst.yahoo.net:4600/user/crawler/docd_inc_v1/bigColumn/_ > > tmp/_tmp/_tmp/fae6ef95297635e32e24c572bec9056e.top no longer fits > > inside a single region. Splitting... > > 10/09/07 01:57:59 INFO mapreduce.LoadIncrementalHFiles: Successfully > > split into new HFiles > > hdfs://b3130080.yst.yahoo.net:4600/user/crawler/docd_inc_v1/bigColumn/_ > > tmp/_tmp/_tmp/_tmp/fae6ef95297635e32e24c572bec9056e.bottom and > > hdfs://b3130080.yst.yahoo.net:4600/user/crawler/docd_inc_v1/bigColumn/_ > > tmp/_tmp/_tmp/_tmp/fae6ef95297635e32e24c572bec9056e.top > > 10/09/07 01:57:59 INFO mapreduce.LoadIncrementalHFiles: Trying to load > > hfile=hdfs://b3130080.yst.yahoo.net:4600/user/crawler/docd_inc_v1/bigCo > > lumn/_tmp/_tmp/_tmp/_tmp/fae6ef95297635e32e24c572bec9056e.top > > first=0000003511885973 last=0000003511999994 > > 10/09/07 01:57:59 INFO mapreduce.LoadIncrementalHFiles: HFile at > > hdfs://b3130080.yst.yahoo.net:4600/user/crawler/docd_inc_v1/bigColumn/_ > > tmp/_tmp/_tmp/_tmp/fae6ef95297635e32e24c572bec9056e.top no longer fits > > inside a single region. Splitting... > > 10/09/07 01:58:06 INFO mapreduce.LoadIncrementalHFiles: Successfully > > split into new HFiles > > hdfs://b3130080.yst.yahoo.net:4600/user/crawler/docd_inc_v1/bigColumn/_ > > tmp/_tmp/_tmp/_tmp/_tmp/fae6ef95297635e32e24c572bec9056e.bottom and > > hdfs://b3130080.yst.yahoo.net:4600/user/crawler/docd_inc_v1/bigColumn/_ > > tmp/_tmp/_tmp/_tmp/_tmp/fae6ef95297635e32e24c572bec9056e.top > > > -- Todd Lipcon Software Engineer, Cloudera
