Hi, I got it resolved. https://issues.apache.org/jira/browse/HADOOP-5750 was causing this, even though I supplied a custom total ordering partitioner, it didnt use that.
Now the regions looks properly sorted, but facing a new issue. The last key of the each region is not retrievable. The table.jsp page shows the start and end key wrongly. for eg, take first 2 regions region1: start : end: 333305184e0f7c3e region2: start: 333305184e0f7c3e end: 666629fe4378c096 The end key of first region = start key of second ?? If I get the first and last key using HFile.Reader, it shows as follows: HFileUtil /hbase/test12/98766318/image/9052388247118781160 FirstKey:00000d7d4f36c112imagevalue������� LastKey:333305184e0f7c3eimagevalue������� HFileUtil /hbase/test12/170637321/image/7602871928600243730 FirstKey:33338d45cc2491b8imagevalue������� LastKey:666629fe4378c096imagevalue������� So, according to this first key of 2nd region is 33338d45cc2491b8 not 333305184e0f7c3e which is correct! Now when I do a get on 333305184e0f7c3e with debug on, it is loading the second region which is wrong! Some thing went wrong with the index? Thanks, Murali Krishna ________________________________ From: stack <[email protected]> To: [email protected] Sent: Sat, 7 November, 2009 6:26:03 AM Subject: Re: Issue with bulk loader tool On Fri, Nov 6, 2009 at 12:58 AM, Murali Krishna. P <[email protected]>wrote: > Hi, > If I increase hbase.hregion.max.filesize so that all the records holds in > one region (and one reducer ), all the records as retrievable. If one > reducer creates multiple hfile or multiple reducer creates one hfile each, > the problem occurs. > > Multiple hfiles in a region? Or are you saying if a reducer creates multiple regions? There is supposed to be one file per region only when done. Thanks for digging in, St.Ack > Does that give any clue? > > Thanks, > Murali Krishna > > > > > ________________________________ > From: Murali Krishna. P <[email protected]> > To: [email protected] > Sent: Thu, 5 November, 2009 6:34:20 PM > Subject: Re: Issue with bulk loader tool > > Hi Stack, > Sorry, could not look into this last week... > > I got problem with the Htable interface as well. Some records i am not > retrieve from Htable as well. > I lost the old table, but reproduced the problem with a different table. > > I cannot send the region since it is very huge. will try to give as much > info as possible here :) > > There are total 5 regions as below in that table: > Name > > Encoded Name > Start Key > End Key > test1,,1257414794600 > 106817540 > fffe9c7f87c8332a > test1,fffe9c7f87c8332a,1257414794616 > 1346846599 fffe9c7f87c8332a fffebe279c0ac4d2 > test1,fffebe279c0ac4d2,1257414794628 > 1835851728 fffebe279c0ac4d2 fffec418284d6fbc > test1,fffec418284d6fbc,1257414794637 > 1078205908 fffec418284d6fbc fffef7a12ea22498 > test1,fffef7a12ea22498,1257414794647 > 1515378663 fffef7a12ea22498 > > I am looking for a key, say 000011d1bc8cd6fe . This should be in the first > region ? > > using hfile tool, > org.apache.hadoop.hbase.io.hfile.HFile -k -f > /hbase/test1/106817540/image/3828859735461759684 -v -m -p | grep > 000011d1bc8cd6fe > The first region doesn't have it. Not sure what happened to that record. > > For a working key, it gives the record properly as below > K: > \x00\x100003bdd08ca88ee2\x05imagevalue\x7F\xFF\xFF\xFF\xFF\xFF\xFF\xFF\x04 > V: \xFF... > > Please let me know if you need more information > > Thanks, > Murali Krishna > > > > > ________________________________ > From: stack <[email protected]> > To: [email protected] > Sent: Mon, 2 November, 2009 11:05:43 PM > Subject: Re: Issue with bulk loader tool > > Murali: > > Any developments worth mentioning? > > St.Ack > > > On Fri, Oct 30, 2009 at 10:14 AM, stack <[email protected]> wrote: > > > That is interesting. It'd almost point to a shell issue. Enable DEBUG > so > > client can see it. Then rerun shell. Is it at least loading the right > > region? (The regions start and end keys span the asked for key?). I > took a > > look at your attached .META. scan. All looks good there. The region > > specifications look right. If you want to bundle up the region that is > > failing -- the one that the failing key comes out of, I can take a look > > here. You could also try playing with the HFile tool: ./bin/hbase > > org.apache.hadoop.hbase.io.hfile.HFile. Run the former and it'll output > > usage. You should be able to get it to dump content of the region (You > need > > to supply flags like -v to see actual keys to the HFile tool else it just > > runs its check silently). Check for your key. Check things like > > timestamp on it. Maybe its 100 years in advance of now or something? > > > > Yours, > > St.Ack > > > > > > On Fri, Oct 30, 2009 at 9:01 AM, Murali Krishna. P < > [email protected] > > > wrote: > > > >> Attached ".META" > >> > >> Interesting, I was able to get the row from HTable via java code. But > from > >> the shell, still getting following > >> > >> hbase(main):004:0> get 'TestTable2', 'ffffef95bcbf2638' > >> 0 row(s) in 1.2250 seconds > >> > >> Thanks, > >> Murali Krishna > >> > >> Thanks, > >> Murali Krishna > >> > >> > >> ------------------------------ > >> *From:* stack <[email protected]> > >> *To:* [email protected] > >> *Sent:* Fri, 30 October, 2009 8:39:46 PM > >> *Subject:* Re: Issue with bulk loader tool > >> > >> Can you send a listing of ".META."? > >> > >> hbase> scan ".META." > >> > >> Also, can you bring a region down from hdfs, tar and gzip it, and then > put > >> it someplace I can pull so I can take a look? > >> > >> Thanks, > >> St.Ack > >> > >> > >> On Fri, Oct 30, 2009 at 3:31 AM, Murali Krishna. P > >> <[email protected]>wrote: > >> > >> > Hi guys, > >> > I created a table according to hbase-48. A mapreduce job which > creates > >> > HFiles and then used loadtable.rb script to create the table. > Everything > >> > worked fine and i was able to scan the table. But when i do a get for > a > >> key > >> > displayed in the scan output, it is not retrieving the row. shell says > 0 > >> > row. > >> > > >> > I tried using one reducer to ensure total ordering, but still same > >> issue. > >> > > >> > > >> > My mapper is like: > >> > context.write(new > >> > ImmutableBytesWritable(((Text)key).toString().getBytes()), new > >> > KeyValue(((Text)key).toString().getBytes(), "family1".getBytes(), > >> > "column1".getBytes(), getValueBytes())); > >> > > >> > > >> > Please help me investigate this. > >> > > >> > Thanks, > >> > Murali Krishna > >> > > >> > > > > >
