Now I'm getting the wrong region exception on the new table that I'm copying the old table to. Running hbck reveals an inconsistency in the new table. The frustration is unbelievable. Like I said before, it doesn't appear that HBase is ready for prime time. I don't know how companies are using this successfully, it doesn't appear plausible.
Robert -----Original Message----- From: Robert Gonzalez [mailto:[email protected]] Sent: Tuesday, May 31, 2011 11:03 AM To: '[email protected]' Subject: RE: wrong region exception I'm trying my "nuclear" option: basically copy the data from the old db to a new one, skipping over bad regions. The bad news is that it is taking forever. I get a stack trace just trying to run check_meta.rb: maxpoint@c1-m02:/usr/lib/hbase/bin$ ./hbase org.jruby.Main check_meta.rb Writables.java:75:in `org.apache.hadoop.hbase.util.Writables.getWritable': java.lang.NullPointerException: null (NativeException) from Writables.java:119:in `org.apache.hadoop.hbase.util.Writables.getHRegionInfo' from NativeMethodAccessorImpl.java:-2:in `sun.reflect.NativeMethodAccessorImpl.invoke0' from NativeMethodAccessorImpl.java:39:in `sun.reflect.NativeMethodAccessorImpl.invoke' from DelegatingMethodAccessorImpl.java:25:in `sun.reflect.DelegatingMethodAccessorImpl.invoke' from Method.java:597:in `java.lang.reflect.Method.invoke' from JavaMethod.java:196:in `org.jruby.javasupport.JavaMethod.invokeWithExceptionHandling' from JavaMethod.java:182:in `org.jruby.javasupport.JavaMethod.invoke_static' from JavaClass.java:371:in `org.jruby.javasupport.JavaClass$StaticMethodInvoker.execute' ... 17 levels... from Main.java:183:in `org.jruby.Main.runInterpreter' from Main.java:120:in `org.jruby.Main.run' from Main.java:95:in `org.jruby.Main.main' Complete Java stackTrace java.lang.NullPointerException at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:75) at org.apache.hadoop.hbase.util.Writables.getHRegionInfo(Writables.java:119) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.jruby.javasupport.JavaMethod.invokeWithExceptionHandling(JavaMethod.java:196) at org.jruby.javasupport.JavaMethod.invoke_static(JavaMethod.java:182) at org.jruby.javasupport.JavaClass$StaticMethodInvoker.execute(JavaClass.java:371) at org.jruby.internal.runtime.methods.SimpleCallbackMethod.call(SimpleCallbackMethod.java:81) at org.jruby.evaluator.EvaluationState.callNode(EvaluationState.java:571) at org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:207) at org.jruby.evaluator.EvaluationState.localAsgnNode(EvaluationState.java:1254) at org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:286) at org.jruby.evaluator.EvaluationState.blockNode(EvaluationState.java:533) at org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:201) at org.jruby.evaluator.EvaluationState.whileNode(EvaluationState.java:1793) at org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:387) at org.jruby.evaluator.EvaluationState.blockNode(EvaluationState.java:533) at org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:201) at org.jruby.evaluator.EvaluationState.rootNode(EvaluationState.java:1628) at org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:356) at org.jruby.evaluator.EvaluationState.eval(EvaluationState.java:164) at org.jruby.Ruby.eval(Ruby.java:278) at org.jruby.Ruby.compileOrFallbackAndRun(Ruby.java:306) at org.jruby.Main.runInterpreter(Main.java:238) at org.jruby.Main.runInterpreter(Main.java:183) at org.jruby.Main.run(Main.java:120) at org.jruby.Main.main(Main.java:95) -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Stack Sent: Friday, May 27, 2011 12:43 AM To: [email protected] Subject: Re: wrong region exception Robert: Looks like script already exists. Check bin/check_meta.rb. If you pass it --fix it should plug the hole. Read the head of the script for how to run it. Good luck, St.Ack On Thu, May 26, 2011 at 1:06 PM, Robert Gonzalez <[email protected]> wrote: > I sent the meta.txt to your saint.ack@gmail .com account due to the > attachment. > > -----Original Message----- > From: [email protected] [mailto:[email protected]] On Behalf Of > Stack > Sent: Thursday, May 26, 2011 1:35 PM > To: [email protected] > Subject: Re: wrong region exception > > On Thu, May 26, 2011 at 8:06 AM, Robert Gonzalez > <[email protected]> wrote: >> It looks like an entire region is missing, here is the online table: >> >> urlhashv2,7FF1A5BF839C37078083B4F8267008F6,1303028235302.b0d55566fd0e >> 0 2ae98541618396aa7b1. >> c1-s03.atxd.maxpointinteractive.com:60030 >> 7FF1A5BF839C37078083B4F8267008F6 >> 80116D7E506D87ED39EAFFE784B5B590 >> urlhashv2,8031483E0B3B7F587020FCBB764272D9,1305226123483.3ed065ad87f8 >> 9 aece6b994dd31b42b2a. c1-s33.atxd.maxpointinteractive.com:60030 >> 8031483E0B3B7F587020FCBB764272D9 >> 8041346D0B05617FA4B9152BFE9B18B9 >> >> One ends at 80116D7E506D87ED39EAFFE784B5B590, but the next one doesn't start >> there. >> > > So, make sure you actually have a hole. Dump out your meta table: > > echo "scan '.META.'"| ./bin/hbase shell &> /tmp/meta.txt > > Then look ensure that there is a hole between the above regions (compare > start and end keys... the end key of one region needs to match the start key > of the next). > > If indeed a hole, you need to do a little surgery inserting a new missing > region (hbck should fix this but it doesn't have the smarts just yet). > > Basically, you create a new region with start and end keys to fill the hole > then you insert it into .META. and then assign it. There are some scripts in > our bin directory that do various parts of this. I'm pretty sure its beyond > any but a few figuring this mess out so if you do the above foot work and > provide a few more details, I'll hack up something for you (and hopefully > something generalized to be use by others later, and later to be integrated > into hbck). > > St.Ack >
