The ruby scripts have a tendency to rot since they aren't tested as part of our unit test suite... we should probably just delete copy_table.rb since IIRC it was a hacked up script that Stack needed for Powerset in 2008.
There's no practical way of copying a table, in the most recent version there's the CopyTable mapreduce job that you could use but in 0.20 I think you could distcp the table's folder to another cluster and try it there or have another hbase instance using the same HDFS but root to a different folder. In both cases you'll have to use the add_table.rb script. J-D On Fri, Jun 10, 2011 at 10:40 AM, James Hammerton <[email protected]> wrote: > Hi, > > I did some digging around in the HBase source and found that HLogEdit last > existed in version 0.18 (we're on 0.20.6) and > HConstants.COL_REGIONINFO_ARRAY last existed in version 0.2. > > After this digging around and making some educated guesses I made a new > version of the script which I've attached to this message. > > I've tried running this script on a small 525M (on disc) table in a test > instance of HBase and it's been running for over 12 mins, and is repeatedly > outputting messages like the following: > > 11/06/10 11:27:05 DEBUG hfile.LruBlockCache: Cache Stats: Sizes: > Total=1.6478577MB (1727904), Free=197.52715MB (207122224), Max=199.175MB > (208850128), Counts: Blocks=3, Access=3, Hit=0, Miss=3, Evictions=0, > Evicted=0, Ratios: Hit Ratio=0.0%, Miss Ratio=100.0%, Evicted/Run=NaN > 11/06/10 11:28:05 DEBUG hfile.LruBlockCache: Cache Stats: Sizes: > Total=1.6478577MB (1727904), Free=197.52715MB (207122224), Max=199.175MB > (208850128), Counts: Blocks=3, Access=3, Hit=0, Miss=3, Evictions=0, > Evicted=0, Ratios: Hit Ratio=0.0%, Miss Ratio=100.0%, Evicted/Run=NaN > > The du command tells me that the copied table's directory so far takes up > just 4k on disk. Is my hacked script actually doing anything useful? > > James > > On Fri, Jun 10, 2011 at 11:19 AM, James Hammerton > <[email protected]> wrote: >> >> [Re-sending as I'm not sure this got through] >> >> Hi, >> >> Before trying to merge regions on a table in our live database we decided >> to copy the table and merge the regions on the copy first to test the >> merging code works before risking our live data. >> >> However when we try to run the copy_table.rb script it fails due to the >> following error (this is actually from a test HBase instance but the error >> from the live instance is the same, modulo different paths): >> >> >> file:/home/james/hbase-0.20.6/lib/jruby-complete-1.2.0.jar!/META-INF/jruby.home/lib/ruby/site_ruby/1.8/builtin/javasupport/core_ext/object.rb:33:in >> `get_proxy_or_package_under_package': cannot load Java class >> org.apache.hadoop.hbase.regionserver.HLogEdit (NameError) >> from >> file:/home/james/hbase-0.20.6/lib/jruby-complete-1.2.0.jar!/META-INF/jruby.home/lib/ruby/site_ruby/1.8/builtin/javasupport/java.rb:51:in >> `method_missing' >> from /home/james/hbase/bin/copy_table.rb:40 >> >> Looking through the copy_table.rb file I could so no reference to HLogEdit >> other than the import, so I tried removing it. Now running the script >> produces the error below: >> >> >> file:/home/james/hbase-0.20.6/lib/jruby-complete-1.2.0.jar!/META-INF/jruby.home/lib/ruby/site_ruby/1.8/builtin/java/ast.rb:80:in >> `const_missing': uninitialized constant >> Java::OrgApacheHadoopHbase::HConstants::COL_REGIONINFO_ARRAY (NameError) >> from /home/james/hbase/bin/copy_table.rb:124 >> from >> file:/home/james/hbase-0.20.6/lib/jruby-complete-1.2.0.jar!/META-INF/jruby.home/lib/ruby/site_ruby/1.8/builtin/java/collections.rb:47:in >> `call' >> from >> file:/home/james/hbase-0.20.6/lib/jruby-complete-1.2.0.jar!/META-INF/jruby.home/lib/ruby/site_ruby/1.8/builtin/java/collections.rb:47:in >> `each' >> from /home/james/hbase/bin/copy_table.rb:118 >> >> I looked at the latest version of the copy_table.rb script and it also >> references both HLogEdit and HConstants::COL_REGIONINFO_ >> ARRAY... >> Any ideas how to fix this? >> >> Regards, >> >> James >> >> -- >> James Hammerton | Senior Data Mining Engineer >> www.mendeley.com/profiles/james-hammerton >> >> Mendeley Limited | London, UK | www.mendeley.com >> Registered in England and Wales | Company Number 6419015 >> >> >> > > > > -- > James Hammerton | Senior Data Mining Engineer > www.mendeley.com/profiles/james-hammerton > > Mendeley Limited | London, UK | www.mendeley.com > Registered in England and Wales | Company Number 6419015 > > > >
