hbase org.apache.hadoop.hbase.snapshot.SnapshotInfo -snapshot test_snapshot -stats -schema
On Thu, Sep 24, 2015 at 3:43 PM, Sandeep Nemuri <[email protected]> wrote: > You can check snapshot state if it is healthy or not using below command. > > > On Thu, Sep 24, 2015 at 2:55 PM, 核弹头す <[email protected]> wrote: > >> Hi all, >> >> >> I am using hive to query on base snapshot. But I got the following error: >> >> FAILED: IllegalArgumentException >> org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException: Couldn't read >> snapshot info >> from:/tmp/hbase-huser/hbase/.hbase-snapshot/test_table_snap0/.snapshotinfo >> >> The following is the steps that I do: >> >> 1, on the hbase we already have an table: test_table. And I use command: >> snapshot >> ‘test_table’ ，’test_table_snap0’ to create a snapshot。 >> >> 2，on the hive create an external table like： >> >> CREATE EXTERNAL TABLE `test_table_snapshot`( >> >> `id` int, >> >> `alias` string, >> >> `kdt_id` int, >> >> `created_time` string, >> >> `update_time` string, >> >> `title` string, >> >> `price` bigint, >> >> `goods_platform` int, >> >> `buy_url` string, >> >> `class1` int, >> >> `class2` string, >> >> `goods_type` tinyint, >> >> `sold_status` tinyint, >> >> `is_display` tinyint, >> >> `is_delete` tinyint, >> >> `num` bigint, >> >> `buy_way` boolean, >> >> `source` tinyint, >> >> `content` string, >> >> `picture` string, >> >> `is_virtual` tinyint) >> >> ROW FORMAT SERDE >> >> 'org.apache.hadoop.hive.hbase.HBaseSerDe' >> >> STORED BY >> >> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' >> >> WITH SERDEPROPERTIES ( >> >> 'field.delim'='\u0001', >> >> >> 'hbase.columns.mapping'=':key,cf1:alalis,cf1:kdt_id,cf1:created_time,cf1:update_time,cf1:title,cf1:price,cf1:goods_platform,cf1:buy_url,cf1:class1,cf1:class2,cf1:goods_type,cf1:sold_stattus,cf1:is_display,cf1:is_delete,cf1:num,cf1:buy_way,cf1:source,cf1:content,cf1:picture,cf1:is_virtual', >> >> 'line.delim'='\n', >> >> 'serialization.format'='\u0001') >> >> TBLPROPERTIES ( >> >> ‘hbase.table.name'='test_table') >> >> >> 4, on Hive, execute “select * from test_table_snapshot”, then I get the >> above error. >> >> >> I have set the “hive.hbase.snapshot.restoredir” value which is same as >> the root dir on hbase, and I also set the zookeep server. I checked the >> base root directory on hdfs and I can see the snapshot files. But when I >> use the above query on hive. it seems that the base.tmp.dir is used to find >> the snapshot info. BTW, if I directly query on hbase, there is no problem. >> I do not know what happened? >> >> >> The following is the hive error log: >> >> FAILED: SemanticException >> org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException: Couldn't read >> snapshot info from:/tmp >> /hbase-huser/hbase/.hbase-snapshot/goods_v3_hbase_snap0/.snapshotinfo >> >> org.apache.hadoop.hive.ql.parse.SemanticException: >> org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException: Couldn't read >> snapshot info from:/tmp >> /hbase-huser/hbase/.hbase-snapshot/goods_v3_hbase_snap0/.snapshotinfo >> >> at >> org.apache.hadoop.hive.ql.optimizer.SimpleFetchOptimizer.transform(SimpleFetchOptimizer.java:117) >> >> at >> org.apache.hadoop.hive.ql.optimizer.Optimizer.optimize(Optimizer.java:182) >> >> at >> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java: >> 10207) >> >> at >> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:192) >> >> at >> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:222) >> >> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:421) >> >> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:307) >> >> at >> org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1112) >> >> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1160) >> >> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049) >> >> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1039) >> >> at >> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:207) >> >> at >> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:159) >> >> at >> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:370) >> >> at >> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:754) >> >> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675) >> >> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:615) >> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> >> at java.lang.reflect.Method.invoke(Method.java:606) >> >> at org.apache.hadoop.util.RunJar.run(RunJar.java:221) >> >> at org.apache.hadoop.util.RunJar.main(RunJar.java:136) >> >> Caused by: java.lang.IllegalArgumentException: >> org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException: Couldn't read >> snapshot info from:/tmp >> /hbase-huser/hbase/.hbase-snapshot/goods_v3_hbase_snap0/.snapshotinfo >> >> at >> org.apache.hadoop.hive.hbase.HBaseStorageHandler.configureTableJobProperties(HBaseStorageHandler.java:401) >> >> at >> org.apache.hadoop.hive.hbase.HBaseStorageHandler.configureInputJobProperties(HBaseStorageHandler.java:317) >> >> at >> org.apache.hadoop.hive.ql.plan.PlanUtils.configureJobPropertiesForStorageHandler(PlanUtils.java:809) >> >> at >> org.apache.hadoop.hive.ql.plan.PlanUtils.configureInputJobPropertiesForStorageHandler(PlanUtils.java:779) >> >> at >> org.apache.hadoop.hive.ql.optimizer.SimpleFetchOptimizer$FetchData.convertToWork(SimpleFetchOptimizer.java:379) >> >> at >> org.apache.hadoop.hive.ql.optimizer.SimpleFetchOptimizer$FetchData.access$000(SimpleFetchOptimizer.java:319) >> >> at >> org.apache.hadoop.hive.ql.optimizer.SimpleFetchOptimizer.optimize(SimpleFetchOptimizer.java:135) >> >> at >> org.apache.hadoop.hive.ql.optimizer.SimpleFetchOptimizer.transform(SimpleFetchOptimizer.java:106) >> >> ... 22 more >> >> Caused by: org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException: >> Couldn't read snapshot info from:/tmp >> /hbase-huser/hbase/.hbase-snapshot/goods_v3_hbase_snap0/.snapshotinfo >> >> at >> org.apache.hadoop.hbase.snapshot.SnapshotDescriptionUtils.readSnapshotInfo(SnapshotDescriptionUtils.java:307) >> >> at >> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.copySnapshotForScanner(RestoreSnapshotHelper.java:727) >> >> at >> org.apache.hadoop.hbase.mapreduce.TableSnapshotInputFormatImpl.setInput(TableSnapshotInputFormatImpl.java:364) >> >> at >> org.apache.hadoop.hive.hbase.HBaseTableSnapshotInputFormatUtil.configureJob(HBaseTableSnapshotInputFormatUtil.java:77) >> >> at >> org.apache.hadoop.hive.hbase.HBaseStorageHandler.configureTableJobProperties(HBaseStorageHandler.java:387) >> >> ... 29 more >> > > > > -- > * Regards* > * Sandeep Nemuri* > -- * Regards* * Sandeep Nemuri*
