Dieter De Paepe created HBASE-28412:
---------------------------------------
Summary: TableNotFoundException while restoring out-of-system
incremental backup with mapping
Key: HBASE-28412
URL: https://issues.apache.org/jira/browse/HBASE-28412
Project: HBase
Issue Type: Bug
Components: backup&restore
Affects Versions: 2.6.0
Reporter: Dieter De Paepe
It appears that restoring a non-existing table from an incremental backup with
the "-m" parameter results in an error in the restore client.
Reproduction steps:
Start from a table with some data
{code:java}
hbase:006:0> scan 'lily_tenant_acme:LILY_SETTINGS'
ROW COLUMN+CELL
row1 column=0:a,
timestamp=2024-02-27T18:49:46.489, value=val1
row1 column=0:test,
timestamp=2024-02-29T15:46:29.956, value=value2 {code}
Create a full backup.
{code:java}
hbase backup create full /tmp/backup_hbase2 {code}
Change the table (eg using hbase shell):
{code:java}
put 'lily_tenant_acme:LILY_SETTINGS', 'row1', '0:test', 'value3' {code}
Create an incremental backup
{code:java}
hbase backup create incremental /tmp/backup_hbase2 {code}
Delete the table:
{code:java}
disable 'lily_tenant_acme:LILY_SETTINGS'
drop 'lily_tenant_acme:LILY_SETTINGS' {code}
Restore of the incremental backup with the "-m" parameter results in an error:
{code:java}
hbase restore hdfs:///tmp/backup_hbase2 backup_1709222166873 -t
"lily_tenant_acme:LILY_SETTINGS" -m "lily_tenant_acme:LILY_SETTINGS-restored"
24/02/29 15:59:20 WARN org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil:
The addDependencyJars(Configuration, Class<?>...) method has been deprecated
since it is easy to use incorrectly. Most users should rely on
addDependencyJars(Job) instead. See HBASE-8386 for more details.
24/02/29 15:59:35 WARN org.apache.hadoop.hbase.tool.LoadIncrementalHFiles:
Skipping non-directory
hdfs://hdfsns/user/lily/hbase-staging/bulk_output-lily_tenant_acme-LILY_SETTINGS-restored-1709222360030/_SUCCESS
24/02/29 15:59:36 ERROR
org.apache.hadoop.hbase.backup.mapreduce.MapReduceRestoreJob:
org.apache.hadoop.hbase.TableNotFoundException: lily_tenant_acme:LILY_SETTINGS
org.apache.hadoop.hbase.TableNotFoundException: lily_tenant_acme:LILY_SETTINGS
at
org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:635)
at org.apache.hadoop.hbase.client.HTable.getDescriptor(HTable.java:244)
at
org.apache.hadoop.hbase.backup.mapreduce.MapReduceHFileSplitterJob.createSubmittableJob(MapReduceHFileSplitterJob.java:117)
at
org.apache.hadoop.hbase.backup.mapreduce.MapReduceHFileSplitterJob.run(MapReduceHFileSplitterJob.java:165)
at
org.apache.hadoop.hbase.backup.mapreduce.MapReduceRestoreJob.run(MapReduceRestoreJob.java:84)
at
org.apache.hadoop.hbase.backup.util.RestoreTool.incrementalRestoreTable(RestoreTool.java:205)
at
org.apache.hadoop.hbase.backup.impl.RestoreTablesClient.restoreImages(RestoreTablesClient.java:185)
at
org.apache.hadoop.hbase.backup.impl.RestoreTablesClient.restore(RestoreTablesClient.java:228)
at
org.apache.hadoop.hbase.backup.impl.RestoreTablesClient.execute(RestoreTablesClient.java:264)
at
org.apache.hadoop.hbase.backup.impl.BackupAdminImpl.restore(BackupAdminImpl.java:518)
at
org.apache.hadoop.hbase.backup.RestoreDriver.parseAndRun(RestoreDriver.java:176)
at
org.apache.hadoop.hbase.backup.RestoreDriver.doWork(RestoreDriver.java:216)
at org.apache.hadoop.hbase.backup.RestoreDriver.run(RestoreDriver.java:252)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:82)
at org.apache.hadoop.hbase.backup.RestoreDriver.main(RestoreDriver.java:224)
24/02/29 15:59:36 ERROR org.apache.hadoop.hbase.backup.RestoreDriver: Error
while running restore backup
java.io.IOException: Can not restore from backup directory
hdfs://hdfsns/tmp/backup_hbase2/backup_1709222166873/lily_tenant_acme/LILY_SETTINGS/0/f011305e78cd4196bf67d4dcca859ea1,hdfs://hdfsns/tmp/backup_hbase2/backup_1709222166873/lily_tenant_acme/LILY_SETTINGS/c6f5124f51d797fde1a43093747d04fa/0/800bad45ccd049fda6a439b60954cd06_SeqId_4_,hdfs://hdfsns/tmp/backup_hbase2/backup_1709222166873/lily_tenant_acme/LILY_SETTINGS/c6f5124f51d797fde1a43093747d04fa/0/e334bbc7089e4c3f8dab84b9b704824c_SeqId_6_
(check Hadoop and HBase logs)
at
org.apache.hadoop.hbase.backup.mapreduce.MapReduceRestoreJob.run(MapReduceRestoreJob.java:103)
at
org.apache.hadoop.hbase.backup.util.RestoreTool.incrementalRestoreTable(RestoreTool.java:205)
at
org.apache.hadoop.hbase.backup.impl.RestoreTablesClient.restoreImages(RestoreTablesClient.java:185)
at
org.apache.hadoop.hbase.backup.impl.RestoreTablesClient.restore(RestoreTablesClient.java:228)
at
org.apache.hadoop.hbase.backup.impl.RestoreTablesClient.execute(RestoreTablesClient.java:264)
at
org.apache.hadoop.hbase.backup.impl.BackupAdminImpl.restore(BackupAdminImpl.java:518)
at
org.apache.hadoop.hbase.backup.RestoreDriver.parseAndRun(RestoreDriver.java:176)
at
org.apache.hadoop.hbase.backup.RestoreDriver.doWork(RestoreDriver.java:216)
at org.apache.hadoop.hbase.backup.RestoreDriver.run(RestoreDriver.java:252)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:82)
at org.apache.hadoop.hbase.backup.RestoreDriver.main(RestoreDriver.java:224)
Caused by: org.apache.hadoop.hbase.TableNotFoundException:
lily_tenant_acme:LILY_SETTINGS
at
org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:635)
at org.apache.hadoop.hbase.client.HTable.getDescriptor(HTable.java:244)
at
org.apache.hadoop.hbase.backup.mapreduce.MapReduceHFileSplitterJob.createSubmittableJob(MapReduceHFileSplitterJob.java:117)
at
org.apache.hadoop.hbase.backup.mapreduce.MapReduceHFileSplitterJob.run(MapReduceHFileSplitterJob.java:165)
at
org.apache.hadoop.hbase.backup.mapreduce.MapReduceRestoreJob.run(MapReduceRestoreJob.java:84)
... 10 more {code}
After this command, HBase has restored the table to the state of the full
backup:
{code:java}
hbase:031:0> scan 'lily_tenant_acme:LILY_SETTINGS-restored'
ROW COLUMN+CELL
row1 column=0:a,
timestamp=2024-02-27T18:49:46.489, value=val1
row1 column=0:test,
timestamp=2024-02-29T15:46:29.956, value=value2 {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
