[
https://issues.apache.org/jira/browse/HBASE-12245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170021#comment-14170021
]
Elliott Clark commented on HBASE-12245:
---------------------------------------
Of course the audit logs don't go back that far on this test cluster..... I'll
be looking at that.
Here's what I'm seeing on the region server. So it looks like the file was
created, compacted, added to a snapshot, and then archived fine.
{code}
2014-10-10 12:52:53,628 DEBUG
org.apache.hadoop.hbase.regionserver.HRegionFileSystem: Committing store file
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/.tmp/31c3d53620ad402cbf44232013834f7b
as
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/31c3d53620ad402cbf44232013834f7b
2014-10-10 12:59:37,709 DEBUG
org.apache.hadoop.hbase.snapshot.SnapshotManifest: Adding snapshot references
for
[hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/80231f01c5d14d8c84f75bea293af26c,
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/303bfa9fcd3649b18605627503d12296,
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/1ca21e865db049aea1d5538aaee7940a,
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/31c3d53620ad402cbf44232013834f7b]
hfiles
2014-10-10 12:59:37,709 DEBUG
org.apache.hadoop.hbase.snapshot.SnapshotManifest: Adding reference for file
(4/4):
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/31c3d53620ad402cbf44232013834f7b
2014-10-10 13:26:53,310 DEBUG
org.apache.hadoop.hbase.snapshot.SnapshotManifest: Adding snapshot references
for
[hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/80231f01c5d14d8c84f75bea293af26c,
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/303bfa9fcd3649b18605627503d12296,
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/1ca21e865db049aea1d5538aaee7940a,
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/31c3d53620ad402cbf44232013834f7b,
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/64dce75224c74da48f2dfd23580eb350]
hfiles
2014-10-10 13:26:53,310 DEBUG
org.apache.hadoop.hbase.snapshot.SnapshotManifest: Adding reference for file
(4/5):
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/31c3d53620ad402cbf44232013834f7b
2014-10-10 13:31:10,991 DEBUG
org.apache.hadoop.hbase.snapshot.SnapshotManifest: Adding snapshot references
for
[hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/80231f01c5d14d8c84f75bea293af26c,
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/303bfa9fcd3649b18605627503d12296,
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/1ca21e865db049aea1d5538aaee7940a,
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/31c3d53620ad402cbf44232013834f7b,
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/64dce75224c74da48f2dfd23580eb350]
hfiles
2014-10-10 13:31:10,992 DEBUG
org.apache.hadoop.hbase.snapshot.SnapshotManifest: Adding reference for file
(4/5):
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/31c3d53620ad402cbf44232013834f7b
2014-10-10 13:36:05,742 DEBUG
org.apache.hadoop.hbase.snapshot.SnapshotManifest: Adding snapshot references
for
[hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/80231f01c5d14d8c84f75bea293af26c,
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/303bfa9fcd3649b18605627503d12296,
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/1ca21e865db049aea1d5538aaee7940a,
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/31c3d53620ad402cbf44232013834f7b,
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/64dce75224c74da48f2dfd23580eb350]
hfiles
2014-10-10 13:36:05,742 DEBUG
org.apache.hadoop.hbase.snapshot.SnapshotManifest: Adding reference for file
(4/5):
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/31c3d53620ad402cbf44232013834f7b
2014-10-10 15:25:45,114 DEBUG
org.apache.hadoop.hbase.regionserver.compactions.Compactor: Compacting
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/31c3d53620ad402cbf44232013834f7b,
keycount=11797, bloomtype=ROW, size=2.7 MB, encoding=NONE, seqNum=30762619024
2014-10-10 15:25:45,926 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
Finished archiving from class
org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
file:hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/data/default/TABLE/d437461577613a475160ee579ea1e203/S/31c3d53620ad402cbf44232013834f7b,
to
hdfs://hbase-dfs.data.facebook.com:9000/hbase-HBASE/archive/data/default/TABLE/d437461577613a475160ee579ea1e203/S/31c3d53620ad402cbf44232013834f7b
{code}
Then there is no mention of that file in the master at all.
> Snapshots going corrupt after creation
> --------------------------------------
>
> Key: HBASE-12245
> URL: https://issues.apache.org/jira/browse/HBASE-12245
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.98.7
> Reporter: Elliott Clark
> Priority: Critical
>
> Took a snapshot Fri Oct 10 12:59:35 PDT 2014. Everything reported good and
> all files were there.
> Ran an export snapshot job over the weekend.
> Came back this morning and all the jobs had failed and now the snapshot is
> corrupt.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
