[
https://issues.apache.org/jira/browse/HBASE-15657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15241793#comment-15241793
]
Ted Yu commented on HBASE-15657:
--------------------------------
Will attach master log after redaction.
The symptom of this issue was that client side timed out waiting for snapshot
to finish:
{code}
2016-04-13
07:45:53,018|machine|INFO|20241|139633983403840|MainThread|2016-04-13
07:45:53,012 INFO [adlfs2] sink.AdlAzureIaasSink: inside put
metricsadl_server_errors 0
2016-04-13 07:46:03,153|machine|INFO|20241|139633983403840|MainThread|
2016-04-13 07:46:03,153|machine|INFO|20241|139633983403840|MainThread|ERROR:
Snapshot 'snapshot_tb1_create' wasn't completed in expectedTime:300000 ms
2016-04-13 07:46:03,154|machine|INFO|20241|139633983403840|MainThread|
2016-04-13 07:46:03,160|machine|INFO|20241|139633983403840|MainThread|Here is
some help for this command:
2016-04-13 07:46:03,160|machine|INFO|20241|139633983403840|MainThread|Take a
snapshot of specified table. Examples:
{code}
> Failed snapshot verification may not be detected by TakeSnapshotHandler
> -----------------------------------------------------------------------
>
> Key: HBASE-15657
> URL: https://issues.apache.org/jira/browse/HBASE-15657
> Project: HBase
> Issue Type: Bug
> Reporter: Ted Yu
> Assignee: Ted Yu
> Attachments: 15657.v1.patch
>
>
> Encountered a case where snapshot verification failed:
> {code}
> 2016-04-13 07:41:12,308 ERROR [MASTER_TABLE_OPERATIONS-10.0.0.75:16000-0]
> executor.EventHandler: Caught throwable while processing event
> C_M_SNAPSHOT_TABLE
> java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos
> at
> org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotDataManifest.internalGetFieldAccessorTable(SnapshotProtos.java:3883)
> at
> com.google.protobuf.GeneratedMessage.getDescriptorForType(GeneratedMessage.java:98)
> at
> com.google.protobuf.AbstractMessage$Builder.findMissingFields(AbstractMessage.java:789)
> at
> com.google.protobuf.AbstractMessage$Builder.findMissingFields(AbstractMessage.java:780)
> at
> com.google.protobuf.AbstractMessage$Builder.newUninitializedMessageException(AbstractMessage.java:770)
> at
> com.google.protobuf.AbstractMessage.newUninitializedMessageException(AbstractMessage.java:237)
> at
> com.google.protobuf.AbstractParser.newUninitializedMessageException(AbstractParser.java:57)
> at
> com.google.protobuf.AbstractParser.checkMessageInitialized(AbstractParser.java:71)
> at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:217)
> at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:223)
> at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:49)
> at
> org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotDataManifest.parseFrom(SnapshotProtos.java:4094)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotManifest.readDataManifest(SnapshotManifest.java:433)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:273)
> at
> org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119)
> at
> org.apache.hadoop.hbase.master.snapshot.MasterSnapshotVerifier.verifySnapshot(MasterSnapshotVerifier.java:108)
> at
> org.apache.hadoop.hbase.master.snapshot.TakeSnapshotHandler.process(TakeSnapshotHandler.java:200)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Note NoClassDefFoundError is not Exception. So it was not caught by the
> following clause in TakeSnapshotHandler :
> {code}
> } catch (Exception e) {
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
