Ted Yu created HBASE-14783:
------------------------------
Summary: Master aborts when downgrading from 1.3 to 1.1
Key: HBASE-14783
URL: https://issues.apache.org/jira/browse/HBASE-14783
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
I was running ITBLL with 1.3 deployed on a 6 node cluster.
Then I stopped the cluster, deployed 1.1 release and tried to start cluster.
However, master failed to start due to:
{code}
2015-11-06 00:58:40,351 FATAL [eval-test-2:20000.activeMasterManager]
master.HMaster: Failed to become active master
java.io.IOException: The procedure class
org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure must be
accessible and have an empty constructor
at
org.apache.hadoop.hbase.procedure2.Procedure.newInstance(Procedure.java:548)
at org.apache.hadoop.hbase.procedure2.Procedure.convert(Procedure.java:640)
at
org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormatReader.read(ProcedureWALFormatReader.java:105)
at
org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.load(ProcedureWALFormat.java:82)
at
org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.load(WALProcedureStore.java:298)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.load(ProcedureExecutor.java:275)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.start(ProcedureExecutor.java:434)
at
org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1208)
at
org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1107)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:694)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:186)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1713)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:191)
at
org.apache.hadoop.hbase.procedure2.Procedure.newInstance(Procedure.java:536)
... 12 more
{code}
The cause was that ServerCrashProcedure, written in some WAL file under
MasterProcWALs from first run, was absent in 1.1 release.
After a brief discussion with Stephen, I am logging this JIRA to solicit
discussion on how customer experience can be improved if downgrade of hbase is
performed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
