[
https://issues.apache.org/jira/browse/HBASE-7683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13563744#comment-13563744
]
Lars Hofhansl commented on HBASE-7683:
--------------------------------------
I think this log snippet is interesting.
The sequence is this:
# updating /observed_table/.tableinfo.0000000003
# updating /observed_table/.tableinfo.0000000004
# updating /observed_table/.tableinfo.0000000005
# updating /observed_table/.tableinfo.0000000006
# failed to delete /observed_table/.tableinfo.0000000006
# updating /observed_table/.tableinfo.0000000002
# cleaned up /observed_table/.tableinfo.0000000002
So it seems version 2 was somehow updated out of sync
{code}
2013-01-27 05:22:57,737 DEBUG [IPC Server handler 1 on 41253]
client.ClientScanner(185): Finished with scanning at {NAME => '.META.,,1',
STARTKEY => '', ENDKEY => '', ENCODED => 1028785192,}
2013-01-27 05:22:57,737 INFO [IPC Server handler 1 on 41253]
master.MasterFileSystem(541): AddColumn. Table = observed_table HCD = {NAME =>
'fam2', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'NONE', REPLICATION_SCOPE
=> '0', VERSIONS => '3', COMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL =>
'2147483647', KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536', IN_MEMORY =>
'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}
2013-01-27 05:22:57,740 DEBUG [IPC Server handler 1 on 41253]
util.FSTableDescriptors(501):
hdfs://localhost:40948/user/jenkins/hbase/observed_table/.tmp/.tableinfo.0000000002
exists; retrying up to 10 times
2013-01-27 05:22:57,750 INFO [IPC Server handler 1 on 41253]
util.FSTableDescriptors(452): Updated
tableinfo=hdfs://localhost:40948/user/jenkins/hbase/observed_table/.tableinfo.0000000003
2013-01-27 05:22:57,761 INFO [IPC Server handler 1 on 41253]
util.FSTableDescriptors(452): Updated
tableinfo=hdfs://localhost:40948/user/jenkins/hbase/observed_table/.tableinfo.0000000004
2013-01-27 05:22:57,764 DEBUG [IPC Server handler 0 on 41253]
client.ClientScanner(90): Creating scanner over .META. starting at key
'observed_table,,'
2013-01-27 05:22:57,764 DEBUG [IPC Server handler 0 on 41253]
client.ClientScanner(198): Advancing internal scanner to startKey at
'observed_table,,'
2013-01-27 05:22:57,767 INFO [IPC Server handler 0 on 41253]
handler.TableEventHandler(91): Handling table operation C_M_MODIFY_FAMILY on
table observed_table
2013-01-27 05:22:57,768 DEBUG [IPC Server handler 0 on 41253]
client.ClientScanner(90): Creating scanner over .META. starting at key
'observed_table,,'
2013-01-27 05:22:57,768 DEBUG [IPC Server handler 0 on 41253]
client.ClientScanner(198): Advancing internal scanner to startKey at
'observed_table,,'
2013-01-27 05:22:57,771 DEBUG [IPC Server handler 0 on 41253]
client.ClientScanner(185): Finished with scanning at {NAME => '.META.,,1',
STARTKEY => '', ENDKEY => '', ENCODED => 1028785192,}
2013-01-27 05:22:57,771 INFO [IPC Server handler 0 on 41253]
master.MasterFileSystem(518): AddModifyColumn. Table = observed_table HCD =
{NAME => 'fam2', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'NONE',
REPLICATION_SCOPE => '0', VERSIONS => '25', COMPRESSION => 'NONE', MIN_VERSIONS
=> '0', TTL => '2147483647', KEEP_DELETED_CELLS => 'false', BLOCKSIZE =>
'65536', IN_MEMORY => 'false', ENCODE_ON_DISK => 'true', BLOCKCACHE => 'true'}
2013-01-27 05:22:57,787 INFO [IPC Server handler 0 on 41253]
util.FSTableDescriptors(452): Updated
tableinfo=hdfs://localhost:40948/user/jenkins/hbase/observed_table/.tableinfo.0000000005
2013-01-27 05:22:57,799 INFO [IPC Server handler 0 on 41253]
util.FSTableDescriptors(452): Updated
tableinfo=hdfs://localhost:40948/user/jenkins/hbase/observed_table/.tableinfo.0000000006
2013-01-27 05:22:57,801 DEBUG [IPC Server handler 1 on 41253]
client.ClientScanner(90): Creating scanner over .META. starting at key
'observed_table,,'
2013-01-27 05:22:57,801 DEBUG [IPC Server handler 1 on 41253]
client.ClientScanner(198): Advancing internal scanner to startKey at
'observed_table,,'
2013-01-27 05:22:57,808 INFO [pool-1-thread-1] client.HBaseAdmin(727): Started
enable of observed_table
2013-01-27 05:22:57,809 DEBUG [pool-1-thread-1] zookeeper.ZKUtil(1571):
hconnection-0x13c7a752b7c0006 Retrieved 8 byte(s) of data from znode
/hbase/table/observed_table; data=ENABLING
2013-01-27 05:22:57,809 DEBUG [pool-1-thread-1] client.HBaseAdmin(684):
Sleeping= 1000ms, waiting for all regions to be enabled in observed_table
2013-01-27 05:22:58,136 WARN
[MASTER_TABLE_OPERATIONS-hemera.apache.org,41253,1359264165304-0]
util.FSTableDescriptors(522): Failed delete of
hdfs://localhost:40948/user/jenkins/hbase/observed_table/.tableinfo.0000000001;
continuing
2013-01-27 05:22:58,136 INFO
[MASTER_TABLE_OPERATIONS-hemera.apache.org,41253,1359264165304-0]
util.FSTableDescriptors(452): Updated
tableinfo=hdfs://localhost:40948/user/jenkins/hbase/observed_table/.tableinfo.0000000002
2013-01-27 05:22:58,138 DEBUG
[MASTER_TABLE_OPERATIONS-hemera.apache.org,41253,1359264165304-0]
util.FSTableDescriptors(286): Cleaned up old tableinfo file
hdfs://localhost:40948/user/jenkins/hbase/observed_table/.tableinfo.0000000002
{code}
> TestMasterObserver fails occasionally
> -------------------------------------
>
> Key: HBASE-7683
> URL: https://issues.apache.org/jira/browse/HBASE-7683
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
>
> {code}
> org.apache.hadoop.hbase.InvalidFamilyOperationException: Column family 'fam2'
> does not exist at
> org.apache.hadoop.hbase.master.handler.TableEventHandler.hasColumnFamily(TableEventHandler.java:182)
> at
> org.apache.hadoop.hbase.master.handler.TableDeleteFamilyHandler.<init>(TableDeleteFamilyHandler.java:43)
> at
> org.apache.hadoop.hbase.master.HMaster.deleteColumn(HMaster.java:1303) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at
> java.lang.reflect.Method.invoke(Method.java:597) at
> org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java:372)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
