[
https://issues.apache.org/jira/browse/PHOENIX-4176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156520#comment-16156520
]
Samarth Jain commented on PHOENIX-4176:
---------------------------------------
Yes, we do this when online schema update is false:
{code}
if (!allowOnlineTableSchemaUpdate()) {
admin.disableTable(tableName);
admin.modifyTable(tableName, newDesc);
admin.enableTable(tableName);
}
{code}
Yes, it is potentially a real production issue if we are using HBase 1.x. If
there is sufficient time between such kind of DDL operations and the queries
that rely on them, then the risk of running into this is low. Of course, doing
offline schema update isn't ideal since we need to disable the table which is
taking a downtime and a no-go. We likely will have to come up with a general
solution for this.
Below is the failure stacktrace for reference:
{code}
org.apache.phoenix.exception.PhoenixIOException:
org.apache.phoenix.exception.PhoenixIOException:
org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column family
CF does not exist in region
T000225,\x06\x00\x00,1504753367780.d3e683408d7801ced0ea2940400cbc2e. in table
'T000225', {TABLE_ATTRIBUTES => {coprocessor$1 =>
'|org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|', coprocessor$2
=>
'|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|805306366|',
coprocessor$3 =>
'|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|805306366|',
coprocessor$4 =>
'|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|805306366|',
coprocessor$5 =>
'|org.apache.phoenix.hbase.index.Indexer|805306366|index.builder=org.apache.phoenix.index.PhoenixIndexBuilder,org.apache.hadoop.hbase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec'},
{NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'NONE',
REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
'FOREVER', MIN_VERSIONS => '0', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE =>
'65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
at
org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:8031)
at
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2675)
at
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2660)
at
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2654)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.newRegionScanner(RSRpcServices.java:2551)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2809)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34950)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2347)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)
at
org.apache.phoenix.end2end.FlappingAlterTableIT.testAddColumnForNewColumnFamily(FlappingAlterTableIT.java:60)
{code}
> Reduce chances of tests flapping with ColumnFamilyNotFoundException for HBase
> 1.x
> ---------------------------------------------------------------------------------
>
> Key: PHOENIX-4176
> URL: https://issues.apache.org/jira/browse/PHOENIX-4176
> Project: Phoenix
> Issue Type: Bug
> Reporter: Samarth Jain
> Assignee: Samarth Jain
> Attachments: PHOENIX-4176.patch
>
>
> In HBase 1.x, when adding a new column family, the check to detect whether
> the HTableDescriptor is updated isn't enough. Tests that add a new column
> family run the risk of flapping when number of regions on the table are high
> (since the column family has to be added to all the regions). Till we figure
> out a permanent solution for this, we can possibly decrease chances of such
> tests from flapping by reducing number of regions/pre-splits and possibly
> configuring hbase.online.schema.update.enable to false in our tests.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
