[
https://issues.apache.org/jira/browse/PHOENIX-6953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth updated PHOENIX-6953:
---------------------------------
Description:
New indexes created on tables that are using the old indexing code have the
GlobalIndexChecker coprocessor defined.
This results in excessive index rebuilds and bad performance.
We need to check whether the data table has been upgraded to the new indexing
when creating a new index.
Repro steps on master:
Create the data table and index
{noformat}
0: jdbc:phoenix:> create table bubu (k integer primary key, v1 integer, v2
integer);
0: jdbc:phoenix:> create index bubu_1 on bubu(v1);{noformat}
Roll back to the old indexing to simulate upgrade from old code
{noformat}
# hbase org.apache.phoenix.mapreduce.index.IndexUpgradeTool -o rollback -tb
BUBU{noformat}
Create another index
{noformat}
0: jdbc:phoenix:> create index BUBU_2 on BUBU(v2); {noformat}
check the tables:
{noformat}
hbase:002:0> describe 'BUBU'
...
coprocessor$7 =>
'|org.apache.phoenix.hbase.index.Indexer|805306366|index.builder=org.apache.phoenix.index.PhoenixIndexBuilder,org.apache.hadoop.hbase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec',
...
hbase:002:0> describe 'BUBU_1'
...
coprocessor$2 =>
'|org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|', coprocessor$3
=>
'|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|805306366|',
coprocessor$4 =>
'|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|805306366|',
coprocessor$5 =>
'|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|805306366|',
coprocessor$6 =>
'|org.apache.phoenix.coprocessor.PhoenixTTLRegionObserver|805306364|',
...
hbase:003:0> describe 'BUBU_2'
...
coprocessor$1 => '|org.apache.phoenix.index.GlobalIndexChecker|805306365|',
coprocessor$2 =>
'|org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|', coprocessor$3
=>
'|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|805306366|',
coprocessor$4 =>
'|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|805306366|',
coprocessor$5 =>
'|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|805306366|',
coprocessor$6 =>
'|org.apache.phoenix.coprocessor.PhoenixTTLRegionObserver|805306364|',
...{noformat}
The data tables has {_}Indexer{_}, BUBU_1 doesn't have any index related
coprocessors, BUBU_2 has _GlobalIndexChecker_
was:
New indexes created on tables that are using the old indexing code have the
GlobalIndexChecker coprocessor defined.
This results in excessive index rebuilds and bad performance.
We need to check whether the data table has been upgraded to the new indexing
when creating a new index.
Repro steps on master:
Create the data table and index
{noformat}
0: jdbc:phoenix:> create table bubu (k integer primary key, v1 integer, v2
integer);
0: jdbc:phoenix:> create index bubu_1 on bubu(v1);{noformat}
Roll back the old indexing to simulate upgrade from old code
{noformat}
# hbase org.apache.phoenix.mapreduce.index.IndexUpgradeTool -o rollback -tb
BUBU{noformat}
Create another index
{noformat}
0: jdbc:phoenix:> create index BUBU_2 on BUBU(v2); {noformat}
check the tables:
{noformat}
hbase:002:0> describe 'BUBU'
...
coprocessor$7 =>
'|org.apache.phoenix.hbase.index.Indexer|805306366|index.builder=org.apache.phoenix.index.PhoenixIndexBuilder,org.apache.hadoop.hbase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec',
...
hbase:002:0> describe 'BUBU_1'
...
coprocessor$2 =>
'|org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|', coprocessor$3
=>
'|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|805306366|',
coprocessor$4 =>
'|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|805306366|',
coprocessor$5 =>
'|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|805306366|',
coprocessor$6 =>
'|org.apache.phoenix.coprocessor.PhoenixTTLRegionObserver|805306364|',
...
hbase:003:0> describe 'BUBU_2'
...
coprocessor$1 => '|org.apache.phoenix.index.GlobalIndexChecker|805306365|',
coprocessor$2 =>
'|org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|', coprocessor$3
=>
'|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|805306366|',
coprocessor$4 =>
'|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|805306366|',
coprocessor$5 =>
'|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|805306366|',
coprocessor$6 =>
'|org.apache.phoenix.coprocessor.PhoenixTTLRegionObserver|805306364|',
...{noformat}
The data table has {_}Indexer{_}, BUBU_1 doesn't have any index related
coprocessors, BUBU_2 has _GlobalIndexChecker._
> Creating indexes on a table with old indexing leads to inconsistent
> co-processors
> ---------------------------------------------------------------------------------
>
> Key: PHOENIX-6953
> URL: https://issues.apache.org/jira/browse/PHOENIX-6953
> Project: Phoenix
> Issue Type: Bug
> Components: core
> Affects Versions: 5.2.0, 5.1.3
> Reporter: Istvan Toth
> Assignee: Istvan Toth
> Priority: Major
>
> New indexes created on tables that are using the old indexing code have the
> GlobalIndexChecker coprocessor defined.
> This results in excessive index rebuilds and bad performance.
> We need to check whether the data table has been upgraded to the new indexing
> when creating a new index.
> Repro steps on master:
> Create the data table and index
> {noformat}
> 0: jdbc:phoenix:> create table bubu (k integer primary key, v1 integer, v2
> integer);
> 0: jdbc:phoenix:> create index bubu_1 on bubu(v1);{noformat}
> Roll back to the old indexing to simulate upgrade from old code
> {noformat}
> # hbase org.apache.phoenix.mapreduce.index.IndexUpgradeTool -o rollback -tb
> BUBU{noformat}
> Create another index
> {noformat}
> 0: jdbc:phoenix:> create index BUBU_2 on BUBU(v2); {noformat}
> check the tables:
> {noformat}
> hbase:002:0> describe 'BUBU'
> ...
> coprocessor$7 =>
> '|org.apache.phoenix.hbase.index.Indexer|805306366|index.builder=org.apache.phoenix.index.PhoenixIndexBuilder,org.apache.hadoop.hbase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec',
> ...
> hbase:002:0> describe 'BUBU_1'
> ...
> coprocessor$2 =>
> '|org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|',
> coprocessor$3 =>
> '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|805306366|',
> coprocessor$4 =>
> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|805306366|',
> coprocessor$5 =>
> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|805306366|',
> coprocessor$6 =>
> '|org.apache.phoenix.coprocessor.PhoenixTTLRegionObserver|805306364|',
> ...
> hbase:003:0> describe 'BUBU_2'
> ...
> coprocessor$1 => '|org.apache.phoenix.index.GlobalIndexChecker|805306365|',
> coprocessor$2 =>
> '|org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|',
> coprocessor$3 =>
> '|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|805306366|',
> coprocessor$4 =>
> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|805306366|',
> coprocessor$5 =>
> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|805306366|',
> coprocessor$6 =>
> '|org.apache.phoenix.coprocessor.PhoenixTTLRegionObserver|805306364|',
> ...{noformat}
> The data tables has {_}Indexer{_}, BUBU_1 doesn't have any index related
> coprocessors, BUBU_2 has _GlobalIndexChecker_
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
