Hi
I'm trying to upgrade phoenix 2.2.3-incubating to phoenix 3.2.2 on my local
computer first in order to gain confidence that it will work on the
production cluster. We use HBase 0.94.6 CDH 4.4.0.
1) My first question is what release to pick? There is no phoenix 3.2.2 jar
in maven central (only 3.2.1 jar) and there is no 3.2.1 tar.gz distribution
is available from phoenix.apache.org (only 3.2.2).
Anyway, I tried replacing phoenix-2.2.3-incubating.jar with
phoenix-core-3.2.2.jar
(from phoenix-3.2.2-bin.tar.gz) in $HBASE_HOME/lib and restarted HBase.
This triggered warnings in HBase log [2], which is understandable since
phoenix-core-3.2.2.jar does not include
org.apache.phoenix.join.HashJoiningRegionObserver.
2) Next step I updated the client from phoenix-2.2.3-incubating.jar to
phoenix-core-3.2.1.jar and added the following to hbase-site.xml.
<configuration>
<property>
<name>phoenix.client.autoUpgradeWhiteList</name>
<value>*</value>
</property>
</configuration>
I think this triggered the upgrade process as soon as the client contacted
HBase. But all tables are inaccessible after this process [2]. Now there
are also periodic warnings occurring periodically in HBase log [3].
I also tried to install 3.2.2 manually into maven and run same client and
server version, but this did not change the behavior.
I'm not sure what have gone wrong?
Cheers,
-Kristoffer
[1]
15/01/12 14:25:12 WARN regionserver.RegionCoprocessorHost: attribute
'coprocessor$6' has invalid coprocessor specification
'|org.apache.hbase.index.Indexer|1073741823|org.apache.hbase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec,index.builder=org.apache.phoenix.index.PhoenixIndexBuilder'
15/01/12 14:25:12 WARN regionserver.RegionCoprocessorHost:
java.io.IOException: No jar path specified for
org.apache.hbase.index.Indexer
15/01/12 14:25:12 WARN regionserver.RegionCoprocessorHost: attribute
'coprocessor$4' has invalid coprocessor specification
'|org.apache.phoenix.join.HashJoiningRegionObserver|1|'
15/01/12 14:25:12 WARN regionserver.RegionCoprocessorHost:
java.io.IOException: No jar path specified for
org.apache.phoenix.join.HashJoiningRegionObserver
[2]
Caused by: org.apache.phoenix.schema.TableNotFoundException: ERROR 1012
(42M03): Table undefined. tableName=TRACKING_COUNTER
[3]
15/01/12 14:25:24 WARN coprocessor.MetaDataRegionObserver:
ScheduledBuildIndexTask failed!
org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column
family 0 does not exist in region
SYSTEM.TABLE,,1421069004810.6b41a24a11a4f106b85d6ae76334baf6. in table
{NAME => 'SYSTEM.TABLE', SPLIT_POLICY =>
'org.apache.phoenix.schema.MetaDataSplitPolicy', UpgradeTo21 => 'true',
UpgradeTo20 => 'true', coprocessor$7 =>
'|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', UpgradeTo22 =>
'true', coprocessor$6 =>
'|org.apache.phoenix.coprocessor.MetaDataEndpointImpl|1|', coprocessor$5 =>
'|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
coprocessor$4 => '|org.apache.phoenix.join.HashJoiningRegionObserver|1|',
coprocessor$3 =>
'|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1|',
coprocessor$2 =>
'|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
coprocessor$1 => '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|',
FAMILIES => [{NAME => '_0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER
=> 'NONE', REPLICATION_SCOPE => '0', VERSIONS => '1000', COMPRESSION =>
'NONE', TTL => '2147483647', MIN_VERSIONS => '0', KEEP_DELETED_CELLS =>
'true', BLOCKSIZE => '65536', ENCODE_ON_DISK => 'true', IN_MEMORY =>
'false', BLOCKCACHE => 'true'}]}
at
org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:5341)
at
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1744)
at
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1722)
at
org.apache.phoenix.coprocessor.MetaDataRegionObserver$BuildIndexScheduleTask.run(MetaDataRegionObserver.java:174)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
