[
https://issues.apache.org/jira/browse/HBASE-20592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16490737#comment-16490737
]
Peter Somogyi commented on HBASE-20592:
---------------------------------------
Example running against a cluster which have some PREFIX_TREE encodings:
{noformat}
bash-3.2$ bin/hbase org.apache.hadoop.hbase.tool.DataBlockEncodingValidator
2018-05-25 15:33:26,093 WARN [main] util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
2018-05-25 15:33:26,414 INFO [main] zookeeper.ReadOnlyZKClient: Connect
0x4387b79e to psomogyi.vpc.cloudera.com:2181 with session timeout=90000ms,
retries 30, retry interval 1000ms, keepAlive=60000ms
2018-05-25 15:33:26,424 INFO
[ReadOnlyZKClient-psomogyi.vpc.cloudera.com:2181@0x4387b79e]
zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f,
built on 03/23/2017 10:13 GMT
2018-05-25 15:33:26,425 INFO
[ReadOnlyZKClient-psomogyi.vpc.cloudera.com:2181@0x4387b79e]
zookeeper.ZooKeeper: Client environment:host.name=172.30.65.195
2018-05-25 15:33:26,425 INFO
[ReadOnlyZKClient-psomogyi.vpc.cloudera.com:2181@0x4387b79e]
zookeeper.ZooKeeper: Client environment:java.version=1.8.0_171
2018-05-25 15:33:26,425 INFO
[ReadOnlyZKClient-psomogyi.vpc.cloudera.com:2181@0x4387b79e]
zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
2018-05-25 15:33:26,425 INFO
[ReadOnlyZKClient-psomogyi.vpc.cloudera.com:2181@0x4387b79e]
zookeeper.ZooKeeper: Client
environment:java.home=/Users/peter.somogyi/.sdkman/candidates/java/8.0.171-oracle/jre
2018-05-25 15:33:26,425 INFO
[ReadOnlyZKClient-psomogyi.vpc.cloudera.com:2181@0x4387b79e]
zookeeper.ZooKeeper:
/target/hbase-examples-3.0.0-SNAPSHOT.jar:/Users/peter.somogyi/Cloudera/hbase/hbase-endpoint/target/hbase-endpoint-3.0.0-SNAPSHOT.jar:/Users/peter.somogyi/.m2/repository/org/apache/curator/curator-framework/4.0.0/curator-framework-4.0.0.jar:/Users/peter.somogyi/.m2/repository/org/apache/curator/curator-client/4.0.0/curator-client-4.0.0.jar:/Users/peter.somogyi/.m2/repository/org/apache/curator/curator-recipes/4.0.0/curator-recipes-4.0.0.jar:/Users/peter.somogyi/Cloudera/hbase/hbase-zookeeper/target/hbase-zookeeper-3.0.0-SNAPSHOT.jar:/Users/peter.somogyi/Cloudera/hbase/hbase-rsgroup/target/hbase-rsgroup-3.0.0-SNAPSHOT.jar:/Users/peter.somogyi/Cloudera/hbase/hbase-annotations/target/hbase-annotations-3.0.0-SNAPSHOT-tests.jar:/Users/peter.somogyi/.m2/repository/org/apache/yetus/audience-annotations/0.5.0/audience-annotations-0.5.0.jar:/Users/peter.somogyi/.m2/repository/junit/junit/4.12/junit-4.12.jar:/Users/peter.somogyi/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar
2018-05-25 15:33:26,425 INFO
[ReadOnlyZKClient-psomogyi.vpc.cloudera.com:2181@0x4387b79e]
zookeeper.ZooKeeper: Client
environment:java.library.path=/Users/peter.somogyi/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.
2018-05-25 15:33:26,425 INFO
[ReadOnlyZKClient-psomogyi.vpc.cloudera.com:2181@0x4387b79e]
zookeeper.ZooKeeper: Client
environment:java.io.tmpdir=/var/folders/rh/jstfvrls7rz6323hl2rft5_r0000gr/T/
2018-05-25 15:33:26,425 INFO
[ReadOnlyZKClient-psomogyi.vpc.cloudera.com:2181@0x4387b79e]
zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
2018-05-25 15:33:26,425 INFO
[ReadOnlyZKClient-psomogyi.vpc.cloudera.com:2181@0x4387b79e]
zookeeper.ZooKeeper: Client environment:os.name=Mac OS X
2018-05-25 15:33:26,425 INFO
[ReadOnlyZKClient-psomogyi.vpc.cloudera.com:2181@0x4387b79e]
zookeeper.ZooKeeper: Client environment:os.arch=x86_64
2018-05-25 15:33:26,425 INFO
[ReadOnlyZKClient-psomogyi.vpc.cloudera.com:2181@0x4387b79e]
zookeeper.ZooKeeper: Client environment:os.version=10.13.3
2018-05-25 15:33:26,425 INFO
[ReadOnlyZKClient-psomogyi.vpc.cloudera.com:2181@0x4387b79e]
zookeeper.ZooKeeper: Client environment:user.name=peter.somogyi
2018-05-25 15:33:26,425 INFO
[ReadOnlyZKClient-psomogyi.vpc.cloudera.com:2181@0x4387b79e]
zookeeper.ZooKeeper: Client environment:user.home=/Users/peter.somogyi
2018-05-25 15:33:26,425 INFO
[ReadOnlyZKClient-psomogyi.vpc.cloudera.com:2181@0x4387b79e]
zookeeper.ZooKeeper: Client
environment:user.dir=/Users/peter.somogyi/Cloudera/hbase
2018-05-25 15:33:26,427 INFO
[ReadOnlyZKClient-psomogyi.vpc.cloudera.com:2181@0x4387b79e]
zookeeper.ZooKeeper: Initiating client connection,
connectString=psomogyi.vpc.cloudera.com:2181 sessionTimeout=90000
watcher=org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$10/452103994@68a56ab6
2018-05-25 15:33:26,450 INFO
[ReadOnlyZKClient-psomogyi.vpc.cloudera.com:2181@0x4387b79e-SendThread(10.14.91.150:2181)]
zookeeper.ClientCnxn: Opening socket connection to server
10.14.91.150/10.14.91.150:2181. Will not attempt to authenticate using SASL
(unknown error)
2018-05-25 15:33:26,496 INFO
[ReadOnlyZKClient-psomogyi.vpc.cloudera.com:2181@0x4387b79e-SendThread(10.14.91.150:2181)]
zookeeper.ClientCnxn: Socket connection established to
10.14.91.150/10.14.91.150:2181, initiating session
2018-05-25 15:33:26,524 INFO
[ReadOnlyZKClient-psomogyi.vpc.cloudera.com:2181@0x4387b79e-SendThread(10.14.91.150:2181)]
zookeeper.ClientCnxn: Session establishment complete on server
10.14.91.150/10.14.91.150:2181, sessionid = 0x163879c5513001b, negotiated
timeout = 90000
2018-05-25 15:33:27,224 INFO [main] client.ConnectionImplementation: Closing
master protocol: MasterService
2018-05-25 15:33:27,224 INFO [main] zookeeper.ReadOnlyZKClient: Close
zookeeper connection 0x4387b79e to psomogyi.vpc.cloudera.com:2181
2018-05-25 15:33:27,225 WARN [main] tool.DataBlockEncodingValidator: There are
15 column families with Data Block Encodings that are not compatible with HBase
2.0+. Do not upgrade until these encodings are converted to a supported one.
2018-05-25 15:33:27,226 WARN [main] tool.DataBlockEncodingValidator:
Incompatible DataBlockEncoding: Table: custom, cf: pref
2018-05-25 15:33:27,226 WARN [main] tool.DataBlockEncodingValidator:
Incompatible DataBlockEncoding: Table: custom, cf: pref2
2018-05-25 15:33:27,226 WARN [main] tool.DataBlockEncodingValidator:
Incompatible DataBlockEncoding: Table: second:other, cf: pref
2018-05-25 15:33:27,226 WARN [main] tool.DataBlockEncodingValidator:
Incompatible DataBlockEncoding: Table: second:other, cf: pref2
2018-05-25 15:33:27,226 WARN [main] tool.DataBlockEncodingValidator:
Incompatible DataBlockEncoding: Table: t0, cf: pref
2018-05-25 15:33:27,226 WARN [main] tool.DataBlockEncodingValidator:
Incompatible DataBlockEncoding: Table: t1, cf: pref
2018-05-25 15:33:27,226 WARN [main] tool.DataBlockEncodingValidator:
Incompatible DataBlockEncoding: Table: t10, cf: pref
2018-05-25 15:33:27,226 WARN [main] tool.DataBlockEncodingValidator:
Incompatible DataBlockEncoding: Table: t2, cf: pref
2018-05-25 15:33:27,226 WARN [main] tool.DataBlockEncodingValidator:
Incompatible DataBlockEncoding: Table: t3, cf: pref
2018-05-25 15:33:27,226 WARN [main] tool.DataBlockEncodingValidator:
Incompatible DataBlockEncoding: Table: t4, cf: pref
2018-05-25 15:33:27,226 WARN [main] tool.DataBlockEncodingValidator:
Incompatible DataBlockEncoding: Table: t5, cf: pref
2018-05-25 15:33:27,226 WARN [main] tool.DataBlockEncodingValidator:
Incompatible DataBlockEncoding: Table: t6, cf: pref
2018-05-25 15:33:27,226 WARN [main] tool.DataBlockEncodingValidator:
Incompatible DataBlockEncoding: Table: t7, cf: pref
2018-05-25 15:33:27,226 WARN [main] tool.DataBlockEncodingValidator:
Incompatible DataBlockEncoding: Table: t8, cf: pref
2018-05-25 15:33:27,226 WARN [main] tool.DataBlockEncodingValidator:
Incompatible DataBlockEncoding: Table: t9, cf: pref
bash-3.2$ echo $?
1
bash-3.2$
{noformat}
> Create a tool to verify tables do not have prefix tree encoding
> ---------------------------------------------------------------
>
> Key: HBASE-20592
> URL: https://issues.apache.org/jira/browse/HBASE-20592
> Project: HBase
> Issue Type: New Feature
> Components: Operability, tooling
> Reporter: Peter Somogyi
> Assignee: Peter Somogyi
> Priority: Minor
> Fix For: 2.1.0
>
> Attachments: HBASE-20592.master.001.patch
>
>
> HBase 2.0.0 removed PREFIX_TREE encoding so users need to modify data block
> encoding to something else before upgrading to HBase 2.0+. A tool would help
> users to verify that there are no tables left with PREFIX_TREE encoding.
> The tool needs to check the following:
> * There are no tables where DATA_BLOCK_ENCODING => 'PREFIX_TREE'
> * -Check existing hfiles that none of them have PREFIX_TREE encoding (in
> case table description is changed but hfiles were not rewritten)-
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
