[
https://issues.apache.org/jira/browse/HBASE-19116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363276#comment-16363276
]
stack commented on HBASE-19116:
-------------------------------
Here is reading a hbase2 hfile with an hbase1 reader:
stack@ve0524:~$ ./hbase/bin/hbase --config ~/conf_hbase/
org.apache.hadoop.hbase.io.hfile.HFile --printmeta -f
/hbase/archive/data/default/IntegrationTestBigLinkedList/25eb09e8ddb00ea240407061e776a289/big/8e54a03ba0c14e458c57290f0b25373d
2018-02-13 16:02:54,864 WARN [main] util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
2018-02-13 16:02:55,284 INFO [main] hfile.CacheConfig: Created cacheConfig:
CacheConfig:disabled
Block index size as per heapsize: 53152
reader=/hbase/archive/data/default/IntegrationTestBigLinkedList/25eb09e8ddb00ea240407061e776a289/big/8e54a03ba0c14e458c57290f0b25373d,
compression=none,
cacheConf=CacheConfig:disabled,
firstKey=\xC7\x1Cr&Y((?\x0E$\x1F\xAF\x966%1/big:big/1518565482300/Put,
lastKey=\xD5Vr_\x13\x9C\x10]\xAE\x19\xDE_9\x1A]</big:big/1518565491617/Put,
avgKeyLen=34,
avgValueLen=160,
entries=281491,
length=58175284
Trailer:
fileinfoOffset=58170565,
loadOnOpenDataOffset=58145710,
dataIndexCount=880,
metaIndexCount=0,
totalUncomressedBytes=58132754,
entryCount=281491,
compressionCodec=NONE,
uncompressedDataIndexSize=24777,
numDataIndexLevels=1,
firstDataBlockOffset=0,
lastDataBlockOffset=57971269,
comparatorClassName=org.apache.hadoop.hbase.KeyValue$KeyComparator,
encryptionKey=NONE,
majorVersion=3,
minorVersion=3
Fileinfo:
BLOOM_FILTER_TYPE = ROW
DELETE_FAMILY_COUNT = \x00\x00\x00\x00\x00\x00\x00\x00
EARLIEST_PUT_TS = \x00\x00\x01a\x91\x8EM\x97
KEY_VALUE_VERSION = \x00\x00\x00\x01
LAST_BLOOM_KEY = \xD5Vr_\x13\x9C\x10]\xAE\x19\xDE_9\x1A]<
MAJOR_COMPACTION_KEY = \x00
MAX_MEMSTORE_TS_KEY = \x00\x00\x00\x00\x00\x00\xD6\x0A
MAX_SEQ_ID_KEY = 54795
TIMERANGE = 5783278630777363385....-8315566786350564724
hfile.AVG_KEY_LEN = 34
hfile.AVG_VALUE_LEN = 160
hfile.CREATE_TIME_TS = \x00\x00\x01a\x91\x8E\x84\xD6
hfile.LASTKEY =
\x00\x10\xD5Vr_\x13\x9C\x10]\xAE\x19\xDE_9\x1A]<\x03bigbig\x00\x00\x01a\x91\x8E\x83\xA1\x04
Mid-key: \x00\x03\xCE7\x12\x00\x7F\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF
Bloom filter:
BloomSize: 393216
No of Keys in bloom: 281491
Max Keys for bloom: 327918
Percentage filled: 86%
Number of chunks: 3
Comparator: RawBytesComparator
Delete Family Bloom filter:
Not present
Note the comparator name.
Seems to read keys fine.
This will have to do for now. Trying to open an hbase1 over an hbase2 cluster
does this:
429519 2018-02-13 16:07:32,526 INFO
[PriorityRpcServer.handler=14,queue=0,port=16020] regionserver.RSRpcServices:
Open hbase:namespace,,1516822761006.9e8d0aeab209a39fde17cf8901d89252.
429520 2018-02-13 16:07:32,528 WARN
[PriorityRpcServer.handler=14,queue=0,port=16020] regionserver.RSRpcServices:
Failed opening region
hbase:namespace,,1516822761006.9e8d0aeab209a39fde17cf8901d89252.
429521 java.io.IOException: missing table descriptor for
9e8d0aeab209a39fde17cf8901d89252
429522 at
org.apache.hadoop.hbase.regionserver.RSRpcServices.openRegion(RSRpcServices.java:1591)
429523 at
org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:22239)
429524 at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2196)
429525 at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
429526 at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
429527 at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
429528 at java.lang.Thread.run(Thread.java:748)
429529 2018-02-13 16:07:32,528 DEBUG
[PriorityRpcServer.handler=14,queue=0,port=16020] ipc.RpcServer:
PriorityRpcServer.handler=14,queue=0,port=16020: callId: 160 service:
AdminService methodName: OpenRegion size: 78 connection: 10.17.240.22:48844
429530 java.io.IOException: missing table descriptor for
9e8d0aeab209a39fde17cf8901d89252
429531 at
org.apache.hadoop.hbase.regionserver.RSRpcServices.openRegion(RSRpcServices.java:1591)
429532 at
org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:22239)
429533 at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2196)
429534 at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
429535 at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
429536 at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
429537 at java.lang.Thread.run(Thread.java:748)
Seems like there is an incompatibility between how hbase1 and hbase2 do schema
in FS; the hbase1 is not picking up the schemas written by hbase2. Fun, fun,
fun. Will worry about it later when rolling upgrade (if we have too...)
> Currently the tail of hfiles with CellComparator* classname makes it so
> hbase1 can't open hbase2 written hfiles; fix
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-19116
> URL: https://issues.apache.org/jira/browse/HBASE-19116
> Project: HBase
> Issue Type: Sub-task
> Components: HFile, migration
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19116.branch-2.001.patch
>
>
> See tail of HBASE-19052 for discussion which concludes we should try and make
> it so operators do not have to go to latest hbase version before they
> upgrade, at least if we can avoid it.
> The necessary change of our default comparator from KV to Cell naming has
> hfiles with tails that have the classname CellComparator in them in place of
> KeyValueComparator. If an hbase1 tries to open them, it will fail not having
> a CellComparator in its classpath (We have name of comparator in tail because
> different files require different comparators... perhaps we write an alias
> instead of a class one day... TODO). HBASE-16189 and HBASE-19052 are about
> trying to carry knowledge of hbase2 back to hbase1, a brittle approach making
> it so operators will have to upgrade to the latest branch-1 before they can
> go to hbase2.
> This issue is about undoing our writing of an incompatible (to hbase1) tail,
> not unless we really have to (and it sounds like we could do without writing
> an incompatible tail) to see if we can avoid requiring operators go to
> lastest branch-1 (we may end up needing this but lets a have a really good
> reason for it if we do).
> Oh, let this filing be an answer to our [~anoop.hbase]'s old high-level
> question over in HBASE-16189:
> bq. ...means when rolling upgrade done to 2.0, first users have to upgrade to
> some 1.x versions which is having this fix and then to 2.0.. What do you guys
> think Whether we should avoid this kind of indirection? cc Enis Soztutar,
> Stack, Ted Yu, Matteo Bertozzi
> Yeah, lets try to avoid this if we can...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
