[ 
https://issues.apache.org/jira/browse/HBASE-19116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363276#comment-16363276
 ] 

stack commented on HBASE-19116:
-------------------------------

Here is reading a hbase2 hfile with an hbase1 reader:

stack@ve0524:~$ ./hbase/bin/hbase --config ~/conf_hbase/ 
org.apache.hadoop.hbase.io.hfile.HFile --printmeta -f 
/hbase/archive/data/default/IntegrationTestBigLinkedList/25eb09e8ddb00ea240407061e776a289/big/8e54a03ba0c14e458c57290f0b25373d
2018-02-13 16:02:54,864 WARN  [main] util.NativeCodeLoader: Unable to load 
native-hadoop library for your platform... using builtin-java classes where 
applicable
2018-02-13 16:02:55,284 INFO  [main] hfile.CacheConfig: Created cacheConfig: 
CacheConfig:disabled
Block index size as per heapsize: 53152
reader=/hbase/archive/data/default/IntegrationTestBigLinkedList/25eb09e8ddb00ea240407061e776a289/big/8e54a03ba0c14e458c57290f0b25373d,
    compression=none,
    cacheConf=CacheConfig:disabled,
    firstKey=\xC7\x1Cr&Y((?\x0E$\x1F\xAF\x966%1/big:big/1518565482300/Put,
    lastKey=\xD5Vr_\x13\x9C\x10]\xAE\x19\xDE_9\x1A]</big:big/1518565491617/Put,
    avgKeyLen=34,
    avgValueLen=160,
    entries=281491,
    length=58175284
Trailer:
    fileinfoOffset=58170565,
    loadOnOpenDataOffset=58145710,
    dataIndexCount=880,
    metaIndexCount=0,
    totalUncomressedBytes=58132754,
    entryCount=281491,
    compressionCodec=NONE,
    uncompressedDataIndexSize=24777,
    numDataIndexLevels=1,
    firstDataBlockOffset=0,
    lastDataBlockOffset=57971269,
    comparatorClassName=org.apache.hadoop.hbase.KeyValue$KeyComparator,
    encryptionKey=NONE,
    majorVersion=3,
    minorVersion=3
Fileinfo:
    BLOOM_FILTER_TYPE = ROW
    DELETE_FAMILY_COUNT = \x00\x00\x00\x00\x00\x00\x00\x00
    EARLIEST_PUT_TS = \x00\x00\x01a\x91\x8EM\x97
    KEY_VALUE_VERSION = \x00\x00\x00\x01
    LAST_BLOOM_KEY = \xD5Vr_\x13\x9C\x10]\xAE\x19\xDE_9\x1A]<
    MAJOR_COMPACTION_KEY = \x00
    MAX_MEMSTORE_TS_KEY = \x00\x00\x00\x00\x00\x00\xD6\x0A
    MAX_SEQ_ID_KEY = 54795
    TIMERANGE = 5783278630777363385....-8315566786350564724
    hfile.AVG_KEY_LEN = 34
    hfile.AVG_VALUE_LEN = 160
    hfile.CREATE_TIME_TS = \x00\x00\x01a\x91\x8E\x84\xD6
    hfile.LASTKEY = 
\x00\x10\xD5Vr_\x13\x9C\x10]\xAE\x19\xDE_9\x1A]<\x03bigbig\x00\x00\x01a\x91\x8E\x83\xA1\x04
Mid-key: \x00\x03\xCE7\x12\x00\x7F\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF
Bloom filter:
    BloomSize: 393216
    No of Keys in bloom: 281491
    Max Keys for bloom: 327918
    Percentage filled: 86%
    Number of chunks: 3
    Comparator: RawBytesComparator
Delete Family Bloom filter:
    Not present

Note the comparator name.

Seems to read keys fine.

This will have to do for now. Trying to open an hbase1 over an hbase2 cluster 
does this:


429519 2018-02-13 16:07:32,526 INFO  
[PriorityRpcServer.handler=14,queue=0,port=16020] regionserver.RSRpcServices: 
Open hbase:namespace,,1516822761006.9e8d0aeab209a39fde17cf8901d89252.
429520 2018-02-13 16:07:32,528 WARN  
[PriorityRpcServer.handler=14,queue=0,port=16020] regionserver.RSRpcServices: 
Failed opening region 
hbase:namespace,,1516822761006.9e8d0aeab209a39fde17cf8901d89252.
429521 java.io.IOException: missing table descriptor for 
9e8d0aeab209a39fde17cf8901d89252
429522   at 
org.apache.hadoop.hbase.regionserver.RSRpcServices.openRegion(RSRpcServices.java:1591)
429523   at 
org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:22239)
429524   at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2196)
429525   at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
429526   at 
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
429527   at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
429528   at java.lang.Thread.run(Thread.java:748)
429529 2018-02-13 16:07:32,528 DEBUG 
[PriorityRpcServer.handler=14,queue=0,port=16020] ipc.RpcServer: 
PriorityRpcServer.handler=14,queue=0,port=16020: callId: 160 service: 
AdminService methodName: OpenRegion size: 78 connection: 10.17.240.22:48844
429530 java.io.IOException: missing table descriptor for 
9e8d0aeab209a39fde17cf8901d89252
429531   at 
org.apache.hadoop.hbase.regionserver.RSRpcServices.openRegion(RSRpcServices.java:1591)
429532   at 
org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:22239)
429533   at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2196)
429534   at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
429535   at 
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
429536   at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
429537   at java.lang.Thread.run(Thread.java:748)


Seems like there is an incompatibility between how hbase1 and hbase2 do schema 
in FS; the hbase1 is not picking up the schemas written by hbase2. Fun, fun, 
fun. Will worry about it later when rolling upgrade (if we have too...)

> Currently the tail of hfiles with CellComparator* classname makes it so 
> hbase1 can't open hbase2 written hfiles; fix
> --------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-19116
>                 URL: https://issues.apache.org/jira/browse/HBASE-19116
>             Project: HBase
>          Issue Type: Sub-task
>          Components: HFile, migration
>            Reporter: stack
>            Assignee: stack
>            Priority: Critical
>             Fix For: 2.0.0-beta-2
>
>         Attachments: HBASE-19116.branch-2.001.patch
>
>
> See tail of HBASE-19052 for discussion which concludes we should try and make 
> it so operators do not have to go to latest hbase version before they 
> upgrade, at least if we can avoid it.
> The necessary change of our default comparator from KV to Cell naming has 
> hfiles with tails that have the classname CellComparator in them in place of 
> KeyValueComparator. If an hbase1 tries to open them, it will fail not having 
> a CellComparator in its classpath (We have name of comparator in tail because 
> different files require different comparators... perhaps we write an alias 
> instead of a class one day... TODO). HBASE-16189 and HBASE-19052 are about 
> trying to carry knowledge of hbase2 back to hbase1, a brittle approach making 
> it so operators will have to upgrade to the latest branch-1 before they can 
> go to hbase2.
> This issue is about undoing our writing of an incompatible (to hbase1) tail, 
> not unless we really have to (and it sounds like we could do without writing 
> an incompatible tail) to see if we can avoid requiring operators go to 
> lastest branch-1 (we may end up needing this but lets a have a really good 
> reason for it if we do).
> Oh, let this filing be an answer to our [~anoop.hbase]'s old high-level 
> question over in HBASE-16189:
> bq. ...means when rolling upgrade done to 2.0, first users have to upgrade to 
> some 1.x versions which is having this fix and then to 2.0.. What do you guys 
> think Whether we should avoid this kind of indirection? cc Enis Soztutar, 
> Stack, Ted Yu, Matteo Bertozzi
> Yeah, lets try to avoid this if we can...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to