[jira] [Comment Edited] (HBASE-26021) HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
[
https://issues.apache.org/jira/browse/HBASE-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17371999#comment-17371999
]
Viraj Jasani edited comment on HBASE-26021 at 6/30/21, 1:30 PM:
Performed a rolling upgrade from 1.7 to 2.4 with this patch, and things look
good. Thanks [~bharathv].
Btw for this upgrade testing, I did not configure ConnectionRegistry as
MasterRegistry. Perhaps we can test rolling upgrade with MasterRegistry on a
cluster with better capacity and traffic.
Upgrade with default configs (except necessary changes for upgrade) look good
so I think we are good.
was (Author: vjasani):
Performed a rolling upgrade from 1.7 to 2.4 with this patch, and things look
good. Thanks [~bharathv].
Btw for this upgrade testing, I did not configure ConnectionRegistry as
MasterRegistry. Perhaps we can test rolling upgrade with MasterRegistry on a
cluster with better capacity and traffic.
> HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
> --
>
> Key: HBASE-26021
> URL: https://issues.apache.org/jira/browse/HBASE-26021
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 1.7.0, 2.4.4
>Reporter: Viraj Jasani
>Assignee: Bharath Vissapragada
>Priority: Major
> Labels: compatibility, incompatibility, serialization, upgrade
> Fix For: 1.7.1
>
> Attachments: Screenshot 2021-06-22 at 12.54.21 PM.png, Screenshot
> 2021-06-22 at 12.54.30 PM.png
>
>
> As of today, if we bring up HBase cluster using branch-1 and upgrade to
> branch-2.4, we are facing issue while parsing namespace from HDFS fileinfo.
> Instead of "*hbase:meta*" and "*hbase:namespace*", parsing using ProtobufUtil
> seems to be producing "*\n hbase meta*" and "*\n hbase namespace*"
> {code:java}
> 2021-06-22 00:05:56,611 INFO
> [RpcServer.priority.RWQ.Fifo.read.handler=3,queue=1,port=16025]
> regionserver.RSRpcServices: Open hbase:meta,,1.1588230740
> 2021-06-22 00:05:56,648 INFO
> [RpcServer.priority.RWQ.Fifo.read.handler=5,queue=1,port=16025]
> regionserver.RSRpcServices: Open
> hbase:namespace,,1624297762817.396cb6cc00cd4334cb1ea3a792d7529a.
> 2021-06-22 00:05:56,759 ERROR
> [RpcServer.priority.RWQ.Fifo.read.handler=5,queue=1,port=16025]
> ipc.RpcServer: Unexpected throwable object
> java.lang.IllegalArgumentException: Illegal character <
> > at 0. Namespaces may only contain 'alphanumeric characters' from any
> > language and digits:
> ^Ehbase^R namespace
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:246)
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:220)
> at org.apache.hadoop.hbase.TableName.(TableName.java:348)
> at
> org.apache.hadoop.hbase.TableName.createTableNameIfNecessary(TableName.java:385)
> at org.apache.hadoop.hbase.TableName.valueOf(TableName.java:508)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toTableName(ProtobufUtil.java:2292)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toTableDescriptor(ProtobufUtil.java:2937)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder$ModifyableTableDescriptor.parseFrom(TableDescriptorBuilder.java:1625)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder$ModifyableTableDescriptor.access$200(TableDescriptorBuilder.java:597)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder.parseFrom(TableDescriptorBuilder.java:320)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.readTableDescriptor(FSTableDescriptors.java:511)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:496)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:482)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:210)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.openRegion(RSRpcServices.java:2112)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:35218)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:395)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
> 2021-06-22 00:05:56,759 ERROR
> [RpcServer.priority.RWQ.Fifo.read.handler=3,queue=1,port=16025]
> ipc.RpcServer: Unexpected throwable object
> java.lang.IllegalArgumentException: Illegal character <
> > at 0.
[jira] [Comment Edited] (HBASE-26021) HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
[
https://issues.apache.org/jira/browse/HBASE-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17369057#comment-17369057
]
Andrew Kyle Purtell edited comment on HBASE-26021 at 6/25/21, 3:44 PM:
---
[~vjasani] [~reidchan] [~zhangduo] [~bharathv]
This is what I would recommend as the least bad option (all options are bad...)
# Withdraw the 1.7.0 release immediately. Delete from the mirrors.
# Apply the fix. I like your simple fix posted on this issue above [~vjasani]
but since we backported a change it makes more sense to backport its revert
too:
[commit|https://github.com/apache/hbase/commit/bdb0cc8808af7c3d08af4a506f34b8341726b58e]
. However, if someone has already upgraded to 1.7.0, has this ship sailed? Do
we need your fix on this issue instead? In any case we clean up the issue.
# Release 1.7.0.1.
The vote and review of the 1.7.0.1 RC should be fast given there have been few
commits since 1.7.0.
When we release 1.7.0.1 we can state in the release notes that 1.7.0 had this
bug and anyone who upgraded to it who wants to upgrade to 2.x must upgrade to
1.7.0.1 first.
was (Author: apurtell):
[~vjasani] [~reidchan] [~zhangduo]
This is what I would recommend as the least bad option (all options are bad...)
# Withdraw the 1.7.0 release immediately. Delete from the mirrors.
# Apply the fix. I like your simple fix posted on this issue above [~vjasani]
but since we backported a change it makes more sense to backport its revert
too:
[commit|https://github.com/apache/hbase/commit/bdb0cc8808af7c3d08af4a506f34b8341726b58e]
. However, if someone has already upgraded to 1.7.0, has this ship sailed? Do
we need your fix on this issue instead? In any case we clean up the issue.
# Release 1.7.0.1.
The vote and review of the 1.7.0.1 RC should be fast given there have been few
commits since 1.7.0.
When we release 1.7.0.1 we can state in the release notes that 1.7.0 had this
bug and anyone who upgraded to it who wants to upgrade to 2.x must upgrade to
1.7.0.1 first.
> HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
> --
>
> Key: HBASE-26021
> URL: https://issues.apache.org/jira/browse/HBASE-26021
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.7.0, 2.4.4
>Reporter: Viraj Jasani
>Priority: Major
> Attachments: Screenshot 2021-06-22 at 12.54.21 PM.png, Screenshot
> 2021-06-22 at 12.54.30 PM.png
>
>
> As of today, if we bring up HBase cluster using branch-1 and upgrade to
> branch-2.4, we are facing issue while parsing namespace from HDFS fileinfo.
> Instead of "*hbase:meta*" and "*hbase:namespace*", parsing using ProtobufUtil
> seems to be producing "*\n hbase meta*" and "*\n hbase namespace*"
> {code:java}
> 2021-06-22 00:05:56,611 INFO
> [RpcServer.priority.RWQ.Fifo.read.handler=3,queue=1,port=16025]
> regionserver.RSRpcServices: Open hbase:meta,,1.1588230740
> 2021-06-22 00:05:56,648 INFO
> [RpcServer.priority.RWQ.Fifo.read.handler=5,queue=1,port=16025]
> regionserver.RSRpcServices: Open
> hbase:namespace,,1624297762817.396cb6cc00cd4334cb1ea3a792d7529a.
> 2021-06-22 00:05:56,759 ERROR
> [RpcServer.priority.RWQ.Fifo.read.handler=5,queue=1,port=16025]
> ipc.RpcServer: Unexpected throwable object
> java.lang.IllegalArgumentException: Illegal character <
> > at 0. Namespaces may only contain 'alphanumeric characters' from any
> > language and digits:
> ^Ehbase^R namespace
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:246)
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:220)
> at org.apache.hadoop.hbase.TableName.(TableName.java:348)
> at
> org.apache.hadoop.hbase.TableName.createTableNameIfNecessary(TableName.java:385)
> at org.apache.hadoop.hbase.TableName.valueOf(TableName.java:508)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toTableName(ProtobufUtil.java:2292)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toTableDescriptor(ProtobufUtil.java:2937)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder$ModifyableTableDescriptor.parseFrom(TableDescriptorBuilder.java:1625)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder$ModifyableTableDescriptor.access$200(TableDescriptorBuilder.java:597)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder.parseFrom(TableDescriptorBuilder.java:320)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.readTableDescriptor(FSTableDescriptors.java:511)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:496)
> at
>
[jira] [Comment Edited] (HBASE-26021) HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
[
https://issues.apache.org/jira/browse/HBASE-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17369057#comment-17369057
]
Andrew Kyle Purtell edited comment on HBASE-26021 at 6/24/21, 7:24 PM:
---
[~vjasani] [~reidchan] [~zhangduo]
This is what I would recommend as the least bad option (all options are bad...)
# Withdraw the 1.7.0 release immediately. Delete from the mirrors.
# Apply the fix. I like your simple fix posted on this issue above [~vjasani]
but since we backported a change it makes more sense to backport its revert
too:
[commit|https://github.com/apache/hbase/commit/bdb0cc8808af7c3d08af4a506f34b8341726b58e]
. However, if someone has already upgraded to 1.7.0, has this ship sailed? Do
we need your fix on this issue instead? In any case we clean up the issue.
# Release 1.7.0.1.
The vote and review of the 1.7.0.1 RC should be fast given there have been few
commits since 1.7.0.
When we release 1.7.0.1 we can state in the release notes that 1.7.0 had this
bug and anyone who upgraded to it who wants to upgrade to 2.x must upgrade to
1.7.0.1 first.
was (Author: apurtell):
[~vjasani] [~reidchan] [~zhangduo]
This is what I would recommend as the least bad option (all options are bad...)
# Withdraw the 1.7.0 release immediately. Delete from the mirrors.
# Apply the fix. I like your simple fix posted on this issue above [~vjasani]
but since we backported a change it makes more sense to backport its revert
too:
[commit|https://github.com/apache/hbase/commit/bdb0cc8808af7c3d08af4a506f34b8341726b58e]
# Release 1.7.0.1.
The vote and review of the 1.7.0.1 RC should be fast given there have been few
commits since 1.7.0.
When we release 1.7.0.1 we can state in the release notes that 1.7.0 had this
bug and anyone who upgraded to it who wants to upgrade to 2.x must upgrade to
1.7.0.1 first.
> HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
> --
>
> Key: HBASE-26021
> URL: https://issues.apache.org/jira/browse/HBASE-26021
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.7.0, 2.4.4
>Reporter: Viraj Jasani
>Priority: Major
> Attachments: Screenshot 2021-06-22 at 12.54.21 PM.png, Screenshot
> 2021-06-22 at 12.54.30 PM.png
>
>
> As of today, if we bring up HBase cluster using branch-1 and upgrade to
> branch-2.4, we are facing issue while parsing namespace from HDFS fileinfo.
> Instead of "*hbase:meta*" and "*hbase:namespace*", parsing using ProtobufUtil
> seems to be producing "*\n hbase meta*" and "*\n hbase namespace*"
> {code:java}
> 2021-06-22 00:05:56,611 INFO
> [RpcServer.priority.RWQ.Fifo.read.handler=3,queue=1,port=16025]
> regionserver.RSRpcServices: Open hbase:meta,,1.1588230740
> 2021-06-22 00:05:56,648 INFO
> [RpcServer.priority.RWQ.Fifo.read.handler=5,queue=1,port=16025]
> regionserver.RSRpcServices: Open
> hbase:namespace,,1624297762817.396cb6cc00cd4334cb1ea3a792d7529a.
> 2021-06-22 00:05:56,759 ERROR
> [RpcServer.priority.RWQ.Fifo.read.handler=5,queue=1,port=16025]
> ipc.RpcServer: Unexpected throwable object
> java.lang.IllegalArgumentException: Illegal character <
> > at 0. Namespaces may only contain 'alphanumeric characters' from any
> > language and digits:
> ^Ehbase^R namespace
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:246)
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:220)
> at org.apache.hadoop.hbase.TableName.(TableName.java:348)
> at
> org.apache.hadoop.hbase.TableName.createTableNameIfNecessary(TableName.java:385)
> at org.apache.hadoop.hbase.TableName.valueOf(TableName.java:508)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toTableName(ProtobufUtil.java:2292)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toTableDescriptor(ProtobufUtil.java:2937)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder$ModifyableTableDescriptor.parseFrom(TableDescriptorBuilder.java:1625)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder$ModifyableTableDescriptor.access$200(TableDescriptorBuilder.java:597)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder.parseFrom(TableDescriptorBuilder.java:320)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.readTableDescriptor(FSTableDescriptors.java:511)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:496)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:482)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:210)
> at
>
[jira] [Comment Edited] (HBASE-26021) HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
[
https://issues.apache.org/jira/browse/HBASE-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17369057#comment-17369057
]
Andrew Kyle Purtell edited comment on HBASE-26021 at 6/24/21, 7:23 PM:
---
[~vjasani] [~reidchan] [~zhangduo]
This is what I would recommend as the least bad option (all options are bad...)
# Withdraw the 1.7.0 release immediately. Delete from the mirrors.
# Apply the fix. I like your simple fix posted on this issue above [~vjasani]
but since we backported a change it makes more sense to backport its revert
too:
[commit|https://github.com/apache/hbase/commit/bdb0cc8808af7c3d08af4a506f34b8341726b58e]
# Release 1.7.0.1.
The vote and review of the 1.7.0.1 RC should be fast given there have been few
commits since 1.7.0.
When we release 1.7.0.1 we can state in the release notes that 1.7.0 had this
bug and anyone who upgraded to it who wants to upgrade to 2.x must upgrade to
1.7.0.1 first.
was (Author: apurtell):
[~vjasani] [~reidchan] [~zhangduo]
This is what I would recommend as the least bad option (all options are bad...)
# Withdraw the 1.7.0 release immediately. Delete from the mirrors.
# Apply the fix.
# Release 1.7.0.1.
The vote and review of the 1.7.0.1 RC should be fast given there have been few
commits since 1.7.0.
When we release 1.7.0.1 we can state in the release notes that 1.7.0 had this
bug and anyone who upgraded to it who wants to upgrade to 2.x must upgrade to
1.7.0.1 first.
> HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
> --
>
> Key: HBASE-26021
> URL: https://issues.apache.org/jira/browse/HBASE-26021
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.7.0, 2.4.4
>Reporter: Viraj Jasani
>Priority: Major
> Attachments: Screenshot 2021-06-22 at 12.54.21 PM.png, Screenshot
> 2021-06-22 at 12.54.30 PM.png
>
>
> As of today, if we bring up HBase cluster using branch-1 and upgrade to
> branch-2.4, we are facing issue while parsing namespace from HDFS fileinfo.
> Instead of "*hbase:meta*" and "*hbase:namespace*", parsing using ProtobufUtil
> seems to be producing "*\n hbase meta*" and "*\n hbase namespace*"
> {code:java}
> 2021-06-22 00:05:56,611 INFO
> [RpcServer.priority.RWQ.Fifo.read.handler=3,queue=1,port=16025]
> regionserver.RSRpcServices: Open hbase:meta,,1.1588230740
> 2021-06-22 00:05:56,648 INFO
> [RpcServer.priority.RWQ.Fifo.read.handler=5,queue=1,port=16025]
> regionserver.RSRpcServices: Open
> hbase:namespace,,1624297762817.396cb6cc00cd4334cb1ea3a792d7529a.
> 2021-06-22 00:05:56,759 ERROR
> [RpcServer.priority.RWQ.Fifo.read.handler=5,queue=1,port=16025]
> ipc.RpcServer: Unexpected throwable object
> java.lang.IllegalArgumentException: Illegal character <
> > at 0. Namespaces may only contain 'alphanumeric characters' from any
> > language and digits:
> ^Ehbase^R namespace
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:246)
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:220)
> at org.apache.hadoop.hbase.TableName.(TableName.java:348)
> at
> org.apache.hadoop.hbase.TableName.createTableNameIfNecessary(TableName.java:385)
> at org.apache.hadoop.hbase.TableName.valueOf(TableName.java:508)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toTableName(ProtobufUtil.java:2292)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toTableDescriptor(ProtobufUtil.java:2937)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder$ModifyableTableDescriptor.parseFrom(TableDescriptorBuilder.java:1625)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder$ModifyableTableDescriptor.access$200(TableDescriptorBuilder.java:597)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder.parseFrom(TableDescriptorBuilder.java:320)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.readTableDescriptor(FSTableDescriptors.java:511)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:496)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:482)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:210)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.openRegion(RSRpcServices.java:2112)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:35218)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:395)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
> at
>
[jira] [Comment Edited] (HBASE-26021) HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
[
https://issues.apache.org/jira/browse/HBASE-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368776#comment-17368776
]
Viraj Jasani edited comment on HBASE-26021 at 6/24/21, 6:47 PM:
Here are HBase proto messages:
{code:java}
/**
* Table Schema
* Inspired by the rest TableSchema
*/
message TableSchema {
optional TableName table_name = 1;
repeated BytesBytesPair attributes = 2;
repeated ColumnFamilySchema column_families = 3;
repeated NameStringPair configuration = 4;
}
/** Denotes state of the table */
message TableState {
// Table's current state
enum State {
ENABLED = 0;
DISABLED = 1;
DISABLING = 2;
ENABLING = 3;
}
// This is the table's state.
required State state = 1;
required TableName table = 2;
optional uint64 timestamp = 3;
}
/** On HDFS representation of table state. */
message TableDescriptor {
required TableSchema schema = 1;
optional TableState.State state = 2 [ default = ENABLED ];
}
{code}
Serialization in FSTableDescriptors:
{code:java}
private static void writeTD(final FileSystem fs, final Path p, final
TableDescriptor htd)
throws IOException {
FSDataOutputStream out = fs.create(p, false);
try {
// We used to write this file out as a serialized HTD Writable followed by
two '\n's and then
// the toString version of HTD. Now we just write out the pb serialization.
out.write(htd.toByteArray());
} finally {
out.close();
}
}
{code}
Before this patch, we used to serialize "HTableDescriptor" in place of
"TableDescriptor" in above method (as [~apurtell] mentioned).
TableDescriptor/HTableDescriptor's toByteArray():
{code:java}
/**
* @return This instance serialized with pb with pb magic prefix
* @see #parseFrom(byte[])
*/
public byte [] toByteArray() {
return ProtobufUtil.prependPBMagic(convert().toByteArray());
}
{code}
Here is the main difference in both convert():
HTableDescriptor#convert converts TD to HBaseProtos.TableSchema (mentioned in
first code block above) whereas TableDescriptor#convert converts TD to
HBaseProtos.TableDescriptor (mentioned in first code block above). As per Proto
message definitions, HBaseProtos.TableDescriptor contains 2 fields:
HBaseProtos.TableSchema and HBaseProtos.TableState.State, so state is the
additional field.
HBase 2 deserializes only HBaseProtos.TableSchema as per this code:
{code:java}
/**
* @param bytes A pb serialized {@link ModifyableTableDescriptor} instance
* with pb magic prefix
* @return An instance of {@link ModifyableTableDescriptor} made from
* bytes
* @throws DeserializationException
* @see #toByteArray()
*/
private static TableDescriptor parseFrom(final byte[] bytes)
throws DeserializationException {
if (!ProtobufUtil.isPBMagicPrefix(bytes)) {
throw new DeserializationException("Expected PB encoded
ModifyableTableDescriptor");
}
int pblen = ProtobufUtil.lengthOfPBMagic();
HBaseProtos.TableSchema.Builder builder =
HBaseProtos.TableSchema.newBuilder();
try {
ProtobufUtil.mergeFrom(builder, bytes, pblen, bytes.length - pblen);
return ProtobufUtil.toTableDescriptor(builder.build());
} catch (IOException e) {
throw new DeserializationException(e);
}
}
{code}
HBase 2's HBase proto does not even have TableDescriptor message, this is odd.
This means that HBASE-7767 introduced HBaseProtos.TableDescriptor and later it
was reverted as well and now it's no longer being used.
Let me dig in why this was removed and for what reason. [~apurtell] [~bharathv]
Although we have released 1.7, but we might not be able to continue with Proto
incompatibility, specifically when HBase 1 has new TD class with it's own Proto
definition but on the other hand HBase 2 has it removed. (Edit: more details in
following comments)
-This brings the question of whether we should really have Protobuf
compatibility guidelines just like source comaptibility guidelines we follow
for backward compatible releases.-
{quote}Unless it actually prevents an upgrade from 1.6, which moots the release
and we will have to withdraw it.
{quote}
Let me test this as well today if possible (else tomorrow) but based on the
code, it seems we should not have any problem with this. This
[commit|https://github.com/apache/hbase/commit/431b8a5383b894381583bbb9ceef5911911b705c]
takes care of backward compatibility.
I assume deserialization should fail from 1.6 to 1.7 and 1.7 has backward
compat code present after DeserializationException is caught:
{code:java}
private static TableDescriptor readTableDescriptor(FileSystem fs, FileStatus
status,
boolean rewritePb) throws IOException {
int len = Ints.checkedCast(status.getLen());
byte [] content = new byte[len];
FSDataInputStream fsDataInputStream = fs.open(status.getPath());
try {
fsDataInputStream.readFully(content);
} finally {
fsDataInputStream.close();
}
TableDescriptor td =
[jira] [Comment Edited] (HBASE-26021) HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
[
https://issues.apache.org/jira/browse/HBASE-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368973#comment-17368973
]
Viraj Jasani edited comment on HBASE-26021 at 6/24/21, 5:11 PM:
Based on the history, this
[commit|https://github.com/apache/hbase/commit/dec0ec229070465ce5a1f7381366a82278149f06]
(HBASE-13016) cleaned up TableState.State from HBaseProto.TableDescriptor and
finally this
[commit|https://github.com/apache/hbase/commit/bdb0cc8808af7c3d08af4a506f34b8341726b58e]
removed TableDescriptor itself from HBaseProtos (HBASE-15467: Remove 1.x/2.0
TableDescriptor incompatibility). And as part of this commit itself, entire
TableDescriptor class also has been removed.
[~apurtell] [~bharathv] Thoughts on backporting
[commit|https://github.com/apache/hbase/commit/bdb0cc8808af7c3d08af4a506f34b8341726b58e]
to branch-1? It's bit weird to resolve this. We backported something to
branch-1 which itself was reverted from branch-2 in order to maintain 1.x/2.x
compatibility.
was (Author: vjasani):
Based on the history, this
[commit|https://github.com/apache/hbase/commit/dec0ec229070465ce5a1f7381366a82278149f06]
(HBASE-13016) cleaned up TableState.State from HBaseProto.TableDescriptor and
finally this
[commit|https://github.com/apache/hbase/commit/bdb0cc8808af7c3d08af4a506f34b8341726b58e]
removed TableDescriptor itself from HBaseProtos (HBASE-15467: Remove 1.x/2.0
TableDescriptor incompatibility). And as part of this commit itself, entire
TableDescriptor class also has been removed.
[~apurtell] [~bharathv] Thoughts on backporting
[commit|https://github.com/apache/hbase/commit/bdb0cc8808af7c3d08af4a506f34b8341726b58e]
to branch-1? It's bit weird to resolve this. We backported something to
branch-1 which itself was removed from branch-2 in order to maintain 1.x/2.x
compatibility.
> HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
> --
>
> Key: HBASE-26021
> URL: https://issues.apache.org/jira/browse/HBASE-26021
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.7.0, 2.4.4
>Reporter: Viraj Jasani
>Priority: Major
> Attachments: Screenshot 2021-06-22 at 12.54.21 PM.png, Screenshot
> 2021-06-22 at 12.54.30 PM.png
>
>
> As of today, if we bring up HBase cluster using branch-1 and upgrade to
> branch-2.4, we are facing issue while parsing namespace from HDFS fileinfo.
> Instead of "*hbase:meta*" and "*hbase:namespace*", parsing using ProtobufUtil
> seems to be producing "*\n hbase meta*" and "*\n hbase namespace*"
> {code:java}
> 2021-06-22 00:05:56,611 INFO
> [RpcServer.priority.RWQ.Fifo.read.handler=3,queue=1,port=16025]
> regionserver.RSRpcServices: Open hbase:meta,,1.1588230740
> 2021-06-22 00:05:56,648 INFO
> [RpcServer.priority.RWQ.Fifo.read.handler=5,queue=1,port=16025]
> regionserver.RSRpcServices: Open
> hbase:namespace,,1624297762817.396cb6cc00cd4334cb1ea3a792d7529a.
> 2021-06-22 00:05:56,759 ERROR
> [RpcServer.priority.RWQ.Fifo.read.handler=5,queue=1,port=16025]
> ipc.RpcServer: Unexpected throwable object
> java.lang.IllegalArgumentException: Illegal character <
> > at 0. Namespaces may only contain 'alphanumeric characters' from any
> > language and digits:
> ^Ehbase^R namespace
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:246)
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:220)
> at org.apache.hadoop.hbase.TableName.(TableName.java:348)
> at
> org.apache.hadoop.hbase.TableName.createTableNameIfNecessary(TableName.java:385)
> at org.apache.hadoop.hbase.TableName.valueOf(TableName.java:508)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toTableName(ProtobufUtil.java:2292)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toTableDescriptor(ProtobufUtil.java:2937)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder$ModifyableTableDescriptor.parseFrom(TableDescriptorBuilder.java:1625)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder$ModifyableTableDescriptor.access$200(TableDescriptorBuilder.java:597)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder.parseFrom(TableDescriptorBuilder.java:320)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.readTableDescriptor(FSTableDescriptors.java:511)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:496)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:482)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:210)
> at
>
[jira] [Comment Edited] (HBASE-26021) HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
[
https://issues.apache.org/jira/browse/HBASE-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368973#comment-17368973
]
Viraj Jasani edited comment on HBASE-26021 at 6/24/21, 5:09 PM:
Based on the history, this
[commit|https://github.com/apache/hbase/commit/dec0ec229070465ce5a1f7381366a82278149f06]
(HBASE-13016) cleaned up TableState.State from HBaseProto.TableDescriptor and
finally this
[commit|https://github.com/apache/hbase/commit/bdb0cc8808af7c3d08af4a506f34b8341726b58e]
removed TableDescriptor itself from HBaseProtos (HBASE-15467: Remove 1.x/2.0
TableDescriptor incompatibility). And as part of this commit itself, entire
TableDescriptor class also has been removed.
[~apurtell] [~bharathv] Thoughts on backporting
[commit|https://github.com/apache/hbase/commit/bdb0cc8808af7c3d08af4a506f34b8341726b58e]
to branch-1? It's bit weird to resolve this. We backported something to
branch-1 which itself was removed from branch-2 in order to maintain 1.x/2.x
compatibility.
was (Author: vjasani):
Based on the history, this
[commit|https://github.com/apache/hbase/commit/dec0ec229070465ce5a1f7381366a82278149f06]
(HBASE-13016) cleaned up TableState.State from HBaseProto.TableDescriptor and
finally this
[commit|https://github.com/apache/hbase/commit/bdb0cc8808af7c3d08af4a506f34b8341726b58e]
removed TableDescriptor itself from HBaseProtos (HBASE-15467: Remove 1.x/2.0
TableDescriptor incompatibility).
> HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
> --
>
> Key: HBASE-26021
> URL: https://issues.apache.org/jira/browse/HBASE-26021
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.7.0, 2.4.4
>Reporter: Viraj Jasani
>Priority: Major
> Attachments: Screenshot 2021-06-22 at 12.54.21 PM.png, Screenshot
> 2021-06-22 at 12.54.30 PM.png
>
>
> As of today, if we bring up HBase cluster using branch-1 and upgrade to
> branch-2.4, we are facing issue while parsing namespace from HDFS fileinfo.
> Instead of "*hbase:meta*" and "*hbase:namespace*", parsing using ProtobufUtil
> seems to be producing "*\n hbase meta*" and "*\n hbase namespace*"
> {code:java}
> 2021-06-22 00:05:56,611 INFO
> [RpcServer.priority.RWQ.Fifo.read.handler=3,queue=1,port=16025]
> regionserver.RSRpcServices: Open hbase:meta,,1.1588230740
> 2021-06-22 00:05:56,648 INFO
> [RpcServer.priority.RWQ.Fifo.read.handler=5,queue=1,port=16025]
> regionserver.RSRpcServices: Open
> hbase:namespace,,1624297762817.396cb6cc00cd4334cb1ea3a792d7529a.
> 2021-06-22 00:05:56,759 ERROR
> [RpcServer.priority.RWQ.Fifo.read.handler=5,queue=1,port=16025]
> ipc.RpcServer: Unexpected throwable object
> java.lang.IllegalArgumentException: Illegal character <
> > at 0. Namespaces may only contain 'alphanumeric characters' from any
> > language and digits:
> ^Ehbase^R namespace
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:246)
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:220)
> at org.apache.hadoop.hbase.TableName.(TableName.java:348)
> at
> org.apache.hadoop.hbase.TableName.createTableNameIfNecessary(TableName.java:385)
> at org.apache.hadoop.hbase.TableName.valueOf(TableName.java:508)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toTableName(ProtobufUtil.java:2292)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toTableDescriptor(ProtobufUtil.java:2937)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder$ModifyableTableDescriptor.parseFrom(TableDescriptorBuilder.java:1625)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder$ModifyableTableDescriptor.access$200(TableDescriptorBuilder.java:597)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder.parseFrom(TableDescriptorBuilder.java:320)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.readTableDescriptor(FSTableDescriptors.java:511)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:496)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:482)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:210)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.openRegion(RSRpcServices.java:2112)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:35218)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:395)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
> at
>
[jira] [Comment Edited] (HBASE-26021) HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
[
https://issues.apache.org/jira/browse/HBASE-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368796#comment-17368796
]
Viraj Jasani edited comment on HBASE-26021 at 6/24/21, 12:13 PM:
-
Here is the patch that resolves this (de)serialization issue, I have confirmed
this. The only thing I am worried about is what else could be compromised with
Proto incompatibilities with missing Proto message.
{code:java}
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptor.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptor.java
index c275f00c72..a2a9b0df0f 100644
--- a/hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptor.java
+++ b/hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptor.java
@@ -96,7 +96,8 @@ public class TableDescriptor {
* @see #parseFrom(byte[])
*/
public byte [] toByteArray() {
- return ProtobufUtil.prependPBMagic(convert().toByteArray());
+ return ProtobufUtil
+ .prependPBMagic(hTableDescriptor.convert().toByteArray());
}
{code}
This patch works for HBase 1.7 as well, because as I mentioned above, hbase-1
takes care of backward compatibility by catching DeserializationException and
trying to deserialize to old TD i.e HTableDescriptor. Confirmed this as well.
What I believe we should consider path forward for this issue:
# Apply this patch and consider rolling out 1.8 release for anyone looking
forward to upgrade from HBase 1 to 2.
# Thoughts on catching any other missing edge cases of HBASE-7767 backport. I
tried generating more traffic in my local testing but it might not be enough.
And I think as long as backport helps MasterRegistry's work to land on
branch-1, we are good and we might not want to consider rework on it.
Edit: From the code, I can see HBaseProtos.TableDescriptor is not used anywhere
else other than the use-case we are focusing on. Other than introducing
TableDescriptor and TableState in HBase proto, I don't see any changes in proto
of big concern. The patch also has changes in Master and Zookeeper protos but
they don't seem problematic from their usage.
I think we should be good with the above patch once we have enough +1 and we
can soon plan for new release from branch-1.
was (Author: vjasani):
Here is the patch that resolves this (de)serialization issue, I have confirmed
this. The only thing I am worried about is what else could be compromised with
Proto incompatibilities with missing Proto message.
{code:java}
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptor.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptor.java
index c275f00c72..a2a9b0df0f 100644
--- a/hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptor.java
+++ b/hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptor.java
@@ -96,7 +96,8 @@ public class TableDescriptor {
* @see #parseFrom(byte[])
*/
public byte [] toByteArray() {
- return ProtobufUtil.prependPBMagic(convert().toByteArray());
+ return ProtobufUtil
+ .prependPBMagic(hTableDescriptor.convert().toByteArray());
}
{code}
This patch works for HBase 1.7 as well, because as I mentioned above, hbase-1
takes care of backward compatibility by catching DeserializationException and
trying to deserialize to old TD i.e HTableDescriptor. Confirmed this as well.
What I believe we should consider path forward for this issue:
# Apply this patch and consider rolling out 1.8 release for anyone looking
forward to upgrade from HBase 1 to 2.
# Thoughts on catching any missing edge cases of HBASE-7767 backport. I think
as long as backport helps MasterRegistry's work to land on branch-1, we are
good and we might not want to consider rework on it. I am already running PE as
of now in my local testing to generate more traffic and add millions of rows
and scanning them, but this might not be enough and we could plan for rigorous
testing on big cluster.
Edit: From the code, I can see HBaseProtos.TableDescriptor is not used anywhere
else other than the usecase we are focusing on. Other than introducing
TableDescriptor and TableState in HBase proto, I don't see any changes in proto
of big concern. The patch also has changes in Master and Zookeeper protos but
they don't seem problematic from their usage.
I think we should be good with the above patch once we have enough +1 and we
can soon plan for new release from branch-1.
> HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
> --
>
> Key: HBASE-26021
> URL: https://issues.apache.org/jira/browse/HBASE-26021
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.7.0, 2.4.4
>Reporter: Viraj Jasani
>Priority: Major
>
[jira] [Comment Edited] (HBASE-26021) HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
[
https://issues.apache.org/jira/browse/HBASE-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17368796#comment-17368796
]
Viraj Jasani edited comment on HBASE-26021 at 6/24/21, 12:04 PM:
-
Here is the patch that resolves this (de)serialization issue, I have confirmed
this. The only thing I am worried about is what else could be compromised with
Proto incompatibilities with missing Proto message.
{code:java}
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptor.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptor.java
index c275f00c72..a2a9b0df0f 100644
--- a/hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptor.java
+++ b/hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptor.java
@@ -96,7 +96,8 @@ public class TableDescriptor {
* @see #parseFrom(byte[])
*/
public byte [] toByteArray() {
- return ProtobufUtil.prependPBMagic(convert().toByteArray());
+ return ProtobufUtil
+ .prependPBMagic(hTableDescriptor.convert().toByteArray());
}
{code}
This patch works for HBase 1.7 as well, because as I mentioned above, hbase-1
takes care of backward compatibility by catching DeserializationException and
trying to deserialize to old TD i.e HTableDescriptor. Confirmed this as well.
What I believe we should consider path forward for this issue:
# Apply this patch and consider rolling out 1.8 release for anyone looking
forward to upgrade from HBase 1 to 2.
# Thoughts on catching any missing edge cases of HBASE-7767 backport. I think
as long as backport helps MasterRegistry's work to land on branch-1, we are
good and we might not want to consider rework on it. I am already running PE as
of now in my local testing to generate more traffic and add millions of rows
and scanning them, but this might not be enough and we could plan for rigorous
testing on big cluster.
Edit: From the code, I can see HBaseProtos.TableDescriptor is not used anywhere
else other than the usecase we are focusing on. Other than introducing
TableDescriptor and TableState in HBase proto, I don't see any changes in proto
of big concern. The patch also has changes in Master and Zookeeper protos but
they don't seem problematic from their usage.
I think we should be good with the above patch once we have enough +1 and we
can soon plan for new release from branch-1.
was (Author: vjasani):
Here is the patch that resolves this (de)serialization issue, I have confirmed
this. The only thing I am worried about is what else could be compromised with
Proto incompatibilities with missing Proto message.
{code:java}
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptor.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptor.java
index c275f00c72..a2a9b0df0f 100644
--- a/hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptor.java
+++ b/hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptor.java
@@ -96,7 +96,8 @@ public class TableDescriptor {
* @see #parseFrom(byte[])
*/
public byte [] toByteArray() {
- return ProtobufUtil.prependPBMagic(convert().toByteArray());
+ return ProtobufUtil
+ .prependPBMagic(hTableDescriptor.convert().toByteArray());
}
{code}
This patch works for HBase 1.7 as well, because as I mentioned above, hbase-1
takes care of backward compatibility by catching DeserializationException and
trying to deserialize to old TD i.e HTableDescriptor. Confirmed this as well.
What I believe we should consider path forward for this issue:
# Apply this patch and consider rolling out 1.8 release for anyone looking
forward to upgrade from HBase 1 to 2.
# Thoughts on catching any missing edge cases of HBASE-7767 backport. I think
as long as backport helps MasterRegistry's work to land on branch-1, we are
good but looks like we requires rigorous testing with this patch. I am already
running pe as of now in my local testing to generate more traffic and add
millions of rows and scanning them, but this might not be enough.
> HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
> --
>
> Key: HBASE-26021
> URL: https://issues.apache.org/jira/browse/HBASE-26021
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.7.0, 2.4.4
>Reporter: Viraj Jasani
>Priority: Major
> Attachments: Screenshot 2021-06-22 at 12.54.21 PM.png, Screenshot
> 2021-06-22 at 12.54.30 PM.png
>
>
> As of today, if we bring up HBase cluster using branch-1 and upgrade to
> branch-2.4, we are facing issue while parsing namespace from HDFS fileinfo.
> Instead of "*hbase:meta*" and "*hbase:namespace*", parsing using ProtobufUtil
> seems to be producing "*\n hbase meta*" and "*\n hbase
