[jira] [Updated] (HBASE-16567) Upgrade to protobuf-3.1.x
[ https://issues.apache.org/jira/browse/HBASE-16567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16567: -- Issue Type: Umbrella (was: Task) > Upgrade to protobuf-3.1.x > - > > Key: HBASE-16567 > URL: https://issues.apache.org/jira/browse/HBASE-16567 > Project: HBase > Issue Type: Umbrella > Components: Protobufs >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: 16567.patch, HBASE-16567.master.001.patch > > > Move master branch on to protobuf3. See > https://github.com/google/protobuf/releases We'd do it because pb3 saves some > on byte copies can work with offheap buffers -- needed for the off-heap write > path project -- though read-time is still a TODO (this means pb3 is not > enough; we'll have to patch it -- or patch pb2.5). > HBASE-15638 has us first shading protobufs before upgrading. Let us list here > issues just going to pb3 without shading if only for completeness sake; i.e. > do we have to shade? > * pb3 is by default wire compatible with pb2. > * protoc3 run against our .protos works fine except pb3 breaks our > HBaseZeroCopyLiteralByteString hack so this has to be removed (possibly > recast using new pb3 types) > * Starting up a cluster that is all pb3 seems to work fine. > * A pb2 branch-1 can read and write against the pb3 master cluster. > What will break if we just upgrade to pb3? > * We should be able to write HDFS messages on our AsyncWAL using pb3; the > pb2 HDFS should be able to read them (not tested). Or maybe not. See policy > here: https://github.com/google/protobuf/issues/1852 which seems to indicate > pb3s will not be able to write compatible pb2 Messages. TODO. > * Core Coprocessor Endpoints such as AccessControl seem to just work (their > protos will have been protoc3'd). I did simple test with a server from master > branch up on pb3 and then going against it with a branch-1 client on pb2. I > was able to add grants. > * For non-core CPEPs where the protos are pb2 still, it might just work. To > test. It would not be the end-of-the-world if they did not. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-16567) Upgrade to protobuf-3.1.x
[ https://issues.apache.org/jira/browse/HBASE-16567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16567: -- Resolution: Fixed Fix Version/s: 2.0.0 Release Note: Core is now up on protobuf 3.1.0 (Coprocessor Endpoints and REST are still on protobuf 2.5.0). Status: Resolved (was: Patch Available) Committed as part of the set that went in under HBASE-15638 Shade protobuf at 95c1dc93fb6780f66fd8ebb57914a050b75b9b11 > Upgrade to protobuf-3.1.x > - > > Key: HBASE-16567 > URL: https://issues.apache.org/jira/browse/HBASE-16567 > Project: HBase > Issue Type: Task > Components: Protobufs >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.0.0 > > Attachments: 16567.patch, HBASE-16567.master.001.patch > > > Move master branch on to protobuf3. See > https://github.com/google/protobuf/releases We'd do it because pb3 saves some > on byte copies can work with offheap buffers -- needed for the off-heap write > path project -- though read-time is still a TODO (this means pb3 is not > enough; we'll have to patch it -- or patch pb2.5). > HBASE-15638 has us first shading protobufs before upgrading. Let us list here > issues just going to pb3 without shading if only for completeness sake; i.e. > do we have to shade? > * pb3 is by default wire compatible with pb2. > * protoc3 run against our .protos works fine except pb3 breaks our > HBaseZeroCopyLiteralByteString hack so this has to be removed (possibly > recast using new pb3 types) > * Starting up a cluster that is all pb3 seems to work fine. > * A pb2 branch-1 can read and write against the pb3 master cluster. > What will break if we just upgrade to pb3? > * We should be able to write HDFS messages on our AsyncWAL using pb3; the > pb2 HDFS should be able to read them (not tested). Or maybe not. See policy > here: https://github.com/google/protobuf/issues/1852 which seems to indicate > pb3s will not be able to write compatible pb2 Messages. TODO. > * Core Coprocessor Endpoints such as AccessControl seem to just work (their > protos will have been protoc3'd). I did simple test with a server from master > branch up on pb3 and then going against it with a branch-1 client on pb2. I > was able to add grants. > * For non-core CPEPs where the protos are pb2 still, it might just work. To > test. It would not be the end-of-the-world if they did not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16567) Upgrade to protobuf-3.1.x
[ https://issues.apache.org/jira/browse/HBASE-16567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16567: -- Attachment: 16567.patch Patch goes in nicely over the top of HBASE-16264 Regenerate all protos in this module with protoc3. Redo ByteStringer to use new pb3.1.0 unsafebytesutil instead of HBaseZeroCopyByteString > Upgrade to protobuf-3.1.x > - > > Key: HBASE-16567 > URL: https://issues.apache.org/jira/browse/HBASE-16567 > Project: HBase > Issue Type: Task > Components: Protobufs >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack >Priority: Critical > Attachments: 16567.patch, HBASE-16567.master.001.patch > > > Move master branch on to protobuf3. See > https://github.com/google/protobuf/releases We'd do it because pb3 saves some > on byte copies can work with offheap buffers -- needed for the off-heap write > path project -- though read-time is still a TODO (this means pb3 is not > enough; we'll have to patch it -- or patch pb2.5). > HBASE-15638 has us first shading protobufs before upgrading. Let us list here > issues just going to pb3 without shading if only for completeness sake; i.e. > do we have to shade? > * pb3 is by default wire compatible with pb2. > * protoc3 run against our .protos works fine except pb3 breaks our > HBaseZeroCopyLiteralByteString hack so this has to be removed (possibly > recast using new pb3 types) > * Starting up a cluster that is all pb3 seems to work fine. > * A pb2 branch-1 can read and write against the pb3 master cluster. > What will break if we just upgrade to pb3? > * We should be able to write HDFS messages on our AsyncWAL using pb3; the > pb2 HDFS should be able to read them (not tested). Or maybe not. See policy > here: https://github.com/google/protobuf/issues/1852 which seems to indicate > pb3s will not be able to write compatible pb2 Messages. TODO. > * Core Coprocessor Endpoints such as AccessControl seem to just work (their > protos will have been protoc3'd). I did simple test with a server from master > branch up on pb3 and then going against it with a branch-1 client on pb2. I > was able to add grants. > * For non-core CPEPs where the protos are pb2 still, it might just work. To > test. It would not be the end-of-the-world if they did not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
