[
https://issues.apache.org/jira/browse/HBASE-15638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-15638:
--------------------------
Resolution: Fixed
Fix Version/s: 2.0.0
Release Note:
Shade/relocate and include the protobuf we use internally. See protobuf chapter
in the refguide for more on how we protobuf in hbase-.2.0.0 and going forward.
See
https://docs.google.com/document/d/1H4NgLXQ9Y9KejwobddCqaVMEDCGbyDcXtdF5iAfDIEk/edit#
for how we arrived at this approach.
See
http://mail-archives.apache.org/mod_mbox/hbase-dev/201610.mbox/%3C07850EDD-7230-431B-9AB0-C5C91B105EEC%40gmail.com%3E
for discussion around merging this change and of how we might revert if an
alternative to this awkward patch presents itself; e.g. an hadoop with
CLASSPATH isolation (and means of dealing with Sparks use of protobuf 2.5.0,
etc.)
Status: Resolved (was: Patch Available)
Above set of patches adds shading of protobufs (and upgrades our internal
protobuf to be 3.1.x). Resolving. See release note for pointers on how we got
to this awkward patch and of how we would unto it if circumstances change.
> Shade protobuf
> --------------
>
> Key: HBASE-15638
> URL: https://issues.apache.org/jira/browse/HBASE-15638
> Project: HBase
> Issue Type: Bug
> Components: Protobufs
> Reporter: stack
> Assignee: stack
> Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 15638v2.patch, HBASE-15638.master.001.patch,
> HBASE-15638.master.002.patch, HBASE-15638.master.003 (1).patch,
> HBASE-15638.master.003 (1).patch, HBASE-15638.master.003 (1).patch,
> HBASE-15638.master.003.patch, HBASE-15638.master.003.patch,
> HBASE-15638.master.004.patch, HBASE-15638.master.005.patch,
> HBASE-15638.master.006.patch, HBASE-15638.master.007.patch,
> HBASE-15638.master.007.patch, HBASE-15638.master.008.patch,
> HBASE-15638.master.009.patch, as.far.as.server.patch
>
>
> We need to change our protobuf. Currently it is pb2.5.0. As is, protobufs
> expect all buffers to be on-heap byte arrays. It does not have facility for
> dealing in ByteBuffers and off-heap ByteBuffers in particular. This fact
> frustrates the off-heaping-of-the-write-path project as
> marshalling/unmarshalling of protobufs involves a copy on-heap first.
> So, we need to patch our protobuf so it supports off-heap ByteBuffers. To
> ensure we pick up the patched protobuf always, we need to relocate/shade our
> protobuf and adjust all protobuf references accordingly.
> Given as we have protobufs in our public facing API, Coprocessor Endpoints --
> which use protobuf Service to describe new API -- a blind relocation/shading
> of com.google.protobuf.* will break our API for CoProcessor EndPoints (CPEP)
> in particular. For example, in the Table Interface, to invoke a method on a
> registered CPEP, we have:
> {code}<T extends com.google.protobuf.Service,R> Map<byte[],R>
> coprocessorService(
> Class<T> service, byte[] startKey, byte[] endKey,
> org.apache.hadoop.hbase.client.coprocessor.Batch.Call<T,R>
> callable)
> throws com.google.protobuf.ServiceException, Throwable{code}
> This issue is how we intend to shade protobuf for hbase-2.0.0 while
> preserving our API as is so CPEPs continue to work on the new hbase.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
