[
https://issues.apache.org/jira/browse/HADOOP-9845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13735286#comment-13735286
]
Daryn Sharp commented on HADOOP-9845:
-------------------------------------
I'm a bit uneasy, but undecided, about upgrading to a new protobuf at this late
hour in the hadoop-2 release. 2.4.1 has 2+ years of proven stability. 2.5's
main benefit seems to be ~25% faster parsing performance, so my questions are:
* Is the parsing improvement for edge cases, or all cases?
* Is the improvement even measurable at scale? 25% might equate to a
micro-optimization.
* If yes, is it substantial enough to risk switching to a much younger release
now, rather than later?
* Are we sure it works across all development (*nix, osx, windows) environments?
* Are we sure it plays nice with Eclipse?
> Update protobuf to 2.5 from 2.4.x
> ---------------------------------
>
> Key: HADOOP-9845
> URL: https://issues.apache.org/jira/browse/HADOOP-9845
> Project: Hadoop Common
> Issue Type: Improvement
> Components: performance
> Affects Versions: 2.0.5-alpha
> Reporter: stack
> Assignee: Alejandro Abdelnur
> Priority: Blocker
> Fix For: 2.1.0-beta
>
> Attachments: HADOOP-9845.patch, HADOOP-9845.patch
>
>
> protobuf 2.5 is a bit faster with a new Parse to avoid a builder step and a
> few other goodies that we'd like to take advantage of over in hbase
> especially now we are all pb all the time. Unfortunately the protoc
> generated files are no longer compatible w/ 2.4.1 generated files. Hadoop
> uses 2.4.1 pb. This latter fact makes it so we cannot upgrade until hadoop
> does.
> This issue suggests hadoop2 move to protobuf 2.5.
> I can do the patch no prob. if there is interest.
> (When we upgraded our build broke with complaints like the below:
> {code}
> java.lang.UnsupportedOperationException: This is supposed to be overridden by
> subclasses.
> at
> com.google.protobuf.GeneratedMessage.getUnknownFields(GeneratedMessage.java:180)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$GetDatanodeReportRequestProto.getSerializedSize(ClientNamenodeProtocolProtos.java:21566)
> at
> com.google.protobuf.AbstractMessageLite.toByteString(AbstractMessageLite.java:49)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.constructRpcRequest(ProtobufRpcEngine.java:149)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:193)
> at com.sun.proxy.$Proxy14.getDatanodeReport(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:164)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:83)
> at com.sun.proxy.$Proxy14.getDatanodeReport(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getDatanodeReport(ClientNamenodeProtocolTranslatorPB.java:488)
> at org.apache.hadoop.hdfs.DFSClient.datanodeReport(DFSClient.java:1887)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.waitActive(MiniDFSCluster.java:1798
> ...
> {code}
> More over in HBASE-8165 if interested.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
