On Tue, Mar 28, 2017 at 4:18 PM, Andrew Wang <andrew.w...@cloudera.com> wrote: > Unfortunately, it sounds like these are intrinsic differences with PB3.
That's too bad... but possibly not fatal: most of the data we proxy through client code is, if not opaque, it's at least immutable (particularly tokens). If PB3 does support reading valid PB fields as bytes, then we could proxy the payload through application code as an opaque blob. That opacity has a drawback: if clients could use that information (e.g., StorageType), we'd need to include it in a redundant field. Ewan Higgs used a technique in HDFS-11026 [1] to handle a transition from Writable to protobuf. This probably could be used for most of our token types. It's not a general solution, but it would be sufficient for existing applications to continue working, with some accommodation for proxy versioning and rolling upgrades. I haven't seen data identifying PB as a bottleneck, but the non-x86/non-Linux and dev setup arguments may make this worthwhile. -C [1] https://issues.apache.org/jira/browse/HDFS-11026 --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
