[
https://issues.apache.org/jira/browse/HADOOP-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985894#action_12985894
]
Todd Lipcon commented on HADOOP-6949:
-------------------------------------
A few comments:
- in readFields/write - add an else { throw new IllegalStateException("no
branch for type: " + componentType); } or something just to be safe?
- For the functions that loop around the arrays, it might be faster to cast the
Object to a foo[] rather than using Array.setFoo/Array.getFoo - might get some
extra bounds check elimination going on and doesn't seem like any lost clarity
in doing so
- For the special case of reading and writing byte[] we can use in.readFully()
and out.write() with the whole array, which might be a bit faster.
- Should we consider using variable length coding for some of these types,
since we have the opportunity to change the format? Could even do something
fancy like delta-code it and then vint-encode for more space savings? Would be
worth a benchmark to see if it helps.
> Reduces RPC packet size for primitive arrays, especially long[], which is
> used at block reporting
> -------------------------------------------------------------------------------------------------
>
> Key: HADOOP-6949
> URL: https://issues.apache.org/jira/browse/HADOOP-6949
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Reporter: Navis
> Assignee: Matt Foley
> Fix For: 0.23.0
>
> Attachments: arrayprim.patch, ObjectWritable.diff
>
> Original Estimate: 0.17h
> Remaining Estimate: 0.17h
>
> Current implementation of oah.io.ObjectWritable marshals primitive array
> types as general object array ; array type string + array length + (element
> type string + value)*n
> It would not be needed to specify each element types for primitive arrays.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
