[
https://issues.apache.org/jira/browse/HADOOP-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12598343#action_12598343
]
Doug Cutting commented on HADOOP-3414:
--------------------------------------
Perhaps we could add a SizedWritable sub-interface or abstract class, then
change Text & BytesWritable to extend that instead. Then
WritableSerializer#getSize(Writable) could use instanceof to decide whether to
call getSize() or simply return -1. It's a little ugly to use instanceof, but
it would be back-compatible, I think.
What we really need is a benchmark that shows this provides some significant
performance improvement. If it does, then it's probably worth finding the best
compromise between back-compatiblity, elegance and performance.
> Facility to query serializable types such as Writables for 'raw length'
> -----------------------------------------------------------------------
>
> Key: HADOOP-3414
> URL: https://issues.apache.org/jira/browse/HADOOP-3414
> Project: Hadoop Core
> Issue Type: Improvement
> Components: io
> Reporter: Arun C Murthy
>
> Currently we need to jump through hoops to get the 'raw length' of
> serializable types for e.g. SequenceFile.Writer.append needs to copy the
> key/value into a buffer and then check the buffer's size to figure the
> record/key/value lenghts. Obviously this can be improved to do away with the
> extra copy if we had types which could be queried for it's raw-length.
> Thoughts?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
