[
https://issues.apache.org/jira/browse/HADOOP-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12551606
]
Tom White commented on HADOOP-1986:
-----------------------------------
> I remain convinced that the
> use of templated types at all in our lowest level API is a mistake and all of
> this templated foo
> should be an optional layer on top of a basic byte oriented API. Other
> famous map-reduce
> implementations make that choice (byte API only) and they get to bypass all
> of this discussion
> and complexity.
Interesting point - it sounds like such a design might be achieved by moving
the serialization stuff into its own layer above core Map Reduce.
>The code doesn't look like it adds any appreciable overhead, a function call
>and a comparison
> per read seems likely to be in the noise, unless the stream class bites us.
> Have you validated that we're not taking on extra cost here?
Not yet, I plan to do so. Also, I need to add javadoc to the new classes and
fix some warnings.
> Add support for a general serialization mechanism for Map Reduce
> ----------------------------------------------------------------
>
> Key: HADOOP-1986
> URL: https://issues.apache.org/jira/browse/HADOOP-1986
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Reporter: Tom White
> Assignee: Tom White
> Fix For: 0.16.0
>
> Attachments: hadoop-serializer-v2.tar.gz, SerializableWritable.java,
> serializer-v1.patch, serializer-v2.patch
>
>
> Currently Map Reduce programs have to use WritableComparable-Writable
> key-value pairs. While it's possible to write Writable wrappers for other
> serialization frameworks (such as Thrift), this is not very convenient: it
> would be nicer to be able to use arbitrary types directly, without explicit
> wrapping and unwrapping.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
