[
https://issues.apache.org/jira/browse/HADOOP-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12572180#action_12572180
]
Doug Cutting commented on HADOOP-1986:
--------------------------------------
I spoke with Tom over the weekend and he reported that, on micro benchmarks,
the performance impact was only a few percent. It is counter-intuitive that
this would balloon to 10% on a macro benchmark. The above results are not very
consistent either: sort on 100 nodes was 20% faster with the patch, while
randomwriter was 100% slower with the patch, both of which show little
correlation with the 20 and 500 node benchmarks. It seems possible that,
rather than a consistent slowdown, we're just seeing noise that should not be
used to judge this patch.
> Add support for a general serialization mechanism for Map Reduce
> ----------------------------------------------------------------
>
> Key: HADOOP-1986
> URL: https://issues.apache.org/jira/browse/HADOOP-1986
> Project: Hadoop Core
> Issue Type: New Feature
> Components: mapred
> Reporter: Tom White
> Assignee: Tom White
> Fix For: 0.17.0
>
> Attachments: hadoop-serializer-v2.tar.gz,
> SequenceFileWriterBenchmark.java, SerializableWritable.java,
> serializer-v1.patch, serializer-v2.patch, serializer-v3.patch,
> serializer-v4.patch, serializer-v5.patch
>
>
> Currently Map Reduce programs have to use WritableComparable-Writable
> key-value pairs. While it's possible to write Writable wrappers for other
> serialization frameworks (such as Thrift), this is not very convenient: it
> would be nicer to be able to use arbitrary types directly, without explicit
> wrapping and unwrapping.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
