[
https://issues.apache.org/jira/browse/HADOOP-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534738
]
Vivek Ratan commented on HADOOP-1986:
-------------------------------------
After some offline discussions, I think there is some convergence. It seems
like most folks have implicitly assumed that DDLs are involved. If a user
defines a class that they want to use as a key or value, they (or somebody)
would either implement Writable or define it using a DDL, run a Thrift or
Record I/O compiler, and use the generated stub. They would also pick the right
_Serializable<T>_ implementation. I've been arguing that there is an additional
way where someone may not want to go through the pain of writing a DDL and
compiling it. I don't have any real use cases for assuming that some folks
might find DDLs and compilers to be a pain. A fair compromise is to stick with
the _Serializable<T>_ interface that Tom and Owen have been talking about.
There will be classes that implement _Serializable<RecordI/O record>_ and
_Serializable<Thrift record>_. There can be an additional implementation for a
general-purpose reflection-based serializer, something like _class
ReflectionSerializer imlements Serializable<Object>_. I don't know if such an
implementation is required right away: it's useful only if someone doesn't want
to deal with DDL hassles. So maybe we can keep this as an option for later, if
required.
> Add support for a general serialization mechanism for Map Reduce
> ----------------------------------------------------------------
>
> Key: HADOOP-1986
> URL: https://issues.apache.org/jira/browse/HADOOP-1986
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Reporter: Tom White
> Assignee: Tom White
> Fix For: 0.16.0
>
> Attachments: SerializableWritable.java
>
>
> Currently Map Reduce programs have to use WritableComparable-Writable
> key-value pairs. While it's possible to write Writable wrappers for other
> serialization frameworks (such as Thrift), this is not very convenient: it
> would be nicer to be able to use arbitrary types directly, without explicit
> wrapping and unwrapping.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
