[
https://issues.apache.org/jira/browse/HADOOP-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534982
]
Tom White commented on HADOOP-1986:
-----------------------------------
I've now run into WritableComparator which needs to be generalized too. I think
what's needed is a RawComparator interface (I'm not too attached to the name)
which has the pertinent method of WritableComparator:
{code}
public interface RawComparator extends Comparator<T> {
public int compare(byte[] b1, int s1, int l1, byte[] b2, int s2, int l2);
}
{code}
Users will be free to implement optimized versions of RawComparator for their
serialization framework, just as Writables do today.
Then WritableComparator will implement RawComparator, and there'll be a new
class DeserializerComparator, also implementing RawComparator, which will use a
Deserializer to implement the compare method above. In JobConf the
WritableComparators will become RawComparators.
Does this sound about right?
> Add support for a general serialization mechanism for Map Reduce
> ----------------------------------------------------------------
>
> Key: HADOOP-1986
> URL: https://issues.apache.org/jira/browse/HADOOP-1986
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Reporter: Tom White
> Assignee: Tom White
> Fix For: 0.16.0
>
> Attachments: SerializableWritable.java
>
>
> Currently Map Reduce programs have to use WritableComparable-Writable
> key-value pairs. While it's possible to write Writable wrappers for other
> serialization frameworks (such as Thrift), this is not very convenient: it
> would be nicer to be able to use arbitrary types directly, without explicit
> wrapping and unwrapping.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
