[ http://issues.apache.org/jira/browse/HADOOP-558?page=comments#action_12437681 ] Doug Cutting commented on HADOOP-558: -------------------------------------
Part of this, I assume, will include: 1.In SequenceFile, adding to the header the version map for key and value classes (& all classes referenced by their non-transient fields), and passing this map to keys and values as they're read. 2. In RPC, Transmitting the version map for all parameter & return classes (& those refernced by their fields) from the client to the server as a handshake on connect. > Adding versioning to Writable classes > ------------------------------------- > > Key: HADOOP-558 > URL: http://issues.apache.org/jira/browse/HADOOP-558 > Project: Hadoop > Issue Type: Improvement > Components: io > Affects Versions: 0.6.2 > Reporter: Owen O'Malley > Assigned To: Owen O'Malley > > We currently don't have a way to support versioned Writables, which among > other problems means that it is very difficult to change the serialization of > any types. (A recent example is that we can't change any of the Writables > that currently use UTF8.writeString to use Text.writeString with out breaking > everything.) Just changing the file version doesn't work because you can't > read the old versions. > Therefore, I propose adding a new interface: > public interface VersionMapWritable extends Writable { > /** > * Destructively read into this object from in based on the version map. > * @param versionMap a map from each class to its version in the DataInput > (version 0 classes are omitted) > */ > public void readFields(DataInput in, Map<Class<VersionMapWritable>, int> > versionMap) throws IOException; > /** > * Classes with non-zero versions should register themselves in static > blocks. > */ > public static void registerCurrentVersion(Class<VersionMapWritable> class, > int version) {...} > /** > * If a version map includes the parent type, always include the child type > as well. > */ > public static void addDependence(Class<VersionMapWritable> parent, > Class<VersionMapWritable> child) { ... } > /** > * Build a version map for a given list of classes, including any dependent > types. > */ > public static Map<Class<VersionMapWritable>, int> > buildVersionMap(Set<Class<VersionMapWritable>> classes) {...} > /** > * Add the types in the parameter/result types to the list of classes. > * Recursively adds the field types for any new types that are added to the > set. > */ > public static void addMethodTypes(Set<Class<VersionMapWritable>> result, > Class<VersionedProtocol> protocol) {...} > /** > * Add the non-transient fields to the list of classes. > */ > public static void addFieldTypes(Set<Class<VersionMapWritable>> result, > Class<Writable> writable) {...} > public static Map<Class<VersionMapWritable>, int> readVersionMap(DataInput > in) throws IOException { ... } > public static void writeVersionMap(DataOutput out, > Map<Class<VersionMapWritable> versionMap) throws > IOException {...} > } > VersionedWritable, which stored a version byte within each object, will be > depriciated. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
