Hi, I am trying to multiplex GenericRecord objects to a set of DataFileWriter objects. Each DataFileWriter expects its own schema, and the GenericRecord objects are written with a "superset" schema that is compatible with the schemas assigned to the DataFileWriters. Is there a way to convert individual GenericRecord objects between schemas without re-reading them from their source (in this case a DataFileStream)? If I simply call append() on a DataFileWriter with a GenericRecord, I get an exception complaining that the schema of the GenericRecord does not match the DataFileWriter. If I set the reader schema in the DataFileStream to the schema expected by a given DataFileWriter, I do not get these errors, but I would prefer to not have to open the stream multiple times to write out the records to the different writers. I feel like there should be some obvious tool that leverages schema evolution that could do this, but I cannot find it if it exists.
Thank you!
