[
https://issues.apache.org/jira/browse/AVRO-1199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493714#comment-13493714
]
Libing Sun commented on AVRO-1199:
----------------------------------
Why not use the serialize and deserialize, How about the performance of the two
way?
I use the avro in M/R for years, I allways use a method likes next to clone a
object:
static DataOutputBuffer out = new DataOutputBuffer();
static DataInputBuffer in = new DataInputBuffer();
public static <V> V clone(V v) throws IOException {
out.reset();
BinaryEncoder encoder = EncoderFactory.get().directBinaryEncoder(out, null);
Schema schema;
if (GenericContainer.class.isAssignableFrom(v.getClass())) schema =
((GenericContainer) v)
.getSchema();
else schema = ReflectData.get().getSchema(v.getClass());
GenericDatumWriter<V> writer = new ReflectDatumWriter<V>(schema);
GenericDatumReader<V> reader = new ReflectDatumReader<V>(schema);
writer.write(v, encoder);
in.reset(out.getData(), out.getLength());
BinaryDecoder decoder = DecoderFactory.get().directBinaryDecoder(in, null);
return reader.read(null, decoder);
}
Another question, I don't know it's appropriate to say it here or not, if it's
inappropriate, I am sorry.
Hadoop's MapFile use a arry to store the index, but here use a TreeMap, why ?
how about the time and space effciency?
I ask it just for the hadoop project is very closely with avor.
> SortedKeyValueFile$Writer.append method have a puzzle for the sorted key
> ------------------------------------------------------------------------
>
> Key: AVRO-1199
> URL: https://issues.apache.org/jira/browse/AVRO-1199
> Project: Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.7.3
> Reporter: Libing Sun
> Labels: patch
>
> At the SortedKeyValueFile.java 539 lines like next:
> mPreviousKey = key;
> This class is same as Hadoop's MapFile, at the MapFile the same methon will
> keep a copy for this key, but not use "=".
> If use "=" at here, when user append a reuse key object to this file. will
> cause key sorted not valid.
> I think next code will fix it:
> private DataInputBuffer inBuf = new DataInputBuffer();
> private DataOutputBuffer outBuf = new DataOutputBuffer();
> GenericDatumWriter<K> writer = new ReflectDatumWriter<K>(schema);
> GenericDatumReader<K> reader = new ReflectDatumReader<K>(schema);
> writer.write(key, encoder);
> inBuf.reset(outBuf.getData(), outBuf.getLength());
> BinaryDecoder decoder = DecoderFactory.get().directBinaryDecoder(inBuf,
> null);
> lastKey = reader.read(null, decoder);
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
