[
https://issues.apache.org/jira/browse/AVRO-557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12874847#action_12874847
]
Scott Carey commented on AVRO-557:
----------------------------------
Interesting. DirectBinaryDecoder is the same as the 1.2 decoders, so I suspect
that the schema parsing from new GenericDatumReader is the main culprit.
What if you use your GenericDatumReaderWithOptionalResolver but the 1.3
BinaryDecoder. Something like:
{code}
private static class GenericReaderOneTimeUsage13Test extends
GenericReaderOneTimeUsageTest {
private final DecoderFactory factory;
protected GenericReaderOneTimeUsage13Test() throws IOException {
super("GenericReaderOneTimeUsage13Test");
factory = new DecoderFactory().configureDecoderBufferSize(256);
}
@Override protected DatumReader<Object> getReader() {
GenericDatumReaderWithOptionalResolver<Object> reader = new
GenericDatumReaderWithOptionalResolver<Object>(writerSchema);
reader.setUseResolvingDecoder(false);
return reader;
}
@Override protected Decoder getDecoder() {
return factory.createBinaryDecoder(data, null);
}
}
{code}
There is room for improvement in how GenericDatumReader works with resolvers.
-- we could cache them per (expected, actual) schema pair, for example, to
avoid too much parsing. But that won't be free either, equals() on a Schema is
not trivial.
> Speed up one-time data decoding
> -------------------------------
>
> Key: AVRO-557
> URL: https://issues.apache.org/jira/browse/AVRO-557
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.3.2
> Reporter: Kevin Oliver
> Assignee: Kevin Oliver
> Fix For: 1.4.0
>
> Attachments: AVRO-557.patch
>
>
> There are big gains to be had in performance when using a BinaryDecoder and a
> GenericDatumReader just one time. This is due to the relatively expensive
> parsing and initialization that came with 1.3. Patch with example code and a
> Perf harness to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
