[
https://issues.apache.org/jira/browse/AVRO-557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12897057#action_12897057
]
Scott Carey commented on AVRO-557:
----------------------------------
+1
Performance results for all GenericReader tests using updated test suite.
Before:
{quote}
GenericReaderTest: 4316 ms, 0.9652518597286222 million entries/sec.
37.46562600631532 million bytes/sec
GenericReaderOneTimeUseReaderTest: 12438 ms, 0.33496174128331707 million
entries/sec. 13.002931005679518 million bytes/sec
GenericReaderOneTimeUseDecoderTest: 4541 ms, 0.9175101858715102 million
entries/sec. 35.60243412710682 million bytes/sec
GenericReaderOneTimeUseTest: 12994 ms, 0.3206316809649748 million entries/sec.
12.444687860652925 million bytes/sec
GenericReaderTestWithDefaultTest: 18814 ms, 0.22144625531538778 million
entries/sec. 8.594070597799654 million bytes/sec
GenericReaderTestWithOutOfOrderTest: 13612 ms, 0.3060743396647221 million
entries/sec. 11.87857138764328 million bytes/sec
GenericReaderTestWithPromotionTest: 13968 ms, 0.29828822008881134 million
entries/sec. 11.577040834808965 million bytes/sec
{quote}
After:
{quote}
GenericReaderTest: 1038 ms, 4.010383780204211 million entries/sec.
155.65178722683382 million bytes/sec
GenericReaderOneTimeUseReaderTest: 1315 ms, 3.1672704992679512 million
entries/sec. 122.93213306717061 million bytes/sec
GenericReaderOneTimeUseDecoderTest: 1089 ms, 3.82440681077608 million
entries/sec. 148.44554591766487 million bytes/sec
GenericReaderOneTimeUseTest: 1278 ms, 3.258017977159077 million entries/sec.
126.44808401330889 million bytes/sec
GenericReaderTestWithDefaultTest: 2281 ms, 1.826564690595329 million
entries/sec. 70.89078839731388 million bytes/sec
GenericReaderTestWithOutOfOrderTest: 1313 ms, 3.1731271581301055 million
entries/sec. 123.14764465849947 million bytes/sec
GenericReaderTestWithPromotionTest: 1627 ms, 2.559650956129309 million
entries/sec. 99.33645049856828 million bytes/sec
{quote}
About a 5x to 9x performance improvement.
A quick summary of changes to Perf.java -- Changed all GenericReader tests to
inherit from one parent class. GenericReader tests write a series of objects,
other tests write an array of Records. All GenericReader tests should be
consistent now and it should be much easier to add more variations later. The
MB/sec measurement was broken before, it should be correct now.
> Speed up one-time data decoding
> -------------------------------
>
> Key: AVRO-557
> URL: https://issues.apache.org/jira/browse/AVRO-557
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.3.2
> Reporter: Kevin Oliver
> Assignee: Doug Cutting
> Fix For: 1.4.0
>
> Attachments: AVRO-557-partial.patch, AVRO-557.patch, AVRO-557.patch,
> AVRO-557.patch, AVRO-557.patch, AVRO-557.patch
>
>
> There are big gains to be had in performance when using a BinaryDecoder and a
> GenericDatumReader just one time. This is due to the relatively expensive
> parsing and initialization that came with 1.3. Patch with example code and a
> Perf harness to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
