[
https://issues.apache.org/jira/browse/AVRO-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13610342#comment-13610342
]
Leo Romanoff commented on AVRO-1277:
------------------------------------
The updated version of the patch has some perf. tests. And they are looking
quite good.
This is what I get with the vanilla Avro:
Executing tests:
[ReflectArrayTest, ReflectSmallFloatArrayTest, ReflectMediumFloatArrayTest]
readTests:true
writeTests:true
cycles=800
test name time M entries/sec M bytes/sec
bytes/cycle
ReflectArrayRead: 26948 ms 7,421 29,686 1000006
ReflectArrayWrite: 24391 ms 8,200 32,799 1000006
ReflectSmallFloatArrayRead: 68081 ms 2,938 13,220 1125004
ReflectSmallFloatArrayWrite: 38192 ms 5,237 23,565 1125004
ReflectMediumFloatArrayRead: 25759 ms 7,764 31,368 1010003
ReflectMediumFloatArrayWrite: 25045 ms 7,986 32,262 1010003
And this is what I get with my patch applied:
Executing tests:
[ReflectArrayTest, ReflectSmallFloatArrayTest, ReflectMediumFloatArrayTest]
readTests:true
writeTests:true
cycles=800
test name time M entries/sec M bytes/sec
bytes/cycle
ReflectArrayRead: 25050 ms 7,984 31,936 1000006
ReflectArrayWrite: 20039 ms 9,980 39,921 1000006
ReflectSmallFloatArrayRead: 28119 ms 7,113 32,007 1125004
ReflectSmallFloatArrayWrite: 16043 ms 12,466 56,096 1125004
ReflectMediumFloatArrayRead: 9153 ms 21,851 88,276 1010003
ReflectMediumFloatArrayWrite: 4715 ms 42,416 171,361 1010003
As you can see, the patch improves performance overall and gives real boost for
objects containing arrays of primitive types.
Future ideas: According to the profiler, a lot of time is spent in
java.lang.reflect.Field.get/set methods. They are very slow. To get the
ultimate speed, one could use java.misc.Unsafe-based approach to access fields.
The drawback is that it would make the source dependent on Sun/Oracle/IBM JVM,
because other JVMs do not implement Unsafe. So, such a feature should be
optional, if it is included, so that people e.g. using Android could disable
it.
> Improve performance of reflection-based serializers
> ---------------------------------------------------
>
> Key: AVRO-1277
> URL: https://issues.apache.org/jira/browse/AVRO-1277
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.7.4
> Reporter: Leo Romanoff
> Priority: Minor
> Attachments: AVRO-1277.patch, AVRO-1277-v2.patch
>
>
> I've played a bit with Avro serialization based on reflection, i.e. with the
> classes from org.apache.avro.generic package.
> It works fine in this mode, but is rather slow compared to such frameworks
> like protostuff or kryo.
> Quick look at the source code has shown that a lot of reflection-based
> operations and class lookups are not cached. And such operations are usually
> pretty expensive when executed by a JVM.
> So I changed some of org.apache.avro.generic classes, introduced caching and
> a few other optimizations. Now it seems to perform much better.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
