[
https://issues.apache.org/jira/browse/AVRO-1282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13643923#comment-13643923
]
Scott Carey commented on AVRO-1282:
-----------------------------------
Leo: Using a variation of your GenericRecordAccessor was the plan, but after a
little more work I'm not sure it will go much faster.
Regarding caching by schema -- it caches first by class, then for each class it
has both a lookup of Accessors by field name, and one by schema. The looup by
schema returns a FieldAccessor[], the one by name returns the FieldAccessor for
that specific named field.
All of the main remaining performance issues are now similar to Generic and
Specific code:
We are traversing and inspecting objects and other data structures on the fly
far too often -- e.g. Schema, Parser, and various instanceof checks, and most
of it could be pre computed if the code was structured radically differently.
That will have to wait for another time. I've got some old prototypes around
elsewhere for Generic/Specific, and after digging in this deep in Reflect for
the first time I'm convinced they all share the same fundamental performance
barriers now. Of course there is room for some tweaks here and there, but for
major wins we need to make bigger changes.
I am finalizing a patch that gets performance back up to the level you had it
or better -- a little faster in most cases and a little slower in others. I've
also rearranged and streamlined Perf.java, incorporating your more recent
versions and refactoring to share more code and make it simpler.
There is still one fundamental flaw: reading blocked encoding is not supported
(it would trigger an array bounds check exception). I have isolated the code
that loops and writes arrays, so we can add that more easily from here.
> Make use of the sun.misc.Unsafe class during serialization if a JDK supports
> it
> -------------------------------------------------------------------------------
>
> Key: AVRO-1282
> URL: https://issues.apache.org/jira/browse/AVRO-1282
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.7.4
> Reporter: Leo Romanoff
> Priority: Minor
> Attachments: AVRO-1282-s1.patch, AVRO-1282-s2.patch,
> AVRO-1282-s3.patch, avro-1282-v1.patch, avro-1282-v2.patch,
> avro-1282-v3.patch, avro-1282-v4.patch, avro-1282-v5.patch,
> avro-1282-v6.patch, avro-1282-v7.patch, avro-1282-v8.patch,
> TestUnsafeUtil.java
>
>
> Unsafe can be used to significantly speed up serialization process, if a JDK
> implementation supports java.misc.Unsafe properly. Most JDKs running on PCs
> support it. Some platforms like Android lack a proper support for Unsafe yet.
> There are two possibilities to use Unsafe for serialization:
> 1) Very quick access to the fields of objects. It is way faster than with the
> reflection-based approach using Field.get/set
> 2) Input and Output streams can be using Unsafe to perform very quick
> input/output.
>
> 3) More over, Unsafe makes it possible to serialize to/deserialize from
> off-heap memory directly and very quickly, without any intermediate buffers
> allocated on heap. There is virtually no overhead compared to the usual byte
> arrays.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
