[
https://issues.apache.org/jira/browse/AVRO-1282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Scott Carey updated AVRO-1282:
------------------------------
Attachment: AVRO-1282-s2.patch
Here is my work in progress. It successfully passes all tests and only has a
slight performance decrease, largely due to commenting out the bits that avoid
boxing in some cases.
I plan to use a different approach to solving that.
There are some cases that I think the tests do not cover that I worry about.
Are we supposed to be able to read a short[] as an avro array of int? What
about a widening conversion from float to double?
The ArrayAccessor code makes strong assumptions about the relationship of java
types to avro types in its current state.
I think I can solve that and reduce boxing at the same time with ArrayRead and
ArrayWrite strategies decoupled from each other and decoupled from boxed get()
and set().
We need to remove the static ReflectData cache because it will cause
classloader leaks. That is in here, but I did not make enough attempts to
address the performance drop due to it. We probably need to have two places we
store context: ReflectData can store only Class + field name -> FieldAccessor
information. ReflectDatumReader and ReflectDatumWriter may have their own
cache or state to improve performance that differs.
> Make use of the sun.misc.Unsafe class during serialization if a JDK supports
> it
> -------------------------------------------------------------------------------
>
> Key: AVRO-1282
> URL: https://issues.apache.org/jira/browse/AVRO-1282
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.7.4
> Reporter: Leo Romanoff
> Priority: Minor
> Attachments: AVRO-1282-s1.patch, AVRO-1282-s2.patch,
> avro-1282-v1.patch, avro-1282-v2.patch, avro-1282-v3.patch,
> avro-1282-v4.patch, avro-1282-v5.patch, avro-1282-v6.patch,
> avro-1282-v7.patch, avro-1282-v8.patch, TestUnsafeUtil.java
>
>
> Unsafe can be used to significantly speed up serialization process, if a JDK
> implementation supports java.misc.Unsafe properly. Most JDKs running on PCs
> support it. Some platforms like Android lack a proper support for Unsafe yet.
> There are two possibilities to use Unsafe for serialization:
> 1) Very quick access to the fields of objects. It is way faster than with the
> reflection-based approach using Field.get/set
> 2) Input and Output streams can be using Unsafe to perform very quick
> input/output.
>
> 3) More over, Unsafe makes it possible to serialize to/deserialize from
> off-heap memory directly and very quickly, without any intermediate buffers
> allocated on heap. There is virtually no overhead compared to the usual byte
> arrays.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
