[
https://issues.apache.org/jira/browse/AVRO-1282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641962#comment-13641962
]
Scott Carey commented on AVRO-1282:
-----------------------------------
I'll have a look at your more recent changes for Perf.java again -- mine were
based on v8 + the performance tests from v6 since v8's performance tests don't
execute for me at all.
Re: GenericRecord uniqueness:
Either a compound key including the schema would work (but also be slower for
each record, so fields would speed up and records would go slower -- wide
records vs narrow. We can still avoid IndexedRecords. Perhaps if instead of
an array or map, the state was the actual strategy used to access fields for
the whole record, so we don't have to check instanceof IndexedRecord repeatedly
either.
We probably should not cache any accessors for GenericRecords (or any
IndexedRecord for that matter) since they have a fast setter / getter already.
I'm trying out some changes to my patch to incorporate that.
The performande idea for arrays is to avoid iterators and simplify the loops so
the compiler will be able to avoid more branches and virtual method calls per
item. On the write side:
{code}
if (!array.getClass().getComponentType().isPrimitive()) {
// Handle arrays of objects
return new Iterator<Object>() {
{code} is unnecessary, calling into getArrayElements from writeArray is
optional, we don't need to create a stateful iterator.
> Make use of the sun.misc.Unsafe class during serialization if a JDK supports
> it
> -------------------------------------------------------------------------------
>
> Key: AVRO-1282
> URL: https://issues.apache.org/jira/browse/AVRO-1282
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.7.4
> Reporter: Leo Romanoff
> Priority: Minor
> Attachments: AVRO-1282-s1.patch, avro-1282-v1.patch,
> avro-1282-v2.patch, avro-1282-v3.patch, avro-1282-v4.patch,
> avro-1282-v5.patch, avro-1282-v6.patch, avro-1282-v7.patch,
> avro-1282-v8.patch, TestUnsafeUtil.java
>
>
> Unsafe can be used to significantly speed up serialization process, if a JDK
> implementation supports java.misc.Unsafe properly. Most JDKs running on PCs
> support it. Some platforms like Android lack a proper support for Unsafe yet.
> There are two possibilities to use Unsafe for serialization:
> 1) Very quick access to the fields of objects. It is way faster than with the
> reflection-based approach using Field.get/set
> 2) Input and Output streams can be using Unsafe to perform very quick
> input/output.
>
> 3) More over, Unsafe makes it possible to serialize to/deserialize from
> off-heap memory directly and very quickly, without any intermediate buffers
> allocated on heap. There is virtually no overhead compared to the usual byte
> arrays.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
