[
https://issues.apache.org/jira/browse/AVRO-1268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexandre Normand updated AVRO-1268:
------------------------------------
Attachment: AVRO-1268.sh
AVRO-1268-needs-work.patch
This needs some more love than I wasn't able to provide this last weekend.
I'm still attaching the most recent version of the patch to keep the
conversation moving and, hopefully, some momentum. In this latest patch, the
tests using SpecificRecords have been moved to {{idl}} like we discussed.
There's also a first performance tweak with a {{#getState}} before reading
elements of arrays/maps which saves the classes for both the keys and elements
which are then reused on reading each one.
There are still some optimizations left to do:
* I haven't touched at the {{#readString}} optimizations.
* I haven't started to look into the _"pre-computed state tree could be
walked in parallel to the schema where all lookups would be pre-computed"_
I made changes to the {{FooBarSpecificRecord}} to include strings and here are
the latest before/after results for that updated version, including the small
optimization I made to keep the state prior to reading arrays/maps.
Before
{code}
Executing tests:
[FooBarSpecificRecordTest]
readTests:true
writeTests:true
cycles=800
test name time M entries/sec M bytes/sec
bytes/cycle
FooBarSpecificRecordTestRead: 31546 ms 0.528 88.284 3481319
FooBarSpecificRecordTestWrite: 30538 ms 0.546 91.197
3481319
{code}
After
{code}
Executing tests:
[FooBarSpecificRecordTest]
readTests:true
writeTests:true
cycles=800
test name time M entries/sec M bytes/sec
bytes/cycle
FooBarSpecificRecordTestRead: 40579 ms 0.411 68.632 3481319
FooBarSpecificRecordTestWrite: 34651 ms 0.481 80.373 3481319
{code}
I'll probably be unable to do much active work on this in the next few days
(tax preparation time) but I'll be back.
> Add java-class, java-key-class and java-element-class support for stringable
> types to SpecificData
> --------------------------------------------------------------------------------------------------
>
> Key: AVRO-1268
> URL: https://issues.apache.org/jira/browse/AVRO-1268
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.7.4
> Reporter: Alexandre Normand
> Assignee: Alexandre Normand
> Priority: Minor
> Fix For: 1.7.5
>
> Attachments: AVRO-1268-needs-work.patch, AVRO-1268.patch,
> AVRO-1268.patch, AVRO-1268.sh, GenericStringsPerf.patch
>
>
> Stringable types are java classes that can be serialized through strings
> (which require a single string constructor and a valid toString()
> implementation). ReflectData currently has support from stringable types but
> it would be desirable to get this feature with SpecificData.
> The work involves changes to the SpecificCompiler (depends on {{@java-class}}
> support in AVRO-1267) to generate the specific sources with the proper java
> type as well as moving the ReflectDatumReader and ReflectDatumWriter to read
> the java-class/java-key-class and java-element-class properties.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
