[
https://issues.apache.org/jira/browse/AVRO-1268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13607115#comment-13607115
]
Alexandre Normand commented on AVRO-1268:
-----------------------------------------
I worked on a patch inspired by your approach although the implementation
details are rough and it needs cleaning up.
I was just wondering what you think of these numbers. Performance is better
than with previous versions of my patch but it's probably never going to be
better than "before":
Before:
{code}
Executing tests:
[IntTest, SmallLongTest, LongTest, FloatTest, DoubleTest, BoolTest, BytesTest,
StringTest, ArrayTest, MapTest, RecordTest, ValidatingRecord, ResolvingRecord,
RecordWithDefault, RecordWithOutOfOrder, RecordWithPromotion, GenericTest,
GenericStrings, GenericNested, GenericNestedFake, GenericWithDefault,
GenericWithOutOfOrder, GenericWithPromotion, GenericOneTimeDecoderUse,
GenericOneTimeReaderUse, GenericOneTimeUse, FooBarSpecificRecordTest]
readTests:true
writeTests:true
cycles=800
test name time M entries/sec M bytes/sec
bytes/cycle
IntRead: 687 ms 290.855 732.169 629325
IntWrite: 1490 ms 134.208 337.841 629325
SmallLongRead: 734 ms 272.301 685.465 629325
SmallLongWrite: 1493 ms 133.922 337.122 629325
LongRead: 1652 ms 121.024 528.802 1092353
LongWrite: 2594 ms 77.097 336.868 1092353
FloatRead: 376 ms 531.422 2125.687 1000000
FloatWrite: 1211 ms 165.028 660.112 1000000
DoubleRead: 358 ms 557.637 4461.099 2000000
DoubleWrite: 1940 ms 103.057 824.458 2000000
BooleanRead: 244 ms 818.090 818.090 250000
BooleanWrite: 524 ms 381.058 381.058 250000
BytesRead: 1629 ms 24.550 872.489 1776937
BytesWrite: 2164 ms 18.476 656.624 1776937
StringRead: 8490 ms 4.711 167.801 1780910
StringWrite: 8300 ms 4.819 171.641 1780910
ArrayRead: 411 ms 485.936 1943.755 1000006
ArrayWrite: 1161 ms 172.159 688.639 1000006
MapRead: 1361 ms 146.907 734.536 1250004
MapWrite: 2177 ms 91.837 459.186 1250004
RecordRead: 645 ms 51.669 2005.278 1617069
RecordWrite: 1999 ms 16.668 646.898 1617069
ValidatingRecordRead: 3834 ms 8.693 337.391 1617069
ValidatingRecordWrite: 3677 ms 9.065 351.822 1617069
ResolvingRecordRead: 4223 ms 7.892 306.280 1617069
RecordWithDefaultRead: 11246 ms 2.964 115.031 1617069
RecordWithOutOfOrderRead: 3273 ms 10.182 395.157 1617069
RecordWithPromotionRead: 3607 ms 9.241 358.649 1617069
GenericRead: 4717 ms 3.533 137.102 808498
GenericWrite: 3212 ms 5.187 201.318 808498
GenericStringsRead: 7117 ms 2.342 249.629 2220873
GenericStringsWrite: 13290 ms 1.254 133.683 2220873
GenericNested_Read: 7968 ms 2.092 81.169 808498
GenericNested_Write: 4800 ms 3.472 134.731 808498
GenericNestedFake_Read: 3424 ms 4.867 188.878 808498
GenericNestedFake_Write: 1571 ms 10.604 411.528 808498
GenericWithDefault_Read: 10052 ms 1.658 64.343 808498
GenericWithOutOfOrder_Read: 4821 ms 3.456 134.139 808498
GenericWithPromotion_Read: 5124 ms 3.253 126.227 808498
GenericOneTimeDecoderUse_Read: 4725 ms 3.527 136.873 808498
GenericOneTimeReaderUse_Read: 6392 ms 2.607 101.182 808498
GenericOneTimeUse_Read: 6254 ms 2.665 103.414 808498
FooBarSpecificRecordTestRead: 35180 ms 0.474 79.165 3481319
FooBarSpecificRecordTestWrite: 28941 ms 0.576 96.229 3481319
{code}
After:
{code}
Executing tests:
[IntTest, SmallLongTest, LongTest, FloatTest, DoubleTest, BoolTest, BytesTest,
StringTest, ArrayTest, MapTest, RecordTest, ValidatingRecord, ResolvingRecord,
RecordWithDefault, RecordWithOutOfOrder, RecordWithPromotion, GenericTest,
GenericStrings, GenericNested, GenericNestedFake, GenericWithDefault,
GenericWithOutOfOrder, GenericWithPromotion, GenericOneTimeDecoderUse,
GenericOneTimeReaderUse, GenericOneTimeUse, FooBarSpecificRecordTest]
readTests:true
writeTests:true
cycles=800
test name time M entries/sec M bytes/sec
bytes/cycle
IntRead: 702 ms 284.702 716.680 629325
IntWrite: 1503 ms 133.012 334.831 629325
SmallLongRead: 773 ms 258.570 650.898 629325
SmallLongWrite: 1489 ms 134.232 337.903 629325
LongRead: 1652 ms 121.035 528.853 1092353
LongWrite: 2653 ms 75.362 329.289 1092353
FloatRead: 373 ms 534.960 2139.838 1000000
FloatWrite: 1183 ms 168.949 675.796 1000000
DoubleRead: 357 ms 559.525 4476.201 2000000
DoubleWrite: 2009 ms 99.537 796.300 2000000
BooleanRead: 253 ms 789.933 789.933 250000
BooleanWrite: 520 ms 384.196 384.196 250000
BytesRead: 1623 ms 24.645 875.850 1776937
BytesWrite: 2178 ms 18.365 652.673 1776937
StringRead: 8350 ms 4.790 170.614 1780910
StringWrite: 8286 ms 4.827 171.935 1780910
ArrayRead: 403 ms 495.061 1980.254 1000006
ArrayWrite: 1202 ms 166.268 665.074 1000006
MapRead: 1342 ms 148.945 744.729 1250004
MapWrite: 2144 ms 93.255 466.277 1250004
RecordRead: 634 ms 52.542 2039.176 1617069
RecordWrite: 1984 ms 16.793 651.726 1617069
ValidatingRecordRead: 3807 ms 8.755 339.801 1617069
ValidatingRecordWrite: 3683 ms 9.050 351.246 1617069
ResolvingRecordRead: 4199 ms 7.937 308.031 1617069
RecordWithDefaultRead: 11242 ms 2.965 115.065 1617069
RecordWithOutOfOrderRead: 3313 ms 10.060 390.412 1617069
RecordWithPromotionRead: 3575 ms 9.324 361.861 1617069
GenericRead: 5043 ms 3.304 128.233 808498
GenericWrite: 3213 ms 5.186 201.255 808498
GenericStringsRead: 6645 ms 2.508 267.358 2220873
GenericStringsWrite: 13025 ms 1.280 136.400 2220873
GenericNested_Read: 7923 ms 2.104 81.634 808498
GenericNested_Write: 4883 ms 3.413 132.436 808498
GenericNestedFake_Read: 3392 ms 4.912 190.628 808498
GenericNestedFake_Write: 1541 ms 10.811 419.574 808498
GenericWithDefault_Read: 9962 ms 1.673 64.923 808498
GenericWithOutOfOrder_Read: 5046 ms 3.302 128.163 808498
GenericWithPromotion_Read: 5177 ms 3.219 124.926 808498
GenericOneTimeDecoderUse_Read: 5211 ms 3.198 124.101 808498
GenericOneTimeReaderUse_Read: 7531 ms 2.213 85.877 808498
GenericOneTimeUse_Read: 7243 ms 2.301 89.295 808498
FooBarSpecificRecordTestRead: 40359 ms 0.413 69.007 3481319
FooBarSpecificRecordTestWrite: 31255 ms 0.533 89.105 3481319
{code}
> Add java-class, java-key-class and java-element-class support for stringable
> types to SpecificData
> --------------------------------------------------------------------------------------------------
>
> Key: AVRO-1268
> URL: https://issues.apache.org/jira/browse/AVRO-1268
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.7.4
> Reporter: Alexandre Normand
> Assignee: Alexandre Normand
> Priority: Minor
> Fix For: 1.7.5
>
> Attachments: AVRO-1268-needs-work.patch, AVRO-1268.patch,
> AVRO-1268.patch, AVRO-1268.sh, GenericStringsPerf.patch
>
>
> Stringable types are java classes that can be serialized through strings
> (which require a single string constructor and a valid toString()
> implementation). ReflectData currently has support from stringable types but
> it would be desirable to get this feature with SpecificData.
> The work involves changes to the SpecificCompiler (depends on {{@java-class}}
> support in AVRO-1267) to generate the specific sources with the proper java
> type as well as moving the ReflectDatumReader and ReflectDatumWriter to read
> the java-class/java-key-class and java-element-class properties.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
