[jira] [Commented] (AVRO-2090) Improve encode/decode time for SpecificRecord using code generation
[ https://issues.apache.org/jira/browse/AVRO-2090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16195955#comment-16195955 ] ASF GitHub Bot commented on AVRO-2090: -- GitHub user rstata opened a pull request: https://github.com/apache/avro/pull/256 AVRO-2090: Improve encode/decode time for SpecificRecord using code generation Initial patch for AVRO-2090 You can merge this pull request into a Git repository by running: $ git pull https://github.com/rstata-projects/avro AVRO-2090 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/avro/pull/256.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #256 commit d2127c7a4051bf7efa56cb0d7e8d9de6ead31c16 Author: rstata Date: 2017-07-13T08:36:39Z Saving initial work, still have more to do. commit 474cc97315ecfeb5bc79dc366424c342d57d83e2 Author: rstata Date: 2017-07-14T04:22:46Z Finished initial implementation (not tested). commit 456d667c58df1493190b99bea40b24408e969679 Author: rstata Date: 2017-07-14T05:51:24Z Poorly done feature flag, and formatting improvements (incl proper indentation). commit b1caba57a90a7fe9a7779c137137315d1d6a99ec Author: rstata Date: 2017-07-16T08:40:03Z Added Reader/Decoder code commit 9f8c853f6f43c9ce07eb9ad10da0d6acf9263c5e Author: rstata Date: 2017-09-16T01:33:16Z Updated output files to reflect new specific-compiler strategy. commit 83698d9e3ea04e00cd7da87373409b74f77d708a Author: rstata Date: 2017-10-03T21:44:05Z Reverting changes to SpecificFixed commit 84e4cbb1ada1ceb97dcec0364a231055cd25142a Author: rstata Date: 2017-10-04T01:54:24Z Change name of feature from Encodable to CustomCoders commit e57289bae26683ba4ea3ed30f863be5a79983bc0 Author: rstata Date: 2017-10-04T04:49:15Z Fixed bugs in codegen template commit f8fae7bc307fae7d51afab7a99025b4213937d40 Author: rstata Date: 2017-10-04T05:32:20Z Added feature flag for custom coders commit d5b45607ace5fbaf9ee526df2fa285a047365548 Author: rstata Date: 2017-10-08T00:18:34Z Remove stale TODO comment > Improve encode/decode time for SpecificRecord using code generation > --- > > Key: AVRO-2090 > URL: https://issues.apache.org/jira/browse/AVRO-2090 > Project: Avro > Issue Type: Improvement > Components: java >Reporter: Raymie Stata > Attachments: customcoders.md > > > Compared to GenericRecords, SpecificRecords offer type-safety plus the > performance of traditional getters/setters/instance variables. But these are > only beneficial to Java code accessing those records. SpecificRecords > inherit serialization and deserialization code from GenericRecords, which is > dynamic and thus slow (in fact, benchmarks show that serialization and > deserialization is _slower_ for SpecificRecord than for GenericRecord). > This patch extends record.vm to generate custom, higher-performance encoder > and decoder functions for SpecificRecords. We've run a public benchmark > showing that the new code reduces serialization time by 2/3 and > deserialization time by close to 50%. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] avro pull request #256: AVRO-2090: Improve encode/decode time for SpecificRe...
GitHub user rstata opened a pull request: https://github.com/apache/avro/pull/256 AVRO-2090: Improve encode/decode time for SpecificRecord using code generation Initial patch for AVRO-2090 You can merge this pull request into a Git repository by running: $ git pull https://github.com/rstata-projects/avro AVRO-2090 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/avro/pull/256.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #256 commit d2127c7a4051bf7efa56cb0d7e8d9de6ead31c16 Author: rstata Date: 2017-07-13T08:36:39Z Saving initial work, still have more to do. commit 474cc97315ecfeb5bc79dc366424c342d57d83e2 Author: rstata Date: 2017-07-14T04:22:46Z Finished initial implementation (not tested). commit 456d667c58df1493190b99bea40b24408e969679 Author: rstata Date: 2017-07-14T05:51:24Z Poorly done feature flag, and formatting improvements (incl proper indentation). commit b1caba57a90a7fe9a7779c137137315d1d6a99ec Author: rstata Date: 2017-07-16T08:40:03Z Added Reader/Decoder code commit 9f8c853f6f43c9ce07eb9ad10da0d6acf9263c5e Author: rstata Date: 2017-09-16T01:33:16Z Updated output files to reflect new specific-compiler strategy. commit 83698d9e3ea04e00cd7da87373409b74f77d708a Author: rstata Date: 2017-10-03T21:44:05Z Reverting changes to SpecificFixed commit 84e4cbb1ada1ceb97dcec0364a231055cd25142a Author: rstata Date: 2017-10-04T01:54:24Z Change name of feature from Encodable to CustomCoders commit e57289bae26683ba4ea3ed30f863be5a79983bc0 Author: rstata Date: 2017-10-04T04:49:15Z Fixed bugs in codegen template commit f8fae7bc307fae7d51afab7a99025b4213937d40 Author: rstata Date: 2017-10-04T05:32:20Z Added feature flag for custom coders commit d5b45607ace5fbaf9ee526df2fa285a047365548 Author: rstata Date: 2017-10-08T00:18:34Z Remove stale TODO comment ---
[jira] [Updated] (AVRO-2090) Improve encode/decode time for SpecificRecord using code generation
[ https://issues.apache.org/jira/browse/AVRO-2090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymie Stata updated AVRO-2090: --- Attachment: customcoders.md Attaching a design document for (forthcoming) patch. > Improve encode/decode time for SpecificRecord using code generation > --- > > Key: AVRO-2090 > URL: https://issues.apache.org/jira/browse/AVRO-2090 > Project: Avro > Issue Type: Improvement > Components: java >Reporter: Raymie Stata > Attachments: customcoders.md > > > Compared to GenericRecords, SpecificRecords offer type-safety plus the > performance of traditional getters/setters/instance variables. But these are > only beneficial to Java code accessing those records. SpecificRecords > inherit serialization and deserialization code from GenericRecords, which is > dynamic and thus slow (in fact, benchmarks show that serialization and > deserialization is _slower_ for SpecificRecord than for GenericRecord). > This patch extends record.vm to generate custom, higher-performance encoder > and decoder functions for SpecificRecords. We've run a public benchmark > showing that the new code reduces serialization time by 2/3 and > deserialization time by close to 50%. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AVRO-2094) Extend "custom coders" to support logical types
Raymie Stata created AVRO-2094: -- Summary: Extend "custom coders" to support logical types Key: AVRO-2094 URL: https://issues.apache.org/jira/browse/AVRO-2094 Project: Avro Issue Type: Improvement Reporter: Raymie Stata The initial implementation of "custom coders" (AVRO-2090) does not support Avro's logical types. This JIRA extends that implementation to remove this limitation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AVRO-2093) Extend "custom coders" to fully support union types
Raymie Stata created AVRO-2093: -- Summary: Extend "custom coders" to fully support union types Key: AVRO-2093 URL: https://issues.apache.org/jira/browse/AVRO-2093 Project: Avro Issue Type: Improvement Reporter: Raymie Stata The initial implementation of "custom coders" for SpecificRecord (AVRO-2090) only supports "nullable unions" (two-branch unions where one branch is the null type). This JIRA extends that implementation to support all forms of unions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AVRO-2092) Flip default value of org.apache.avro.specific.use_custom_coder to true
Raymie Stata created AVRO-2092: -- Summary: Flip default value of org.apache.avro.specific.use_custom_coder to true Key: AVRO-2092 URL: https://issues.apache.org/jira/browse/AVRO-2092 Project: Avro Issue Type: Improvement Components: java Reporter: Raymie Stata Priority: Minor The initial implementation of "custom coders" for SpecificRecord is incomplete (it didn't initially handle logical types) and hasn't been battle-tested. Thus, it includes a feature flag (org.apache.avro.specific.use_custom_coder) to toggle between the new code and the old code. The initial default for this feature flag is false -- defaulting to the old code -- but when the implementation of SpecificRecord is completed and it's seen more production use, we should switch the default to false, on the way to eliminating the flag altogether (AVRO-2091). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AVRO-2091) Eliminate org.apache.avro.specific.use_custom_coder feature flag
Raymie Stata created AVRO-2091: -- Summary: Eliminate org.apache.avro.specific.use_custom_coder feature flag Key: AVRO-2091 URL: https://issues.apache.org/jira/browse/AVRO-2091 Project: Avro Issue Type: Improvement Components: java Reporter: Raymie Stata Priority: Minor After the implementation of "custom coders" (AVRO-2090) is complete and seen more production usage, this feature flag should be eliminated. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AVRO-2090) Improve encode/decode time for SpecificRecord using code generation
Raymie Stata created AVRO-2090: -- Summary: Improve encode/decode time for SpecificRecord using code generation Key: AVRO-2090 URL: https://issues.apache.org/jira/browse/AVRO-2090 Project: Avro Issue Type: Improvement Components: java Reporter: Raymie Stata Compared to GenericRecords, SpecificRecords offer type-safety plus the performance of traditional getters/setters/instance variables. But these are only beneficial to Java code accessing those records. SpecificRecords inherit serialization and deserialization code from GenericRecords, which is dynamic and thus slow (in fact, benchmarks show that serialization and deserialization is _slower_ for SpecificRecord than for GenericRecord). This patch extends record.vm to generate custom, higher-performance encoder and decoder functions for SpecificRecords. We've run a public benchmark showing that the new code reduces serialization time by 2/3 and deserialization time by close to 50%. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
