[jira] [Commented] (IGNITE-13618) Provide generated and reflection-based class (de)serializers
[ https://issues.apache.org/jira/browse/IGNITE-13618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17303218#comment-17303218 ] Alexey Goncharuk commented on IGNITE-13618: --- [~amashenkov] I've added a README file to the module with schema management description from the IEP. Good to merge from my side. > Provide generated and reflection-based class (de)serializers > > > Key: IGNITE-13618 > URL: https://issues.apache.org/jira/browse/IGNITE-13618 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Alexey Goncharuk >Assignee: Andrey Mashenkov >Priority: Major > Labels: iep-54, ignite-3 > Fix For: 3.0.0-alpha2 > > Attachments: benchmark-results.txt > > Time Spent: 1h > Remaining Estimate: 0h > > h3. Motivation. > It may worth having generated serializer code for performance reasons. > However, this should be proved with benchmarks. > h3. Description. > Let's prototype object serializer for type-system described in IEP-54, and > benchmark them to check if generated code approach is a better one. > * As we go with Java11 then VarHandles must be used instead of Unsafe. > * For generated serializer we can use: JDK compiler + Javapoet project (Java > code generator) or Janino compiler or even Prestodb-bytecode module of > PrestoDB project > *UPD*: JDK compile is too slow, Janino doesn't support Java9+ and VarHandles, > Prestodb-bytecode has an unwanted Guava dependency. > So, forking Prestodb without Guava looks like a preferable way. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13618) Provide generated and reflection-based class (de)serializers
[ https://issues.apache.org/jira/browse/IGNITE-13618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295090#comment-17295090 ] Konstantin Orlov commented on IGNITE-13618: --- [~amashenkov], LGTM! > Provide generated and reflection-based class (de)serializers > > > Key: IGNITE-13618 > URL: https://issues.apache.org/jira/browse/IGNITE-13618 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Alexey Goncharuk >Assignee: Andrey Mashenkov >Priority: Major > Labels: iep-54, ignite-3 > Fix For: 3.0 > > Attachments: benchmark-results.txt > > Time Spent: 1h > Remaining Estimate: 0h > > h3. Motivation. > It may worth having generated serializer code for performance reasons. > However, this should be proved with benchmarks. > h3. Description. > Let's prototype object serializer for type-system described in IEP-54, and > benchmark them to check if generated code approach is a better one. > * As we go with Java11 then VarHandles must be used instead of Unsafe. > * For generated serializer we can use: JDK compiler + Javapoet project (Java > code generator) or Janino compiler or even Prestodb-bytecode module of > PrestoDB project > *UPD*: JDK compile is too slow, Janino doesn't support Java9+ and VarHandles, > Prestodb-bytecode has an unwanted Guava dependency. > So, forking Prestodb without Guava looks like a preferable way. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13618) Provide generated and reflection-based class (de)serializers
[ https://issues.apache.org/jira/browse/IGNITE-13618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17264112#comment-17264112 ] Andrey Mashenkov commented on IGNITE-13618: --- So, Janino serializer doesn't fit our needs. JDK serializer requires tools.jar and compilation takes 10-12 times slow. I've reworked generated serializer with using prestodb-bytecode. Now it works as fast as Janino and supports varhandles. > Provide generated and reflection-based class (de)serializers > > > Key: IGNITE-13618 > URL: https://issues.apache.org/jira/browse/IGNITE-13618 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Alexey Goncharuk >Assignee: Andrey Mashenkov >Priority: Major > Labels: iep-54, ignite-3 > Attachments: benchmark-results.txt > > > Let's prototype object serializer for type-system described in IEP-54, and > benchmark them to check if generated code approach is a better one. > * For reflection-based serializer we can use Unsafe (see GridUnsafe class) > which works fine for previour versions of Ignite. > * To keep focus on the ticket needs mainly, we can use Janino project for > code generator. > * In addition, reflection-based serialized could have a version using Java9+ > VanHandles and it's could be hidden under 'java-9+' maven-profile. > Hints: > Javapoet can be used as codegenerator, Janino as a compiler, class loader. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13618) Provide generated and reflection-based class (de)serializers
[ https://issues.apache.org/jira/browse/IGNITE-13618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248442#comment-17248442 ] Andrey Mashenkov commented on IGNITE-13618: --- Unfortunatelly, VarHandles/MethodHandle support is limited in Janino. Seems, it can't process polymorphic signatures and we can't use MethodHandles. As we can't write plain code and access fields/call method directly therefore generated code will be very similar to the reflection-based code and hardly possible it could be more efficient. > Provide generated and reflection-based class (de)serializers > > > Key: IGNITE-13618 > URL: https://issues.apache.org/jira/browse/IGNITE-13618 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Alexey Goncharuk >Assignee: Andrey Mashenkov >Priority: Major > Labels: iep-54, ignite-3 > Attachments: benchmark-results.txt > > > Let's prototype object serializer for type-system described in IEP-54, and > benchmark them to check if generated code approach is a better one. > * For reflection-based serializer we can use Unsafe (see GridUnsafe class) > which works fine for previour versions of Ignite. > * To keep focus on the ticket needs mainly, we can use Janino project for > code generator. > * In addition, reflection-based serialized could have a version using Java9+ > VanHandles and it's could be hidden under 'java-9+' maven-profile. > Hints: > Javapoet can be used as codegenerator, Janino as a compiler, class loader. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13618) Provide generated and reflection-based class (de)serializers
[ https://issues.apache.org/jira/browse/IGNITE-13618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247293#comment-17247293 ] Andrey Mashenkov commented on IGNITE-13618: --- As we a going with Java 11 it is worth to rewrite reflection-based serialized with using VarHandles. I've found at a PR [1] guys use JavaPoet to generate class code and it looks great. Let's try JavaPoet to generate serializer. [1] https://github.com/apache/ignite-3/pull/5 > Provide generated and reflection-based class (de)serializers > > > Key: IGNITE-13618 > URL: https://issues.apache.org/jira/browse/IGNITE-13618 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Alexey Goncharuk >Assignee: Andrey Mashenkov >Priority: Major > Labels: iep-54, ignite-3 > Attachments: benchmark-results.txt > > > Let's prototype object serializer for type-system described in IEP-54, and > benchmark them to check if generated code approach is a better one. > * For reflection-based serializer we can use Unsafe (see GridUnsafe class) > which works fine for previour versions of Ignite. > * To keep focus on the ticket needs mainly, we can use Janino project for > code generator. > * In addition, reflection-based serialized could have a version using Java9+ > VanHandles and it's could be hidden under 'java-9+' maven-profile. > Hints: > Javapoet can be used as codegenerator, Janino as a compiler, class loader. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13618) Provide generated and reflection-based class (de)serializers
[ https://issues.apache.org/jira/browse/IGNITE-13618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17236408#comment-17236408 ] Andrey Mashenkov commented on IGNITE-13618: --- I've pushed benchmark test "SerializerBenchmarkTest". Please, note Reflection-based serialized is already optimized and uses Unsafe directly bypassing some reflection methods. Still, generated serialized show noticeable better results. PFA benchmark results attached. > Provide generated and reflection-based class (de)serializers > > > Key: IGNITE-13618 > URL: https://issues.apache.org/jira/browse/IGNITE-13618 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Alexey Goncharuk >Assignee: Andrey Mashenkov >Priority: Major > Labels: iep-54, ignite-3 > Attachments: benchmark-results.txt > > > Let's prototype object serializer for type-system described in IEP-54, and > benchmark them to check if generated code approach is a better one. > * For reflection-based serializer we can use Unsafe (see GridUnsafe class) > which works fine for previour versions of Ignite. > * To keep focus on the ticket needs mainly, we can use Janino project for > code generator. > * In addition, reflection-based serialized could have a version using Java9+ > VanHandles and it's could be hidden under 'java-9+' maven-profile. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13618) Provide generated and reflection-based class (de)serializers
[ https://issues.apache.org/jira/browse/IGNITE-13618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235572#comment-17235572 ] Andrey Mashenkov commented on IGNITE-13618: --- [~agoncharuk], I've done the implementation both serializers, and start with benchmark implementation. Code can be found at ignite-13618 branch. I'm not sure if it make sense to add VarHandle-based version right now? What benchmark are you expect to be? I see next cases # Key-val pairs of simple types: ** fixed-len, e.g. (long, long) as most popular cases or (uuid, uuid)? ** varlen string (64,256, 1k), e.g. encoding may costs. ** varlen bytes (64,256, 1k ,4k, 16k? or more?), e.g. just cover large entries case. # Key-val pairs of simple object with single field: ** Same cases as above to verify the overhead? or may be just a part of them? # Key-val pairs of complex multi-fields objects. Varlen fields should be short: ** with non-null fields (for ommited null map) ** with no varlen fields (with ommited offset table) ** fix-len non-null fields (minimal overhead). ** any fields. It look like a large task, may be it make sense to implement few of them especially there is no support for date-time, collection/arrays yet. What framework can be used for this purpose? JMH or Jardstick? I doubt the last one will be easier. > Provide generated and reflection-based class (de)serializers > > > Key: IGNITE-13618 > URL: https://issues.apache.org/jira/browse/IGNITE-13618 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Alexey Goncharuk >Assignee: Andrey Mashenkov >Priority: Major > Labels: iep-54, ignite-3 > > Let's prototype object serializer for type-system described in IEP-54, and > benchmark them to check if generated code approach is a better one. > * For reflection-based serializer we can use Unsafe (see GridUnsafe class) > which works fine for previour versions of Ignite. > * To keep focus on the ticket needs mainly, we can use Janino project for > code generator. > * In addition, reflection-based serialized could have a version using Java9+ > VanHandles and it's could be hidden under 'java-9+' maven-profile. -- This message was sent by Atlassian Jira (v8.3.4#803005)
