[jira] [Commented] (FLINK-3673) Annotations for code generation
[ https://issues.apache.org/jira/browse/FLINK-3673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17334328#comment-17334328 ] Flink Jira Bot commented on FLINK-3673: --- This issue was marked "stale-assigned" and has not received an update in 7 days. It is now automatically unassigned. If you are still working on it, you can assign it to yourself again. Please also give an update about the status of the work. > Annotations for code generation > --- > > Key: FLINK-3673 > URL: https://issues.apache.org/jira/browse/FLINK-3673 > Project: Flink > Issue Type: Sub-task > Components: API / Type Serialization System >Reporter: Gabor Horvath >Assignee: Gabor Horvath >Priority: Major > Labels: gsoc2016, stale-assigned > > Annotations should be utilized to generate more efficient serialization code. > Planned improvements: > * Using never null annotations on a field, the serialized representation can > omit the 1 byte null tags and the serializer code handling this tag. > * Using never null annotiation on the POJO, we can omit the top level null > tag. > * Making a POJO final we can omit the subclass tag. > The very same annotations can be used to make the getLength method much > smarter. > Code generation is a prerequisite, to avoid runtime checks which could make > the common codepath (without annotations) slower. > I could also annotate some internal Flink types to make them more efficient. > The main risk: it would break savepoints created with a Flink version that > did not have annotation. We could either introduce a compatibility mode, or > force users to recreate those save points. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-3673) Annotations for code generation
[ https://issues.apache.org/jira/browse/FLINK-3673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17323723#comment-17323723 ] Flink Jira Bot commented on FLINK-3673: --- This issue is assigned but has not received an update in 7 days so it has been labeled "stale-assigned". If you are still working on the issue, please give an update and remove the label. If you are no longer working on the issue, please unassign so someone else may work on it. In 7 days the issue will be automatically unassigned. > Annotations for code generation > --- > > Key: FLINK-3673 > URL: https://issues.apache.org/jira/browse/FLINK-3673 > Project: Flink > Issue Type: Sub-task > Components: API / Type Serialization System >Reporter: Gabor Horvath >Assignee: Gabor Horvath >Priority: Major > Labels: gsoc2016, stale-assigned > > Annotations should be utilized to generate more efficient serialization code. > Planned improvements: > * Using never null annotations on a field, the serialized representation can > omit the 1 byte null tags and the serializer code handling this tag. > * Using never null annotiation on the POJO, we can omit the top level null > tag. > * Making a POJO final we can omit the subclass tag. > The very same annotations can be used to make the getLength method much > smarter. > Code generation is a prerequisite, to avoid runtime checks which could make > the common codepath (without annotations) slower. > I could also annotate some internal Flink types to make them more efficient. > The main risk: it would break savepoints created with a Flink version that > did not have annotation. We could either introduce a compatibility mode, or > force users to recreate those save points. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-3673) Annotations for code generation
[ https://issues.apache.org/jira/browse/FLINK-3673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366425#comment-15366425 ] Greg Hogan commented on FLINK-3673: --- Could we file a ticket for this with a fix version of 2.0.0 and (if simple) an associated PR? > Annotations for code generation > --- > > Key: FLINK-3673 > URL: https://issues.apache.org/jira/browse/FLINK-3673 > Project: Flink > Issue Type: Sub-task > Components: Type Serialization System >Reporter: Gabor Horvath >Assignee: Gabor Horvath > Labels: gsoc2016 > > Annotations should be utilized to generate more efficient serialization code. > Planned improvements: > * Using never null annotations on a field, the serialized representation can > omit the 1 byte null tags and the serializer code handling this tag. > * Using never null annotiation on the POJO, we can omit the top level null > tag. > * Making a POJO final we can omit the subclass tag. > The very same annotations can be used to make the getLength method much > smarter. > Code generation is a prerequisite, to avoid runtime checks which could make > the common codepath (without annotations) slower. > I could also annotate some internal Flink types to make them more efficient. > The main risk: it would break savepoints created with a Flink version that > did not have annotation. We could either introduce a compatibility mode, or > force users to recreate those save points. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-3673) Annotations for code generation
[ https://issues.apache.org/jira/browse/FLINK-3673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15365925#comment-15365925 ] Gabor Horvath commented on FLINK-3673: -- According to our discussion at DataArtisans, we do not want to break the serialization format right now. The null checks for primitive types and the subclass checks for final types are already eliminated from the generated code, but the tags are still written out. In the future, when it is desirable to change the format, it might give some performance advantage to not to write those tags out. This way spills to the disk might happen less frequently. > Annotations for code generation > --- > > Key: FLINK-3673 > URL: https://issues.apache.org/jira/browse/FLINK-3673 > Project: Flink > Issue Type: Sub-task > Components: Type Serialization System >Reporter: Gabor Horvath >Assignee: Gabor Horvath > Labels: gsoc2016 > > Annotations should be utilized to generate more efficient serialization code. > Planned improvements: > * Using never null annotations on a field, the serialized representation can > omit the 1 byte null tags and the serializer code handling this tag. > * Using never null annotiation on the POJO, we can omit the top level null > tag. > * Making a POJO final we can omit the subclass tag. > The very same annotations can be used to make the getLength method much > smarter. > Code generation is a prerequisite, to avoid runtime checks which could make > the common codepath (without annotations) slower. > I could also annotate some internal Flink types to make them more efficient. > The main risk: it would break savepoints created with a Flink version that > did not have annotation. We could either introduce a compatibility mode, or > force users to recreate those save points. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-3673) Annotations for code generation
[ https://issues.apache.org/jira/browse/FLINK-3673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15217940#comment-15217940 ] Gabor Horvath commented on FLINK-3673: -- Sure, I have extended the description of the ticket. > Annotations for code generation > --- > > Key: FLINK-3673 > URL: https://issues.apache.org/jira/browse/FLINK-3673 > Project: Flink > Issue Type: Sub-task > Components: Type Serialization System >Reporter: Gabor Horvath >Assignee: Gabor Horvath > Labels: gsoc2016 > > Annotations should be utilized to generate more efficient serialization code. > Planned improvements: > * Using never null annotations on a field, the serialized representation can > omit the 1 byte null tags and the serializer code handling this tag. > * Using never null annotiation on the POJO, we can omit the top level null > tag. > * Making a POJO final we can omit the subclass tag. > The very same annotations can be used to make the getLength method much > smarter. > Code generation is a prerequisite, to avoid runtime checks which could make > the common codepath (without annotations) slower. > I could also annotate some internal Flink types to make them more efficient. > The main risk: it would break savepoints created with a Flink version that > did not have annotation. We could either introduce a compatibility mode, or > force users to recreate those save points. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-3673) Annotations for code generation
[ https://issues.apache.org/jira/browse/FLINK-3673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15216988#comment-15216988 ] Fabian Hueske commented on FLINK-3673: -- Hi [~horvgab], can you add more detail about the annotations you are proposing with this issue? What would the semantics be and how could they be used to generate more efficient code for serializers? Thanks > Annotations for code generation > --- > > Key: FLINK-3673 > URL: https://issues.apache.org/jira/browse/FLINK-3673 > Project: Flink > Issue Type: Sub-task > Components: Type Serialization System >Reporter: Gabor Horvath >Assignee: Gabor Horvath > Labels: gsoc2016 > > Annotations should be utilized to generate more efficient serialization code. > The very same annotations can be used to make the getLength method much > smarter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
