Re: GSoC Project Proposal Draft: Code Generation in Serializers

Hi! I would like to give you some status updates on the Google Summer of Code project. I started to implement the proposed features [1]. Status of code generation in general: * I can compile the generated code using Janino compiler * I can load the compiled classes and use them * For some

Re: GSoC Project Proposal Draft: Code Generation in Serializers

On the second thought I think you are right. I had the impression that there is cyclic dependency between TypeInformation and the serializers but that is not the case. So there is no rewrite needed for TypeInformation in order to be able to use Scala for serializers. According to the proposal

Re: GSoC Project Proposal Draft: Code Generation in Serializers

Why would you need to rewrite the TypeInformation in Scala? I think we need a way to replace Serializer implementations anyway unless the generated serializers are compatible to the current ones. 2016-04-20 15:53 GMT+02:00 Gábor Horváth : > Hi Fabian, > > I agree that it

Re: GSoC Project Proposal Draft: Code Generation in Serializers

Hi Gabor, you are right, a codegen serializer module would depend on flink-core and in the current design flink-core would need to know about the type infos / serializers / comparators. Decoupling implementations of type info, serializers, and comparators from flink-core and resolving the cyclic

Re: GSoC Project Proposal Draft: Code Generation in Serializers

Unfortunately making code generation a separate module would introduce cyclic dependency. Code generation requires the TypeInfo which is available in flink-core and flink-core requires the generated serializers from the code generation module. Do you have a solution for this? I think if we can

Re: GSoC Project Proposal Draft: Code Generation in Serializers

+1 for not mixing Java and Scala in flink-core. Maybe it makes sense to implement the code generated serializers / comparators as a separate module which can be plugged-in. This could be pure Scala. In general, I think it would be good to have some kind of "version management" for serializers in

Re: GSoC Project Proposal Draft: Code Generation in Serializers

Yes, I know Janino is a pure Java project. I meant if we add Scala code to flink-core, we should add Scala dependency to flink-core and it could be confusing. Regards, Chiwan Park > On Apr 18, 2016, at 2:49 PM, Márton Balassi wrote: > > Chiwan, just to clarify

Re: GSoC Project Proposal Draft: Code Generation in Serializers

Hi! Table API already uses code generation and the Janino compiler [1]. Is it a dependency that is ok to add to flink-core? In case it is ok, I think I will use the same in order to be consistent with the other code generation efforts. I started to look at the Table API code generation [2] and

Re: GSoC Project Proposal Draft: Code Generation in Serializers

Thanks Gábor, now I also see it on the internal GSoC interface. I have indicated that I wish to mentor your project, I think you can hit finalize on your project there. On Mon, Mar 14, 2016 at 11:16 AM, Gábor Horváth wrote: > Hi, > > I have updated this draft to include

Re: GSoC Project Proposal Draft: Code Generation in Serializers

Hi, I have updated this draft to include preliminary benchmarks, mentioned the interaction of annotations with savepoints, extended it with a timeline, and some notes about scala case classes. Regards, Gábor On 9 March 2016 at 16:12, Gábor Horváth wrote: > Hi! > > As far

Re: GSoC Project Proposal Draft: Code Generation in Serializers

Hi! As far as I can see the formatting was not correct in my previous mail. A better formatted version is available here: https://docs.google.com/document/d/1VC8lCeErx9kI5lCMPiUn625PO0rxR-iKlVqtt3hkVnk Sorry for that. Regards, Gábor On 9 March 2016 at 15:51, Gábor Horváth