Hi Jochen, Thanks for raising that issue. It has been a limitation for some folks for quite a while. I am very keen to improve our joint compilation process if we can.
I did try one experiment to improve the V0 process by making transforms smarter and idempotent. Essentially, I was trying to get more transform results into the stubs and I was duplicating more of the process. I was basically regenerating stubs after each phase and repeating earlier Groovy phases for files affected by any changes. I was trying to make more things incremental otherwise that approach doesn't scale at all. But I ran out of cycles to explore before getting anything that I would call usable. I would be interested in looking at earlier attempts like V2 to see if advances in the meantime now make them more feasible. Cheers, Paul. <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Virus-free.www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> On Mon, Jun 26, 2023 at 10:16 PM Jochen Theodorou <blackd...@gmx.org> wrote: > Hi all, > > yesterday I was playing around with joint compilation and I want to > share some thoughts: > > current state (afaik): V0 > > Groovy creates stubs, Javac compiles Java and stubs, Groov compiles > against generated classes. Annotationprocessing on Java should work, on > Groovy it depends on the phase. Reading constants from Java compiled > classes should work for Groovy, I the stubs contain these too, therefore > Java can read the constants as well. Since this runs the Groovy compiler > and the Java compiler together, this consumes quite a bit of memory. > > In summary this works quite well with some problems in the stubs here > and there, memory consumption is not so nice, limitations on Groovy > annotations are not so nice. > > The stubs also allow annotation processors to run on them. Of course > this does not make sense for Lombok, but if a processor is collecting > meta data for example then this is a different matter altogether. > > > Now I will list a few variants I will not specifically mention the > eclipse plugin, I would leave that to Eric to add if he wants to. > > > V1: > In this version I use javaparser to ClassNodes the Groovy compiler does > understand and then let the Groovy compiler proceed normally. The result > is a compilation of only the Groovy classes. All information relevant in > the java classes (modifiers, referenced classes, members, field > constants, imports) have to be resolved in a limited way. Limited means > that we assume any class we cannot resolve comes from Groovy and is to > be resolved by the Groovy compiler later on. In a second step we would > call Javac to compile against the files Groovyc produced. This approach > is less memory consuming than V0, since we can completely end the groovy > compilation. In fact this architecture would be somewhat extensible to > support other languages as well. Also the Java compilation would always > see the fully applied Groovy annotations. A potential problem is > annotations processors working on Java, since there is no Javac > compatible source tree. I am not sure this problem can be solved > satisfactory. An example would be compiling against a Java class using > Lombok and then not seeing methods generated by Lombok. > > V2: > Collect class data from javac using annotation processing and feed that > to groovyc. This would then be a 3 stage process. First we use javac to > get the class information, but javac cannot complete compilation, since > not all classes are resolved. Then we use the extracted information to > compile Groovy. And finally we compile normally with Java. > > I tried this years ago and did not manage to get this working properly. > I would have to investigate what exactly was the problem since I tried > this maybe 8 years ago. But I think I actually was able to extract the > information and my problem was more making the classes then known to > Groovy. If not done in-memory this can probably be done easily. > > V3: > in-process joint compilation by sharing ASTs with Javac. This we tried > more or less with a the Google Summer of Code project. The idea is that > we let both compilers progress in parallel and concurrently resolve > missing types against each other. > > This has numerous problems, mostly the lack of control over Javac, but > even Groovyc does not work as required for this. Because for this to > work you would need to progress a unit at a time as much as possible. > Otherwise Java cannot see the changes by transforms on Groovy. And then > you are basically in the same position as V0 with additional drawbacks. > > V4: > Mix of V3 and V2. Use javac to extract information instead of annotation. > > Frankly I am not sure we get all that many advantages of V3 and V4. If > the goal is to have completely transformed Groovy classes available to > javac, then I see possible advantages. Memorywise I don't see them. But > annotation processing would then work on both, just Groovy seeing a > method added by lombok probably not. > > V5/V6: > Use ECJ/JDT instead of javac. > > V7: > maybe there is a variant with language server infrastructure? I did not > really look into that the last years. Last time I spoke with the guys I > suggested adding some kind of type resolving bridge where multiple > language server can exchange type data, but they did not see a need for > this back then. Today's situation might be different. > > > Well... what do others think? Are we happy with our joint compilation. > Did I list something that sounds like a better alternative? > > bye Jochen >
