Hi Antonio, I am in favour of adding features to the cpplite branch, primarily because the cnd code has no documentation at all and is hard to follow with way too many indirections (been spending 2 months understanding it). I also found some design quirks like indexing happening in the Token producer. It would be much easier to write new code and copy over some existing code.
Clang/clangd are the best bet for getting AST. Cnd has its own ast model which seems to be inspired from clang, so integration should be easy I guess. Clangd has everything to make an IDE [1], though according to their docs, the indexing is not incremental [2]. I would prefer indexing to be done by Netbeans along with code completion and refactoring. This allows us to add value to what is already available. Siddhesh [1] https://clangd.llvm.org/design/code.html [2] https://clangd.llvm.org/design/indexing.html Get Outlook for Android<https://aka.ms/ghei36> ________________________________ From: antonio <[email protected]> Sent: Wednesday, April 14, 2021 2:48:47 PM To: [email protected] <[email protected]> Subject: Re: [DISCUSS] CND Next Steps Hi all, So I tried to replace the "cnd.antlr" runtime with "antlr" runtime as promised. This required removing some stuff in the grammars that is not recognized with the plain antlr compiler. The result of this exercise is that there're indeed differences with the antlr runtime (see [1] for an excerpt) that will require us to fork antlr. Inheritance alone is not enough to have the job done, internal antlr classes have to be modified. For example, the aptBitIntegerExpr.g grammar in cnd.apt builds a Parser object (https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fnetbeans%2Fblob%2Fe6d890a5f0bfff8ccee7c3bb49a821fb5e875900%2Fcnd%2Fcnd.apt%2Fsrc%2Forg%2Fnetbeans%2Fmodules%2Fcnd%2Fapt%2Fimpl%2Fsupport%2FaptBigIntegerExpr.g%23L50&data=04%7C01%7Crane.si%40northeastern.edu%7C49477a5caa6e4534492e08d8ff75f82c%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637540229434505157%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=80DVma7mb2r37orBeQXNmpCEjN6WtDLk%2BzsSDIj8MkA%3D&reserved=0) and then uses a constructor that doesn't exist in antlr.LLkParser (https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fnetbeans%2Fblob%2Fe6d890a5f0bfff8ccee7c3bb49a821fb5e875900%2Fcnd%2Fcnd.apt%2Fsrc%2Forg%2Fnetbeans%2Fmodules%2Fcnd%2Fapt%2Fimpl%2Fsupport%2FaptBigIntegerExpr.g%23L69&data=04%7C01%7Crane.si%40northeastern.edu%7C49477a5caa6e4534492e08d8ff75f82c%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637540229434505157%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UQco82GbedrKxtI%2FnbwRi7BM1lBb04meRf8cZVVKdYY%3D&reserved=0). So forking antlr is inevitable (if anyone has another idea please speak up). The steps to fork antlr in the ASF would be, I think: 1. Forking antlr As far as I understand it's possible to fork Antlr 2.7.7 even though it's not an Apache License based project. The problem could be that license is somewhat blurry, and we may bump into requests with ASF-Legal to acknowledge the licese as category A. (see https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2Ffoundation%2Flicense-faq.html%23code-developed-elsewhere-received-under-a-category-a-license-incorporated-into-apache-projects-distributed-by-apache-and-licensed-to-downstream-users-under-its-original-license&data=04%7C01%7Crane.si%40northeastern.edu%7C49477a5caa6e4534492e08d8ff75f82c%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637540229434515142%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=SLTfRkPLbaL3bRgbZ5kDB16Y6X6a%2BuiXZpDKc4F7BtA%3D&reserved=0 ) 2. Hosting the fork in ASF infra In order to host a fork of antlr we should open a JIRA ticket (to legal?). (see https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.apache.org%2Ffoundation%2Flicense-faq.html%23licenses&data=04%7C01%7Crane.si%40northeastern.edu%7C49477a5caa6e4534492e08d8ff75f82c%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637540229434515142%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=dbjsk6pZ2PR37Np%2BL26JDGZhfnC49WGhHrL5Yj6tY8k%3D&reserved=0 ) 3. Building the fork Once the fork is under ASF Infra the idea would be to enhance antlr.LLKParser and friends to try to reproduce the cnd.antlr runtime behaviour. All of this without seeing what cnd.antlr runtime does (i.e., we won't copy and paste CDDL code). This, of course, would require research and testing other modules. 4. Asking for permissions for the C++ grammar etc. Once alternative runtime is compiling and running, it may be difficult to incorporate the "public license" C++ parser grammar that was long ago incorporated in NetBeans 8.2 code. See https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapache.org%2Flegal%2Fresolved.html%23handling-public-domain-licensed-works&data=04%7C01%7Crane.si%40northeastern.edu%7C49477a5caa6e4534492e08d8ff75f82c%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637540229434515142%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=YYxJMdxEt%2FNesK1dO%2Fu8OlD11ywCqoLTgkEC8JLppdk%3D&reserved=0 for details on incorporating public domains work in ASF projects. 5. Summary After following all the steps above (correct me if I'm wrong) we'll end up with a running alternative CND antlr runtime, and grammars that are difficult to maintain and evolve, as we've discussed in other emails in this thread. After all this research I think we should start thinking of a new CND, possibly based on the current CPPLITE branch, taking advantage of whatever the donated CND offers us. So, what say? Shall we open a vote to decide what to do? Any questions, anyone? Any other ideas you may come up with? Thanks, Antonio [1] Some of the errors found when replacing the "cnd.antlr" runtime with the "antlr" runtime, and then trying to compile the "cnd.apt" module. Note that the generated parser "APTBigIntegerExprParser extends antlr.LLKParser" behaves differently: netbeans/cnd/cnd.apt/src/org/netbeans/modules/cnd/apt/impl/support/generated/APTBigIntegerExprParser.java:752: error: cannot find symbol r=r.mod(b); ^ symbol: variable b location: class APTBigIntegerExprParser netbeans/cnd/cnd.apt/src/org/netbeans/modules/cnd/apt/impl/support/generated/APTExprParser.java:48: error: no suitable constructor found for LLkParser(TokenStream,int,int) super(lexer, 1, 16); constructor LLkParser.LLkParser(int) is not applicable (actual and formal argument lists differ in length) constructor LLkParser.LLkParser(ParserSharedInputState,int) is not applicable (actual and formal argument lists differ in length) constructor LLkParser.LLkParser(TokenBuffer,int) is not applicable (actual and formal argument lists differ in length) constructor LLkParser.LLkParser(TokenStream,int) is not applicable (actual and formal argument lists differ in length) El 12/4/21 a las 22:51, antonio escribió: > [...] > > So, to summarize: it may be possible to replace cnd.antlr with antlr, so > I think I'll explore this possibility. Will report here afterwards. > > Kind regards, > Antonio > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected] For further information about the NetBeans mailing lists, visit: https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FNETBEANS%2FMailing%2Blists&data=04%7C01%7Crane.si%40northeastern.edu%7C49477a5caa6e4534492e08d8ff75f82c%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637540229434515142%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=itC78igK4fxk75beeZoMgNkkZmgDU%2Fkxvv01iMTwg8g%3D&reserved=0
