Hi all,
As you may remember from previous threads, the NetBeans 8.2 "cnd.antlr"
module contained a fork of Antlr 2.7.7 (compiler & runtime) that was not
donated to the ASF. As a substitute Oracle donated a
"cnd/libs.cnd.antlr" library wrapper that wrapped the NetBeans 8.2 jar,
that we cannot use in Apache NetBeans due to licensing problems.
The "cnd.apt" package contains an old "aptlexer.g" (on top of this Antlr
2.7.7 runtime) lexer grammar that is responsible for lexing C/C++ and
Fortran files.
I've done a small experiment (see [1]) transforming this old "aptexpr.g"
grammar with 1403 lines [2] to a brand new "APTLexer.g4" lexer grammar
[3] with 480 lines.
This is still on early stages, but it seems the resulting Antlrv4 lexer
outperforms the old Antlrv2 lexer (we go from ~2600 files/second to
~4000 files/second).
I think we can continue the path of removing the old "libs.antlr.cnd"
module from CND and then refactor "cnd.apt" (and "cnd.asm" et al) on top
of the new lexer and Antlrv4. The idea is to upgrade all existing
grammars (including the CXX parser at
https://github.com/apache/netbeans/blob/cnd/cnd/cnd.modelimpl/src/org/netbeans/modules/cnd/modelimpl/parser/CXXParser.g)
to Antlrv4 on top of this new lexer.
Since the "libs.antlr.cnd" API permeates different modules of CND this
may take quite a long time.
I think we can create a brand new "cnd2" branch in the main repository
(forked from the "cnd" branch and refreshed with latest 12.4 version)
and do this CND refactoring on top of Antlrv4 there, right? Any
objections to this? Or shall I continue this experiment in a repo of
mine? Any other plans/suggestions to have CND back?
Thanks,
Antonio
P.S.: In order to test the lexer I'd appreciate if you run the new lexer
(instructions at [1]) on some of your include/source directories and
report back any syntax errors you may see (the lexer will halt on any
lexer error), so we can fine-tune any extra lexer rules we may need.
[1]
https://github.com/vieiro/cnd.nextapt
To lex a whole directory of (*.h/*.c files) run the command:
java -Dlexer.print=false -jar
target/cnd.apt.antlr4-1.0-SNAPSHOT-jar-with-dependencies.jar /usr/include/
To lex a specific file (dumping all tokens to stdout by removing
-Dlexer.print=false):
java -jar target/cnd.apt.antlr4-1.0-SNAPSHOT-jar-with-dependencies.jar
your-file.[ch]
[2]
https://github.com/vieiro/cnd.nextapt/blob/master/cnd.apt.nb82/aptlexer.g
[3]
https://github.com/vieiro/cnd.nextapt/blob/master/cnd.apt.antlr4/src/main/antlr4/org/netbeans/modules/cnd/apt/impl/support/generated/APTLexer.g4
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
