Robert Roessler wrote:
Before I wrote the one for OCaml, I had some of the same issues as you -
why isn't this more flexible/useful/what I want? But then you realize
after studying it that real languages are full of special and/or weird
cases (WRT syntax coloring) and are not really amenable to being handled
by some sort of regular expression -based solution (which is what some
of the "universal" lexers constructed have tried to do).
Well, RegExes are not the only way to make generic lexers:
- Lua (or other scripting language) based -- slow but flexible and powerful;
- Generic rules ("a comment starts with this sequence and ends with this
sequence (or EOL)") -- rigid and primitive, but enough for lot of cases,
at least to get started and have a quick satisfactory result; what lot
of users is looking for;
- A description language, defining a behaviour of a generic automaton --
seems to be an original way, not seen elsewhere (well, it is more or
less a derivative of the previous way) still primitive, but enough for
most languages, perhaps harder to write for users (and to implement),
but more complete and flexible than the previous way.
I started to thing about the third way, but not hard enough. One
difficulty is to find an elegant way to translate this description
language, as the user see it, to something understandable to Scintilla
(some kind of bytecode, independent of the chosen syntax).
I think, like you, that specialized lexers are a good thing, as they are
powerful and able to handle subtle syntax rules.
But having a generic mechanism to create new lexers, either temporary
(waiting somebody to write a compiled lexer) or definitively (if
language is simple -- most LexOthers lexers for example, or if the users
are happy enough with it) would be really beneficial for Scintilla (and
SciTE!), making it more usable for those wanting to use it for some rare
language but having no C++ knowledge, even less a C++ compiler, to
create yet another lexer.
Well, if this generic lexer would be good enough, we could even remove
lot of rarely used, hard-coded lexers, to reduce overall size of
SciLexer.dll (at least for private use), like do Notepad2 and Notepad++
(creative names...).
I know I can already, but I never know when I need to look at a crontab,
an Eiffel listing or an Inno script.
Now, if making this generic lexer business was so easy, we would have it
since a long time in Scintilla!
--
Philippe Lhoste
-- (near) Paris -- France
-- http://Phi.Lho.free.fr
-- -- -- -- -- -- -- -- -- -- -- -- -- --
_______________________________________________
Scite-interest mailing list
[email protected]
http://mailman.lyra.org/mailman/listinfo/scite-interest
