Thomas Guest <thomas.guest <at> gmail.com> writes:
>
> Joel de Guzman <joel <at> boost-consulting.com> writes:
>
> >
> > Eric Niebler wrote:
> > >
> > >> - maybe the syntax highlighting could be done dynamically with
> > >> informations from an external source. this would allow to define
> > >> keywords and comment rules for other languages without having to write
> > >> a new parser.
> > >>
> > >
> > > Ya, that would be cool. I don't know how to do that with Spirit, though.
> > > (It would be pretty straightforward if we were using xpressive, since
> > > xpressive let's you parse strings into rules, but I don't think Spirit
> > > lets you do that. Joel?)
> >
> > Oops, almost missed this...
> >
> > The syntax highlighter rules (e.g. comment, identifier... etc) can
> > certainly use regex (boost/spirit/utility/regex.hpp), or xpressive.
> > That way, we can have different syntax files for different languages
> > which can be loaded at runtime, say from a specified directory in the
> > quickbook path. The syntax files can be something as simple as:
> >
> > c++
> > {
> > preprocessor = ...some regex...;
> > comment = ...some regex...;
> > keyword = ...some regex...;
> > special = ...some regex...;
> > string = ...some regex...;
> > char = ...some regex...;
> > number = ...some regex...;
> > identifier = ...some regex...;
> > }
> >
> > boost::file_system is used to iterate over the contents of the
> > quickbook_syntax directory. A simple spirit parser is the used
> > to parse the contents and instantiate different syntax highlighters
> > for each language (c++, python, etc).
> >
> > I welcome anyone who wish to hack on this. Thomas? :)
>
> Sounds like this would be a fun way for me to get into Boost regex (or
> xpressive) and file system so I'll investigate. This will certainly take me a
> while since 1) learning curve and 2) it's a spare time activity for me, so
> please don't expect anything soon!
>
I've been making (slow) progress with this and thought I should report back
before proceeding any further. The proposal is to replace the hardcoded
syntax_highlight.hpp with a dynamically loaded syntax highlight scheme.
This scheme is based around boost::regex's capabilities. A first draft of the
C++ syntax highlight scheme is appended to this email.
Pros:
As Eric says, users can create highlight schemes for other languages without
having to write a new parser.
Cons:
An extra build dependency (on boost regex)
Creating the regexes is no easier than defining a new parser
Need to work out where to put the built-in highlight scheme
Any comments or thoughts?
Regards, Thomas
------ highlight_scheme -------
# DRAFT FOR COMMENT
# Example syntax highlighting scheme.
#
# Lines on which the first non-whitespace character is '#' are
# treated as comments.
#
# The double-quoted strings look like C/C++ strings, but they are
# handled like Python's raw strings.
# See http://python.org/doc/2.4/ref/strings.html
# The data they enclose is either:
# 1) scheme name or,
# 2) scheme element name or,
# 3) a regex associated with that element.
#
# It is important the elements within a scheme are placed in order
# of preference: otherwise (for example) keywords will be processed
# as identifiers.
"c++"
{
# Some of the regex strings which follow are based on those in
# the Boost regex merge example.
"preprocessor"
=
"^\s*#\s*\w\[\w\d]*"
;
"comment"
=
"//[^\n]*"
"|/\*.*?\*/"
;
"number"
=
"\<[+-]?"
"(?:(?:0x[[:xdigit:]]+)|"
"(?:(?:[[:digit:]]*\.)?[[:digit:]]+(?:[eE][+-]?[[:digit:]]+)?))[uUfFlL]*\>"
;
"char"
=
"[lL]?'[^']*'"
;
"string"
=
"[lL]?\"[^\"]*\""
;
"special"
=
"[~!%\^&\*\(\)\+=\{\[\}\]:;,<\.>\?/\|\-]"
;
"keyword"
=
"\<"
"and_eq|and|asm|auto|bitand|bitor|"
"bool|break|case|catch|char|class|"
"compl|const_cast|const|continue|default|"
"delete|do|double|dynamic_cast|else|"
"enum|explicit|export|extern|false|"
"float|for|friend|goto|if|inline|"
"int|long|mutable|namespace|new|not_eq|"
"not|operator|or_eq|or|private|"
"protected|public|register|reinterpret_cast|"
"return|short|signed|sizeof|static|"
"static_cast|struct|switch|template|this|"
"throw|true|try|typedef|typeid|"
"typename|union|unsigned|using|virtual|"
"void|volatile|wchar_t|while|xor_eq|xor"
"\>"
;
"identifier"
=
"\<"
"\w[[:alnum:]_]*"
"\>"
;
}
# Python highlighting scheme IN PROGRESS
"python"
{
"comment"
=
"#[^\n]*"
;
"keyword"
=
"\<"
"and|del|for|is|raise|"
"assert|elif|from|lambda|return|"
"break|else|global|not|try|"
"class|except|if|or|while|"
"continue|exec|import|pass|yield|"
"def|finally|in|print|"
# Technically "as" and "None" are not yet keywords (at Python
# 2.4). They are destined to become keywords, and we treat them
# as such for syntax highlighting purposes.
"as|None"
"\>"
;
}
> --
> Thomas Guest
> http://homepage.ntlworld.com/thomas.guest
>
-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
Boost-docs mailing list
[email protected]
Unsubscribe and other administrative requests:
https://lists.sourceforge.net/lists/listinfo/boost-docs
