Re: [Boost-docs] Re: Dynamically Loadable Syntax Highighters [ Was Re: [ANN] QuickBook 1.0 now in boost/tools]

[James Fowler]

This is my main problem with the proposed extension: when I wanted to extend QuickBook to cope with a new source mode, I chose to add a Spirit parser and recompile. If I wanted to add yet another source mode, I'd probably find it easier to create a new Spirit grammar for that mode than to get the regexes going. The capability to create dynamic syntax highlighters could/would augment (not necessarily replace) the existing Spirit-grammar-based source modes, right?  Or is this referring to some other problem? Need to work out where to put the built-in highlight scheme Details please? I just meant that the quickbook executable needs to locate (on start up) the built in syntax highlight scheme file. The user could extend this scheme by supplying their own scheme via the command line, or indeed by editing the default one. It's not really a problem... Why at startup?   Why not actually load (include) dynamic syntax definitions explicitly, something like "[sourcemode foo]" pulling in "foo.highlight_def_extension"?   This could look first in the local directory, then search relative to a some standard lookup path (perhaps $BOOST/tools/QuickBook/highlighters ?).  It might allow for more intelligent feedback on errors (i.e., dependency on a definition file that isn't found).  I think it could even be leveraged to allow BBv2 to treat referenced definitions as dependencies, triggering a QuickBook "rebuild" if the definition is updated (which might be much harder to do otherwise, short of any change to any definition file triggering every .qbk to get reprocessed whether or not it used that definition). Any comments or thoughts? As a means of extending QuickBook to deal with new grammars, it looks like a great idea to me.  I'm not sure I would rush to replace the existing Spirit C++ and Python grammars, but this would be great for adding in other oddball stuff.  When I was updating quickbook.qbk,  it occurred to me how nice it would be to have syntax highlighting for showing 'literal' xml/html and even Quickbook syntax.   This seems to be a great way to get that in there. The Spirit parser and the dynamic grammar Thomas created appear to be using consistent names so far, which is great.  If we keep it consistent (and maybe even document the intentions for the various atomic bits more formally) it should make things easier over time.  One issue to consider is how to create "highlighted output" from a dynamic scheme.  If the syntax definition file only deals with classifying the source code ("here's an identifier, there's a keyword"),  leaving the mechanics of formatting the output (i.e., "void" becomes "<phrase role="keyword">void</phrase>") to the QuickBook internals, the overall system should be more robust and the dynamic definitions easier to create.  The consistent use of named regular expressions would be very helpful there, as QuickBook could determine the output format for a given span by simply using the name of the _expression_ which it matched.  I like it. A bit pythonish, but it's ok Is that supposed to be a problem? ; )  I had never looked at python until I started working with Boost, so I just kind of consider it the Boost scripting language...   I'll keep working on this, then. I already have a standalone dynamically configurable syntax highlighter based on Spirit, Boost Regex and the scheme I posted. I just need to integrate it into QuickBook -- I think those QuickBook macros might give me trouble again! hmm... which macros would those be? - james -- __________________________________________________________ James Fowler, Open Sea Consulting http://www.OpenSeaConsulting.com, Marietta, Georgia, USA Do C++ Right. http://www.OpenCpp.org, opening soon!