Re: [Boost-docs] Re: Dynamically Loadable Syntax Highighters [ Was Re: [ANN] QuickBook 1.0 now in boost/tools]

[James Fowler]

Joel de Guzman wrote: We need a program options flag for this which defaults to 4 spaces. Thoughts? It's converting tabs to four spaces now, which seems to be the "preferred" indent scheme for Boost.  The recommended approach to tabs in Boost code is to simply avoid them, which IMHO is the right thing to do for Boost.  I'm a little wary of adding flags or options which make it easier to support tabs which probably shouldn't be there in the first place in any code included in Boost docs, but we can also deal with that by just issuing nasty warnings whenever the highlighting process encounters tabs...  Anyhow, the "flaky" part is that the current tab expansion is simply "tab" -> "space space space space", which works pretty well for leading indentation, but thrashes stuff like int foo( ) {                         // here are some aligned     void *pointless_pointer = 0;     // comments which might need     if( pointless_pointer ) {        // smarter tab handling to        ;                             // stay that way if there are     }                                // tabs in any of the whitespace  }                                    // immediately preceding "//" because it's not smart enough to replace a tab with "enough spaces to reach the next tab stop".  We can handle this through doing tab expansion just before we process the highlighting, but this isn't really a high priority for me.  If anyone is interested, I can point them to where it would need to go, or graft it in given a tab expansion algorithm which works on standard iterators or std::string. What's macroref? I seem to have missed this? Just like [classref ...] and [funcref ...], but for macros. Don't you think we ought to just standardize on a quickbook inline syntax for this? Probably, with a few comments: It will probably be fine "as is" for a while, especially for internal (i.e., QuickBook developers) use.  As long as specific projects aren't trying to do they're own local highlighting definitions, this is a good idea but not IMHO an urgent necessity. The inline syntax needs to maintain the level of clarity and simplicity of Thomas's implementation.  Perhaps the best approach is to just keep using what Thomas put together, but add higher level directives (maybe [begin_highlight_def ...] ... [end_highlight_def]) which can extract the contents and pass them to the highlight "load" logic.  This should be pretty simple, if the "load" logic is smart enough to append to existing schemes. I _think_ that's the case, but I don't know that anyone's ever tried it yet.  FWIW, this can be tested with [include more_highlights.hlt] - just make sure that afterwards both the new and the prior (C++ and python) highlight schemes work properly.   If they do, then the rest is easy. I think we ought to have the extensions in the syntax definition. For example:     "cpp" { extensions = "cpp"|"hpp"|.... } Agreed.  But that was more than I had time for, so I hard-coded the basics for now. Besides, it gave me another excuse to play with the Boost preprocessor library: #define HIGHLIGHT_EXTENSION( r, mode, ext )             \     if( extension == "." BOOST_PP_STRINGIZE(ext) ) {    \         actions.source_mode = BOOST_PP_STRINGIZE(mode); \     } else BOOST_PP_SEQ_FOR_EACH( HIGHLIGHT_EXTENSION, c++,    (h)(hpp)(hxx)(c)(cpp)(cxx) ) BOOST_PP_SEQ_FOR_EACH( HIGHLIGHT_EXTENSION, python, (py) ) { // else - falls through if no extensions matched     issue_warning(first, "unrecognized extension \"" + extension +             "\", no automatic source mode selection" ); } This little snippet doesn't even scratch the surface demonstrating how cool it can get...  If you haven't explored the Boost preprocessor library yet (soapbox mode, I suspect Joel's already knows this ;) ),  I suggest doing so when you can.  Like QuickBook, it has proven very helpful in applying my favorite three principles of software elegance.           o -I / --include_path command line option to add additional             directories for file search Are we using boost::file_system now? No.  Maybe we could - I am jumping through some untested hoops to deal with the infamous directory separator character issue.  I did take a peek, but haven't used it yet.  One problem I ran into is that I couldn't find any portable way in Boost to get the path used for the current executable.  Anyone know if boost::filesystem (or anything else in boost or the C++ Standard Library) can portably find this, given the contents of argv[0]? Cheers and Nice work! Thanks! - james -- __________________________________________________________ James Fowler, Open Sea Consulting http://www.OpenSeaConsulting.com, Marietta, Georgia, USA Do C++ Right. http://www.OpenCpp.org, opening soon!