Thu Sep 18 09:52:56 PDT 2008 Chaddai Fouche <[EMAIL PROTECTED]>
* RichTokenStream support
This patch adds support for raw token streams, that contain more
information than normal token streams (they contains comments at
least). The "lexTokenStream" function brings this support to the
Lexer module. In addition to that, functions have been added to
the GHC module to make easier to recover of the token stream of
a module ("getTokenStream").
Building on that, I added what could be called "rich token
stream": token stream to which have been added the source string
corresponding to each token, the function addSourceToToken takes
a StringBuffer and a starting SrcLoc and a token stream and build
this rich token stream. getRichTokenStream is a convenience
function to get a module rich token stream. "showRichTokenStream"
use the SrcLoc information in such a token stream to get a string
similar to the original source (except unsignificant
whitespaces). Thus "putStrLn . showRichTokenStream =<<
getRichTokenStream s mod" should print a valid module source, the
interesting part being to modify the token stream between the get
and the show of course.
M ./compiler/main/DynFlags.hs +1
M ./compiler/main/GHC.hs -6 +85
M ./compiler/parser/Lexer.x -17 +41
View patch online:
http://darcs.haskell.org/ghc/_darcs/patches/20080918165256-f4941-96b22cff0938f854fd9e1304d81d844694afa488.gz
_______________________________________________
Cvs-ghc mailing list
[email protected]
http://www.haskell.org/mailman/listinfo/cvs-ghc
