[Haskell-cafe] Re: Case-insensitive lexing with Alex

2008-01-28 Thread Joel Reymont


On Jan 28, 2008, at 2:44 PM, Geoffrey Mainland wrote:

map toLower onto your input before you pass it to your lexer? Or do  
you

only want keywords to be case-insensitive?



Just keywords. You can have Array or array or aRrAy.

--
http://wagerlabs.com





___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


[Haskell-cafe] Re: Case-insensitive lexing with Alex

2008-01-28 Thread Geoffrey Mainland
Joel Reymont wrote:
 
 On Jan 28, 2008, at 2:44 PM, Geoffrey Mainland wrote:
 
 map toLower onto your input before you pass it to your lexer? Or do you
 only want keywords to be case-insensitive?
 
 
 Just keywords. You can have Array or array or aRrAy.

One old trick for reducing the size of lexers is to not put keyword
recognition into the lexer rules (that is, don't write rules to match
keywords explicitly), but to look up identifiers in a map from strings
to keyword tokens. In your case, apply toLower to the identifier before
you look it up in the map from keywords to tokens.

The C parsing library I gave you a link to off-list does its lexing
using this technique if you want a concrete example.

Geoff
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe