Thomas Maier-Komor wrote:
Hi,
I am using Solaris' lex for tokenization and I am wondering why I get
the following error message, although I use %pointer in my .l file:
Input string too long, limit 1024
Looking at the source code emitted by lex, I have to find out that the
buffer, although referenced via pointer, is still statically allocated
and fixed in size. Is there any way to get it working dynamically? Is
there an RFE open concerning this?
BTW: flex does not seem to have this limitation.
I'd be grateful for any comments or hints concerning this.
Lex was written a long long time ago; one of the major
design constraints for both lex and yacc IIRC was fitting
into the PDP-11's split 64K text 64K data address spaces.
There are a lot of fixed limits in both lex and yacc that can
be awkward to work around. Hence, both flex and bison
appear in /usr/sfw/bin.
I might also suggest that restructuring your grammar to avoid
huge tokens is another possible approach. For example, you might
handle C's begin comment /end comment tokens separately rather than
attempting to parse the entire comment as a single lexical token.
There are probably RFE's open on this; given the availability of
flex I'm sure this would be fairly low priority.
- Bart
--
Bart Smaalders Solaris Kernel Performance
[EMAIL PROTECTED] http://blogs.sun.com/barts
_______________________________________________
tools-discuss mailing list
[email protected]
