Hi Jim,
What you think about this idea to resolve everything on the LEXER level?
So we must resolve tokens as
* STRING_LITERAL 'aa'
* STRING_LITERAL 'aa' ws* 'bb' => Token( "aabb" )
* STRING_LITERAL 'aa\'bb' => Token( "aa'bb" )
* STRING_LITERAL 'aa''bb' => Token( "aa'bb" )
* STRING_LITERAL 'aa''bb''cc' => Token( "aa'bb'cc" )
* HEX_LITERAL x'aa' => Token( "aabb" )
* HEX_LITERAL x'aa' ws* 'bb' => Token( "aabb" )
Do you think we can do this in [C] without copying buffers?
I think not.
Then question is:
how this can be solved using minimal copies?
Or you think that better really use
Lexer -> Parser -> TreeParser combination ?
On 9/28/11 1:34 AM, "Ruslan Zasukhin" <ruslan_zasuk...@valentina-db.com>
wrote:
> On 9/28/11 12:46 AM, "Douglas Godfrey" <douglasgodf...@gmail.com> wrote:
>
> Hi Douglas,
>
> Yes, I have thinked about this way also.
>
> But in your solution you use helper functions as
> RemoveQuotePairs()
>
> Which, I guess do some coping in additional ram buffers.
> This is fine for Java guys, but in C code, as Jim likes underline each time,
> we tend to use only pointers to input buffer, as long as possible.
>
>
>> You need to modify your string lexing rules to use sub-rules for the
>> elementary
>> strings and return the concatenated string as the lexer token value.
>>
>> The value of
>>
>> StringConstant: QuotedString
>> {RemoveQuotePairs($QuotedString);};
>>
>> fragment
>> QuotedString: ( StringTerm )+;
>>
>> fragment
>> StringTerm: Dquote ( Character )* Dquote;
>>
>> fragment
>> Character: ( ' ' | AlphaChar | Punctuation | Digit );
--
Best regards,
Ruslan Zasukhin
VP Engineering and New Technology
Paradigma Software, Inc
Valentina - Joining Worlds of Information
http://www.paradigmasoft.com
[I feel the need: the need for speed]
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
