Hi again,
I came up with this now which made my tests happy:
do
{
tokens.Consume();
} while (tokens.LastToken.Type != MyLexer.EOF);
Afterwards I can go forward with tokens.GetTokens(). Is that how it is
supposed to be used ?
Lars
On Wed, May 11, 2011 at 3:22 PM, Lars von Wedel <[email protected]>wrote:
> Thanks for the answer. However, calling GetTokens() yields an empty list
> for me. This is my code (C#):
>
> var inputStream = new ANTLRStringStream(input);
> var lexer = new MyLexer(inputStream);
> var tokens = new CommonTokenStream(lexer,
> BaseRecognizer.DefaultTokenChannel);
>
> foreach (var token in tokens.GetTokens())
> {
> System.Console.WriteLine(token);
> }
>
> Interestingly, it works, if I inspect the token stream variable in the VS
> debugger.
>
> Lars
>
>
> On Wed, May 11, 2011 at 3:03 PM, Bart Kiers <[email protected]> wrote:
>
>> CommonTokenStream inherits getTokens() which returns a List of Tokens.
>> You'll need to cast them to a Token (or something that extends a Token),
>> since it's a non-generics List list:
>>
>> CommonTokenStream tokens = new CommonTokenStream(lexer);
>> for(Object o : tokens.getTokens()) {
>> Token t = (Token)o;
>> System.out.println(t);
>> }
>>
>>
>> Regards,
>>
>> Bart.
>>
>> On Wed, May 11, 2011 at 1:40 PM, Lars von Wedel
>> <[email protected]>wrote:
>>
>>> Hello,
>>>
>>> I am writing an interactive interpreter and I would like to obtain all
>>> tokens from a lexer of token stream to test, whether the input is
>>> complete
>>> or continued on the next line.
>>>
>>> What is the easiest approach to do this ? I tried a CommonTokenStream but
>>> I
>>> am not sure how to tell it to pull all tokens from the lexer.
>>>
>>> Thanks and Regards,
>>> Lars
>>>
>>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
>>> Unsubscribe:
>>> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>>>
>>
>>
>
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
