You have to remember that the lexer "automatically" chooses which token to parse next. It doesn't know anything about your block_1 grammar rule, so when it sees 10 digits in a row, it won't know to lex a session number before a sequence number. I believe if the lexer has two rules with the same left prefix it will try to lex the one which matches the most characters. (http://www.antlr.org/wiki/pages/viewpage.action?pageId=4882470) I think your best option is to make application_id, service_id, address, session_number, and sequence number grammar rules. Dave
_____ From: [email protected] [mailto:[email protected]] On Behalf Of [email protected] Sent: Wednesday, December 02, 2009 11:15 AM To: [email protected] Subject: [antlr-interest] How can I parse length defined tokens? Hello, I'm relatively new to antlr v3. I'm trying to write a parser that should be able to parse the following (SWIFT block 1) input: {1:F01PARBDEFFXXXX0001000000} I wish the parser to be able to then work with the following rule: block_1: '{1:' APPLICATION_ID SERVICE_ID ADDRESS SESSION_NUMBER SEQUENCE_NUMBER '}' ; with tokens: APPLICATION_ID: 'F' | 'A' | 'L'; SERVICE_ID: '01' | '02'; fragment CHAR: 'A' .. 'Z' | '0' .. '9'; ADDRESS: CHAR CHAR CHAR CHAR CHAR CHAR CHAR CHAR CHAR CHAR CHAR CHAR; fragment DIGIT: '0' .. '1'; SESSION_NUMBER: DIGIT DIGIT DIGIT DIGIT; SEQUENCE_NUMBER: DIGIT DIGIT DIGIT DIGIT DIGIT DIGIT; However it doesn't work. Do you know an elegant way to work with this kind of tokens? I mean to define length defined tokens and to be able to work without "token recognition overlapp". (I'm open to combined grammar or not solution) Could the "filter" Lexer option help me there? Thanks in advance. Regards, Loïc This message and any attachments (the "message") is intended solely for the addressees and is confidential. If you receive this message in error, please delete it and immediately notify the sender. Any use not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. The internet can not guarantee the integrity of this message. BNP PARIBAS (and its subsidiaries) shall (will) not therefore be liable for the message if modified. Do not print this message unless it is necessary, consider the environment. --------------------------------------------- Ce message et toutes les pieces jointes (ci-apres le "message") sont etablis a l'intention exclusive de ses destinataires et sont confidentiels. Si vous recevez ce message par erreur, merci de le detruire et d'en avertir immediatement l'expediteur. Toute utilisation de ce message non conforme a sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. L'internet ne permettant pas d'assurer l'integrite de ce message, BNP PARIBAS (et ses filiales) decline(nt) toute responsabilite au titre de ce message, dans l'hypothese ou il aurait ete modifie. N'imprimez ce message que si necessaire, pensez a l'environnement. -- You received this message because you are subscribed to the Google Groups "il-antlr-interest" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
