The WhitespaceTokenizer fixed the problem, so that'll do as a stop gap until I can figure out how to write our own COBOL tokenizer.
Thanks for the help, Iain ***************************************** * Micro Focus Developer Forum 2004 * * 3 days that will make a difference * * www.microfocus.com/devforum * ***************************************** -----Original Message----- From: Erik Hatcher [mailto:[EMAIL PROTECTED] Sent: 16 December 2003 12:31 To: Lucene Users List Subject: Re: Disabling modifiers? On Tuesday, December 16, 2003, at 07:28 AM, Erik Hatcher wrote: > And yes, if you are using StandardTokenizer, you are probably not > tokenizing COBOL quite like you expect. Is there a COBOL parser you > could tap into that could give you the tokens you want? Ummm..... nevermind that last question... I just realized where you work! :) So, my recommendation would be to tap into some parser for the COBOL language that you have handy and have it feed your Analyzer appropriately. Or, use something very very simple like the WhitespaceAnalyzer as a first try. Erik --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] ________________________________________________________________________ This e-mail has been scanned for viruses by MCI's Internet Managed Scanning Services - powered by MessageLabs. For further information visit http://www.mci.com ________________________________________________________________________ --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
