RE: splitting a search string into tokens

2004-04-05 Thread Robert Taylor
: Daniel F. Savarese [mailto:[EMAIL PROTECTED] Sent: Sunday, April 04, 2004 4:05 PM To: ORO Users List Subject: Re: splitting a search string into tokens In message [EMAIL PROTECTED], Robert Taylor writes: I need to parse the search string into tokens in the manner that search engine s

Re: splitting a search string into tokens

2004-04-04 Thread Daniel F. Savarese
In message [EMAIL PROTECTED], Robert Taylor writes: I need to parse the search string into tokens in the manner that search engine s would. Lexical analysis (i.e., tokenization) and parsing are two separate activities. Sometimes you can get away with combining the two, but you'll find you can