StandardAnalyzer & e-mail addresses

[Erik Hatcher]

I'm not JavaCC-savvy enough (yet), but it seems there is a flaw in the 
StandardTokenizer and its determination of e-mail addresses.

If I analyze "[EMAIL PROTECTED]", it splits into two tokens: 
"[EMAIL PROTECTED]" and "com".  Shouldn't this rule:

  // email addresses
| <EMAIL: <ALPHANUM> ("."|"-"|"_" <ALPHANUM>)+ "@" <ALPHANUM> ("."|"-" 
<ALPHANUM>)+ >

Be clever enough to keep the .com with it?  Perhaps some other parsing 
is taking precedence?

Thanks,
        Erik
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]