Hi,
This is what I've tried:
https://gist.github.com/anonymous/7383104
So far so good except that something is definitely wrong in my code as the
synonym is not emitted as a valid token it seems. This is how my indexing
analyzer is built:
private static final class MyIndexAnalyzer extends
Replying to self: silly me. I am obviously creating the array with the
wrong length.
final String term = new String(buffer, 1, length);
should be replaced by
final String term = new String(buffer, 1, length -1);
and the silly trim can go away. I guess I need more coffee.
S.
On Sat, Nov 9,
This is a parts-of-speech analyzer for tweets. It would make your index
far more useful.
http://www.ark.cs.cmu.edu/TweetNLP/
On 11/04/2013 11:40 PM, Stéphane Nicoll wrote:
Hi,
I am building an application that indexes tweet and offer some basic
search facilities on them.
I am trying to find
If your universe of items you want to match this way is small,
consider something akin to synonyms. Your indexing process
emits two tokens, with and without the @ or # which should
cover your situation.
FWIW,
Erick
On Tue, Nov 5, 2013 at 2:40 AM, Stéphane Nicoll
stephane.nic...@gmail.comwrote:
Hi,
Thanks for the reply. It's an index with tweets so any word really is a
target for this. This would mean a significant increase of the index. My
volumes are really small so that shouldn't be a problem (but
performance/scalability is a concern).
I have the control over the query. Another
You have to get the values _into_ the index with the special characters,
that's where the issue is. Depending on your analysis chain special
characters may or may not be even in your index to search in the first
place.
So it's not how many different words are after the special characters as
much
protWords)
See:
http://lucene.apache.org/core/4_5_0/analyzers-common/org/apache/lucene/analysis/miscellaneous/WordDelimiterFilter.html
-- Jack Krupansky
-Original Message-
From: Stéphane Nicoll
Sent: Tuesday, November 05, 2013 2:40 AM
To: java-user@lucene.apache.org
Subject: Twitter analyser
Hi,
I am building an application that indexes tweet and offer some basic
search facilities on them.
I am trying to find a combination where the following would work:
* foo matches the foo word, a mention (@foo) or the hashtag (#foo)
* @foo only matches the mention
* #foo matches only the
