Synonym filter with support for phrases?

2009-04-22 Thread Dawid Weiss
Hello everyone, I'm looking for feedback and thoughts on the following problem (it's more of development than user-centered problem, hope the dev list is appropriate): - a token stream is given, - a set of synonyms is given, where synonyms are token sequences to be matched and token

Re: Synonym filter with support for phrases?

2009-04-22 Thread Earwin Burrfoot
Hello everyone, I'm looking for feedback and thoughts on the following problem (it's more of development than user-centered problem, hope the dev list is appropriate): - a token stream is given, - a set of synonyms is given, where synonyms are token sequences to be matched and token

Re: Synonym filter with support for phrases?

2009-04-22 Thread Dawid Weiss
Your synonyms will break if you try searching for phrases. Good point, I did write that filter, but I never actually got to searching for exact phrases in it (there was a very specific scenario and we used prefix queries which worked quite well). Building on your example, food place in

Re: Synonym filter with support for phrases?

2009-04-22 Thread Earwin Burrfoot
Building on your example, food place in new york will find nothing, because 'place' and 'in' share the same position. You're right, but is it such a big problem in real life? Well, everyone has his own requirements for the search quality. For us it was a problem. User enters a query, then

New TokenStream API usage

2009-04-22 Thread Grant Ingersoll
Has anyone started using the new TokenStream/AttributeSource API? I'm wondering how it is turning out in practice. -Grant - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail:

Re: Synonym filter with support for phrases?

2009-04-22 Thread Dawid Weiss
Well, everyone has his own requirements for the search quality. For us it was a problem. The topic is subjective... I don't see this as a deterioration in search quality. Let me explain. Your example concerns phrase queries, so somebody would have to keep adding terms to a phrase. My

Re: Synonym filter with support for phrases?

2009-04-22 Thread Earwin Burrfoot
Your example concerns phrase queries, so somebody would have to keep adding terms to a phrase. My experience with open search queries (I had access to a larger slice of queries from Microsoft Live) is that phrases are a minority of all searches. In the most common case, people will look for a

Re: Synonym filter with support for phrases?

2009-04-22 Thread Dawid Weiss
engine. So guys looking for MSU CMC really want to get Московский Государственный Университет, факультет ВМиК and his friends. And? How often do they extend this particular phrase with further terms? It must be fun to have an index running concurrently on multi language synonyms, mixing the

[jira] Assigned: (LUCENE-1608) CustomScoreQuery should support arbitrary Queries

2009-04-22 Thread Doron Cohen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doron Cohen reassigned LUCENE-1608: --- Assignee: Doron Cohen CustomScoreQuery should support arbitrary Queries

Create an index from known terms and frequencies

2009-04-22 Thread Johnny B21
Hi! I want to create an index with lucene but i want to do it without having to analyze the text since i already have the terms and term frequencies. How can i create an index like that? I am searching the source of lucene but i can't find where the terms and term frequencies are stored. Please

[jira] Commented: (LUCENE-1607) String.intern() faster alternative

2009-04-22 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12701626#action_12701626 ] Earwin Burrfoot commented on LUCENE-1607: - I tried it out. Works a little bit

Question around LOM | Lucene Ontology

2009-04-22 Thread Rangan Gupta
Hi Am a newbie to Lucene and hence this question about how to implement Ontology based search using Lucene (LOM). It would be useful to guide to any useful books, white papers etc. detailing out the same. Thanks R

[jira] Commented: (LUCENE-1608) CustomScoreQuery should support arbitrary Queries

2009-04-22 Thread Doron Cohen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12701683#action_12701683 ] Doron Cohen commented on LUCENE-1608: - I thought I had written a class exactly for

[jira] Updated: (LUCENE-1608) CustomScoreQuery should support arbitrary Queries

2009-04-22 Thread Doron Cohen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doron Cohen updated LUCENE-1608: Attachment: LUCENE-1608.patch Patch for passing arbitrary queries to custom-score-query. Not

Re: Spatial package plans

2009-04-22 Thread Wouter Heijke
The amount of replies and the state of the code make me think making my own distance filter using a real GIS solution like geotools is the way to go. I wonder anyway if GIS code should be in any Lucene package.. Wouter Yeah it's hard coded to use miles, 5 years in the US gets to you.. But the

Re: Spatial package plans

2009-04-22 Thread patrick o'leary
Free world, help yourself :-) On Wed, Apr 22, 2009 at 6:39 PM, Wouter Heijke whei...@xs4all.nl wrote: The amount of replies and the state of the code make me think making my own distance filter using a real GIS solution like geotools is the way to go. I wonder anyway if GIS code should be in

[jira] Commented: (LUCENE-1252) Avoid using positions when not all required terms are present

2009-04-22 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12701726#action_12701726 ] Jason Rutherglen commented on LUCENE-1252: -- When flexible indexing goes in, users

Re: Future projects

2009-04-22 Thread Jason Rutherglen
Hey Michael, You're in San Jose? Feel free to come by one of these days on our pizza days. Also, can you post what you have of LUCENE-1231? I got a lot more familiar with IndexWriter internals with LUCENE-1516 and could to a good whack at getting LUCENE-1231 integrated. Cheers! Jason On

[jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation

2009-04-22 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12701751#action_12701751 ] Jason Rutherglen commented on LUCENE-831: - I'm trying to figure out how to

Re: Spatial package plans

2009-04-22 Thread Ryan McKinley
Patrick's original version of localluce included geotools -- to make it Apache license compatible we took that out and make the distance calculations pluggable. The hardcoded miles part should be changeable -- feel free to post any patches and we can make it a better solution. best ryan

[jira] Commented: (LUCENE-1539) Improve Benchmark

2009-04-22 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12701768#action_12701768 ] Jason Rutherglen commented on LUCENE-1539: -- {quote} I think it should mean delete

Greetings and questions about patches

2009-04-22 Thread Erick Erickson
Hi all: I've been participating in the user list for some time, and I'd like to start helping maintain/enhance the code. So I thought I'd start with something small, mostly to get the process down. Unit tests sure fit the bill it seems to me, less chance of introducing errors through ignorance

Re: Greetings and questions about patches

2009-04-22 Thread Chris Miller
Issues: 1 none of these methods is ever called. Note that Yonik's suggested patch for LUCENE-1607 contains the following code: + public SimpleStringInterner(int sz) { +cache = new String[BitUtil.nextHighestPowerOfTwo(sz)]; + } ...so the int flavour of nextHighestPowerOfTwo() might be