custom sort and out of memory

2008-02-17 Thread Brian Doyle
We've written our own custom sorter to be able to sort on the latitude and longitude fields from the results. We have an index that is about 18million records and 12GB on disk in size. We allocated about 3GB of heap to the index and with about 1 request to the index every 2 or 3 seconds we would

RE: remote stored index

2008-02-17 Thread John Griffin
Don't forget old, venerable RMI. We use it for multiple remote indexes and it works well. John G. -Original Message- From: Jan Pieper [mailto:[EMAIL PROTECTED] Sent: Sunday, February 17, 2008 5:20 AM To: Lucene Mailinglist Subject: remote stored index Hi guys, I want to use lucene for

Re: remote stored index

2008-02-17 Thread Shalin Shekhar Mangar
I think using Apache Solr could be a good solution. It builds on top of lucene and provides HTTP APIs to query/store data. On Feb 17, 2008 5:50 PM, Jan Pieper <[EMAIL PROTECTED]> wrote: > Hi guys, > > I want to use lucene for my new application and I have to store my > lucene index on another serv

Re: remote stored index

2008-02-17 Thread Erick Erickson
RAMdirectories are transient, they go away when your program ends. They are suitable for small indexes that are created on the fly etc. An index written to disk (i.e. an FSDIr) is just a set of files. You can freely copy them anywhere you want. Assuming you have an index phase where the index is c

remote stored index

2008-02-17 Thread Jan Pieper
Hi guys, I want to use lucene for my new application and I have to store my lucene index on another server than my client is running on. What is the best way to use the index? Only found RAMDirectory and FSDirectory in lucene package. I read something about "download, manipulate and upload i

Re: Using lucene with a Geospatial catalog

2008-02-17 Thread Stephane Nicoll
Hi, Thanks for the fast answer. That's also what I sort of figured out by searching on the web but it's good to know someone has implemented it :) The "rough" positioning is a very good idea, thanks. I am pretty sure we have the kind of algorithm you are looking for but it is in a commercial prod

RE: Using lucene with a Geospatial catalog

2008-02-17 Thread Max Metral
We're doing this for our site (http://boston.povo.com) the simple way: have Lucene return all matches based on non-geo criteria and then fetch the items from the db by id and run our geo logic. We store some "rough" positioning in Lucene, such as the region and use that for first level rejectio

Using lucene with a Geospatial catalog

2008-02-17 Thread Stephane Nicoll
Hi, I've been browsing the archive and the documentation about Lucene. It really seems that it could help implementing my use case but I would like to be sure first. What I need is to be able to search data in a "catalog" which is geo-enabled. The data is stored in a database. A record has namely