Hi Michael, One issue to consider is how long it's going to take to rebuild the indexes at the frequency that you plan. One option is to not store any data on the indexes just a docId that references to another repository. This will keep the processes a little lighter. Also consider using Solid State Drives to improve the indexing speed.
Marilson Campos On 8/16/10 12:08 PM, "Michel Nadeau" <aka...@gmail.com> wrote: > Hi, > > we are building an application using Lucene and we have HUGE data sets (our > index contains millions and millions and millions of documents), which > obviously cause us very important problems when sorting. In fact, we > disabled sorting completely because the servers were just exploding when > trying to sort results in RAM. The users using the system can search for > whatever they want, so we never know how many results will be returned - a > search can return 10 documents (no problem with sorting) or 10,000,000 (huge > sorting problems). > > I woke up this morning and had a flash : is it possible with Lucene to have > a "natural sorting" of documents? For example, let's say I have 3 columns I > want to be able to sort by : first name, last name, email; I would have 3 > different indexes with the very same data but with a different primary key > for sorting. I know it's far fetched, and I have never seen anything like > that since I use Lucene, but we're just desperate... how people do to have > huge data sets, a lot of users, and sort!? > > Thanks, > > Mike --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
