Re: Query:Adding all docs at once or creating smaller indexes and merge

2009-06-05 Thread Ian Lea
Hi My guess is that one big index would be more efficient since the total IO read and write load would be less. The big reason for creating smaller intermediate indexes is that you could spread their creation over multiple jobs/disks/servers. There is lots of good advice in http://wiki.apache.o

Query:Adding all docs at once or creating smaller indexes and merge

2009-06-04 Thread Tarandeep Singh
Hi, >From efficiency point of view, what will be more efficient- Creating a single big index (big enough for one machine) by adding all documents in it at once or Creating smaller indexes and then merge them to make one bigger index? And if there is a performance penalty, then any rough estima