I have huge lucene index. On disk it's about 24Gb.
I have a purging routine that is supposed to run and purge old docs. There are about 650 million docs in there and through testing I have determined that about 1/3 of these need to be purged. During the purge, every so often it's apparently doing some flushing and applying deletes. This causes the process to hang. I know it's not hanging, but actually doing work because I have enabled infostream and I am getting messages every so often (every 5 minutes). Is there some trick (index config) I can employ to get this to work faster. Stuart M Goldberg