Hi, Could you post more info about your environment?
OS, JVM version, Total RAM and how much allocated to JVM? JVM options such as GC settings, other applications running on same box? document type and size, size of your "index" folder on disk, schema & solrconfig changes (have you migrated schema since 1.4?), what have you changed in solrconfig.xml? What method/API/UpdateHandler do you use to feed documents? You should also try to use Java VisualVM (http://visualvm.java.net/index.html) and connect to your running JVM to analyze heap size, GC activity and profile mem/cpu usage. Report back what you find. -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com Solr Training - www.solrtraining.com On 7. okt. 2011, at 15:38, Jan Høydahl wrote: > Hi, > > Have you tried to do a commit after the deleteByQuery only? > Also, what seems to cause the slowdown? Any hints from the logs? > > -- > Jan Høydahl, search solution architect > Cominvent AS - www.cominvent.com > Solr Training - www.solrtraining.com > > On 7. okt. 2011, at 10:04, Willem Basson wrote: > >> Hi there >> >> We are currently moving from Solr 1.4 to 3.4 and we are seeing a few issues >> with adding documents. >> We do a delete by query and then do a lot of adds, about 100k before we do a >> commit and optimise. >> With 1.4 this was all fine, not super quick but didn't see any problems. >> With 3.4 the rate of adding documents seriously degrades. For our one index >> at about 80% it severely slows down but struggles and completes. >> For the other index which has quite a few more fields (up to 6000+ for some >> documents) it slows down at about 20%. >> >> If we do periodic commits then we don't see the slowdown, but that causes us >> some other issues with replication etc. and while we can go down that route >> if we really must we would like to know what has changed from 1.4 to 3.4 to >> cause this behaviour. I have tried changing the LuceneMatchVersion to >> LUCENE_34 and upped to memory from 2GB to 4GB on a machine with 8GB ram but >> it really doesn't make any difference to the behaviour. Don't see any errors >> in the log files. >> >> Any ideas of what we could try to diagnose or fix the problem? >> >> -- >> Willem Basson >
