Sorry, I see now you mentioned 56K docs which is pretty small. On Mon, Feb 22, 2016 at 8:30 AM, Susheel Kumar <susheel2...@gmail.com> wrote:
> Adam - how many documents you have in your index? > > Thanks, > Susheel > > On Mon, Feb 22, 2016 at 4:37 AM, Adam Neal [Extranet] <an...@mass.co.uk> > wrote: > >> Well I got the numbers wrong, there are actually around 66000 fields on >> the index. I have restructured the index and there are now around 1500 >> fiields. This has resulted in the commit taking 34 seconds which is >> acceptable for my usage however it is still significantly slower than the >> 4.10.2 commit on the original 66000 fields which was around 1 second. >> ________________________________________ >> From: Adam Neal [Extranet] [an...@mass.co.uk] >> Sent: 19 February 2016 17:43 >> To: solr-user@lucene.apache.org >> Subject: RE: Slow commits >> >> I'm out of the office now so I don't have the numbers to hand but from >> memory I think there are probably around 800-1000 fields or so. I will >> confirm on Monday. >> >> If i have time over the weekend I will try and recreate the problem at >> home and see if I can post up a sample. >> ________________________________________ >> From: Yonik Seeley [ysee...@gmail.com] >> Sent: 19 February 2016 16:25 >> To: solr-user@lucene.apache.org >> Subject: Re: Slow commits >> >> On Fri, Feb 19, 2016 at 8:51 AM, Adam Neal [Extranet] <an...@mass.co.uk> >> wrote: >> > I've recently upgraded from 4.10.2 to 5.3.1 and I've hit an issue with >> slow commits on one of my cores. The core in question is relatively small >> (56k docs) and the issue only shows when commiting after a number of >> deletes, commiting after additions is fine. As an example commiting after >> deleting approximately 10% of the documents takes around 25mins. The same >> test on the 4.10.2 instance takes around 1 second. >> > >> > I have done some investigation and the problem appears to be caused by >> having dynamic fields, the core in question has a large number, performing >> the same operation on this core with the dynamic fields removed sees a big >> improvement on the performance with the commit taking 11 seconds (still not >> quite on a par with 4.10.2). >> >> Dynamic fields is a Solr schema concept, and does not translate to any >> differences in Lucene. >> You may be hitting something due to a large number of fields (at the >> lucene level, each field name is a different field). How many >> different fields (i.e. fieldnames) do you have across the entire >> index? >> >> -Yonik >> >> >> ##################################################################################### >> >> This E-mail is the property of Mass Consultants Ltd. It is confidential >> and intended only for the use of the addressee or with its permission. Use >> by anyone else for any purpose is prohibited. If you are not the >> addressee, you should not use, disclose, copy or distribute this e-mail and >> should notify us of receipt immediately by return e-mail to the address >> where the e-mail originated. >> >> This E-mail may not have been sent through a secure system and >> accordingly (i) its contents should not be relied upon by any person >> without independent verification from Mass Consultants Ltd and (ii) it is >> the responsibility of the recipient to ensure that the onward transmission, >> opening or use of this message and any attachments will not adversely >> affect its systems or data. No responsibility is accepted by Mass >> Consultants Ltd in this regard. >> >> Any e-mails that are sent to Mass Consultants Ltd's e-mail addresses may >> be monitored by systems or persons other than the addressee, for the >> purposes of ascertaining whether the communication complies with the law >> and Mass Consultants Ltd's policies. >> >> Mass Consultants Ltd is registered in England No. 1705804, Enterprise >> House, Great North Road, Little Paxton, Cambs., PE19 6BN, United Kingdom. >> Tel: +44 (0) 1480 222600. >> >> >> ##################################################################################### >> >> >> ##################################################################################### >> >> This E-mail is the property of Mass Consultants Ltd. It is confidential >> and intended only for the use of the addressee or with its permission. Use >> by anyone else for any purpose is prohibited. If you are not the >> addressee, you should not use, disclose, copy or distribute this e-mail and >> should notify us of receipt immediately by return e-mail to the address >> where the e-mail originated. >> >> This E-mail may not have been sent through a secure system and >> accordingly (i) its contents should not be relied upon by any person >> without independent verification from Mass Consultants Ltd and (ii) it is >> the responsibility of the recipient to ensure that the onward transmission, >> opening or use of this message and any attachments will not adversely >> affect its systems or data. No responsibility is accepted by Mass >> Consultants Ltd in this regard. >> >> Any e-mails that are sent to Mass Consultants Ltd's e-mail addresses may >> be monitored by systems or persons other than the addressee, for the >> purposes of ascertaining whether the communication complies with the law >> and Mass Consultants Ltd's policies. >> >> Mass Consultants Ltd is registered in England No. 1705804, Enterprise >> House, Great North Road, Little Paxton, Cambs., PE19 6BN, United Kingdom. >> Tel: +44 (0) 1480 222600. >> >> >> ##################################################################################### >> > >
