Adam - how many documents you have in your index? Thanks, Susheel
On Mon, Feb 22, 2016 at 4:37 AM, Adam Neal [Extranet] <an...@mass.co.uk> wrote: > Well I got the numbers wrong, there are actually around 66000 fields on > the index. I have restructured the index and there are now around 1500 > fiields. This has resulted in the commit taking 34 seconds which is > acceptable for my usage however it is still significantly slower than the > 4.10.2 commit on the original 66000 fields which was around 1 second. > ________________________________________ > From: Adam Neal [Extranet] [an...@mass.co.uk] > Sent: 19 February 2016 17:43 > To: solr-user@lucene.apache.org > Subject: RE: Slow commits > > I'm out of the office now so I don't have the numbers to hand but from > memory I think there are probably around 800-1000 fields or so. I will > confirm on Monday. > > If i have time over the weekend I will try and recreate the problem at > home and see if I can post up a sample. > ________________________________________ > From: Yonik Seeley [ysee...@gmail.com] > Sent: 19 February 2016 16:25 > To: solr-user@lucene.apache.org > Subject: Re: Slow commits > > On Fri, Feb 19, 2016 at 8:51 AM, Adam Neal [Extranet] <an...@mass.co.uk> > wrote: > > I've recently upgraded from 4.10.2 to 5.3.1 and I've hit an issue with > slow commits on one of my cores. The core in question is relatively small > (56k docs) and the issue only shows when commiting after a number of > deletes, commiting after additions is fine. As an example commiting after > deleting approximately 10% of the documents takes around 25mins. The same > test on the 4.10.2 instance takes around 1 second. > > > > I have done some investigation and the problem appears to be caused by > having dynamic fields, the core in question has a large number, performing > the same operation on this core with the dynamic fields removed sees a big > improvement on the performance with the commit taking 11 seconds (still not > quite on a par with 4.10.2). > > Dynamic fields is a Solr schema concept, and does not translate to any > differences in Lucene. > You may be hitting something due to a large number of fields (at the > lucene level, each field name is a different field). How many > different fields (i.e. fieldnames) do you have across the entire > index? > > -Yonik > > > ##################################################################################### > > This E-mail is the property of Mass Consultants Ltd. It is confidential > and intended only for the use of the addressee or with its permission. Use > by anyone else for any purpose is prohibited. If you are not the > addressee, you should not use, disclose, copy or distribute this e-mail and > should notify us of receipt immediately by return e-mail to the address > where the e-mail originated. > > This E-mail may not have been sent through a secure system and accordingly > (i) its contents should not be relied upon by any person without > independent verification from Mass Consultants Ltd and (ii) it is the > responsibility of the recipient to ensure that the onward transmission, > opening or use of this message and any attachments will not adversely > affect its systems or data. No responsibility is accepted by Mass > Consultants Ltd in this regard. > > Any e-mails that are sent to Mass Consultants Ltd's e-mail addresses may > be monitored by systems or persons other than the addressee, for the > purposes of ascertaining whether the communication complies with the law > and Mass Consultants Ltd's policies. > > Mass Consultants Ltd is registered in England No. 1705804, Enterprise > House, Great North Road, Little Paxton, Cambs., PE19 6BN, United Kingdom. > Tel: +44 (0) 1480 222600. > > > ##################################################################################### > > > ##################################################################################### > > This E-mail is the property of Mass Consultants Ltd. It is confidential > and intended only for the use of the addressee or with its permission. Use > by anyone else for any purpose is prohibited. If you are not the > addressee, you should not use, disclose, copy or distribute this e-mail and > should notify us of receipt immediately by return e-mail to the address > where the e-mail originated. > > This E-mail may not have been sent through a secure system and accordingly > (i) its contents should not be relied upon by any person without > independent verification from Mass Consultants Ltd and (ii) it is the > responsibility of the recipient to ensure that the onward transmission, > opening or use of this message and any attachments will not adversely > affect its systems or data. No responsibility is accepted by Mass > Consultants Ltd in this regard. > > Any e-mails that are sent to Mass Consultants Ltd's e-mail addresses may > be monitored by systems or persons other than the addressee, for the > purposes of ascertaining whether the communication complies with the law > and Mass Consultants Ltd's policies. > > Mass Consultants Ltd is registered in England No. 1705804, Enterprise > House, Great North Road, Little Paxton, Cambs., PE19 6BN, United Kingdom. > Tel: +44 (0) 1480 222600. > > > ##################################################################################### >
