Hi Manuela, These queries *are* cached as long as you have "webui.strengths.cache=true" set in your dspace.cfg file. So, that might be a first step here is to ensure you are caching these item counts.
Offhand, I'm not sure myself why the performance would be significantly different between 1.8 and 5.8, but there have been a lot of code changes between those releases (so it's possible something is affecting performance). If you or someone on your end is able to step through the code and/or determine the performance cause, we do welcome code contributions / bug fixes. DSpace is primarily supported through community collaboration, and therefore any bug fixes you may find would help others throughout the world. You are also welcome to log a bug in our Issue Tracker, and we can then look for a volunteer to try to reproduce the issue and look for a possible fix: https://jira.duraspace.org/projects/DS/issues In case it is of help, in DSpace 5, this Item counter functionality is performed by ItemCounter.getCount(): https://github.com/DSpace/DSpace/blob/dspace-5_x/dspace-api/src/main/java/org/dspace/browse/ItemCounter.java#L110 If you run across other things you would like advice on, we'd be glad to try and give tips/hints. You may wish to look at other Support options (especially Slack) as well, if you plan to dig deeper into the code here * https://wiki.duraspace.org/display/DSPACE/Support * https://wiki.duraspace.org/display/DSPACE/Slack Good luck, and let us know if you discover anything else as you dig into this issue. Tim On Fri, Sep 22, 2017 at 1:07 PM Manuela Ferreira <[email protected]> wrote: > Hi! > > Today we use DSpace 1.8 in our repository (http://lume.ufrgs.br), with > *webui.strengths.show=true* and *webui.strengths.cache = true*, and we > have more than *150 thousands of items*. The performance of communities, > collections and items pages is very good. > > Now, we will migrate to 5.8 version. > > However when showing the item counter of communities/collections the page > load time is increased in 100% of current time. In the solr.log we can see > where this time is expended. The solr queries take between 1 and 2 (in the > first page load) seconds. If we set *webui.strengths.show=false *the page > load time is the same of 1.8 version. (both the 1.8 and 5.8 versions are > populated with the same items/communities/collections) > > [solr.log] > 2017-09-22 14:25:57,520 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=3 > 2017-09-22 14:25:58,103 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=5 > 2017-09-22 14:25:58,116 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=3 > 2017-09-22 14:25:58,123 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=2 > 2017-09-22 14:25:58,130 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=2 > 2017-09-22 14:25:58,136 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=2 > 2017-09-22 14:25:58,142 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=2 > 2017-09-22 14:25:58,160 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={q=*:*&fl=dateIssued.year,handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=dateIssued.year:[*+TO+*]&fq=read:(g0+OR+g0)&sort=dateIssued.year_sort+asc&rows=1&wt=javabin&version=2} > hits=157785 status=0 QTime=1 > 2017-09-22 14:25:58,165 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={q=*:*&fl=dateIssued.year,handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=dateIssued.year:[*+TO+*]&fq=read:(g0+OR+g0)&sort=dateIssued.year_sort+desc&rows=1&wt=javabin&version=2} > hits=157785 status=0 QTime=1 > 2017-09-22 14:25:58,172 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={facet.field=author_filter&facet.field=subject_tax_0_filter&facet.field=tipo_filter&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=read:(g0+OR+g0)&f.author_filter.facet.sort=count&rows=0&version=2&f.subject_tax_0_filter.facet.limit=4&facet.query=dateIssued.year:[2000+TO+2017]&facet.query=dateIssued.year:[1900+TO+1999]&facet.query=dateIssued.year:[1800+TO+1899]&facet.query=dateIssued.year:[1700+TO+1799]&facet.query=dateIssued.year:[1600+TO+1699]&facet.query=dateIssued.year:[1500+TO+1599]&facet.query=dateIssued.year:[1400+TO+1499]&facet.query=dateIssued.year:[1300+TO+1399]&facet.query=dateIssued.year:[1200+TO+1299]&facet.query=dateIssued.year:[1100+TO+1199]&facet.query=dateIssued.year:[1017+TO+1099]&q=*:*&f.subject_tax_0_filter.facet.sort=count&f.tipo_filter.facet.limit=4&f.tipo_filter.facet.sort=count&facet.mincount=1&facet=true&wt=javabin&facet.offset=0&f.author_filter.facet.limit=6} > hits=159010 status=0 QTime=3 > 2017-09-22 14:25:58,176 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={q=*:*&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=search.resourcetype:2&fq=read:(g0+OR+g0)&sort=dc.date.issued_dt+desc&rows=10&wt=javabin&version=2} > hits=159010 status=0 QTime=1 > 2017-09-22 14:25:58,271 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=2 > 2017-09-22 14:25:58,281 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=2 > 2017-09-22 14:25:58,291 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=2 > 2017-09-22 14:25:58,301 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=3 > 2017-09-22 14:25:58,311 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=2 > 2017-09-22 14:25:58,323 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=3 > 2017-09-22 14:25:58,496 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=5 > 2017-09-22 14:25:58,508 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=3 > 2017-09-22 14:25:58,520 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=2 > 2017-09-22 14:25:58,530 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=2 > 2017-09-22 14:25:58,549 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=2 > 2017-09-22 14:25:58,566 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=3 > 2017-09-22 14:25:58,648 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=2 > 2017-09-22 14:25:58,658 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=3 > 2017-09-22 14:25:58,666 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=2 > 2017-09-22 14:25:58,682 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=4 > 2017-09-22 14:25:58,695 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=4 > 2017-09-22 14:25:58,710 INFO org.apache.solr.core.SolrCore @ [search] > webapp=/solr path=/select > params={f.location.coll.facet.sort=count&facet.field=location.comm&facet.field=location.coll&fl=handle,search.resourcetype,search.resourceid&start=0&fq=NOT(withdrawn:true)&fq=NOT(discoverable:false)&fq=search.resourcetype:2&fq=NOT(discoverable:false)&fq=read:(g0+OR+g0)&rows=0&version=2&q=*:*&f.location.coll.facet.limit=-1&facet.mincount=1&facet=true&f.location.comm.facet.sort=count&wt=javabin&facet.offset=0&f.location.comm.facet.limit=-1} > hits=159010 status=0 QTime=4 > > Are these queries cached in some way (it seems that they aren't)? > Is there a way to cache the item counter value of communities/collections? > How can be possible that in 5.8 version the item counter takes more time > than in the 1.8 version? > > We want to upgrade to DSpace 5.8 version without performance loses and > showing the item counter information. > > Please, help us. > > Manuela Klanovicz Ferreira > System Analyst > UFRGS - Federal University of Rio Grande do Sul / Brazil > > > -- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/dspace-tech. > For more options, visit https://groups.google.com/d/optout. > -- Tim Donohue Technical Lead for DSpace & DSpaceDirect DuraSpace.org | DSpace.org | DSpaceDirect.org -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
