Re: [dspace-tech] Re: DSpace 6.2 with collection home page too slow to load if items have lots of bitsteams

[kardeiz]

Hi Bill,

Thanks for your note. A quick follow up question:

On the discovery pages you tested, do most of the search result items have 
1 or 2 or 3 bitstreams (e.g., an ORIGINAL file and a PREVIEW and a 
THUMBNAIL), or do they have many?

I think my issue is that many of my search result items have dozens, 
sometimes hundreds, of bitstreams associated with them, and the discovery 
page is loading metadata individually for each of these bitstreams (which 
for a search results page with 20 items could easily be many thousands).

Thanks,

Jacob



On Thursday, November 1, 2018 at 9:40:11 AM UTC-5, Bill T wrote:
>
> In my case, I see no performance problem on discovery pages.  It is 
> only on pages with a large number of records, for instance 
> community-list or on community pages with a large number of 
> sub-communities and collections. 
>
> For me, it is most apparent for /community-list, which takes a couple 
> minutes to display. 
> -- Bill 
> On Wed, Oct 31, 2018 at 3:53 PM > wrote: 
> > 
> > Hi all, 
> > 
> > As Ying suggests in his second post here, this seems to affect search 
> (i.e. /discover) pages as well. The fix mentioned by Tim (
> https://github.com/DSpace/DSpace/pull/2016) doesn't seem to affect search 
> pages. 
> > 
> > I've encountered this on both 5.9 and 6.3. I created another post (
> https://groups.google.com/forum/#!topic/dspace-tech/PbKbDfQlhok) before I 
> saw this one. 
> > 
> > Jacob 
> > 
> > On Wednesday, February 21, 2018 at 4:02:06 PM UTC-6, Ying Jin wrote: 
> >> 
> >> Dear All, 
> >> 
> >> We are experiencing a performance issue with DSpace 6.2. Some of our 
> collections will time out/take several minutes to load. They are not big 
> collections, one of them only have 20 items, but each item contains 100+ 
> bitstreams (one PDF in Original bundle and 100+ of JP2 files in customized 
> MASTER bundle which are hidden from end users). The postgresql left so many 
> "idle in transaction" processes that will slow down the overall site. The 
> tomcat begin to take most of the CPU time too. 
> >> 
> >> We used to use 16G of memory for tomcat under v5.x, and the performance 
> has been ok. Now, even I increased to 64G(half of our server memory) under 
> 6.2, the performance didn't improve. 
> >> 
> >> We are using Tomcat 8.0.13, Java 1.8, Redhat Linux 6.7, and XMLUI. We 
> have about 80+ communities, 280+ collections, 60,000+ items and 490,000 
> bitstreams. 
> >> 
> >> To determine the cause of the slowness, I duplicate the same collection 
> with two copies. In one collection, I removed all MASTER files and leave 
> PDF only. The other collection, I zipped all MASTER files as one zip file 
> and uploaded with PDF. It turns out they all have no performance issues. 
> Seems like the number of bitstreams will affect the performance. 
> >> 
> >> After turning on the debugging, I got following information. It is too 
> big to upload the whole log so I just put hibernate stat here. 
> >> 
> >> == 
> >> The collection has 100+ bitstreams in it 
> >> == 
> >> 
> >> 2018-02-07 00:43:17,724 DEBUG 
> org.hibernate.stat.internal.ConcurrentStatisticsImpl @ HHH000117: HQL: 
> null, time: 1ms, rows: 1 
> >> 
> >> 2018-02-07 00:43:17,882 INFO 
>  org.hibernate.engine.internal.StatisticalLoggingSessionEventListener @ 
> Session Metrics { 
> >> 
> >> 237746 nanoseconds spent acquiring 1 JDBC connections; 
> >> 
> >> 0 nanoseconds spent releasing 0 JDBC connections; 
> >> 
> >> 552698751 nanoseconds spent preparing 48017 JDBC statements; 
> >> 
> >> 12590561333 nanoseconds spent executing 48017 JDBC statements; 
> >> 
> >> 0 nanoseconds spent executing 0 JDBC batches; 
> >> 
> >> 929992 nanoseconds spent performing 52 L2C puts; 
> >> 
> >> 188492 nanoseconds spent performing 10 L2C hits; 
> >> 
> >> 1935100 nanoseconds spent performing 42 L2C misses; 
> >> 
> >> 133422494 nanoseconds spent executing 2 flushes (flushing a total 
> of 43868 entities and 58348 collections); 
> >> 
> >> 562373235433 nanoseconds spent executing 20136 partial-flushes 
> (flushing a total of 235915143 entities and 235915143 collections) 
> >> 
> >> } 
> >> 
> >> 2018-02-07 00:43:17,884 INFO 
>  org.hibernate.engine.internal.StatisticalLoggingSessionEventListener @ 
> Session Metrics { 
> >> 
> >> 0 nanoseconds spent acquiring 0 JDBC connections; 
> >> 
> >> 0 nanoseconds spent releasing 0 JDBC connections; 
> >> 
> >> 0 nanoseconds spent preparing 0 JDBC statements; 
> >> 
> >> 0 nanoseconds spent executing 0 JDBC statements; 
> >> 
> >> 0 nanoseconds spent executing 0 JDBC batches; 
> >> 
> >> 0 nanoseconds spent performing 0 L2C puts; 
> >> 
> >> 0 nanoseconds spent performing 0 L2C hits; 
> >> 
> >> 0 nanoseconds spent performing 0 L2C misses; 
> >> 
> >> 0 nanoseconds spent executing 0 flushes (flushing a total of 0 
> entities and 0 collections); 
> >> 
> >> 0 nanoseconds spent executing 0 partial-flushes (flushing a total 
> 

Re: [dspace-tech] Re: DSpace 6.2 with collection home page too slow to load if items have lots of bitsteams

[Bill Tantzen]

In my case, I see no performance problem on discovery pages.  It is
only on pages with a large number of records, for instance
community-list or on community pages with a large number of
sub-communities and collections.

For me, it is most apparent for /community-list, which takes a couple
minutes to display.
-- Bill
On Wed, Oct 31, 2018 at 3:53 PM  wrote:
>
> Hi all,
>
> As Ying suggests in his second post here, this seems to affect search (i.e. 
> /discover) pages as well. The fix mentioned by Tim 
> (https://github.com/DSpace/DSpace/pull/2016) doesn't seem to affect search 
> pages.
>
> I've encountered this on both 5.9 and 6.3. I created another post 
> (https://groups.google.com/forum/#!topic/dspace-tech/PbKbDfQlhok) before I 
> saw this one.
>
> Jacob
>
> On Wednesday, February 21, 2018 at 4:02:06 PM UTC-6, Ying Jin wrote:
>>
>> Dear All,
>>
>> We are experiencing a performance issue with DSpace 6.2. Some of our 
>> collections will time out/take several minutes to load. They are not big 
>> collections, one of them only have 20 items, but each item contains 100+ 
>> bitstreams (one PDF in Original bundle and 100+ of JP2 files in customized 
>> MASTER bundle which are hidden from end users). The postgresql left so many 
>> "idle in transaction" processes that will slow down the overall site. The 
>> tomcat begin to take most of the CPU time too.
>>
>> We used to use 16G of memory for tomcat under v5.x, and the performance has 
>> been ok. Now, even I increased to 64G(half of our server memory) under 6.2, 
>> the performance didn't improve.
>>
>> We are using Tomcat 8.0.13, Java 1.8, Redhat Linux 6.7, and XMLUI. We have 
>> about 80+ communities, 280+ collections, 60,000+ items and 490,000 
>> bitstreams.
>>
>> To determine the cause of the slowness, I duplicate the same collection with 
>> two copies. In one collection, I removed all MASTER files and leave PDF 
>> only. The other collection, I zipped all MASTER files as one zip file and 
>> uploaded with PDF. It turns out they all have no performance issues. Seems 
>> like the number of bitstreams will affect the performance.
>>
>> After turning on the debugging, I got following information. It is too big 
>> to upload the whole log so I just put hibernate stat here.
>>
>> ==
>> The collection has 100+ bitstreams in it
>> ==
>>
>> 2018-02-07 00:43:17,724 DEBUG 
>> org.hibernate.stat.internal.ConcurrentStatisticsImpl @ HHH000117: HQL: null, 
>> time: 1ms, rows: 1
>>
>> 2018-02-07 00:43:17,882 INFO  
>> org.hibernate.engine.internal.StatisticalLoggingSessionEventListener @ 
>> Session Metrics {
>>
>> 237746 nanoseconds spent acquiring 1 JDBC connections;
>>
>> 0 nanoseconds spent releasing 0 JDBC connections;
>>
>> 552698751 nanoseconds spent preparing 48017 JDBC statements;
>>
>> 12590561333 nanoseconds spent executing 48017 JDBC statements;
>>
>> 0 nanoseconds spent executing 0 JDBC batches;
>>
>> 929992 nanoseconds spent performing 52 L2C puts;
>>
>> 188492 nanoseconds spent performing 10 L2C hits;
>>
>> 1935100 nanoseconds spent performing 42 L2C misses;
>>
>> 133422494 nanoseconds spent executing 2 flushes (flushing a total of 
>> 43868 entities and 58348 collections);
>>
>> 562373235433 nanoseconds spent executing 20136 partial-flushes (flushing 
>> a total of 235915143 entities and 235915143 collections)
>>
>> }
>>
>> 2018-02-07 00:43:17,884 INFO  
>> org.hibernate.engine.internal.StatisticalLoggingSessionEventListener @ 
>> Session Metrics {
>>
>> 0 nanoseconds spent acquiring 0 JDBC connections;
>>
>> 0 nanoseconds spent releasing 0 JDBC connections;
>>
>> 0 nanoseconds spent preparing 0 JDBC statements;
>>
>> 0 nanoseconds spent executing 0 JDBC statements;
>>
>> 0 nanoseconds spent executing 0 JDBC batches;
>>
>> 0 nanoseconds spent performing 0 L2C puts;
>>
>> 0 nanoseconds spent performing 0 L2C hits;
>>
>> 0 nanoseconds spent performing 0 L2C misses;
>>
>> 0 nanoseconds spent executing 0 flushes (flushing a total of 0 entities 
>> and 0 collections);
>>
>> 0 nanoseconds spent executing 0 partial-flushes (flushing a total of 0 
>> entities and 0 collections)
>>
>> }
>>
>>
>> =
>>
>> The collection has PDF only
>> =
>>
>> 2018-02-17 20:30:27,534 INFO  
>> org.hibernate.engine.internal.StatisticalLoggingSessionEventListener @ 
>> Session Metrics {
>>
>> 341778 nanoseconds spent acquiring 1 JDBC connections;
>>
>> 0 nanoseconds spent releasing 0 JDBC connections;
>>
>> 15974134 nanoseconds spent preparing 1189 JDBC statements;
>>
>> 271327429 nanoseconds spent executing 1189 JDBC statements;
>>
>> 0 nanoseconds spent executing 0 JDBC batches;
>>
>> 717412 nanoseconds spent performing 45 L2C puts;
>>
>> 3906724 nanoseconds spent performing 247 L2C hits;
>>
>> 2081066 nanoseconds spent performing 855 L2C misses;
>>
>> 6829028 nanoseconds spent executing 2 flushes (flushing a total of 3916 
>> entities and 5072 

Re: [dspace-tech] Re: DSpace 6.2 with collection home page too slow to load if items have lots of bitsteams

[Tim Donohue]

Hi Bill,

The issue you are seeing doesn't sound to be the same as the original issue
in this thread.  The behavior that was fixed in 6.3 was a behavior where
the *Collection homepage* loaded extremely slowly if you had Items that had
large number of bitstreams/thumbnails.  The fix was to be much smarter
about when to load Thumbnails/bitstreams into memory (and it was a fix to
the Java code under the XMLUI, so it applied to all themes):
https://github.com/DSpace/DSpace/pull/2016

The behavior you seem to be describing sounds different... you are talking
about the /community-list page with a large number of
collections/communities, and a Community page with a large number of
sub-communities/collections.  So, I'm not surprised that this specific 6.3
optimization had no effect -- as it was only for Collection homepages.

I cannot say for certain what is behind the behavior you are seeing -- it's
very possible that it's semi-related in that DSpace may be loading too many
objects into memory at once (which slows down the application, especially
with the addition of Hibernate in 6.x).  But, in your situation, it sounds
like maybe DSpace is loading too many Communities/Collections at once
(whereas the bug fixed was about ensuring DSpace didn't load too many
bitstreams into memory at once)

All in all, I think we may need to create a new ticket & begin a new
investigation into why this other performance behavior is occurring.
Hopefully it's something we can track down and fix in a 6.4 release.

- Tim

On Fri, Sep 28, 2018 at 10:11 AM Bill Tantzen  wrote:

> I'm very tardy jumping back into this party, but here's an update:  I just
> upgraded my dev site to version 6.3, and the speed problems persist.  I
> have made sure to optimize the mets requests in my theme, but that's
> actually a moot point, because the slowness is equally apparent even in the
> Default Reference Theme and the Atmire Mirage2 Theme (which I assume has
> been optimized, in v6.3, correct?).
>
> For /community-list with ~1500 collections + communities and 62K items,
> the page load time is minutes.  For our main community page, which contains
> the bulk of those sub-communities and collections (maybe 90%), the page
> load time is about 15 seconds.  That's still too slow to put into
> production, but makes me wonder why the difference in speed is so vast
> compared with the number of children in each case.
>
> Also, in one respect, the performance is even worse now.  If you dig up an
> earlier thread that I started, you will remember that the speed is just
> fine when logged in as an admin.  This is no longer the case, as there is
> no difference in speed between admin and anonymous users.
>
> Naturally, this is causing us to sit on v5.x for now, but since the rest
> half of v7 is based on v6, we need to figure this out!
>
> Cheers,
> Bill
>
> On Thu, Apr 12, 2018 at 1:02 AM Ying Jin  wrote:
>
>> Hi Tim,
>>
>> I have the fileSec updated so it only retrieve THUMBNAILs and it
>> significantly improved the page loading. Though some of our static
>> thumbnails broken, I am sure I can find another way to fix that.
>>
>> Cheers,
>> Ying
>>
>> On Wednesday, April 11, 2018 at 10:19:10 AM UTC-5, Tim Donohue wrote:
>>>
>>> Hi Ying,
>>>
>>> Looking forward to hearing what you find out.
>>>
>>> In the meantime, I'd like to better understand this statement:
>>>
>>> "For the thumbnail part, it is still a problem for us as I mentioned
>>> earlier, we have special theme needs to load all the thumbnails for the
>>> Gallery theme and TEI theme."
>>>
>>> Could you describe how you are using/displaying thumbnails in the
>>> Gallery theme and TEI theme? Are you displaying multiple thumbnails *per
>>> item* on the Community/Collection homepages (e.g. in Recent Submissions) in
>>> these themes?  If so, is there a limit to the number of thumbnails you
>>> display per Item (or a limit to the number of thumbnails that tend to exist
>>> per item)? Or is each item still represented by a single thumbnail on the
>>> Community/Collection homepage?  Do you have links you could share with us
>>> regarding these themes, so that we can better understand the use case
>>> here?
>>>
>>> Obviously, we don't want to include changes in DSpace 6.3 that would
>>> improve performance but also break specific theme use cases.
>>>
>>> Thanks
>>>
>>> - Tim
>>>
>>> On Wed, Apr 11, 2018 at 10:06 AM Ying Jin  wrote:
>>>
 Hi Tim,

 Thanks very much for the detailed information and great suggestion! It
 makes sense to not load original bundles or any other bundles that are not
 related to the page view.

 Now I remembered I customized the "sections=
 dmdSec,fileSec=THUMBNAIL" part long time ago for some
 customizations as we need more info from mets.xml than what it is provided
 by default. I need to go and check what I have changed and report back!

 For the thumbnail part, it is still a problem for us as I mentioned
 earlier, we have special 

Re: [dspace-tech] Re: DSpace 6.2 with collection home page too slow to load if items have lots of bitsteams

[Tim Donohue]

Hi Ying,

Looking forward to hearing what you find out.

In the meantime, I'd like to better understand this statement:

"For the thumbnail part, it is still a problem for us as I mentioned
earlier, we have special theme needs to load all the thumbnails for the
Gallery theme and TEI theme."

Could you describe how you are using/displaying thumbnails in the Gallery
theme and TEI theme? Are you displaying multiple thumbnails *per item* on
the Community/Collection homepages (e.g. in Recent Submissions) in these
themes?  If so, is there a limit to the number of thumbnails you display
per Item (or a limit to the number of thumbnails that tend to exist per
item)? Or is each item still represented by a single thumbnail on the
Community/Collection homepage?  Do you have links you could share with us
regarding these themes, so that we can better understand the use case
here?

Obviously, we don't want to include changes in DSpace 6.3 that would
improve performance but also break specific theme use cases.

Thanks

- Tim

On Wed, Apr 11, 2018 at 10:06 AM Ying Jin  wrote:

> Hi Tim,
>
> Thanks very much for the detailed information and great suggestion! It
> makes sense to not load original bundles or any other bundles that are not
> related to the page view.
>
> Now I remembered I customized the "sections=dmdSec,fileSec=
> THUMBNAIL" part long time ago for some customizations as we need more
> info from mets.xml than what it is provided by default. I need to go and
> check what I have changed and report back!
>
> For the thumbnail part, it is still a problem for us as I mentioned
> earlier, we have special theme needs to load all the thumbnails for the
> Gallery theme and TEI theme.
>
> I posted a code update on ticket about the READ_ONLY mode. For read-only
> transactions, we should set Hibernate flush mode to MANUAL. However, DSpace
> missed several places to change the default AUTO to MANUAL, that's why we
> got lots of partial-flushing in the log. I will post more findings on the
> tickets.
>
> Best,
> Ying
>
>
>
> On Wednesday, April 11, 2018 at 9:13:10 AM UTC-5, Tim Donohue wrote:
>
>> Hi Ying (I'm copying back in DSpace-Tech, as I'd like this discussion to
>> stay on-list),
>>
>> Looking back at this thread, I see you mentioned the custom MASTER bundle
>> earlier as well. I had forgotten that.
>>
>> The core issue (that I resolved in
>> https://github.com/DSpace/DSpace/pull/2016) is actually not directly
>> related to Hibernate. The issue we discovered is the XMLUI themes/aspects
>> are loading and parsing bundles that *are unnecessary to display the
>> Collection homepage*.   So, in PR#2016, I resolved this by ensuring that,
>> on the Collection/Community homepages, the XMLUI themes/aspects never load
>> the ORIGINAL bundle content (and only load up the first bitstream in the
>> THUMBNAIL bundle).
>>
>> Since this didn't improve your performance, I'd venture to guess that the
>> XMLUI theme you are using is either purposefully or accidentally loading up
>> you custom MASTER bundle (and looping through all the objects in that
>> bundle).  Assuming you don't need any information about the MASTER
>> bitstreams on the Collection homepage, the fix would be to ensure your
>> theme is *only* loading/using the THUMBNAIL bundle (as likely your
>> Collection homepages will want to display thumbnails for items that have
>> them).
>>
>> This is usually controlled at the *theme* level in the XMLUI.
>>
>> For example, our demo.dspace.org site uses the Mirage 2 theme.  The
>> Mirage 2 theme specifically only loads the THUMBNAIL bundle for items on
>> the Collection homepage.  You can see this by viewing the HTML source of a
>> Collection's homepage, e.g. http://demo.dspace.org/xmlui/handle/10673/2
>>
>> In the HTML source, where each Item is displayed (in the Recent
>> Submissions section) you'll see an HTML comment that says:
>> 
>>
>>
>> This comment shows where the metadata was gathered for this Item.  In
>> fact, you can paste that URL into your browser to see the Item's metadata
>> (in METS format):
>>
>> http://demo.dspace.org/xmlui/metadata/handle/10673/80/mets.xml?sections=dmdSec,fileSec=THUMBNAIL
>>
>> The key point here is that URL includes a parameter
>> "fileGrpTypes=THUMBNAIL".  That parameter restricts the theme to ONLY using
>> the THUMBNAIL bundle of Items when creating the Recent Submissions lis on a
>> Collection/Community homepage.
>>
>> My guess here is that your local theme may NOT include this key
>> optimization.  If instead, the URL were to just point at the "mets.xml",
>> then by default your theme would load ALL bitstreams in ALL bundles for ALL
>> items displayed in that Recent Submissions listing. As you probably can
>> guess, that would take a VERY LONG time if you have hundreds of
>> bitstreams...and that time is wasted unless your theme actually needs to
>> display something from those bitstreams. The reason this behavior occurs in
>> the XMLUI is that the XMLUI itself 

Re: [dspace-tech] Re: DSpace 6.2 with collection home page too slow to load if items have lots of bitsteams

[Ying Jin]

Hi Tim,

Thanks very much for the detailed information and great suggestion! It 
makes sense to not load original bundles or any other bundles that are not 
related to the page view. 

Now I remembered I customized the "sections=dmdSec,fileSec=
THUMBNAIL" part long time ago for some customizations as we need more info 
from mets.xml than what it is provided by default. I need to go and check 
what I have changed and report back!

For the thumbnail part, it is still a problem for us as I mentioned 
earlier, we have special theme needs to load all the thumbnails for the 
Gallery theme and TEI theme.

I posted a code update on ticket about the READ_ONLY mode. For read-only 
transactions, we should set Hibernate flush mode to MANUAL. However, DSpace 
missed several places to change the default AUTO to MANUAL, that's why we 
got lots of partial-flushing in the log. I will post more findings on the 
tickets.

Best,
Ying



On Wednesday, April 11, 2018 at 9:13:10 AM UTC-5, Tim Donohue wrote:
>
> Hi Ying (I'm copying back in DSpace-Tech, as I'd like this discussion to 
> stay on-list),
>
> Looking back at this thread, I see you mentioned the custom MASTER bundle 
> earlier as well. I had forgotten that.
>
> The core issue (that I resolved in 
> https://github.com/DSpace/DSpace/pull/2016) is actually not directly 
> related to Hibernate. The issue we discovered is the XMLUI themes/aspects 
> are loading and parsing bundles that *are unnecessary to display the 
> Collection homepage*.   So, in PR#2016, I resolved this by ensuring that, 
> on the Collection/Community homepages, the XMLUI themes/aspects never load 
> the ORIGINAL bundle content (and only load up the first bitstream in the 
> THUMBNAIL bundle).
>
> Since this didn't improve your performance, I'd venture to guess that the 
> XMLUI theme you are using is either purposefully or accidentally loading up 
> you custom MASTER bundle (and looping through all the objects in that 
> bundle).  Assuming you don't need any information about the MASTER 
> bitstreams on the Collection homepage, the fix would be to ensure your 
> theme is *only* loading/using the THUMBNAIL bundle (as likely your 
> Collection homepages will want to display thumbnails for items that have 
> them).
>
> This is usually controlled at the *theme* level in the XMLUI.  
>
> For example, our demo.dspace.org site uses the Mirage 2 theme.  The 
> Mirage 2 theme specifically only loads the THUMBNAIL bundle for items on 
> the Collection homepage.  You can see this by viewing the HTML source of a 
> Collection's homepage, e.g. http://demo.dspace.org/xmlui/handle/10673/2
>
> In the HTML source, where each Item is displayed (in the Recent 
> Submissions section) you'll see an HTML comment that says:
> 
>  
>
> This comment shows where the metadata was gathered for this Item.  In 
> fact, you can paste that URL into your browser to see the Item's metadata 
> (in METS format): 
>
> http://demo.dspace.org/xmlui/metadata/handle/10673/80/mets.xml?sections=dmdSec,fileSec=THUMBNAIL
>
> The key point here is that URL includes a parameter 
> "fileGrpTypes=THUMBNAIL".  That parameter restricts the theme to ONLY using 
> the THUMBNAIL bundle of Items when creating the Recent Submissions lis on a 
> Collection/Community homepage.
>
> My guess here is that your local theme may NOT include this key 
> optimization.  If instead, the URL were to just point at the "mets.xml", 
> then by default your theme would load ALL bitstreams in ALL bundles for ALL 
> items displayed in that Recent Submissions listing. As you probably can 
> guess, that would take a VERY LONG time if you have hundreds of 
> bitstreams...and that time is wasted unless your theme actually needs to 
> display something from those bitstreams. The reason this behavior occurs in 
> the XMLUI is that the XMLUI itself doesn't know which "data" the theme 
> requires. If the theme just asks for the "mets.xml", then the XMLUI will 
> assume the theme needs *all information about the item* (including all 
> bitstreams/bundles).  So, it's up to the theme to properly tell the XMLUI 
> which data to load.
>
> In the Mirage2 theme, this setting can be found here: 
> https://github.com/DSpace/DSpace/blob/dspace-6_x/dspace-xmlui-mirage2/src/main/webapp/xsl/aspect/artifactbrowser/common.xsl#L214
>  
>
> In the Mirage theme, this setting can be found here (in the dri2xhtml-alt 
> base theme): 
> https://github.com/DSpace/DSpace/blob/dspace-6_x/dspace-xmlui/src/main/webapp/themes/dri2xhtml-alt/aspect/artifactbrowser/common.xsl#L157
>
> If your theme is based on the original dri2xhtml.xsl base theme (which is 
> less likely), you'd find that setting here: 
> https://github.com/DSpace/DSpace/blob/dspace-6_x/dspace-xmlui/src/main/webapp/themes/dri2xhtml/structural.xsl#L3600
>  
>
> Notice how in all of these themes, the theme always makes sure to append 
> "fileGrpTypes=THUMBNAIL" when in the "summaryList" mode (this mode is the 
> one used for Recent Submissions 

Re: [dspace-tech] Re: DSpace 6.2 with collection home page too slow to load if items have lots of bitsteams

[Ying Jin]

Hi Tom,

Sorry for late reply. Somehow I missed your posting and just found it by
doing a search.

"would it be possible for any of you to provide an anonymised and cleaned
database that you can share with us? Make sure the database does not
contain any sensitive information. I'm sure such a database will also be
useful for other (demo and testing) purposes and the development of DSpace
7."

 Yes, I would love to have an anonymised and cleaned database to share
with you. I can't include everything we have, however, it would include
several collections that we are experiencing the performance problems. I am
wondering if it is easier sending them out as simple archive format and you
can ingest into whatever database you have?


"for the Collection and Community pages can you try setting the context
mode to READ_ONLY in the getValidity(), addPageMeta() and addBody() methods
of classes org.dspace.app.xmlui.aspect.artifactbrowser.CollectionViewer and
org.dspace.app.xmlui.aspect.artifactbrowser.CommunityViewer. Here's an
example of how to do that: https://github.com/DSpace/DSpace/pull/1694/files#
diff-fb7249e19a6a8652860795d8fd59cd9e"

 I will give it a try and see if it is indeed make any difference of
our problem. Although I think the ultimate solution might be to not loading
unnecessary objects (in our case, it may not necessary to load all the
bitstreams for a collection home page; )

"unfortunately the class responsible for the /community-list page already
uses the READ_ONLY mode (https://github.com/dspace/
DSpace/blob/dspace-6_x/dspace-xmlui/src/main/java/org/
dspace/app/xmlui/aspect/artifactbrowser/CommunityBrowser.java#L245). But
the fact that your number of (partial-)flushes is still so high, I wonder
if there is still another place hibernate is loading so many objects.
However as a workaround, you could enable caching of that /community-list
page by setting the xmlui.community-list.cache property in dspace.cfg to
true (https://github.com/DSpace/DSpace/blob/dspace-6_x/dspace/
config/dspace.cfg#L1917)."

 Is there any configuration for the cache of collection page? I thought
they are enabled by default but from the page loading, it doesn't seem like
the pages are cached ...

Best,
Ying


On Sun, Mar 11, 2018 at 5:39 PM, Tom Desair  wrote:

> Thanks all, this is really useful information!
>
> Since in both cases most time is lost on executing "flushes" or
> "partial-flushes", I think the problem is similar to
> https://jira.duraspace.org/browse/DS-3552: In AUTO mode
> 
> Hibernate keeps track of all objects loaded from the database and tries to
> "auto-save" them in the database before executing a new query. But the more
> objects Hibernate loads into its session, the more objects it needs to
> check for modification and possible interference with the new query that is
> being executed. Hence the more time it takes.
>
> *@Bill and @Ying*, would it be possible for any of you to provide an
> anonymised and cleaned database that you can share with us? Make sure the
> database does not contain any sensitive information. I'm sure such a
> database will also be useful for other (demo and testing) purposes and the
> development of DSpace 7.
>
> *@Ying*, for the Collection and Community pages can you try setting the
> context mode to READ_ONLY in the getValidity(), addPageMeta() and addBody()
> methods of classes 
> org.dspace.app.xmlui.aspect.artifactbrowser.CollectionViewer
> and org.dspace.app.xmlui.aspect.artifactbrowser.CommunityViewer. Here's
> an example of how to do that: https://github.com/
> DSpace/DSpace/pull/1694/files#diff-fb7249e19a6a8652860795d8fd59cd9e
>
> *@Bill*, unfortunately the class responsible for the /community-list page
> already uses the READ_ONLY mode (https://github.com/dspace/
> DSpace/blob/dspace-6_x/dspace-xmlui/src/main/java/org/
> dspace/app/xmlui/aspect/artifactbrowser/CommunityBrowser.java#L245). But
> the fact that your number of (partial-)flushes is still so high, I wonder
> if there is still another place hibernate is loading so many objects.
> However as a workaround, you could enable caching of that /community-list
> page by setting the xmlui.community-list.cache property in dspace.cfg to
> true (https://github.com/DSpace/DSpace/blob/dspace-6_x/dspace/
> config/dspace.cfg#L1917).
>
>
> Best regards,
> Tom
>
> [image: logo] Tom Desair
> 250-B Suite 3A, Lucius Gordon Drive, West Henrietta, NY 14586
> 
> Gaston Geenslaan 14, Leuven 3001, Belgium
> 
> www.atmire.com
> 
>
> 2018-03-11 22:43 GMT+01:00 Kim Shepherd :
>
>> Hi Bill and Ying, just a note to say I haven't managed to reproduce /
>> 

Re: [dspace-tech] Re: DSpace 6.2 with collection home page too slow to load if items have lots of bitsteams

[Tom Desair]

Thanks all, this is really useful information!

Since in both cases most time is lost on executing "flushes" or
"partial-flushes", I think the problem is similar to
https://jira.duraspace.org/browse/DS-3552: In AUTO mode

Hibernate keeps track of all objects loaded from the database and tries to
"auto-save" them in the database before executing a new query. But the more
objects Hibernate loads into its session, the more objects it needs to
check for modification and possible interference with the new query that is
being executed. Hence the more time it takes.

*@Bill and @Ying*, would it be possible for any of you to provide an
anonymised and cleaned database that you can share with us? Make sure the
database does not contain any sensitive information. I'm sure such a
database will also be useful for other (demo and testing) purposes and the
development of DSpace 7.

*@Ying*, for the Collection and Community pages can you try setting the
context mode to READ_ONLY in the getValidity(), addPageMeta() and addBody()
methods of classes
org.dspace.app.xmlui.aspect.artifactbrowser.CollectionViewer and
org.dspace.app.xmlui.aspect.artifactbrowser.CommunityViewer. Here's an
example of how to do that:
https://github.com/DSpace/DSpace/pull/1694/files#diff-fb7249e19a6a8652860795d8fd59cd9e

*@Bill*, unfortunately the class responsible for the /community-list page
already uses the READ_ONLY mode (
https://github.com/dspace/DSpace/blob/dspace-6_x/dspace-xmlui/src/main/java/org/dspace/app/xmlui/aspect/artifactbrowser/CommunityBrowser.java#L245).
But the fact that your number of (partial-)flushes is still so high, I
wonder if there is still another place hibernate is loading so many
objects. However as a workaround, you could enable caching of that
/community-list page by setting the xmlui.community-list.cache property in
dspace.cfg to true (
https://github.com/DSpace/DSpace/blob/dspace-6_x/dspace/config/dspace.cfg#L1917
).


Best regards,
Tom

[image: logo] Tom Desair
250-B Suite 3A, Lucius Gordon Drive, West Henrietta, NY 14586
Gaston Geenslaan 14, Leuven 3001, Belgium
www.atmire.com


2018-03-11 22:43 GMT+01:00 Kim Shepherd :

> Hi Bill and Ying, just a note to say I haven't managed to reproduce /
> investigate this problem thoroughly yet -- i keep getting sidetracked with
> other work! And I'm struggling a bit to reproduce with test data now that
> Bill is reporting the same problem with a fairly small / typical amount of
> items and bitstreams, so it's obviously not just connected to number of
> bitstreams attached to a single item.
>
> However I still have this on my radar, and if you discover anything else
> please let the list know! Once we know exactly which conditions are needed
> to reproduce this / identify where things are going wrong, exactly, we'll
> log a JIRA issue so we can get a fix in for a future DSpace version
>
> Cheers!
>
> Kim
>
> M: k...@shepherd.nz
> T: @kimshepherd
> P: +6421883635 <+64%2021%20883%20635>
> W: www.shepherd.nz 
>
> 0CCB D957 0C35 F5C1 497E CDCF FC4B ABA3 2A1A FAEC
>
> On 23 February 2018 at 09:33, Bill T  wrote:
>
>> Here is my hibernate log for /community-list (page takes about 90 seconds
>> to load);  We have about 200 communities, 1200 collections, and 64,000
>> items.  Unlike Ying, we essentially have one original bitstream per item,
>> primarily pdf files.  Redhat 7, Java 8, Tomcat 8.5, Postgresql 9.6.2;  32G
>> Ram, 8 CPU  This setup worked well with previous versions up to and
>> including 5.8
>>
>> I'm not trying to pile on, but hopefully some additional info will be
>> helpful!
>>
>>
>> 2018-02-22 13:52:05,871 INFO  org.hibernate.engine.internal
>> .StatisticalLoggingSessionEventListener @ Session Metrics {
>> 372441 nanoseconds spent acquiring 1 JDBC connections;
>> 0 nanoseconds spent releasing 0 JDBC connections;
>> 233248 nanoseconds spent preparing 3 JDBC statements;
>> 482887 nanoseconds spent executing 1 JDBC statements;
>> 1985778 nanoseconds spent executing 2 JDBC batches;
>> 3867293 nanoseconds spent performing 3 L2C puts;
>> 0 nanoseconds spent performing 0 L2C hits;
>> 0 nanoseconds spent performing 0 L2C misses;
>> 29762475 nanoseconds spent executing 2 flushes (flushing a total of 2
>> entities and 0 collections);
>> 0 nanoseconds spent executing 0 partial-flushes (flushing a total of
>> 0 entities and 0 collections)
>> }
>>
>> 2018-02-22 13:52:05,873 INFO  org.hibernate.engine.internal
>> .StatisticalLoggingSessionEventListener @ Session Metrics {
>> 0 nanoseconds spent acquiring 0 JDBC connections;
>> 0 nanoseconds spent releasing 0 JDBC connections;
>> 0 nanoseconds spent preparing 0 JDBC statements;
>> 0 nanoseconds spent executing 0 JDBC statements;
>> 0 

Re: [dspace-tech] Re: DSpace 6.2 with collection home page too slow to load if items have lots of bitsteams

[Kim Shepherd]

Hi Bill and Ying, just a note to say I haven't managed to reproduce /
investigate this problem thoroughly yet -- i keep getting sidetracked with
other work! And I'm struggling a bit to reproduce with test data now that
Bill is reporting the same problem with a fairly small / typical amount of
items and bitstreams, so it's obviously not just connected to number of
bitstreams attached to a single item.

However I still have this on my radar, and if you discover anything else
please let the list know! Once we know exactly which conditions are needed
to reproduce this / identify where things are going wrong, exactly, we'll
log a JIRA issue so we can get a fix in for a future DSpace version

Cheers!

Kim

M: k...@shepherd.nz
T: @kimshepherd
P: +6421883635
W: www.shepherd.nz 

0CCB D957 0C35 F5C1 497E CDCF FC4B ABA3 2A1A FAEC

On 23 February 2018 at 09:33, Bill T  wrote:

> Here is my hibernate log for /community-list (page takes about 90 seconds
> to load);  We have about 200 communities, 1200 collections, and 64,000
> items.  Unlike Ying, we essentially have one original bitstream per item,
> primarily pdf files.  Redhat 7, Java 8, Tomcat 8.5, Postgresql 9.6.2;  32G
> Ram, 8 CPU  This setup worked well with previous versions up to and
> including 5.8
>
> I'm not trying to pile on, but hopefully some additional info will be
> helpful!
>
>
> 2018-02-22 13:52:05,871 INFO  org.hibernate.engine.internal.
> StatisticalLoggingSessionEventListener @ Session Metrics {
> 372441 nanoseconds spent acquiring 1 JDBC connections;
> 0 nanoseconds spent releasing 0 JDBC connections;
> 233248 nanoseconds spent preparing 3 JDBC statements;
> 482887 nanoseconds spent executing 1 JDBC statements;
> 1985778 nanoseconds spent executing 2 JDBC batches;
> 3867293 nanoseconds spent performing 3 L2C puts;
> 0 nanoseconds spent performing 0 L2C hits;
> 0 nanoseconds spent performing 0 L2C misses;
> 29762475 nanoseconds spent executing 2 flushes (flushing a total of 2
> entities and 0 collections);
> 0 nanoseconds spent executing 0 partial-flushes (flushing a total of 0
> entities and 0 collections)
> }
>
> 2018-02-22 13:52:05,873 INFO  org.hibernate.engine.internal.
> StatisticalLoggingSessionEventListener @ Session Metrics {
> 0 nanoseconds spent acquiring 0 JDBC connections;
> 0 nanoseconds spent releasing 0 JDBC connections;
> 0 nanoseconds spent preparing 0 JDBC statements;
> 0 nanoseconds spent executing 0 JDBC statements;
> 0 nanoseconds spent executing 0 JDBC batches;
> 0 nanoseconds spent performing 0 L2C puts;
> 0 nanoseconds spent performing 0 L2C hits;
> 0 nanoseconds spent performing 0 L2C misses;
> 0 nanoseconds spent executing 0 flushes (flushing a total of 0
> entities and 0 collections);
> 0 nanoseconds spent executing 0 partial-flushes (flushing a total of 0
> entities and 0 collections)
> }
>
> 2018-02-22 13:52:16,450 INFO  org.hibernate.engine.internal.
> StatisticalLoggingSessionEventListener @ Session Metrics {
> 440048 nanoseconds spent acquiring 1 JDBC connections;
> 0 nanoseconds spent releasing 0 JDBC connections;
> 273468 nanoseconds spent preparing 3 JDBC statements;
> 547508 nanoseconds spent executing 1 JDBC statements;
> 1702327 nanoseconds spent executing 2 JDBC batches;
> 3683682 nanoseconds spent performing 3 L2C puts;
> 0 nanoseconds spent performing 0 L2C hits;
> 0 nanoseconds spent performing 0 L2C misses;
> 27298233 nanoseconds spent executing 2 flushes (flushing a total of 2
> entities and 0 collections);
> 0 nanoseconds spent executing 0 partial-flushes (flushing a total of 0
> entities and 0 collections)
> }
>
> 2018-02-22 13:52:16,452 INFO  org.hibernate.engine.internal.
> StatisticalLoggingSessionEventListener @ Session Metrics {
> 0 nanoseconds spent acquiring 0 JDBC connections;
> 0 nanoseconds spent releasing 0 JDBC connections;
> 0 nanoseconds spent preparing 0 JDBC statements;
> 0 nanoseconds spent executing 0 JDBC statements;
> 0 nanoseconds spent executing 0 JDBC batches;
> 0 nanoseconds spent performing 0 L2C puts;
> 0 nanoseconds spent performing 0 L2C hits;
> 0 nanoseconds spent performing 0 L2C misses;
> 0 nanoseconds spent executing 0 flushes (flushing a total of 0
> entities and 0 collections);
> 0 nanoseconds spent executing 0 partial-flushes (flushing a total of 0
> entities and 0 collections)
> }
>
> 2018-02-22 13:52:27,121 INFO  org.hibernate.engine.internal.
> StatisticalLoggingSessionEventListener @ Session Metrics {
> 429299 nanoseconds spent acquiring 1 JDBC connections;
> 0 nanoseconds spent releasing 0 JDBC connections;
> 236026 nanoseconds spent preparing 3 JDBC statements;
> 561023 nanoseconds spent executing 1 JDBC statements;
> 1029554 nanoseconds spent executing 2 JDBC batches;
> 3446462 nanoseconds spent performing 3 L2C puts;
> 0 nanoseconds