Re: [dspace-tech] Re: Dspace limitations and low performance, or just wrong implementation?

[Andreea Oprescu]

Hi Tim,
thank you for your time :).
My machine has 5 GB memory, from which we have allocated half of it to
Tomcat.
I have already took a look on that information (Performance Tuning DSpace)
and it does help improve performance, altough its still far from desirable.

Do you think a possible solution would be to upgrade Dspace to 6.3?

El lun., 20 may. 2019 a las 17:10, Tim Donohue ()
escribió:

> Hello Andreea,
>
> If you are still experiencing less than ideal performance, we have some
> notes on improving the performance of DSpace by providing more memory to
> it:  https://wiki.duraspace.org/display/DSDOC6x/Performance+Tuning+DSpace
>  While the *minimum* values there are good for brand new sites, as you site
> grows in size or has a large number of users, you really should be looking
> at allocating several GB of memory for Tomcat alone. Very large or very
> active sites tend to be in the 2-4GB or more range.  If your memory
> allocation is too small, your site performance will suffer, and DSpace will
> become slower and slower.
>
> It's hard to say if this is your exact issue, but it's usually the first
> thing to check when performance is less than ideal.  Others on this mailing
> list may have more specific advice to add here.
>
> Tim
>
>
>
>
> On Mon, May 20, 2019 at 2:35 AM Andreea Oprescu 
> wrote:
>
>> Thank you for your answers. I will check into your proposed solutions.
>> Yes, its dspace 6.2 with xmlui.
>>
>> I've been reading around a bit, but I still wonder why mydspace is so
>> slow. Is it because of bitstreams? My items don't have more than 4-5
>> bitstreams...
>>
>> El viernes, 17 de mayo de 2019, 11:10:00 (UTC+2), Andreea Oprescu
>> escribió:
>>>
>>> Hello everyone,
>>>
>>> I am currently trying to debug a* very slow* dspace implementation. I
>>> am facing the following issues:
>>>
>>> - organization with more than 2.624 collections, in 4 communities, which
>>> at the same time have 5, 5, 52 and 19 subcommunities.
>>> - users and administrator accounts. Users accounts with less permissions
>>> take a lot longer to process than administrator accounts (does dspace check
>>> all the permissions of an user in ALL the collections?)
>>> - there seem to be a lot of SQL queries. For example, for moving an
>>> item, I ended up with almost 6000 SQL queries, which took a little bit more
>>> than 2 minutes to complete (for admin account).
>>>
>>> I think dspace is a wonderful tool which provides my organization a lot
>>> of value, but it's really, really slow.
>>>
>>>
>>> => Is it normal, because of the amount of data?
>>> => Are we implementing something wrong?
>>> => If this is not normal, do you have any guides or advices on how to
>>> procede with the debugging?
>>>
>>>
>>> I have tried to find information about this, but could only find this
>>> post, with the same problem, but no responses:
>>> http://dspace.2283337.n4.nabble.com/Very-slow-quot-Move-item-quot-button-td4687571.html
>>>
>>>
>>> I really appreciate your time.
>>> Regards,
>>>
>>> Andreea.
>>>
>> --
>> All messages to this mailing list should adhere to the DuraSpace Code of
>> Conduct: https://duraspace.org/about/policies/code-of-conduct/
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "DSpace Technical Support" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to dspace-tech+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/dspace-tech/7d8bbdc8-bd92-493f-8768-ce95f1587011%40googlegroups.com
>> 
>> .
>>
>
>
> --
>
> Tim Donohue
> Technical Lead for DSpace & DSpaceDirect
> DuraSpace.org | DSpace.org | DSpaceDirect.org
>
>

-- 
All messages to this mailing list should adhere to the DuraSpace Code of 
Conduct: https://duraspace.org/about/policies/code-of-conduct/
--- 
You received this message because you are subscribed to the Google Groups 
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dspace-tech+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dspace-tech/CACQbyXH%3DTKYsNhXQ0J4ZDrb4jaQ7zPJ3MCDJoM0U7W8idv%3DeUw%40mail.gmail.com.

Re: [dspace-tech] Re: Dspace limitations and low performance, or just wrong implementation?

[Shaun donovan]

Hi Andreea.

When tracking slow performance, you need to try to discover (on the 
machine level) what is causing the problem. A tool that I find very 
useful for this is atop. Running this gives a very good overview of what 
is happening on your machine. The main things to look for is high 
processor utilization (not very common), high hard disk activity (I find 
this very often with virtual machines) and any swapping.


If you machine is swapping, then you will always have a problem. Put 
more memory in. Atop will show you what processes are utilizing the memory.


If the hard drive are over utilized, then another good tool to trace 
that down further is iotop. I have found that running DSpace in a 
virtual environment is problematic if you are not using very fast (SSD) 
hard drives.


If you do not find the culprit using this, then you should put a probe 
on your servlet container (Tomcat). PSI probe does the job here. You can 
get it at https://github.com/psi-probe/psi-probe. The installation 
instructions are available under the "wiki" tab. When you run this, look 
at how often garbage collection happens (under "system" -> "memory 
utilization"). Here is a graph that I pulled off one of my bigger 
installations.


Here you can see that garbage collection happens about twice in a ten 
minute period. This is good, and it means that your JAVA_OPTS / 
CATALINA_OPTS are set correctly. If it garbage collect too often, then 
you will notice that your site becomes unresponsive (sticky). You need 
to correct this by ensuring the JAVA_OPTS / CATALINA_OPTS parameters are 
set correctly. This is a rather difficult thing to do and there is not 
set method to do this. Experimentation is required.


Lastly (and also very important) is your database configuration. You 
need to ensure that your database (either Oracle or Postgres) is 
configured with enough memory. Once again, this is not a simple thing to 
do, and there are plenty of suggestions of how to do this on the web.


Most often I find that the problems are on the hardware side and not the 
software side


Hope this helps.

Shaun.


On 2019/05/21 08:01, Andreea Oprescu wrote:


Hi Tim,
thank you for your time :).
My machine has 5 GB memory, from which we have allocated half of it to 
Tomcat.
I have already took a look on that information (Performance Tuning 
DSpace) and it does help improve performance, altough its still far 
from desirable.


Do you think a possible solution would be to upgrade Dspace to 6.3?

El lunes, 20 de mayo de 2019, 17:10:06 (UTC+2), Tim Donohue escribió:

Hello Andreea,

If you are still experiencing less than ideal performance, we have
some notes on improving the performance of DSpace by providing
more memory to it:
https://wiki.duraspace.org/display/DSDOC6x/Performance+Tuning+DSpace

 While the *minimum* values there are good for brand new sites, as
you site grows in size or has a large number of users, you really
should be looking at allocating several GB of memory for Tomcat
alone. Very large or very active sites tend to be in the 2-4GB or
more range.  If your memory allocation is too small, your site
performance will suffer, and DSpace will become slower and slower.

It's hard to say if this is your exact issue, but it's usually the
first thing to check when performance is less than ideal.  Others
on this mailing list may have more specific advice to add here.

Tim




On Mon, May 20, 2019 at 2:35 AM Andreea Oprescu
> wrote:

Thank you for your answers. I will check into your proposed
solutions.
Yes, its dspace 6.2 with xmlui.

I've been reading around a bit, but I still wonder why
mydspace is so slow. Is it because of bitstreams? My items
don't have more than 4-5 bitstreams...

El viernes, 17 de mayo de 2019, 11:10:00 (UTC+2), Andreea
Oprescu escribió:

Hello everyone,

I am currently trying to debug a*very slow* dspace
implementation. I am facing the following issues:

- organization with more than 2.624 collections, in 4
communities, which at the same time have 5, 5, 52 and 19
subcommunities.
- users and administrator accounts. Users accounts with
less permissions take a lot longer to process than
administrator accounts (does dspace check all the
permissions of an user in ALL the collections?)
- there seem to be a lot of SQL queries. For example, for
moving an item, I ended up with almost 6000 SQL queries,
which took a little bit more than 2 minutes to complete
(for admin account).

I think dspace is a wonderful tool which provides my
organization a lot of value, but it's really, really slow.


=> Is it normal, because of 

Re: [dspace-tech] Re: Dspace limitations and low performance, or just wrong implementation?

[Claudia Jürgen]

Hello Andreea,

not actually improving the performance itself,  but the user experience
while starting a new publication.
Recommend those who publish to start a new submission from the
collection page and NOT from the My DSpace Page.


Hope this helps

Claudia Jürgen



Am 21.05.2019 um 08:01 schrieb Andreea Oprescu:

Hi Tim,
thank you for your time :).
My machine has 5 GB memory, from which we have allocated half of it to
Tomcat.
I have already took a look on that information (Performance Tuning DSpace)
and it does help improve performance, altough its still far from desirable.

Do you think a possible solution would be to upgrade Dspace to 6.3?

El lunes, 20 de mayo de 2019, 17:10:06 (UTC+2), Tim Donohue escribió:

Hello Andreea,

If you are still experiencing less than ideal performance, we have some
notes on improving the performance of DSpace by providing more memory to
it:  https://wiki.duraspace.org/display/DSDOC6x/Performance+Tuning+DSpace
  While the *minimum* values there are good for brand new sites, as you site
grows in size or has a large number of users, you really should be looking
at allocating several GB of memory for Tomcat alone. Very large or very
active sites tend to be in the 2-4GB or more range.  If your memory
allocation is too small, your site performance will suffer, and DSpace will
become slower and slower.

It's hard to say if this is your exact issue, but it's usually the first
thing to check when performance is less than ideal.  Others on this mailing
list may have more specific advice to add here.

Tim




On Mon, May 20, 2019 at 2:35 AM Andreea Oprescu > wrote:


Thank you for your answers. I will check into your proposed solutions.
Yes, its dspace 6.2 with xmlui.

I've been reading around a bit, but I still wonder why mydspace is so
slow. Is it because of bitstreams? My items don't have more than 4-5
bitstreams...

El viernes, 17 de mayo de 2019, 11:10:00 (UTC+2), Andreea Oprescu
escribió:

Hello everyone,

I am currently trying to debug a* very slow* dspace implementation. I
am facing the following issues:

- organization with more than 2.624 collections, in 4 communities, which
at the same time have 5, 5, 52 and 19 subcommunities.
- users and administrator accounts. Users accounts with less permissions
take a lot longer to process than administrator accounts (does dspace check
all the permissions of an user in ALL the collections?)
- there seem to be a lot of SQL queries. For example, for moving an
item, I ended up with almost 6000 SQL queries, which took a little bit more
than 2 minutes to complete (for admin account).

I think dspace is a wonderful tool which provides my organization a lot
of value, but it's really, really slow.


=> Is it normal, because of the amount of data?
=> Are we implementing something wrong?
=> If this is not normal, do you have any guides or advices on how to
procede with the debugging?


I have tried to find information about this, but could only find this
post, with the same problem, but no responses:
http://dspace.2283337.n4.nabble.com/Very-slow-quot-Move-item-quot-button-td4687571.html


I really appreciate your time.
Regards,

Andreea.


--
All messages to this mailing list should adhere to the DuraSpace Code of
Conduct: https://duraspace.org/about/policies/code-of-conduct/
---
You received this message because you are subscribed to the Google Groups
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to dspac...@googlegroups.com .
To view this discussion on the web visit
https://groups.google.com/d/msgid/dspace-tech/7d8bbdc8-bd92-493f-8768-ce95f1587011%40googlegroups.com

.



--

Tim Donohue
Technical Lead for DSpace & DSpaceDirect
DuraSpace.org | DSpace.org | DSpaceDirect.org




--
Claudia Juergen
Eldorado

Technische Universität Dortmund
Universitätsbibliothek
Vogelpothsweg 76
44227 Dortmund

Tel.: +49 231-755 40 43
Fax: +49 231-755 40 32
claudia.juer...@tu-dortmund.de
www.ub.tu-dortmund.de


Wichtiger Hinweis: Die Information in dieser E-Mail ist vertraulich. Sie ist 
ausschließlich für den Adressaten bestimmt. Sollten Sie nicht der für diese 
E-Mail bestimmte Adressat sein, unterrichten Sie bitte den Absender und 
vernichten Sie diese Mail. Vielen Dank.
Unbeschadet der Korrespondenz per E-Mail, sind unsere Erklärungen 
ausschließlich final rechtsverbindlich, wenn sie in herkömmlicher Schriftform 
(mit eigenhändiger Unterschrift) oder durch Übermittlung eines solchen 
Schriftstücks per Telefax erfolgen.

Important note: The information included in this e-mail is confidential. It is 
solely intended for the recipient. If you are not the intended recipient of 
this e-mail please contact the sender and delete this message. Thank you. 
Without prejudice of e-mail correspondence, our statements are only legally 
binding when they are made in 

Re: [dspace-tech] Re: Dspace limitations and low performance, or just wrong implementation?

[Andreea Oprescu]

Hi Tim,
thank you for your time :).
My machine has 5 GB memory, from which we have allocated half of it to 
Tomcat. 
I have already took a look on that information (Performance Tuning DSpace) 
and it does help improve performance, altough its still far from desirable. 

Do you think a possible solution would be to upgrade Dspace to 6.3? 

El lunes, 20 de mayo de 2019, 17:10:06 (UTC+2), Tim Donohue escribió:
>
> Hello Andreea,
>
> If you are still experiencing less than ideal performance, we have some 
> notes on improving the performance of DSpace by providing more memory to 
> it:  https://wiki.duraspace.org/display/DSDOC6x/Performance+Tuning+DSpace  
>  While the *minimum* values there are good for brand new sites, as you site 
> grows in size or has a large number of users, you really should be looking 
> at allocating several GB of memory for Tomcat alone. Very large or very 
> active sites tend to be in the 2-4GB or more range.  If your memory 
> allocation is too small, your site performance will suffer, and DSpace will 
> become slower and slower.
>
> It's hard to say if this is your exact issue, but it's usually the first 
> thing to check when performance is less than ideal.  Others on this mailing 
> list may have more specific advice to add here.
>
> Tim
>
>
>
>
> On Mon, May 20, 2019 at 2:35 AM Andreea Oprescu  > wrote:
>
>> Thank you for your answers. I will check into your proposed solutions.
>> Yes, its dspace 6.2 with xmlui.
>>
>> I've been reading around a bit, but I still wonder why mydspace is so 
>> slow. Is it because of bitstreams? My items don't have more than 4-5 
>> bitstreams... 
>>
>> El viernes, 17 de mayo de 2019, 11:10:00 (UTC+2), Andreea Oprescu 
>> escribió:
>>>
>>> Hello everyone, 
>>>
>>> I am currently trying to debug a* very slow* dspace implementation. I 
>>> am facing the following issues: 
>>>
>>> - organization with more than 2.624 collections, in 4 communities, which 
>>> at the same time have 5, 5, 52 and 19 subcommunities. 
>>> - users and administrator accounts. Users accounts with less permissions 
>>> take a lot longer to process than administrator accounts (does dspace check 
>>> all the permissions of an user in ALL the collections?) 
>>> - there seem to be a lot of SQL queries. For example, for moving an 
>>> item, I ended up with almost 6000 SQL queries, which took a little bit more 
>>> than 2 minutes to complete (for admin account). 
>>>
>>> I think dspace is a wonderful tool which provides my organization a lot 
>>> of value, but it's really, really slow. 
>>>
>>>
>>> => Is it normal, because of the amount of data? 
>>> => Are we implementing something wrong? 
>>> => If this is not normal, do you have any guides or advices on how to 
>>> procede with the debugging? 
>>>
>>>
>>> I have tried to find information about this, but could only find this 
>>> post, with the same problem, but no responses: 
>>> http://dspace.2283337.n4.nabble.com/Very-slow-quot-Move-item-quot-button-td4687571.html
>>>  
>>>
>>> I really appreciate your time.
>>> Regards,
>>>
>>> Andreea.
>>>
>> -- 
>> All messages to this mailing list should adhere to the DuraSpace Code of 
>> Conduct: https://duraspace.org/about/policies/code-of-conduct/
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "DSpace Technical Support" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to dspac...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/dspace-tech/7d8bbdc8-bd92-493f-8768-ce95f1587011%40googlegroups.com
>>  
>> 
>> .
>>
>
>
> -- 
>
> Tim Donohue
> Technical Lead for DSpace & DSpaceDirect
> DuraSpace.org | DSpace.org | DSpaceDirect.org
>
>

-- 
All messages to this mailing list should adhere to the DuraSpace Code of 
Conduct: https://duraspace.org/about/policies/code-of-conduct/
--- 
You received this message because you are subscribed to the Google Groups 
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dspace-tech+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dspace-tech/11385a73-8188-4d7f-9013-cbfa29e7d60e%40googlegroups.com.

Re: [dspace-tech] Re: Dspace limitations and low performance, or just wrong implementation?

[Tim Donohue]

Hello Andreea,

If you are still experiencing less than ideal performance, we have some
notes on improving the performance of DSpace by providing more memory to
it:  https://wiki.duraspace.org/display/DSDOC6x/Performance+Tuning+DSpace
 While the *minimum* values there are good for brand new sites, as you site
grows in size or has a large number of users, you really should be looking
at allocating several GB of memory for Tomcat alone. Very large or very
active sites tend to be in the 2-4GB or more range.  If your memory
allocation is too small, your site performance will suffer, and DSpace will
become slower and slower.

It's hard to say if this is your exact issue, but it's usually the first
thing to check when performance is less than ideal.  Others on this mailing
list may have more specific advice to add here.

Tim




On Mon, May 20, 2019 at 2:35 AM Andreea Oprescu 
wrote:

> Thank you for your answers. I will check into your proposed solutions.
> Yes, its dspace 6.2 with xmlui.
>
> I've been reading around a bit, but I still wonder why mydspace is so
> slow. Is it because of bitstreams? My items don't have more than 4-5
> bitstreams...
>
> El viernes, 17 de mayo de 2019, 11:10:00 (UTC+2), Andreea Oprescu escribió:
>>
>> Hello everyone,
>>
>> I am currently trying to debug a* very slow* dspace implementation. I am
>> facing the following issues:
>>
>> - organization with more than 2.624 collections, in 4 communities, which
>> at the same time have 5, 5, 52 and 19 subcommunities.
>> - users and administrator accounts. Users accounts with less permissions
>> take a lot longer to process than administrator accounts (does dspace check
>> all the permissions of an user in ALL the collections?)
>> - there seem to be a lot of SQL queries. For example, for moving an item,
>> I ended up with almost 6000 SQL queries, which took a little bit more than
>> 2 minutes to complete (for admin account).
>>
>> I think dspace is a wonderful tool which provides my organization a lot
>> of value, but it's really, really slow.
>>
>>
>> => Is it normal, because of the amount of data?
>> => Are we implementing something wrong?
>> => If this is not normal, do you have any guides or advices on how to
>> procede with the debugging?
>>
>>
>> I have tried to find information about this, but could only find this
>> post, with the same problem, but no responses:
>> http://dspace.2283337.n4.nabble.com/Very-slow-quot-Move-item-quot-button-td4687571.html
>>
>>
>> I really appreciate your time.
>> Regards,
>>
>> Andreea.
>>
> --
> All messages to this mailing list should adhere to the DuraSpace Code of
> Conduct: https://duraspace.org/about/policies/code-of-conduct/
> ---
> You received this message because you are subscribed to the Google Groups
> "DSpace Technical Support" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to dspace-tech+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/dspace-tech/7d8bbdc8-bd92-493f-8768-ce95f1587011%40googlegroups.com
> 
> .
>


-- 

Tim Donohue
Technical Lead for DSpace & DSpaceDirect
DuraSpace.org | DSpace.org | DSpaceDirect.org

-- 
All messages to this mailing list should adhere to the DuraSpace Code of 
Conduct: https://duraspace.org/about/policies/code-of-conduct/
--- 
You received this message because you are subscribed to the Google Groups 
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dspace-tech+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dspace-tech/CACKSJ9Nsi4wE%3DXxkHPRtMyzMJ47Q_OnoLR1s%2BUor_RHBoM5aSg%40mail.gmail.com.

[dspace-tech] Re: Dspace limitations and low performance, or just wrong implementation?

[Andreea Oprescu]

Thank you for your answers. I will check into your proposed solutions.
Yes, its dspace 6.2 with xmlui.

I've been reading around a bit, but I still wonder why mydspace is so slow. 
Is it because of bitstreams? My items don't have more than 4-5 
bitstreams... 

El viernes, 17 de mayo de 2019, 11:10:00 (UTC+2), Andreea Oprescu escribió:
>
> Hello everyone, 
>
> I am currently trying to debug a* very slow* dspace implementation. I am 
> facing the following issues: 
>
> - organization with more than 2.624 collections, in 4 communities, which 
> at the same time have 5, 5, 52 and 19 subcommunities. 
> - users and administrator accounts. Users accounts with less permissions 
> take a lot longer to process than administrator accounts (does dspace check 
> all the permissions of an user in ALL the collections?) 
> - there seem to be a lot of SQL queries. For example, for moving an item, 
> I ended up with almost 6000 SQL queries, which took a little bit more than 
> 2 minutes to complete (for admin account). 
>
> I think dspace is a wonderful tool which provides my organization a lot of 
> value, but it's really, really slow. 
>
>
> => Is it normal, because of the amount of data? 
> => Are we implementing something wrong? 
> => If this is not normal, do you have any guides or advices on how to 
> procede with the debugging? 
>
>
> I have tried to find information about this, but could only find this 
> post, with the same problem, but no responses: 
> http://dspace.2283337.n4.nabble.com/Very-slow-quot-Move-item-quot-button-td4687571.html
>  
>
> I really appreciate your time.
> Regards,
>
> Andreea.
>

-- 
All messages to this mailing list should adhere to the DuraSpace Code of 
Conduct: https://duraspace.org/about/policies/code-of-conduct/
--- 
You received this message because you are subscribed to the Google Groups 
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dspace-tech+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/dspace-tech/7d8bbdc8-bd92-493f-8768-ce95f1587011%40googlegroups.com.