Re: [Archivesspace_Users_Group] Performance tuning

[Blake Carver]

> ASPACE_JAVA_XMX="-Xmx16144m"

If the site needs 16 gigs of RAM to keep going, something is probably wrong.

> AppConfig[:indexer_records_per_thread] = 25
> AppConfig[:indexer_thread_count] = 7
> AppConfig[:indexer_solr_timeout_seconds] = 300

Messing around with those indexer settings is generally a good idea for sites 
with ALLOTTA records and/or a decent number of people adding new records. "It 
Depends" is about the only answer for those settings. If you have really 
deep/complex resources you'll probably want to change those up. Try slowing it 
down a bit. Try 2/50 or 2/25. You could even slow it WAY down to 1/1. It'll 
take forever to finish, but you can see exactly what it's doing each time.

BUT

If your site was OK and now it's not, maybe something else has gone wrong.

Check your logs. Make sure you're running the log_level on debug to catch 
everything possible.

- Look for ERROR or FATAL, maybe Java out of memory errors?
Sometimes there will be an Error or error but it's usually ERROR

-Is your indexer just running forever? Maybe it's caught in a crashing loop?
 "grep Indexed" do those numbers keep going up and down?
You sould see:
 Indexed 102303 of 102303 archival_object records in repository
Only once, you don't want to see that count going up and up and then suddenly
 Indexed 1 of 102303 archival_object records in repository

- Also check your proxy logs, maybe there's a bad bot hitting the PUI way too 
hard.

- Also just check that the server doesn't have anything new running and using 
up all the resources.

The answer is usually in the logs.


From: archivesspace_users_group-boun...@lyralists.lyrasis.org 
 on behalf of Cowing 
[he], Jared 
Sent: Tuesday, September 13, 2022 12:33 PM
To: archivesspace_users_group@lyralists.lyrasis.org 

Subject: [Archivesspace_Users_Group] Performance tuning

Hi all,

I'm currently working with our campus IT to identify the cause(s) of a 
degradation in performance that we've been seeing. I'm relatively new to our 
library and to ArchivesSpace generally, but I've been told it's been slowing 
bit by bit for a few years, and the problem has escalated just in the past few 
weeks. Neither full re-indexes nor our nightly re-starts seem to help.

I'm aware of this page on 
tuning<https://archivesspace.github.io/tech-docs/provisioning/tuning.html>, 
which has been quite helpful in addition to suggestions already posted to this 
list. We're hopeful that moving to external Solr with our next upgrade will 
also help (currently on 2.7.1), but are still trying other measures just in 
case it doesn't. While we look more into general best practices for tuning our 
hosting environment, I'd also like to check with all of you to see if there are 
common issues that are more specific to ArchivesSpace that we have overlooked 
and should focus our attention on.

Here are a few of our key settings. Our Java memory variables are below. I get 
the sense that they are higher than average, is that so?
ASPACE_JAVA_XMX="-Xmx16144m"
ASPACE_JAVA_XSS="-Xss8m"

Indexer settings from config.rb:
AppConfig[:indexer_records_per_thread] = 25
AppConfig[:indexer_thread_count] = 7
AppConfig[:indexer_solr_timeout_seconds] = 300


And copied from our innodb settings:
innodb_file_per_table
innodb_flush_method=O_DIRECT
innodb_log_file_size=1G
innodb_buffer_pool_size=8G
symbolic-links=0
max_allowed_packet=128M
open_files_limit=9182

I appreciate any tips on what we ought to be looking for. I know it's hard to 
give advice from afar when each institution's situation is different, but 
thought it worth asking in case anything jumped out before we turn to technical 
support.

Thanks,
--

Jared Cowing | Systems Librarian | he/him
WILLIAMS COLLEGE LIBRARIES<https://library.williams.edu/>  | Williamstown, MA | 
(413)597-3061
___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

[Archivesspace_Users_Group] Performance tuning

[Cowing [he], Jared]

Hi all,

I'm currently working with our campus IT to identify the cause(s) of a
degradation in performance that we've been seeing. I'm relatively new to
our library and to ArchivesSpace generally, but I've been told it's been
slowing bit by bit for a few years, and the problem has escalated just in
the past few weeks. Neither full re-indexes nor our nightly re-starts seem
to help.

I'm aware of this page on tuning
, which
has been quite helpful in addition to suggestions already posted to this
list. We're hopeful that moving to external Solr with our next upgrade will
also help (currently on 2.7.1), but are still trying other measures just in
case it doesn't. While we look more into general best practices for tuning
our hosting environment, I'd also like to check with all of you to see if
there are common issues that are more specific to ArchivesSpace that we
have overlooked and should focus our attention on.

Here are a few of our key settings. Our Java memory variables are below. I
get the sense that they are higher than average, is that so?
ASPACE_JAVA_XMX="-Xmx16144m"
ASPACE_JAVA_XSS="-Xss8m"

Indexer settings from config.rb:
AppConfig[:indexer_records_per_thread] = 25
AppConfig[:indexer_thread_count] = 7
AppConfig[:indexer_solr_timeout_seconds] = 300


And copied from our innodb settings:
innodb_file_per_table
innodb_flush_method=O_DIRECT
innodb_log_file_size=1G
innodb_buffer_pool_size=8G
symbolic-links=0
max_allowed_packet=128M
open_files_limit=9182

I appreciate any tips on what we ought to be looking for. I know it's hard
to give advice from afar when each institution's situation is different,
but thought it worth asking in case anything jumped out before we turn to
technical support.

Thanks,
-- 

*Jared Cowing* *|* Systems Librarian *| *he/him
*WILLIAMS COLLEGE LIBRARIES *  *|* Williamstown,
MA *| *(413)597-3061
___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group