Dear Filippos, Thank you very much for sharing this helpful information, I will try it on my side and reply.
Regards, Lewatle On Tuesday, 23 March 2021 at 23:20:39 UTC+2 FILIPPOS KOLOVOS wrote: > Dear Lewatle, > > When you say that you have increased memory for Java you mean that apart > from the tomcat settings, you have also increased it in the ./dspace binary > file that performs the indexing? Also, how large is your DB? 24GB RAM is > huge!!! > > I would suggest that you check the ./dspace file of your running dspace > instance editing it with your text editor. Then, in the file search and > find the line: > > JAVA_OPTS="-Xmx256m -Dfile.encoding=UTF-8" > > and change it to something really high: > > JAVA_OPTS="-Xmx22000m -Xms20000m -Dfile.encoding=UTF-8" > > This will specify that the dspace TOOL (which is the file you are running > for the indexing) will use from 20000MB up to 22000MB RAM for its task and > not just 256MB. However, you must have more than 22000MB RAM available on > the server (i.e. 24GB which you have), or the script will fail again and it > might hang your server. If you want to allocate LESS than 22GB, then adjust > these values accordingly (i.e. Xmx16000m Xms14000m, etc). > > The problem with the "dspace" commands is that even if your server has > 32GB RAM, the dspace tool will only use up to the RAM specified in this > file and there are many occasions that lead to this error. > > Now, if you have done that, then there must be something REALLY > inefficient in the code for large DBs that leads to OOM Error and we'll try > to figure it out. > > I hope that I have helped! > > Στις Πέμ, 18 Μαρ 2021, 12:30 ο χρήστης [email protected] < > [email protected]> έγραψε: > >> Dear DSpace Team, >> >> The following is error message received when running index-authority on >> the server, we have increased memory for java to 24 GB and we have very >> huge database as this is happening on production site, please advise as our >> system engineers beleives this error might not be memory related, if this >> is relating to memory how big should we increase the memory to be ably to >> re-index successfully. This setup was for first time done on the server and >> followed this documentation : >> https://wiki.lyrasis.org/display/DSDOC5x/ORCID+Integration >> Your help is appreciated with hints to get this problem resolved. >> >> bash-4.2$ ./dspace index-authority >> Retrieving all data >> Initialize org.dspace.authority.indexer.DSpaceAuthorityIndexer >> Exception: Java heap space >> java.lang.OutOfMemoryError: Java heap space >> at >> org.hibernate.internal.util.collections.IdentityMap.entryArray(IdentityMap.java:160) >> at >> org.hibernate.internal.util.collections.IdentityMap.concurrentEntries(IdentityMap.java:76) >> at >> org.hibernate.event.internal.AbstractFlushingEventListener.prepareCollectionFlushes(AbstractFlushingEventListener.java:191) >> at >> org.hibernate.event.internal.AbstractFlushingEventListener.flushEverythingToExecutions(AbstractFlushingEventListener.java:93) >> at >> org.hibernate.event.internal.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:61) >> at >> org.hibernate.internal.SessionImpl.autoFlushIfRequired(SessionImpl.java:1166) >> at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1223) >> at org.hibernate.internal.QueryImpl.list(QueryImpl.java:101) >> at >> org.dspace.core.AbstractHibernateDAO.list(AbstractHibernateDAO.java:129) >> at >> org.dspace.handle.dao.impl.HandleDAOImpl.getHandlesByDSpaceObject(HandleDAOImpl.java:61) >> at >> org.dspace.handle.HandleServiceImpl.getInternalHandles(HandleServiceImpl.java:371) >> at >> org.dspace.handle.HandleServiceImpl.findHandle(HandleServiceImpl.java:274) >> at >> org.dspace.identifier.VersionedHandleIdentifierProvider.lookup(VersionedHandleIdentifierProvider.java:330) >> at >> org.dspace.identifier.IdentifierServiceImpl.lookup(IdentifierServiceImpl.java:165) >> at >> org.dspace.content.DSpaceObjectServiceImpl.getIdentifiers(DSpaceObjectServiceImpl.java:75) >> at >> org.dspace.content.ItemServiceImpl.updateLastModified(ItemServiceImpl.java:252) >> at >> org.dspace.content.ItemServiceImpl.updateLastModified(ItemServiceImpl.java:61) >> at >> org.dspace.content.MetadataValueServiceImpl.update(MetadataValueServiceImpl.java:88) >> at org.dspace.authority.AuthorityValue.updateItem(AuthorityValue.java:184) >> at >> org.dspace.authority.indexer.DSpaceAuthorityIndexer.prepareNextValue(DSpaceAuthorityIndexer.java:199) >> at >> org.dspace.authority.indexer.DSpaceAuthorityIndexer.hasMore(DSpaceAuthorityIndexer.java:132) >> at >> org.dspace.authority.indexer.DSpaceAuthorityIndexer.hasMore(DSpaceAuthorityIndexer.java:157) >> at >> org.dspace.authority.indexer.AuthorityIndexClient.main(AuthorityIndexClient.java:60) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:498) >> at >> org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:229) >> at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:81) >> -bash-4.2$ >> >> Regards, >> Lewatle >> >> -- >> All messages to this mailing list should adhere to the Code of Conduct: >> https://duraspace.org/about/policies/code-of-conduct/ >> --- >> You received this message because you are subscribed to the Google Groups >> "DSpace Community" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/dspace-community/d00e1bda-8a67-477b-8a67-4b09db450737n%40googlegroups.com >> >> <https://groups.google.com/d/msgid/dspace-community/d00e1bda-8a67-477b-8a67-4b09db450737n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- All messages to this mailing list should adhere to the Code of Conduct: https://duraspace.org/about/policies/code-of-conduct/ --- You received this message because you are subscribed to the Google Groups "DSpace Community" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-community/c5f784aa-2e5f-49be-9668-f992e2bcb3b8n%40googlegroups.com.
