Dear Sir, I am glad that I have helped you.
This error with the XOAI import happened also to our installation a few years ago and sometimes it happens because it tries to index in OAI an item from the SOLR index that *does not have a handle number*. This happens when an item with no handle number has been indexed in SOLR, which should not occur. This might not happen for an ordinary item, but it may occur for an* Item Template* that is specified for a collection, but it is *withdrawn *(which should *NOT *happen). You should check all your *withdrawn *items in DSpace and if you find one or more that are not ordinary items, but* Item Templates*, then you should delete and recreate them for the collection(s) they belong to. *Make sure to keep the pre-filled data, so that you will be able to re-create them.* One query you could run in your postgres, in order to see which *items in general *do not have a handle number is: *SELECT * FROM item WHERE NOT EXISTS (SELECT resource_id FROM handle WHERE handle.resource_id = item.uuid AND handle.resource_type_id = 2);* The query will respond with a number of items with *no handle number*. For most of the items is *normal*, since they might be incomplete submissions from users, or submissions which have not completed the workflow, etc. You will also see item templates in that list. They also do not have handle numbers and it is normal. However, check the "*withdrawn*" column for everyone of these items and to see if it is "t" (true) and for these items check if they are *item templates*. You can do that by pasting the uuid in the "*internal ID*" of the administrative interface of DSpace in "*Content->Items*" If that is the case, then you must delete them and recreate them for the collection they belong to, but this time they must not be withdrawn. When you recreate them, before you run the OAI index script, you will have to recreate the SOLR index with */dspace/bin/index-discovery -b -f *so that the erroneous item will be excluded from the index (the* -b deletes the index *and the *-f forces each item to be re-indexed*, so make sure that your reindexing process works well, or have a backup of the index before attempting this necessary step, or you might end up with no search index if it also crashes). If that does not solve the problem, then another item with no handle number might be the culprit, ordinary or Item template, which mistakenly has been indexed in SOLR. In case you would like to dive into SOLR, there is this query that can be run *directly to SOLR from the console of your Linux server*, which will return* all indexed documents in SOLR that do not have a handle number.* However, the returned file is an XML JSON file, but I guess it will provide enough information about these erroneous items and then you can spot them. The query is (keep the single and double quotes, or the shell might not ran the command).: *wget 'http://127.0.0.1:8080/solr/search/select?q=*:*+AND+-handle:{* <http://127.0.0.1:8080/solr/search/select?q=*:*+AND+-handle:{*> TO *}&wt=json&indent=true'* I hope this also helps you Best Regards, -Fk On Tue, Mar 23, 2021 at 11:56 AM Panyarak Ngamsritragul <[email protected]> wrote: > > Thanks a lot Filippos. That really help solve my problem! > > Anyway, I got another error following the success of importing the items: > 11400 items imported so far... > 11500 items imported so far... > Total: 11505 items > java.lang.NullPointerException > at org.dspace.xoai.app.XOAI.willChangeStatus(XOAI.java:438) > at org.dspace.xoai.app.XOAI.index(XOAI.java:368) > at org.dspace.xoai.app.XOAI.index(XOAI.java:280) > at org.dspace.xoai.app.XOAI.indexAll(XOAI.java:227) > at org.dspace.xoai.app.XOAI.index(XOAI.java:134) > at org.dspace.xoai.app.XOAI.main(XOAI.java:560) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:229) > at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:81) > > I did searches through the web and found that it will be resolved in > 6.4... > https://alanorth.github.io/cgspace-notes/2020-06/ > > Thanks a lot. > Panyarak > > On Mon, Mar 22, 2021 at 11:16 PM FILIPPOS KOLOVOS <[email protected]> wrote: > >> Dear Sir, >> >> This error relates to the Garbage Collection (GC) mechanism of JVM, which >> means that it does not have enough memory available to complete the task. >> More in particular, it means that the Garbage Collector is spending too >> much time clearing the heap space from unused objects, but at the same time >> it does not free more than 2% of the heap space, which is used for the >> instantiated objects. That's wh the GC reports that it does not have any >> more space to do its job. >> >> One thing you can do, if you do not have enough RAM on your server, is >> first shutdown Tomcat (this will lead to some downtime for your server, but >> it will free up some valuable RAM and after OAI completes you can restart >> it) and then go to the */dspace/bin/dspace* file of your running dspace >> instance and edit it with your text editor. Then, in the file search and >> find the line: >> >> *JAVA_OPTS="-Xmx256m -Dfile.encoding=UTF-8"* >> >> and *change it *to: >> >> *JAVA_OPTS="-Xmx2048m -Xms1024m -Dfile.encoding=UTF-8"* >> >> *DO NOT* copy and paste the line, but edit it in place, because you do >> not want to mess up this file. >> >> This will specify that the dspace *TOOL *(which is the file you are >> running for the OAI) will use *from 1024MB up to 2048MB RAM* for its >> task and not just *256MB*. However, you *must have more than 2048MB RAM* >> available on the server (i.e. 3GB), or the script will fail again and it >> might hang your server. If you have *LESS *than 3GB allocated to the >> server, then adjust these values accordingly (i.e. Xmx1024m Xms500m, etc). >> >> The bottomline is that you do not have enough memory to complete the task >> and even if your server has 32GB RAM, the dspace tool will only use up to >> the RAM specified in this file. >> >> I hope that I have helped! >> >> Best Regards, >> >> -Fk >> >> On Mon, Mar 22, 2021 at 4:08 PM Panyarak Ngamsritragul < >> [email protected]> wrote: >> >>> >>> Hi, >>> >>> I am using DSpace 6.3+Apache Tomcat Version 8.0.37+javac 11.0.10 >>> >>> The instance I am managing has 11,428 records. (kb.psu.ac.th) >>> >>> I tried to create indexes for OAI using this command >>> >>> /dspace/bin/dspace oai import -c >>> >>> It worked until 8900 items and crashed with error messages: >>> >>> 8600 items imported so far... >>> 8700 items imported so far... >>> 8800 items imported so far... >>> 8900 items imported so far... >>> java.lang.OutOfMemoryError: GC overhead limit exceeded >>> at java.util.Arrays.copyOfRange(Arrays.java:3664) >>> at java.lang.String.<init>(String.java:207) >>> at java.lang.StringBuilder.toString(StringBuilder.java:407) >>> at >>> org.hibernate.persister.entity.AbstractEntityPersister.selectFragment(AbstractEntityPersister.java:1422) >>> at >>> org.hibernate.persister.entity.AbstractEntityPersister.selectFragment(AbstractEntityPersister.java:4434) >>> at org.hibernate.loader.JoinWalker.selectString(JoinWalker.java:1099) >>> at >>> org.hibernate.loader.AbstractEntityJoinWalker.initStatementString(AbstractEntityJoinWalker.java:123) >>> at >>> org.hibernate.loader.AbstractEntityJoinWalker.initStatementString(AbstractEntityJoinWalker.java:108) >>> at >>> org.hibernate.loader.AbstractEntityJoinWalker.initAll(AbstractEntityJoinWalker.java:90) >>> at >>> org.hibernate.loader.AbstractEntityJoinWalker.initAll(AbstractEntityJoinWalker.java:77) >>> at >>> org.hibernate.loader.criteria.CriteriaJoinWalker.<init>(CriteriaJoinWalker.java:123) >>> at >>> org.hibernate.loader.criteria.CriteriaJoinWalker.<init>(CriteriaJoinWalker.java:92) >>> at >>> org.hibernate.loader.criteria.CriteriaLoader.<init>(CriteriaLoader.java:95) >>> at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1604) >>> at org.hibernate.internal.CriteriaImpl.list(CriteriaImpl.java:374) >>> at >>> org.dspace.core.AbstractHibernateDAO.list(AbstractHibernateDAO.java:158) >>> at >>> org.dspace.authorize.dao.impl.ResourcePolicyDAOImpl.findByDSoAndAction(ResourcePolicyDAOImpl.java:74) >>> at >>> org.dspace.authorize.ResourcePolicyServiceImpl.find(ResourcePolicyServiceImpl.java:103) >>> at >>> org.dspace.authorize.AuthorizeServiceImpl.getPoliciesActionFilter(AuthorizeServiceImpl.java:575) >>> at >>> org.dspace.authorize.AuthorizeServiceImpl.authorize(AuthorizeServiceImpl.java:301) >>> at >>> org.dspace.authorize.AuthorizeServiceImpl.authorizeAction(AuthorizeServiceImpl.java:129) >>> at >>> org.dspace.authorize.AuthorizeServiceImpl.authorizeAction(AuthorizeServiceImpl.java:95) >>> at >>> org.dspace.authorize.AuthorizeServiceImpl.authorizeActionBoolean(AuthorizeServiceImpl.java:181) >>> at >>> org.dspace.authorize.AuthorizeServiceImpl.authorizeActionBoolean(AuthorizeServiceImpl.java:166) >>> at org.dspace.xoai.app.XOAI.isPublic(XOAI.java:458) >>> at org.dspace.xoai.app.XOAI.index(XOAI.java:343) >>> at org.dspace.xoai.app.XOAI.index(XOAI.java:280) >>> at org.dspace.xoai.app.XOAI.indexAll(XOAI.java:227) >>> at org.dspace.xoai.app.XOAI.index(XOAI.java:134) >>> at org.dspace.xoai.app.XOAI.main(XOAI.java:560) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>> >>> Could you please help to solve this problem? >>> >>> Thanks and best regards, >>> Panyarak Ngamsritragul >>> >>> -- >>> All messages to this mailing list should adhere to the Code of Conduct: >>> https://duraspace.org/about/policies/code-of-conduct/ >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "DSpace Community" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/dspace-community/fcc5dfb5-0c01-4665-b490-e915d4662f96n%40googlegroups.com >>> <https://groups.google.com/d/msgid/dspace-community/fcc5dfb5-0c01-4665-b490-e915d4662f96n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >> >> -- All messages to this mailing list should adhere to the Code of Conduct: https://duraspace.org/about/policies/code-of-conduct/ --- You received this message because you are subscribed to the Google Groups "DSpace Community" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-community/CAHEC7xuRii-wjuNCvzy%3DpRXcFK9q9B3TaySGkNNJm9PNLfXudQ%40mail.gmail.com.
