Hi, Jis, >From your description, the key observation is this:
- PostgreSQL is running. - Tomcat is running. - The repository previously worked. - You have 1,352 items in the database. - The error started after Solr permission/read-only filesystem problems. - The JSPUI now only shows *"Internal System Error"*. This strongly suggests that *the JSPUI itself is healthy*, but *one backend component is failing during a request*. In DSpace 6.3 the most common causes are: 1. Solr search core unavailable (*most likely*) 2. Database connection issue 3. Corrupted Assetstore 4. Configuration mismatch 5. Java memory/classpath issues Since PostgreSQL and Tomcat are working, I would investigate *Solr first*. ------------------------------ *Step 1: Check the REAL exception* The JSP page hides the actual error. Run tail -200 /dspace/log/dspace.log and tail -200 /var/log/tomcat9/catalina.out Look for Caused by: or Exception or ERROR The first exception is the important one. Without that exception we're only guessing. ------------------------------ *Step 2: Verify Solr cores* Open http://SERVER:8080/solr/ Do you see something like search statistics oai authority ? If the Solr Admin page loads, execute curl http://localhost:8080/solr/admin/cores?action=STATUS Expected output search status: If instead you get Core search is unavailable or Error loading core then Solr is the problem. ------------------------------ *Step 3: Verify search core exists* Check ls -l /dspace/solr Typical DSpace 6.3 layout authority/ oai/ search/ statistics/ Inside search conf/ core.properties data/ Verify ls /dspace/solr/search Expected conf core.properties data If any are missing, Solr cannot initialize. ------------------------------ *Step 4: Check ownership* Run ls -ld /dspace ls -ld /dspace/solr ls -ld /dspace/log ls -l /dspace/solr/search Usually they should be tomcat:dspace or dspace:dspace depending on your installation. Permissions 775 or 755 Files should not be owned by root. ------------------------------ *Step 5: Check if filesystem is still read-only* Earlier you mentioned Read-only filesystem Sometimes Ubuntu remounts the filesystem read-only after disk errors. Check mount Look for (ro, If you see ro instead of rw then the filesystem is still read-only. Also check touch /dspace/testfile If Permission denied or Read-only file system appears, that's the actual problem. ------------------------------ *Step 6: Check available disk* Run df -h Especially / and /dspace If disk usage is 100% Solr may fail to start. ------------------------------ *Step 7: Check inode exhaustion* df -i Sometimes there is disk space available but no inodes. ------------------------------ *Step 8: Verify Java can write to Solr* Delete only the Solr lock files. find /dspace/solr -name "*.lock" If any exist rm *.lock inside the data directories. Never delete schema.xml or solrconfig.xml ------------------------------ *Step 9: Verify search index* You can safely rebuild the search index. This *does not delete items*. Only Solr indexes are recreated. First stop Tomcat sudo systemctl stop tomcat9 Delete only the index /dspace/solr/search/data/index NOT conf NOT core.properties Restart Tomcat Then run [dspace]/bin/dspace index-discovery -b This rebuilds the Discovery index. For 1352 items it should finish quickly. ------------------------------ *Step 10: Verify database connectivity* Run [dspace]/bin/dspace database test or simply psql -U dspace dspace If the CLI connects, database is fine. ------------------------------ *Step 11: Check local.cfg* Verify solr.server = http://localhost:8080/solr and db.url matches PostgreSQL. Also verify assetstore.dir= points to an existing directory. ------------------------------ *Step 12: Test Discovery manually* Open http://SERVER:8080/solr/search/admin/ping Expected status: OK If instead 500 or Core unavailable then Discovery cannot start. ------------------------------ *Step 13: Check Tomcat deployment* Confirm http://SERVER:8080/xmlui loads. Also verify ls /var/lib/tomcat9/webapps contains xmlui jspui solr depending on your installation. ------------------------------ *Step 14: Check Java memory* Sometimes Solr cannot initialize because of heap exhaustion. Check journalctl -u tomcat9 Look for OutOfMemoryError or GC overhead limit exceeded ------------------------------ *Most likely causes (based on your symptoms)* · Based on the sequence you described, I'd rank the likely root causes as: · Solr search core failed to initialize after the filesystem/permission issue · Corrupted Solr index after the read-only filesystem event · Incorrect ownership or permissions on /dspace/solr or /dspace/log · Disk or filesystem corruption causing incomplete writes · Database problem (less likely if you can query all items) On Mon, Jul 20, 2026 at 5:32 PM JIS PUNNASSERY <[email protected]> wrote: > Dear DSpace Technical Support Team, > > I am writing to request your urgent assistance in resolving a critical > issue with our DSpace repository. > > We are currently using *DSpace 6.3 (JSPUI)* on *Ubuntu 20.04 LTS*. Our > repository was working previously, but it is now displaying an *Internal > System Error*, preventing users from accessing the repository. > System Environment > > - > > DSpace Version: 6.3 (JSPUI) > - > > Operating System: Ubuntu 20.04 LTS > - > > Database: PostgreSQL 12 > - > > Application Server: Apache Tomcat 9 > - > > Search Engine: Apache Solr > > Error Displayed > > When opening the DSpace website, the following message appears: > > *Internal System Error* > > The system has experienced an internal error. Please try to do what you > were doing again, and if the problem persists, please contact us so we can > fix the problem. > > Troubleshooting Performed > > To resolve the issue, I have already performed the following checks: > > - > > Verified that PostgreSQL is running normally. > - > > Verified that Apache Tomcat 9 is running. > - > > Confirmed that the DSpace database is accessible and contains > approximately *1,352 items*. > - > > Checked the DSpace configuration files. > - > > Reviewed Tomcat and DSpace log files. > - > > Corrected ownership and permissions of the /dspace directory. > - > > Corrected permissions on the /dspace/log and /dspace/solr directories. > - > > Added the Tomcat user to the DSpace group. > - > > Restarted Tomcat several times. > - > > Checked the Solr application. > - > > Attempted to rebuild the Solr search index. > - > > Verified that the filesystem is writable. > > Errors Found > > During troubleshooting, I observed the following issues: > > - > > Solr search core initialization failure. > - > > Search core reported as unavailable. > - > > Permission-related errors involving the /dspace/log and /dspace/solr > directories. > - > > Previous log entries showed "Permission denied" and "Read-only file > system" errors. > - > > Although these permission issues have been corrected, the repository > still displays the same *Internal System Error*. > > Current Status > > - > > PostgreSQL service: Running > - > > Apache Tomcat 9: Running > - > > Solr application: Accessible > - > > DSpace Web Interface: Displays *Internal System Error* > > Request for Assistance > > I would greatly appreciate your help in identifying the root cause of this > problem and restoring the repository to normal operation. > > Could you please advise: > > 1. > > What could be causing the Internal System Error? > 2. > > How can I resolve the Solr search core initialization problem? > 3. > > Are there any additional configuration files or services that should > be checked? > 4. > > What diagnostic steps do you recommend to identify the exact cause? > 5. > > Is there any procedure to safely rebuild the DSpace search indexes > without affecting the existing repository? > > If required, I can provide the following information: > > - > > DSpace log (dspace.log) > - > > Tomcat logs (catalina.out) > - > > Solr logs > - > > PostgreSQL logs > - > > local.cfg > - > > Configuration files > - > > Directory permissions > - > > Any additional information needed for diagnosis > > This repository is actively used by our institution, so your assistance in > resolving this issue would be greatly appreciated. > > Thank you very much for your time and support. I look forward to your > guidance. > > > -- > All messages to this mailing list should adhere to the Code of Conduct: > https://lyrasis.org/code-of-conduct/ > --- > You received this message because you are subscribed to the Google Groups > "DSpace Technical Support" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion visit > https://groups.google.com/d/msgid/dspace-tech/94f90a98-6398-43e8-bee3-55105452e576n%40googlegroups.com > <https://groups.google.com/d/msgid/dspace-tech/94f90a98-6398-43e8-bee3-55105452e576n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- Thank you, Titus J Sam -- All messages to this mailing list should adhere to the Code of Conduct: https://lyrasis.org/code-of-conduct/ --- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/dspace-tech/CAKyuAPHi-h%3Dk5YegX33HUc9AbsvfQrfoaQoyzBkb1-EpoWmOpw%40mail.gmail.com.
