Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)
Hi George, Hmmm...it actually looks a bit fishy to me. You have two Host entries defined, which look to me like they are loading the same webapps: 1. Host name=localhost is configured to load *everything* under the webapps subdirectory. 2. Host name=ecommons-test.library.cornell.edu is then configured to load webapps/xmlui and webapps/jspui So, essentially, between these two Host definitions, it looks like you are telling Tomcat to load both the XMLUI and JSPUI *twice*. Could you try commenting out one of your Host definitions and rebooting Tomcat? I wonder if that would resolve the Solr lock issues. In general, with the setup you are using, you really only need to either set appBase='webapps' OR define individual Context tags per webapp. You don't need to do both. As for your other question, I'm not sure why this work work on older versions of DSpace and not in DSpace 5.1. Perhaps it has something to do with the fact that we upgraded Solr to the latest version in DSpace 5.1. Maybe the latest version of Solr is less forgiving about multiple apps writing to it simultaneously. - Tim On 4/9/2015 8:40 AM, George Stanley Kozak wrote: Tim and Graham: Here are excerpts from my tomcat sever.xml file (again, this works OK under DSpace 4.3 and lower): Engine name=Catalina defaultHost=localhost ... Host name=localhost appBase=webapps unpackWARs=true autoDeploy=true ... Host name=ecommons-test.library.cornell.edu debug=0 unpackWARs=true autoDeploy=false ... Context path= docBase=[tomcat]/webapps/xmlui / ... Context path=/jspui docBase=[ tomcat]/webapps/jspui / Do you think that this kind of configuration could cause the problem I am observing? And, if so, why doesn't it happen at a lower version of DSpace? George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -Original Message- From: Tim Donohue [mailto:tdono...@duraspace.org] Sent: Thursday, April 09, 2015 9:24 AM To: George Stanley Kozak; Graham Triggs; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi George, On 4/8/2015 1:48 PM, George Stanley Kozak wrote: By the way, I run my webapps under tomcat, but I know that some of you run the webapps in their dspace directory. Is there any advantage to doing one or another? It's really just a matter of preference, it doesn't matter which way you load your webapps via Tomcat. But, as Graham notes, the one thing you don't want to do is have a single webapp loaded *twice*. So, if your XMLUI (or JSPUI) webapp is both under [tomcat]/webapps/ and configured under [tomcat]/config/Catalina/localhost/ , then Tomcat may be essentially running two copies of it. As Graham notes, this also could be a possible cause of the Solr lock errors. Out of curiosity, are you running both XMLUI and JSPUI in parallel? While I've never heard it reported, I wonder if doing so could be causing that same sort of issue -- one webapp obtains Solr access, and the other then hits lock errors. (Again though, this is complete speculation...just trying to determine what may be unique about your 5.1 setup) - Tim -- BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT Develop your own process in accordance with the BPMN 2 standard Learn Process modeling best practices with Bonita BPM through live exercises http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_ source=Sourceforge_BPM_Camp_5_6_15utm_medium=emailutm_campaign=VA_SF ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)
Tim and Graham: Here are excerpts from my tomcat sever.xml file (again, this works OK under DSpace 4.3 and lower): Engine name=Catalina defaultHost=localhost ... Host name=localhost appBase=webapps unpackWARs=true autoDeploy=true ... Host name=ecommons-test.library.cornell.edu debug=0 unpackWARs=true autoDeploy=false ... Context path= docBase=[tomcat]/webapps/xmlui / ... Context path=/jspui docBase=[ tomcat]/webapps/jspui / Do you think that this kind of configuration could cause the problem I am observing? And, if so, why doesn't it happen at a lower version of DSpace? George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -Original Message- From: Tim Donohue [mailto:tdono...@duraspace.org] Sent: Thursday, April 09, 2015 9:24 AM To: George Stanley Kozak; Graham Triggs; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi George, On 4/8/2015 1:48 PM, George Stanley Kozak wrote: By the way, I run my webapps under tomcat, but I know that some of you run the webapps in their dspace directory. Is there any advantage to doing one or another? It's really just a matter of preference, it doesn't matter which way you load your webapps via Tomcat. But, as Graham notes, the one thing you don't want to do is have a single webapp loaded *twice*. So, if your XMLUI (or JSPUI) webapp is both under [tomcat]/webapps/ and configured under [tomcat]/config/Catalina/localhost/ , then Tomcat may be essentially running two copies of it. As Graham notes, this also could be a possible cause of the Solr lock errors. Out of curiosity, are you running both XMLUI and JSPUI in parallel? While I've never heard it reported, I wonder if doing so could be causing that same sort of issue -- one webapp obtains Solr access, and the other then hits lock errors. (Again though, this is complete speculation...just trying to determine what may be unique about your 5.1 setup) - Tim -- BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT Develop your own process in accordance with the BPMN 2 standard Learn Process modeling best practices with Bonita BPM through live exercises http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_ source=Sourceforge_BPM_Camp_5_6_15utm_medium=emailutm_campaign=VA_SF ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)
Hi George, On 4/8/2015 1:48 PM, George Stanley Kozak wrote: By the way, I run my webapps under tomcat, but I know that some of you run the webapps in their dspace directory. Is there any advantage to doing one or another? It's really just a matter of preference, it doesn't matter which way you load your webapps via Tomcat. But, as Graham notes, the one thing you don't want to do is have a single webapp loaded *twice*. So, if your XMLUI (or JSPUI) webapp is both under [tomcat]/webapps/ and configured under [tomcat]/config/Catalina/localhost/ , then Tomcat may be essentially running two copies of it. As Graham notes, this also could be a possible cause of the Solr lock errors. Out of curiosity, are you running both XMLUI and JSPUI in parallel? While I've never heard it reported, I wonder if doing so could be causing that same sort of issue -- one webapp obtains Solr access, and the other then hits lock errors. (Again though, this is complete speculation...just trying to determine what may be unique about your 5.1 setup) - Tim -- BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT Develop your own process in accordance with the BPMN 2 standard Learn Process modeling best practices with Bonita BPM through live exercises http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_ source=Sourceforge_BPM_Camp_5_6_15utm_medium=emailutm_campaign=VA_SF ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)
Thanks, Brian: I set my tomcat server.xml as you indicated, but I am still having the same problem with the indexes. I'm not sure what could be wrong. As I stated earlier, everything works fine if I am running DSpace 4.3 and I only have problems once I upgrade to 5.1. In any case, I thank you and Tim and Hilton and Andrea for your help so far. This problem has stumped me. -Original Message- From: Brian Freels-Stendel [mailto:bfre...@unm.edu] Sent: Tuesday, April 07, 2015 4:40 PM To: George Stanley Kozak; dspace-tech@lists.sourceforge.net Subject: RE: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hey there, Sorry for not being descriptive enough. In tomcat.xml, there's a Host section where the DSpace app context fragments go: ___ Host name=localhost appBase=/opt/dspace/webapps unpackWARs=true autoDeploy=true xmlValidation=false xmlNamespaceAware=false !-- DEFINE A CONTEXT PATH FOR DSpace XML User Interface (Manakin) -- Context path=/ docBase=/opt/dspace/webapps/xmlui debug=0 reloadable=true cachingAllowed=false allowLinking=true/ ___ In that first line, the appBase may still be pointed to Tomcat's default. For us, it let DSpace start and function semi-well, but none of the SOLR stuff would work. B-- -Original Message- From: George Stanley Kozak [mailto:g...@cornell.edu] Sent: Tuesday, April 7, 2015 2:04 PM To: Brian Freels-Stendel; dspace-tech@lists.sourceforge.net Subject: RE: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi, Brian: Sorry for being dense, but what do you mean by updating the appBase directory? George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -Original Message- From: Brian Freels-Stendel [mailto:bfre...@unm.edu] Sent: Tuesday, April 07, 2015 3:47 PM To: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Good afternoon, We saw very similar errors (particularly Expected mime type application/octet-stream but got text/html) and found that we hadn't updated the appBase directory in tomcat.xml. Another symptom of ours was the code of the entire error page being written to the dspace log. It's a longshot, but perhaps something to check. B-- -Original Message- From: George Stanley Kozak [mailto:g...@cornell.edu] Sent: Tuesday, April 7, 2015 1:28 PM To: Tim Donohue; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi, Tim: To answer one of your questions: my original DSpace 1.8.2 solr statistics core did have data in it. Attached is a snapshot of the index directory. Now, I did as you asked. I deleted the all of the solr cores (actually everything under /cul/app/dspace/solr) I did the mvn and ant update and everything went well. I copied the webapps over to tomcat and restarted tomcat. I watched the DSpace log and the catalina.out log. Everything seemed to move along smoothly. When everything was done, I waited a few minutes before I went to my DSpace home page. When my home page came up, I began to see errors in my DSpace log: ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ anonymous:session_id=B3B12E105B95C8FB8E63E9FB94B5971F:ip_addr=128.84.117.195:Error in Discovery while setting up date facet range:date facet\colon; dateIssued.year SolrCore 'search' is not available due to init failure: Index locked for write for core search,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Index locked for write for core search Any attempt to do any browse or search results with errors. By the way, the errors on the web appear with org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Expected mime type application/octet-stream but got text/html. And then raw HTML after that. What has me stumped is that outside of the problem with the solr statistics, none of these problems appear in my DSpace 4.3 upgrade. I have no write lock problems in DSpace 4.3 install. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -Original Message- From: Tim Donohue [mailto:tdono...@duraspace.org] Sent: Tuesday, April 07, 2015 2:33 PM To: George Stanley Kozak; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi George, On 4/7/2015 1:09 PM, George Stanley Kozak wrote: Hi, Tim: 1) I did the lucene-core-3.5.0.jar manual update of the indexes. For the search index, I got no errors, but for the statistics core, I got: Exception
Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)
Hi George, This really has me stumped. It almost starts to sound like a permissions issue in your index directory (though I know from looking back at this thread, you said you looked at that). Since you essentially started with a fresh index (no content), and it still doesn't work, this doesn't sound like an issue with the index files themselves. While DSpace 5.1 uses a newer version of Solr, it doesn't act that much different from DSpace 4.3. So, it's confusing to me that this works for you on DSpace 4.3 and not on DSpace 5.1. When you encounter that Index locked for write for core search, there should be a write.lock file sitting in your /cul/app/dspace/solr/search/data/index/ folder. What are the permissions on that write.lock file? Is it owned by your Tomcat user? Also is *everything* under /cul/app/dspace/solr/ owned by the Tomcat user (all files and subdirectories)? You may even want to try and do the following: 1. Stop Tomcat 2. Recursively change the ownership (just in case), e.g. chown -R [tomcat-user]:[tomcat-group] /cul/app/dspace/solr/ 3. Delete that write.lock 4. Start Tomcat again. 5. (You may also need to manually reindex: 'dspace index-discovery -b') Yea, I know you checked these permissions before, but I'm rather stumped as to why a fresh, empty index won't even work on your system. I've never seen that before, and it implies that there may be something lower level (like a permissions problem) causing problems with Solr. - Tim On 4/7/2015 2:28 PM, George Stanley Kozak wrote: Hi, Tim: To answer one of your questions: my original DSpace 1.8.2 solr statistics core did have data in it. Attached is a snapshot of the index directory. Now, I did as you asked. I deleted the all of the solr cores (actually everything under /cul/app/dspace/solr) I did the mvn and ant update and everything went well. I copied the webapps over to tomcat and restarted tomcat. I watched the DSpace log and the catalina.out log. Everything seemed to move along smoothly. When everything was done, I waited a few minutes before I went to my DSpace home page. When my home page came up, I began to see errors in my DSpace log: ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ anonymous:session_id=B3B12E105B95C8FB8E63E9FB94B5971F:ip_addr=128.84.117.195:Error in Discovery while setting up date facet range:date facet\colon; dateIssued.year SolrCore 'search' is not available due to init failure: Index locked for write for core search,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Index locked for write for core search Any attempt to do any browse or search results with errors. By the way, the errors on the web appear with org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Expected mime type application/octet-stream but got text/html. And then raw HTML after that. What has me stumped is that outside of the problem with the solr statistics, none of these problems appear in my DSpace 4.3 upgrade. I have no write lock problems in DSpace 4.3 install. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -Original Message- From: Tim Donohue [mailto:tdono...@duraspace.org] Sent: Tuesday, April 07, 2015 2:33 PM To: George Stanley Kozak; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi George, On 4/7/2015 1:09 PM, George Stanley Kozak wrote: Hi, Tim: 1) I did the lucene-core-3.5.0.jar manual update of the indexes. For the search index, I got no errors, but for the statistics core, I got: Exception in thread main org.apache.lucene.index.IndexNotFoundException: org.apache.lucene.store.MMapDirectory@/cul/app/dspace/solr/statistics/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2e888d65 at org.apache.lucene.index.IndexUpgrader.upgrade(IndexUpgrader.java:118) at org.apache.lucene.index.IndexUpgrader.main(IndexUpgrader.java:85) This would imply that the search index upgraded fine (no errors). But, the error for your statistics index almost implies that it's not a valid Solr index. Do you use the statistics index? Is there anything in the /cul/app/dspace/solr/statistics/data/index directory or is it empty? 2) I deleted the statistics core and did the ant update. I brought up my DSpace 5.1 test system and when I tried to do a search, I got no results. When I went to do a browse, I got: ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ anonymous:session_id=5749DBA88953D8B0ABDDA3FFD1DFF5F2:ip_addr=128.84.1 17.195:Error in Discovery while setting up date facet range:date facet\colon; dateIssued.year SolrCore 'search' is not available due to init failure
Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)
Is it at all possible that you have two separate processes / applications attempting to access the same Solr index - so one locks it, and the other is unable to? For example, maybe a context file under tomcat/config/Catalina/localhost and an application in tomcat/webapps? Regards, G On 08/04/2015 17:17:08, George Stanley Kozak g...@cornell.edu wrote: Hi, Tim: Yes, this has me stumped. Here is the permissions on the writelock: -rw-rw-r-- 1 dspace dspace 0 Apr 8 09:59 write.lock Tomcat is owned by the dspace user: drwxr-xr-x 3 dspace dspace 4096 Apr 2 2014 tomcat Solr is owned by the dspace user: drwxrwxr-x 6 dspace dspace 4096 Apr 8 09:58 solr I did as you suggested and stopped tomcat, recursively changed the permissions on the solr indexes: sudo chown -R dspace:dspace /cul/app/dspace/solr I deleted the write.lock and restarted tomcat. I then did /cul/app/dspace/bin/dspace index-discovery -b Immediately, I received the write lock error and the re-index did not work. I restored the postgres database and solr indexes to what they were before and did a mvn build and ant update of my DSpace 4.3 source and everything works fine(?!?!) There is obviously some subtle difference on how DSpace 5.1 and DSpace 4.3 (and lower) are handling the solr indexes and my server set up is just right (or wrong) enough to cause this problem. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -Original Message- From: Tim Donohue [mailto:tdono...@duraspace.org] Sent: Wednesday, April 08, 2015 10:16 AM To: George Stanley Kozak; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi George, This really has me stumped. It almost starts to sound like a permissions issue in your index directory (though I know from looking back at this thread, you said you looked at that). Since you essentially started with a fresh index (no content), and it still doesn't work, this doesn't sound like an issue with the index files themselves. While DSpace 5.1 uses a newer version of Solr, it doesn't act that much different from DSpace 4.3. So, it's confusing to me that this works for you on DSpace 4.3 and not on DSpace 5.1. When you encounter that Index locked for write for core search, there should be a write.lock file sitting in your /cul/app/dspace/solr/search/data/index/ folder. What are the permissions on that write.lock file? Is it owned by your Tomcat user? Also is *everything* under /cul/app/dspace/solr/ owned by the Tomcat user (all files and subdirectories)? You may even want to try and do the following: 1. Stop Tomcat 2. Recursively change the ownership (just in case), e.g. chown -R [tomcat-user]:[tomcat-group] /cul/app/dspace/solr/ 3. Delete that write.lock 4. Start Tomcat again. 5. (You may also need to manually reindex: 'dspace index-discovery -b') Yea, I know you checked these permissions before, but I'm rather stumped as to why a fresh, empty index won't even work on your system. I've never seen that before, and it implies that there may be something lower level (like a permissions problem) causing problems with Solr. - Tim On 4/7/2015 2:28 PM, George Stanley Kozak wrote: Hi, Tim: To answer one of your questions: my original DSpace 1.8.2 solr statistics core did have data in it. Attached is a snapshot of the index directory. Now, I did as you asked. I deleted the all of the solr cores (actually everything under /cul/app/dspace/solr) I did the mvn and ant update and everything went well. I copied the webapps over to tomcat and restarted tomcat. I watched the DSpace log and the catalina.out log. Everything seemed to move along smoothly. When everything was done, I waited a few minutes before I went to my DSpace home page. When my home page came up, I began to see errors in my DSpace log: ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ anonymous:session_id=B3B12E105B95C8FB8E63E9FB94B5971F:ip_addr=128.84.1 17.195:Error in Discovery while setting up date facet range:date facet\colon; dateIssued.year SolrCore 'search' is not available due to init failure: Index locked for write for core search,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Index locked for write for core search Any attempt to do any browse or search results with errors. By the way, the errors on the web appear with org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Expected mime type application/octet-stream but got text/html. And then raw HTML after that. What has me stumped is that outside of the problem with the solr statistics, none of these problems appear in my DSpace 4.3 upgrade. I have no write lock problems in DSpace 4.3 install. George Kozak Digital Library Specialist Cornell University
Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)
Graham: I guess if I had two tomcat processes interfering with each other, I believe it would happen in both my DSpace 4.3 upgrade and my DSpace 5.1 upgrade, but everything works OK under DSpace 4.3 and then breaks down when I go to DSpace 5.1 (no changes to tomcat). However, I will do a bit more digging. Perhaps the older DSpace configs are more “forgiving” of problematic tomcat configurations. By the way, I run my webapps under tomcat, but I know that some of you run the webapps in their dspace directory. Is there any advantage to doing one or another? George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 From: Graham Triggs [mailto:grahamtri...@gmail.com] Sent: Wednesday, April 08, 2015 2:30 PM To: George Stanley Kozak; Tim Donohue; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Is it at all possible that you have two separate processes / applications attempting to access the same Solr index - so one locks it, and the other is unable to? For example, maybe a context file under tomcat/config/Catalina/localhost and an application in tomcat/webapps? Regards, G On 08/04/2015 17:17:08, George Stanley Kozak g...@cornell.edumailto:g...@cornell.edu wrote: Hi, Tim: Yes, this has me stumped. Here is the permissions on the writelock: -rw-rw-r-- 1 dspace dspace 0 Apr 8 09:59 write.lock Tomcat is owned by the dspace user: drwxr-xr-x 3 dspace dspace 4096 Apr 2 2014 tomcat Solr is owned by the dspace user: drwxrwxr-x 6 dspace dspace 4096 Apr 8 09:58 solr I did as you suggested and stopped tomcat, recursively changed the permissions on the solr indexes: sudo chown -R dspace:dspace /cul/app/dspace/solr I deleted the write.lock and restarted tomcat. I then did /cul/app/dspace/bin/dspace index-discovery -b Immediately, I received the write lock error and the re-index did not work. I restored the postgres database and solr indexes to what they were before and did a mvn build and ant update of my DSpace 4.3 source and everything works fine(?!?!) There is obviously some subtle difference on how DSpace 5.1 and DSpace 4.3 (and lower) are handling the solr indexes and my server set up is just right (or wrong) enough to cause this problem. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -Original Message- From: Tim Donohue [mailto:tdono...@duraspace.org] Sent: Wednesday, April 08, 2015 10:16 AM To: George Stanley Kozak; dspace-tech@lists.sourceforge.netmailto:dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi George, This really has me stumped. It almost starts to sound like a permissions issue in your index directory (though I know from looking back at this thread, you said you looked at that). Since you essentially started with a fresh index (no content), and it still doesn't work, this doesn't sound like an issue with the index files themselves. While DSpace 5.1 uses a newer version of Solr, it doesn't act that much different from DSpace 4.3. So, it's confusing to me that this works for you on DSpace 4.3 and not on DSpace 5.1. When you encounter that Index locked for write for core search, there should be a write.lock file sitting in your /cul/app/dspace/solr/search/data/index/ folder. What are the permissions on that write.lock file? Is it owned by your Tomcat user? Also is *everything* under /cul/app/dspace/solr/ owned by the Tomcat user (all files and subdirectories)? You may even want to try and do the following: 1. Stop Tomcat 2. Recursively change the ownership (just in case), e.g. chown -R [tomcat-user]:[tomcat-group] /cul/app/dspace/solr/ 3. Delete that write.lock 4. Start Tomcat again. 5. (You may also need to manually reindex: 'dspace index-discovery -b') Yea, I know you checked these permissions before, but I'm rather stumped as to why a fresh, empty index won't even work on your system. I've never seen that before, and it implies that there may be something lower level (like a permissions problem) causing problems with Solr. - Tim On 4/7/2015 2:28 PM, George Stanley Kozak wrote: Hi, Tim: To answer one of your questions: my original DSpace 1.8.2 solr statistics core did have data in it. Attached is a snapshot of the index directory. Now, I did as you asked. I deleted the all of the solr cores (actually everything under /cul/app/dspace/solr) I did the mvn and ant update and everything went well. I copied the webapps over to tomcat and restarted tomcat. I watched the DSpace log and the catalina.out log. Everything seemed to move along smoothly. When everything was done, I waited a few minutes before I went to my DSpace home page. When my home page
Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)
Hi, Tim: Yes, this has me stumped. Here is the permissions on the writelock: -rw-rw-r-- 1 dspace dspace0 Apr 8 09:59 write.lock Tomcat is owned by the dspace user: drwxr-xr-x 3 dspace dspace 4096 Apr 2 2014 tomcat Solr is owned by the dspace user: drwxrwxr-x 6 dspace dspace 4096 Apr 8 09:58 solr I did as you suggested and stopped tomcat, recursively changed the permissions on the solr indexes: sudo chown -R dspace:dspace /cul/app/dspace/solr I deleted the write.lock and restarted tomcat. I then did /cul/app/dspace/bin/dspace index-discovery -b Immediately, I received the write lock error and the re-index did not work. I restored the postgres database and solr indexes to what they were before and did a mvn build and ant update of my DSpace 4.3 source and everything works fine(?!?!) There is obviously some subtle difference on how DSpace 5.1 and DSpace 4.3 (and lower) are handling the solr indexes and my server set up is just right (or wrong) enough to cause this problem. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -Original Message- From: Tim Donohue [mailto:tdono...@duraspace.org] Sent: Wednesday, April 08, 2015 10:16 AM To: George Stanley Kozak; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi George, This really has me stumped. It almost starts to sound like a permissions issue in your index directory (though I know from looking back at this thread, you said you looked at that). Since you essentially started with a fresh index (no content), and it still doesn't work, this doesn't sound like an issue with the index files themselves. While DSpace 5.1 uses a newer version of Solr, it doesn't act that much different from DSpace 4.3. So, it's confusing to me that this works for you on DSpace 4.3 and not on DSpace 5.1. When you encounter that Index locked for write for core search, there should be a write.lock file sitting in your /cul/app/dspace/solr/search/data/index/ folder. What are the permissions on that write.lock file? Is it owned by your Tomcat user? Also is *everything* under /cul/app/dspace/solr/ owned by the Tomcat user (all files and subdirectories)? You may even want to try and do the following: 1. Stop Tomcat 2. Recursively change the ownership (just in case), e.g. chown -R [tomcat-user]:[tomcat-group] /cul/app/dspace/solr/ 3. Delete that write.lock 4. Start Tomcat again. 5. (You may also need to manually reindex: 'dspace index-discovery -b') Yea, I know you checked these permissions before, but I'm rather stumped as to why a fresh, empty index won't even work on your system. I've never seen that before, and it implies that there may be something lower level (like a permissions problem) causing problems with Solr. - Tim On 4/7/2015 2:28 PM, George Stanley Kozak wrote: Hi, Tim: To answer one of your questions: my original DSpace 1.8.2 solr statistics core did have data in it. Attached is a snapshot of the index directory. Now, I did as you asked. I deleted the all of the solr cores (actually everything under /cul/app/dspace/solr) I did the mvn and ant update and everything went well. I copied the webapps over to tomcat and restarted tomcat. I watched the DSpace log and the catalina.out log. Everything seemed to move along smoothly. When everything was done, I waited a few minutes before I went to my DSpace home page. When my home page came up, I began to see errors in my DSpace log: ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ anonymous:session_id=B3B12E105B95C8FB8E63E9FB94B5971F:ip_addr=128.84.1 17.195:Error in Discovery while setting up date facet range:date facet\colon; dateIssued.year SolrCore 'search' is not available due to init failure: Index locked for write for core search,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Index locked for write for core search Any attempt to do any browse or search results with errors. By the way, the errors on the web appear with org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Expected mime type application/octet-stream but got text/html. And then raw HTML after that. What has me stumped is that outside of the problem with the solr statistics, none of these problems appear in my DSpace 4.3 upgrade. I have no write lock problems in DSpace 4.3 install. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -Original Message- From: Tim Donohue [mailto:tdono...@duraspace.org] Sent: Tuesday, April 07, 2015 2:33 PM To: George Stanley Kozak; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech
Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)
Hi George, Yes, that is what I was suggesting...upgrading directly from 1.8.2 or 3.3 (whichever was your starting point) to 5.1. The ant update should take care of the necessary Solr index upgrades for you. - Tim On 4/6/2015 6:21 PM, George Stanley Kozak wrote: Tim: Thank you for your advice. I did try the 5.1 ant update, but it failed. Maybe if I tried it going from DSpace 3.3 or 1.8.2 directly to DSpace 5.1 instead of doing the intermediate step to 4.3, this might work? I will try a few other things and let everyone know what works. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CULIT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 From: Tim Donohue tdono...@duraspace.org Sent: Monday, April 6, 2015 4:44 PM To: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi George, The error below is essentially saying that your Solr statistics index is too old for the version of Solr packages with DSpace 4.3 to upgrade. However, in the DSpace 5.1 upgrade process, during the ant update step, we've attempted to catch this scenario automatically -- we try to determine the version of a Solr index, and upgrade it automatically to the latest version. So, I'd recommend trying to point DSpace 5.1 at your old Solr index, and running ant update. It is supposed to upgrade an old index automatically. If that doesn't work for some reason, another option is to do a more manual upgrade, as described in the DSpace 5.x documentation. This manual upgrade essentially requires downloading multiple versions of the Solr/Lucene core JAR, and running it against your index in order to upgrade it to the latest compatible version (this is essentially the same process that is now automated by ant update though) https://wiki.duraspace.org/display/DSDOC5x/Upgrading+DSpace#UpgradingDSpace-ManuallyUpgradingSolrIndexes By the way, from your earlier messages, recently I also did encounter that odd Solr write.lock error (Index locked for write..) during a recent upgrade I tried to 5.1. In my situation, it seemed like it was a stale write.lock file which was somehow sitting around. After clearing it out, the upgrade proceeded. Admittedly, I still need to dig a bit further and ensure my analysis is correct. - Tim On 4/1/2015 3:16 PM, George Stanley Kozak wrote: Hi, Everyone: I have discovered that when I try to optimize my Solr indexes when I upgraded from DSpace 3.3 to 4.3 (wget ‘http://localhost:8080/solr/statistics/update?optimize=true’) I get the following errors in the solr logs: ERROR org.apache.solr.core.CoreContainer @ Unable to create core: statistics Format version is not supported (resource: segment _32 in resource ChecksumIndexInput(MMapIndexInput(path=/cul/app/dspace/solr/statistics/data/index/segments_1p0))): 2.x. This version of Lucene only supports indexes created with release 3.0 and later. If I later try to upgrade to 5.1, I see during the ant update: /cul/app/dspace/src/dspace-5.1-src-release/dspace/target/dspace-installer/build.xml:1061: ERROR occurred while checking Solr index version: Exception in thread main java.io.IOException: Could not read Lucene segments files in /cul/app/dspace/solr/statistics/data/index I know the 5.1 documentation states that you can manually update the Solr indexes. Should that be done in my 4.3 upgrade before I try to go to 5.1 or should it be done in the 5.1 upgrade? George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT Develop your own process in accordance with the BPMN 2 standard Learn Process modeling best practices with Bonita BPM through live exercises http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_ source=Sourceforge_BPM_Camp_5_6_15utm_medium=emailutm_campaign=VA_SF ___ DSpace
Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)
Hi George, What happens if you do the following against your old 1.8.2 indexes: 1. wget http://search.maven.org/remotecontent?filepath=org/apache/lucene/lucene-core/3.5.0/lucene-core-3.5.0.jar; 2. java -cp lucene-core-3.5.0.jar org.apache.lucene.index.IndexUpgrader /cul/app/dspace/solr/statistics/data/index 3. java -cp lucene-core-3.5.0.jar org.apache.lucene.index.IndexUpgrader /cul/app/dspace/solr/search/data/index Do either of those IndexUpgrader commands succeed against your *old* 1.8.2 indexes? If not, could you send along the exception stacktrace? I'm just trying to determine if the issue is actually in these old 1.8.2 indexes. It may be either they are indexes from an unexpected, older version of Solr, or there's some minor index corruption here. -- To be honest, another thing to note here is that your *search* indexes can be easily rebuilt in DSpace 5.x (simply run ./dspace index-discovery -b). The only older index that really requires this upgrade process is the *statistics* index (as currently it doesn't have a reindex command). So, if your statistics index is of no importance, you could just delete both of these old indexes and upgrade to 5.1. As part of the 5.1 upgrade, a full reindex of the search index will be kicked off (when you start Tomcat). So, after a few minutes, your content should be fully reindexed and working. - Tim On 4/7/2015 10:12 AM, George Stanley Kozak wrote: Thanks for your help, Tim: I'm afraid that I keep hitting a wall when I try to upgrade to DSpace 5.1 I tried to go directly from DSpace 1.8.2 to DSpace 5.1 and got: ERROR occurred while checking Solr index version: Exception in thread main java.io.IOException: Could not read Lucene segments files in /cul/app/dspace/solr/statistics/data/index We have been using our own stats gathering system instead of the one within DSpace. So, I removed the statistics index but then I got: Exception in thread main java.io.IOException: Could not read Lucene segments files in /cul/app/dspace/solr/search/data/index This version of Lucene only supports indexes created with release 3.0 and later. So, I went ahead and used the DSpace 4.3 version of our solr indexes (without the statistics). It ran fine and everything got upgraded, but when I tried to do any searching, I get the write lock error. If I turn off tomcat, delete the write lock and restart things, I get the same thing. Also, I cannot update the discovery index because of the write lock problem. By the way, I did try the manual upgrade of the Solr indexes defined in the manual, but that didn't work for me, either. So, right now, it looks like I can only upgrade to DSpace 4.3, unless you have some more suggestions for me. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -Original Message- From: Tim Donohue [mailto:tdono...@duraspace.org] Sent: Tuesday, April 07, 2015 9:31 AM To: George Stanley Kozak; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi George, Yes, that is what I was suggesting...upgrading directly from 1.8.2 or 3.3 (whichever was your starting point) to 5.1. The ant update should take care of the necessary Solr index upgrades for you. - Tim On 4/6/2015 6:21 PM, George Stanley Kozak wrote: Tim: Thank you for your advice. I did try the 5.1 ant update, but it failed. Maybe if I tried it going from DSpace 3.3 or 1.8.2 directly to DSpace 5.1 instead of doing the intermediate step to 4.3, this might work? I will try a few other things and let everyone know what works. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CULIT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 From: Tim Donohue tdono...@duraspace.org Sent: Monday, April 6, 2015 4:44 PM To: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi George, The error below is essentially saying that your Solr statistics index is too old for the version of Solr packages with DSpace 4.3 to upgrade. However, in the DSpace 5.1 upgrade process, during the ant update step, we've attempted to catch this scenario automatically -- we try to determine the version of a Solr index, and upgrade it automatically to the latest version. So, I'd recommend trying to point DSpace 5.1 at your old Solr index, and running ant update. It is supposed to upgrade an old index automatically. If that doesn't work for some reason, another option is to do a more manual upgrade, as described in the DSpace 5.x documentation. This manual upgrade essentially requires downloading multiple versions of the Solr/Lucene core JAR, and running
Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)
Hi George, On 4/7/2015 1:09 PM, George Stanley Kozak wrote: Hi, Tim: 1) I did the lucene-core-3.5.0.jar manual update of the indexes. For the search index, I got no errors, but for the statistics core, I got: Exception in thread main org.apache.lucene.index.IndexNotFoundException: org.apache.lucene.store.MMapDirectory@/cul/app/dspace/solr/statistics/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2e888d65 at org.apache.lucene.index.IndexUpgrader.upgrade(IndexUpgrader.java:118) at org.apache.lucene.index.IndexUpgrader.main(IndexUpgrader.java:85) This would imply that the search index upgraded fine (no errors). But, the error for your statistics index almost implies that it's not a valid Solr index. Do you use the statistics index? Is there anything in the /cul/app/dspace/solr/statistics/data/index directory or is it empty? 2) I deleted the statistics core and did the ant update. I brought up my DSpace 5.1 test system and when I tried to do a search, I got no results. When I went to do a browse, I got: ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ anonymous:session_id=5749DBA88953D8B0ABDDA3FFD1DFF5F2:ip_addr=128.84.117.195:Error in Discovery while setting up date facet range:date facet\colon; dateIssued.year SolrCore 'search' is not available due to init failure: Index locked for write for core search,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Index locked for write for core search 3) I stopped tomcat, deleted the search index (as you suggested) and did the mvn build and ant update and then copied the new webapps over to tomcat. Everything looked good. I started tomcat and my site came up. Before doing anything more I ran ./dspace index-discovery -b. It ran to completion, but all searches return no hits and the browses give me the same write lock error as above. I made sure the permissions are correct (dspace user owns the solr indexes and tomcat) and removed the write lock and tried again, but again I get the same error. These other two errors seem rather odd to me...like something else is accessing Solr at the same time in which you are trying to reindex, and those multiple simultaneous writes are throwing errors. Can you try something even more simplistic than what you've done above? 1. Stop Tomcat 2. Copy your 1.8.3 data over to where you will install DSpace 5 3. Immediately delete *everything* under /cul/app/dspace/solr/ (including the entire statistics and search subdirectories). This will wipe out all existing indexes, obviously. 4. Build DSpace 5.1 (mvn package). Then run ant update. This should complete, as you'll have no existing Solr indexes for it to complain about. 5. Finally, start Tomcat back up. Now, sit back and wait. Don't run any other commands. But, feel free to check the logs for any issues as Tomcat boots up. Behind the scenes, DSpace should *automatically* be reindexing your content (and recreating the search index from nothing). You won't need to manually run anything (like 'index-discovery -b'), as this is all automated in DSpace 5 on the first bootup. (After the first bootup though, 'index-discovery -b' is the way to force a reindex.) I'm just curious if starting from *completely empty* indexes and installing DSpace 5 (without running 'index-discovery' manually) will act any different. If you still hit problems here, I'd recommend sending us a full stacktrace of the error messages you encounter. - Tim George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -Original Message- From: Tim Donohue [mailto:tdono...@duraspace.org] Sent: Tuesday, April 07, 2015 11:52 AM To: George Stanley Kozak; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi George, What happens if you do the following against your old 1.8.2 indexes: 1. wget http://search.maven.org/remotecontent?filepath=org/apache/lucene/lucene-core/3.5.0/lucene-core-3.5.0.jar; 2. java -cp lucene-core-3.5.0.jar org.apache.lucene.index.IndexUpgrader /cul/app/dspace/solr/statistics/data/index 3. java -cp lucene-core-3.5.0.jar org.apache.lucene.index.IndexUpgrader /cul/app/dspace/solr/search/data/index Do either of those IndexUpgrader commands succeed against your *old* 1.8.2 indexes? If not, could you send along the exception stacktrace? I'm just trying to determine if the issue is actually in these old 1.8.2 indexes. It may be either they are indexes from an unexpected, older version of Solr, or there's some minor index corruption here. -- BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT Develop your own process
Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)
Hi, Tim: 1) I did the lucene-core-3.5.0.jar manual update of the indexes. For the search index, I got no errors, but for the statistics core, I got: Exception in thread main org.apache.lucene.index.IndexNotFoundException: org.apache.lucene.store.MMapDirectory@/cul/app/dspace/solr/statistics/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2e888d65 at org.apache.lucene.index.IndexUpgrader.upgrade(IndexUpgrader.java:118) at org.apache.lucene.index.IndexUpgrader.main(IndexUpgrader.java:85) 2) I deleted the statistics core and did the ant update. I brought up my DSpace 5.1 test system and when I tried to do a search, I got no results. When I went to do a browse, I got: ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ anonymous:session_id=5749DBA88953D8B0ABDDA3FFD1DFF5F2:ip_addr=128.84.117.195:Error in Discovery while setting up date facet range:date facet\colon; dateIssued.year SolrCore 'search' is not available due to init failure: Index locked for write for core search,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Index locked for write for core search 3) I stopped tomcat, deleted the search index (as you suggested) and did the mvn build and ant update and then copied the new webapps over to tomcat. Everything looked good. I started tomcat and my site came up. Before doing anything more I ran ./dspace index-discovery -b. It ran to completion, but all searches return no hits and the browses give me the same write lock error as above. I made sure the permissions are correct (dspace user owns the solr indexes and tomcat) and removed the write lock and tried again, but again I get the same error. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -Original Message- From: Tim Donohue [mailto:tdono...@duraspace.org] Sent: Tuesday, April 07, 2015 11:52 AM To: George Stanley Kozak; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi George, What happens if you do the following against your old 1.8.2 indexes: 1. wget http://search.maven.org/remotecontent?filepath=org/apache/lucene/lucene-core/3.5.0/lucene-core-3.5.0.jar; 2. java -cp lucene-core-3.5.0.jar org.apache.lucene.index.IndexUpgrader /cul/app/dspace/solr/statistics/data/index 3. java -cp lucene-core-3.5.0.jar org.apache.lucene.index.IndexUpgrader /cul/app/dspace/solr/search/data/index Do either of those IndexUpgrader commands succeed against your *old* 1.8.2 indexes? If not, could you send along the exception stacktrace? I'm just trying to determine if the issue is actually in these old 1.8.2 indexes. It may be either they are indexes from an unexpected, older version of Solr, or there's some minor index corruption here. -- To be honest, another thing to note here is that your *search* indexes can be easily rebuilt in DSpace 5.x (simply run ./dspace index-discovery -b). The only older index that really requires this upgrade process is the *statistics* index (as currently it doesn't have a reindex command). So, if your statistics index is of no importance, you could just delete both of these old indexes and upgrade to 5.1. As part of the 5.1 upgrade, a full reindex of the search index will be kicked off (when you start Tomcat). So, after a few minutes, your content should be fully reindexed and working. - Tim On 4/7/2015 10:12 AM, George Stanley Kozak wrote: Thanks for your help, Tim: I'm afraid that I keep hitting a wall when I try to upgrade to DSpace 5.1 I tried to go directly from DSpace 1.8.2 to DSpace 5.1 and got: ERROR occurred while checking Solr index version: Exception in thread main java.io.IOException: Could not read Lucene segments files in /cul/app/dspace/solr/statistics/data/index We have been using our own stats gathering system instead of the one within DSpace. So, I removed the statistics index but then I got: Exception in thread main java.io.IOException: Could not read Lucene segments files in /cul/app/dspace/solr/search/data/index This version of Lucene only supports indexes created with release 3.0 and later. So, I went ahead and used the DSpace 4.3 version of our solr indexes (without the statistics). It ran fine and everything got upgraded, but when I tried to do any searching, I get the write lock error. If I turn off tomcat, delete the write lock and restart things, I get the same thing. Also, I cannot update the discovery index because of the write lock problem. By the way, I did try the manual upgrade of the Solr indexes defined in the manual, but that didn't work for me, either. So, right now, it looks like I can only upgrade to DSpace 4.3, unless you have some more suggestions for me. George Kozak Digital Library Specialist
Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)
Hi, Tim: To answer one of your questions: my original DSpace 1.8.2 solr statistics core did have data in it. Attached is a snapshot of the index directory. Now, I did as you asked. I deleted the all of the solr cores (actually everything under /cul/app/dspace/solr) I did the mvn and ant update and everything went well. I copied the webapps over to tomcat and restarted tomcat. I watched the DSpace log and the catalina.out log. Everything seemed to move along smoothly. When everything was done, I waited a few minutes before I went to my DSpace home page. When my home page came up, I began to see errors in my DSpace log: ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ anonymous:session_id=B3B12E105B95C8FB8E63E9FB94B5971F:ip_addr=128.84.117.195:Error in Discovery while setting up date facet range:date facet\colon; dateIssued.year SolrCore 'search' is not available due to init failure: Index locked for write for core search,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Index locked for write for core search Any attempt to do any browse or search results with errors. By the way, the errors on the web appear with org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Expected mime type application/octet-stream but got text/html. And then raw HTML after that. What has me stumped is that outside of the problem with the solr statistics, none of these problems appear in my DSpace 4.3 upgrade. I have no write lock problems in DSpace 4.3 install. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -Original Message- From: Tim Donohue [mailto:tdono...@duraspace.org] Sent: Tuesday, April 07, 2015 2:33 PM To: George Stanley Kozak; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi George, On 4/7/2015 1:09 PM, George Stanley Kozak wrote: Hi, Tim: 1) I did the lucene-core-3.5.0.jar manual update of the indexes. For the search index, I got no errors, but for the statistics core, I got: Exception in thread main org.apache.lucene.index.IndexNotFoundException: org.apache.lucene.store.MMapDirectory@/cul/app/dspace/solr/statistics/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2e888d65 at org.apache.lucene.index.IndexUpgrader.upgrade(IndexUpgrader.java:118) at org.apache.lucene.index.IndexUpgrader.main(IndexUpgrader.java:85) This would imply that the search index upgraded fine (no errors). But, the error for your statistics index almost implies that it's not a valid Solr index. Do you use the statistics index? Is there anything in the /cul/app/dspace/solr/statistics/data/index directory or is it empty? 2) I deleted the statistics core and did the ant update. I brought up my DSpace 5.1 test system and when I tried to do a search, I got no results. When I went to do a browse, I got: ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ anonymous:session_id=5749DBA88953D8B0ABDDA3FFD1DFF5F2:ip_addr=128.84.1 17.195:Error in Discovery while setting up date facet range:date facet\colon; dateIssued.year SolrCore 'search' is not available due to init failure: Index locked for write for core search,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Index locked for write for core search 3) I stopped tomcat, deleted the search index (as you suggested) and did the mvn build and ant update and then copied the new webapps over to tomcat. Everything looked good. I started tomcat and my site came up. Before doing anything more I ran ./dspace index-discovery -b. It ran to completion, but all searches return no hits and the browses give me the same write lock error as above. I made sure the permissions are correct (dspace user owns the solr indexes and tomcat) and removed the write lock and tried again, but again I get the same error. These other two errors seem rather odd to me...like something else is accessing Solr at the same time in which you are trying to reindex, and those multiple simultaneous writes are throwing errors. Can you try something even more simplistic than what you've done above? 1. Stop Tomcat 2. Copy your 1.8.3 data over to where you will install DSpace 5 3. Immediately delete *everything* under /cul/app/dspace/solr/ (including the entire statistics and search subdirectories). This will wipe out all existing indexes, obviously. 4. Build DSpace 5.1 (mvn package). Then run ant update. This should complete, as you'll have no existing Solr indexes for it to complain about. 5. Finally, start Tomcat back up. Now, sit back and wait. Don't run any other commands. But, feel free to check the logs for any issues as Tomcat boots
Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)
Good afternoon, We saw very similar errors (particularly Expected mime type application/octet-stream but got text/html) and found that we hadn't updated the appBase directory in tomcat.xml. Another symptom of ours was the code of the entire error page being written to the dspace log. It's a longshot, but perhaps something to check. B-- -Original Message- From: George Stanley Kozak [mailto:g...@cornell.edu] Sent: Tuesday, April 7, 2015 1:28 PM To: Tim Donohue; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi, Tim: To answer one of your questions: my original DSpace 1.8.2 solr statistics core did have data in it. Attached is a snapshot of the index directory. Now, I did as you asked. I deleted the all of the solr cores (actually everything under /cul/app/dspace/solr) I did the mvn and ant update and everything went well. I copied the webapps over to tomcat and restarted tomcat. I watched the DSpace log and the catalina.out log. Everything seemed to move along smoothly. When everything was done, I waited a few minutes before I went to my DSpace home page. When my home page came up, I began to see errors in my DSpace log: ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ anonymous:session_id=B3B12E105B95C8FB8E63E9FB94B5971F:ip_addr=128.84.117.195:Error in Discovery while setting up date facet range:date facet\colon; dateIssued.year SolrCore 'search' is not available due to init failure: Index locked for write for core search,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Index locked for write for core search Any attempt to do any browse or search results with errors. By the way, the errors on the web appear with org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Expected mime type application/octet-stream but got text/html. And then raw HTML after that. What has me stumped is that outside of the problem with the solr statistics, none of these problems appear in my DSpace 4.3 upgrade. I have no write lock problems in DSpace 4.3 install. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -Original Message- From: Tim Donohue [mailto:tdono...@duraspace.org] Sent: Tuesday, April 07, 2015 2:33 PM To: George Stanley Kozak; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi George, On 4/7/2015 1:09 PM, George Stanley Kozak wrote: Hi, Tim: 1) I did the lucene-core-3.5.0.jar manual update of the indexes. For the search index, I got no errors, but for the statistics core, I got: Exception in thread main org.apache.lucene.index.IndexNotFoundException: org.apache.lucene.store.MMapDirectory@/cul/app/dspace/solr/statistics/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2e888d65 at org.apache.lucene.index.IndexUpgrader.upgrade(IndexUpgrader.java:118) at org.apache.lucene.index.IndexUpgrader.main(IndexUpgrader.java:85) This would imply that the search index upgraded fine (no errors). But, the error for your statistics index almost implies that it's not a valid Solr index. Do you use the statistics index? Is there anything in the /cul/app/dspace/solr/statistics/data/index directory or is it empty? 2) I deleted the statistics core and did the ant update. I brought up my DSpace 5.1 test system and when I tried to do a search, I got no results. When I went to do a browse, I got: ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ anonymous:session_id=5749DBA88953D8B0ABDDA3FFD1DFF5F2:ip_addr=128.84.1 17.195:Error in Discovery while setting up date facet range:date facet\colon; dateIssued.year SolrCore 'search' is not available due to init failure: Index locked for write for core search,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Index locked for write for core search 3) I stopped tomcat, deleted the search index (as you suggested) and did the mvn build and ant update and then copied the new webapps over to tomcat. Everything looked good. I started tomcat and my site came up. Before doing anything more I ran ./dspace index-discovery -b. It ran to completion, but all searches return no hits and the browses give me the same write lock error as above. I made sure the permissions are correct (dspace user owns the solr indexes and tomcat) and removed the write lock and tried again, but again I get the same error. These other two errors seem rather odd to me...like something else is accessing Solr at the same time in which you are trying to reindex, and those multiple simultaneous writes are throwing errors. Can you try something
Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)
Hi, Brian: Sorry for being dense, but what do you mean by updating the appBase directory? George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -Original Message- From: Brian Freels-Stendel [mailto:bfre...@unm.edu] Sent: Tuesday, April 07, 2015 3:47 PM To: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Good afternoon, We saw very similar errors (particularly Expected mime type application/octet-stream but got text/html) and found that we hadn't updated the appBase directory in tomcat.xml. Another symptom of ours was the code of the entire error page being written to the dspace log. It's a longshot, but perhaps something to check. B-- -Original Message- From: George Stanley Kozak [mailto:g...@cornell.edu] Sent: Tuesday, April 7, 2015 1:28 PM To: Tim Donohue; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi, Tim: To answer one of your questions: my original DSpace 1.8.2 solr statistics core did have data in it. Attached is a snapshot of the index directory. Now, I did as you asked. I deleted the all of the solr cores (actually everything under /cul/app/dspace/solr) I did the mvn and ant update and everything went well. I copied the webapps over to tomcat and restarted tomcat. I watched the DSpace log and the catalina.out log. Everything seemed to move along smoothly. When everything was done, I waited a few minutes before I went to my DSpace home page. When my home page came up, I began to see errors in my DSpace log: ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ anonymous:session_id=B3B12E105B95C8FB8E63E9FB94B5971F:ip_addr=128.84.117.195:Error in Discovery while setting up date facet range:date facet\colon; dateIssued.year SolrCore 'search' is not available due to init failure: Index locked for write for core search,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Index locked for write for core search Any attempt to do any browse or search results with errors. By the way, the errors on the web appear with org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Expected mime type application/octet-stream but got text/html. And then raw HTML after that. What has me stumped is that outside of the problem with the solr statistics, none of these problems appear in my DSpace 4.3 upgrade. I have no write lock problems in DSpace 4.3 install. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -Original Message- From: Tim Donohue [mailto:tdono...@duraspace.org] Sent: Tuesday, April 07, 2015 2:33 PM To: George Stanley Kozak; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi George, On 4/7/2015 1:09 PM, George Stanley Kozak wrote: Hi, Tim: 1) I did the lucene-core-3.5.0.jar manual update of the indexes. For the search index, I got no errors, but for the statistics core, I got: Exception in thread main org.apache.lucene.index.IndexNotFoundException: org.apache.lucene.store.MMapDirectory@/cul/app/dspace/solr/statistics/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2e888d65 at org.apache.lucene.index.IndexUpgrader.upgrade(IndexUpgrader.java:118) at org.apache.lucene.index.IndexUpgrader.main(IndexUpgrader.java:85) This would imply that the search index upgraded fine (no errors). But, the error for your statistics index almost implies that it's not a valid Solr index. Do you use the statistics index? Is there anything in the /cul/app/dspace/solr/statistics/data/index directory or is it empty? 2) I deleted the statistics core and did the ant update. I brought up my DSpace 5.1 test system and when I tried to do a search, I got no results. When I went to do a browse, I got: ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ anonymous:session_id=5749DBA88953D8B0ABDDA3FFD1DFF5F2:ip_addr=128.84.1 17.195:Error in Discovery while setting up date facet range:date facet\colon; dateIssued.year SolrCore 'search' is not available due to init failure: Index locked for write for core search,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Index locked for write for core search 3) I stopped tomcat, deleted the search index (as you suggested) and did the mvn build and ant update and then copied the new webapps over to tomcat. Everything looked good. I started tomcat and my site came up. Before doing anything more I ran ./dspace index-discovery -b. It ran
Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)
Hey there, Sorry for not being descriptive enough. In tomcat.xml, there's a Host section where the DSpace app context fragments go: ___ Host name=localhost appBase=/opt/dspace/webapps unpackWARs=true autoDeploy=true xmlValidation=false xmlNamespaceAware=false !-- DEFINE A CONTEXT PATH FOR DSpace XML User Interface (Manakin) -- Context path=/ docBase=/opt/dspace/webapps/xmlui debug=0 reloadable=true cachingAllowed=false allowLinking=true/ ___ In that first line, the appBase may still be pointed to Tomcat's default. For us, it let DSpace start and function semi-well, but none of the SOLR stuff would work. B-- -Original Message- From: George Stanley Kozak [mailto:g...@cornell.edu] Sent: Tuesday, April 7, 2015 2:04 PM To: Brian Freels-Stendel; dspace-tech@lists.sourceforge.net Subject: RE: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi, Brian: Sorry for being dense, but what do you mean by updating the appBase directory? George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -Original Message- From: Brian Freels-Stendel [mailto:bfre...@unm.edu] Sent: Tuesday, April 07, 2015 3:47 PM To: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Good afternoon, We saw very similar errors (particularly Expected mime type application/octet-stream but got text/html) and found that we hadn't updated the appBase directory in tomcat.xml. Another symptom of ours was the code of the entire error page being written to the dspace log. It's a longshot, but perhaps something to check. B-- -Original Message- From: George Stanley Kozak [mailto:g...@cornell.edu] Sent: Tuesday, April 7, 2015 1:28 PM To: Tim Donohue; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi, Tim: To answer one of your questions: my original DSpace 1.8.2 solr statistics core did have data in it. Attached is a snapshot of the index directory. Now, I did as you asked. I deleted the all of the solr cores (actually everything under /cul/app/dspace/solr) I did the mvn and ant update and everything went well. I copied the webapps over to tomcat and restarted tomcat. I watched the DSpace log and the catalina.out log. Everything seemed to move along smoothly. When everything was done, I waited a few minutes before I went to my DSpace home page. When my home page came up, I began to see errors in my DSpace log: ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ anonymous:session_id=B3B12E105B95C8FB8E63E9FB94B5971F:ip_addr=128.84.117.195:Error in Discovery while setting up date facet range:date facet\colon; dateIssued.year SolrCore 'search' is not available due to init failure: Index locked for write for core search,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Index locked for write for core search Any attempt to do any browse or search results with errors. By the way, the errors on the web appear with org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Expected mime type application/octet-stream but got text/html. And then raw HTML after that. What has me stumped is that outside of the problem with the solr statistics, none of these problems appear in my DSpace 4.3 upgrade. I have no write lock problems in DSpace 4.3 install. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -Original Message- From: Tim Donohue [mailto:tdono...@duraspace.org] Sent: Tuesday, April 07, 2015 2:33 PM To: George Stanley Kozak; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi George, On 4/7/2015 1:09 PM, George Stanley Kozak wrote: Hi, Tim: 1) I did the lucene-core-3.5.0.jar manual update of the indexes. For the search index, I got no errors, but for the statistics core, I got: Exception in thread main org.apache.lucene.index.IndexNotFoundException: org.apache.lucene.store.MMapDirectory@/cul/app/dspace/solr/statistics/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2e888d65 at org.apache.lucene.index.IndexUpgrader.upgrade(IndexUpgrader.java:118) at org.apache.lucene.index.IndexUpgrader.main(IndexUpgrader.java:85) This would imply that the search index upgraded fine (no errors). But, the error for your statistics index almost implies that it's not a valid Solr index. Do you use the statistics index? Is there anything in the /cul/app/dspace/solr/statistics/data/index directory
Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)
Hi George, The error below is essentially saying that your Solr statistics index is too old for the version of Solr packages with DSpace 4.3 to upgrade. However, in the DSpace 5.1 upgrade process, during the ant update step, we've attempted to catch this scenario automatically -- we try to determine the version of a Solr index, and upgrade it automatically to the latest version. So, I'd recommend trying to point DSpace 5.1 at your old Solr index, and running ant update. It is supposed to upgrade an old index automatically. If that doesn't work for some reason, another option is to do a more manual upgrade, as described in the DSpace 5.x documentation. This manual upgrade essentially requires downloading multiple versions of the Solr/Lucene core JAR, and running it against your index in order to upgrade it to the latest compatible version (this is essentially the same process that is now automated by ant update though) https://wiki.duraspace.org/display/DSDOC5x/Upgrading+DSpace#UpgradingDSpace-ManuallyUpgradingSolrIndexes By the way, from your earlier messages, recently I also did encounter that odd Solr write.lock error (Index locked for write..) during a recent upgrade I tried to 5.1. In my situation, it seemed like it was a stale write.lock file which was somehow sitting around. After clearing it out, the upgrade proceeded. Admittedly, I still need to dig a bit further and ensure my analysis is correct. - Tim On 4/1/2015 3:16 PM, George Stanley Kozak wrote: Hi, Everyone: I have discovered that when I try to optimize my Solr indexes when I upgraded from DSpace 3.3 to 4.3 (wget ‘http://localhost:8080/solr/statistics/update?optimize=true’) I get the following errors in the solr logs: ERROR org.apache.solr.core.CoreContainer @ Unable to create core: statistics Format version is not supported (resource: segment _32 in resource ChecksumIndexInput(MMapIndexInput(path=/cul/app/dspace/solr/statistics/data/index/segments_1p0))): 2.x. This version of Lucene only supports indexes created with release 3.0 and later. If I later try to upgrade to 5.1, I see during the ant update: /cul/app/dspace/src/dspace-5.1-src-release/dspace/target/dspace-installer/build.xml:1061: ERROR occurred while checking Solr index version: Exception in thread main java.io.IOException: Could not read Lucene segments files in /cul/app/dspace/solr/statistics/data/index I know the 5.1 documentation states that you can manually update the Solr indexes. Should that be done in my 4.3 upgrade before I try to go to 5.1 or should it be done in the 5.1 upgrade? George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT Develop your own process in accordance with the BPMN 2 standard Learn Process modeling best practices with Bonita BPM through live exercises http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_ source=Sourceforge_BPM_Camp_5_6_15utm_medium=emailutm_campaign=VA_SF ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)
Tim: Thank you for your advice. I did try the 5.1 ant update, but it failed. Maybe if I tried it going from DSpace 3.3 or 1.8.2 directly to DSpace 5.1 instead of doing the intermediate step to 4.3, this might work? I will try a few other things and let everyone know what works. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CULIT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 From: Tim Donohue tdono...@duraspace.org Sent: Monday, April 6, 2015 4:44 PM To: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information) Hi George, The error below is essentially saying that your Solr statistics index is too old for the version of Solr packages with DSpace 4.3 to upgrade. However, in the DSpace 5.1 upgrade process, during the ant update step, we've attempted to catch this scenario automatically -- we try to determine the version of a Solr index, and upgrade it automatically to the latest version. So, I'd recommend trying to point DSpace 5.1 at your old Solr index, and running ant update. It is supposed to upgrade an old index automatically. If that doesn't work for some reason, another option is to do a more manual upgrade, as described in the DSpace 5.x documentation. This manual upgrade essentially requires downloading multiple versions of the Solr/Lucene core JAR, and running it against your index in order to upgrade it to the latest compatible version (this is essentially the same process that is now automated by ant update though) https://wiki.duraspace.org/display/DSDOC5x/Upgrading+DSpace#UpgradingDSpace-ManuallyUpgradingSolrIndexes By the way, from your earlier messages, recently I also did encounter that odd Solr write.lock error (Index locked for write..) during a recent upgrade I tried to 5.1. In my situation, it seemed like it was a stale write.lock file which was somehow sitting around. After clearing it out, the upgrade proceeded. Admittedly, I still need to dig a bit further and ensure my analysis is correct. - Tim On 4/1/2015 3:16 PM, George Stanley Kozak wrote: Hi, Everyone: I have discovered that when I try to optimize my Solr indexes when I upgraded from DSpace 3.3 to 4.3 (wget ‘http://localhost:8080/solr/statistics/update?optimize=true’) I get the following errors in the solr logs: ERROR org.apache.solr.core.CoreContainer @ Unable to create core: statistics Format version is not supported (resource: segment _32 in resource ChecksumIndexInput(MMapIndexInput(path=/cul/app/dspace/solr/statistics/data/index/segments_1p0))): 2.x. This version of Lucene only supports indexes created with release 3.0 and later. If I later try to upgrade to 5.1, I see during the ant update: /cul/app/dspace/src/dspace-5.1-src-release/dspace/target/dspace-installer/build.xml:1061: ERROR occurred while checking Solr index version: Exception in thread main java.io.IOException: Could not read Lucene segments files in /cul/app/dspace/solr/statistics/data/index I know the 5.1 documentation states that you can manually update the Solr indexes. Should that be done in my 4.3 upgrade before I try to go to 5.1 or should it be done in the 5.1 upgrade? George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT Develop your own process in accordance with the BPMN 2 standard Learn Process modeling best practices with Bonita BPM through live exercises http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_ source=Sourceforge_BPM_Camp_5_6_15utm_medium=emailutm_campaign=VA_SF ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- BPM Camp - Free Virtual Workshop May 6th at 10am PDT
Re: [Dspace-tech] Error after install of DSpace 5.1
Thanks, Andrea: I do backup everything with every upgrade and that has been a help since I have had to go back and forth from 4.x to 5.x multiple times. I might just drop teh stats since I am not using them for my test instance. I am planning on converting our production instance soon, and I may take the the indexes from that and try them and see if I get the same error. Thanks for your help. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CULIT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 From: Andrea Schweer schw...@waikato.ac.nz Sent: Wednesday, April 1, 2015 11:09 PM To: George Stanley Kozak; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 Hi, On 02/04/15 12:50, George Stanley Kozak wrote: ?Andrea: I'm not sure if you saw my follow up post. Yes, I did see errors in the Solr log. It is: ERROR org.apache.solr.core.CoreContainer @ Unable to create core: statistics Format version is not supported (resource: segment _32 in resource ChecksumIndexInput(MMapIndexInput(path=/cul/app/dspace/solr/statistics/data/index/segments_1p0))): 2.x. This version of Lucene only supports indexes created with release 3.0 and later. If I later try to upgrade to 5.1, I see during the ant update: /cul/app/dspace/src/dspace-5.1-src-release/dspace/target/dspace-installer/build.xml:1061: ERROR occurred while checking Solr index version: Exception in thread main java.io.IOException: Could not read Lucene segments files in /cul/app/dspace/solr/statistics/data/index No sorry, I didn't see that. I did however run into a very similar problem with one of my own DSpace 5.1 development instances the other day. I'm not quite sure what state the solr indexes were in at that stage, so I'm not sure how to reproduce it. It does look like we should add a big warning to the upgrade instructions for people to back up their solr index first -- my problem went away when I re-set the usage stats data dir in my dev instance to a fresh copy from (4.2) production. I did try to manually update the Solr statistics using the instructions in the DSpace 5.1 Manual, but that didn't seem to help.. My suspicion is that once the upgrade fails halfway through for some reason, there is no way to recover from the situation. Hence the importance of the back-up! Is there a way to tell the DSpace that we don't want to use the Solr Stats? We have our own statistics gathering so I'm really not using them. It looks like you can disable adding new hits to the Solr statistics by commenting out the SolrLoggerUsageEventListener bean in dspace-xmlui/src/main/webapp/WEB-INF/spring/applicationContext.xml (XMLUI) / dspace-jspui/src/main/webapp/WEB-INF/spring//applicationContext.xml (JSPUI). Then you could delete the contents of [dspace]/solr/statistics/data (for anyone following along on this -- this _will_ wipe all your usage stats data! Do this only when you're sure that's what you want!) and that should be it. But I've never tried this, so it may break lots of things in lots of interesting ways. cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1
Hi, Everyone: I have been working on debugging the problems that I had upgrading from DSpace 4.x to DSpace 5.x for several days. I went ahead and redid my installs on my development server starting with DSpace 1.82, then upgrading to DSpace 3.3 and then to DSpace 4.3. I am now preparing the upgrade from 4.3 to 5.1. However, I found a problem in my upgrade from 3.3 to 4.3. When I do the optimize of the solr indexes, my optimize of core search and oai work fine, but when I try to optimize statistics, I get an internal server error: wget 'http://localhost:8080/solr/statistics/update?optimize=true' results in Connecting to localhost|::1|:8080... connected. HTTP request sent, awaiting response... 500 Internal Server Error Does anyone have suggestions as to what I can do to fix this? (By the way, the 4.3 install seems to run OK even with this error). George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1
?Andrea: I'm not sure if you saw my follow up post. Yes, I did see errors in the Solr log. It is: ERROR org.apache.solr.core.CoreContainer @ Unable to create core: statistics Format version is not supported (resource: segment _32 in resource ChecksumIndexInput(MMapIndexInput(path=/cul/app/dspace/solr/statistics/data/index/segments_1p0))): 2.x. This version of Lucene only supports indexes created with release 3.0 and later. If I later try to upgrade to 5.1, I see during the ant update: /cul/app/dspace/src/dspace-5.1-src-release/dspace/target/dspace-installer/build.xml:1061: ERROR occurred while checking Solr index version: Exception in thread main java.io.IOException: Could not read Lucene segments files in /cul/app/dspace/solr/statistics/data/index I did try to manually update the Solr statistics using the instructions in the DSpace 5.1 Manual, but that didn't seem to help.. Is there a way to tell the DSpace that we don't want to use the Solr Stats? We have our own statistics gathering so I'm really not using them. ? George Kozak Digital Library Specialist Cornell University Library Information Technologies (CULIT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 From: Andrea Schweer schw...@waikato.ac.nz Sent: Wednesday, April 1, 2015 6:11 PM To: George Stanley Kozak; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 Hi George, On 02/04/15 04:22, George Stanley Kozak wrote: I found a problem in my upgrade from 3.3 to 4.3. When I do the optimize of the solr indexes, my optimize of core search and oai work fine, but when I try to optimize statistics, I get an internal server error: wget 'http://localhost:8080/solr/statistics/update?optimize=true' results in Connecting to localhost|::1|:8080... connected. HTTP request sent, awaiting response... 500 Internal Server Error Does anyone have suggestions as to what I can do to fix this? (By the way, the 4.3 install seems to run OK even with this error). Is there any more information about the 500 error? Maybe in the solr log or in the tomcat log, either from the time you made the request or when you started up tomcat? cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1
Hi, On 02/04/15 12:50, George Stanley Kozak wrote: Andrea: I'm not sure if you saw my follow up post. Yes, I did see errors in the Solr log. It is: ERROR org.apache.solr.core.CoreContainer @ Unable to create core: statistics Format version is not supported (resource: segment _32 in resource ChecksumIndexInput(MMapIndexInput(path="/cul/app/dspace/solr/statistics/data/index/segments_1p0"))): 2.x. This version of Lucene only supports indexes created with release 3.0 and later. If I later try to upgrade to 5.1, I see during the ant update: /cul/app/dspace/src/dspace-5.1-src-release/dspace/target/dspace-installer/build.xml:1061: ERROR occurred while checking Solr index version: Exception in thread "main" java.io.IOException: Could not read Lucene segments files in /cul/app/dspace/solr/statistics/data/index No sorry, I didn't see that. I did however run into a very similar problem with one of my own DSpace 5.1 development instances the other day. I'm not quite sure what state the solr indexes were in at that stage, so I'm not sure how to reproduce it. It does look like we should add a big warning to the upgrade instructions for people to back up their solr index first -- my problem went away when I re-set the usage stats data dir in my dev instance to a fresh copy from (4.2) production. I did try to manually update the Solr statistics using the instructions in the DSpace 5.1 Manual, but that didn't seem to help.. My suspicion is that once the upgrade fails halfway through for some reason, there is no way to recover from the situation. Hence the importance of the back-up! Is there a way to tell the DSpace that we don't want to use the Solr Stats? We have our own statistics gathering so I'm really not using them. It looks like you can disable adding new hits to the Solr statistics by commenting out the SolrLoggerUsageEventListener bean in dspace-xmlui/src/main/webapp/WEB-INF/spring/applicationContext.xml (XMLUI) / dspace-jspui/src/main/webapp/WEB-INF/spring//applicationContext.xml (JSPUI). Then you could delete the contents of [dspace]/solr/statistics/data (for anyone following along on this -- this _will_ wipe all your usage stats data! Do this only when you're sure that's what you want!) and that should be it. But I've never tried this, so it may break lots of things in lots of interesting ways. cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1
Hi George, On 02/04/15 04:22, George Stanley Kozak wrote: I found a problem in my upgrade from 3.3 to 4.3. When I do the optimize of the solr indexes, my optimize of core “search” and “oai” work fine, but when I try to optimize “statistics”, I get an internal server error: wget ‘http://localhost:8080/solr/statistics/update?optimize=true’ results in Connecting to localhost|::1|:8080... connected. HTTP request sent, awaiting response... 500 Internal Server Error Does anyone have suggestions as to what I can do to fix this? (By the way, the 4.3 install seems to run OK even with this error). Is there any more information about the 500 error? Maybe in the solr log or in the tomcat log, either from the time you made the request or when you started up tomcat? cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1 (Additional information)
Hi, Everyone: I have discovered that when I try to optimize my Solr indexes when I upgraded from DSpace 3.3 to 4.3 (wget 'http://localhost:8080/solr/statistics/update?optimize=true') I get the following errors in the solr logs: ERROR org.apache.solr.core.CoreContainer @ Unable to create core: statistics Format version is not supported (resource: segment _32 in resource ChecksumIndexInput(MMapIndexInput(path=/cul/app/dspace/solr/statistics/data/index/segments_1p0))): 2.x. This version of Lucene only supports indexes created with release 3.0 and later. If I later try to upgrade to 5.1, I see during the ant update: /cul/app/dspace/src/dspace-5.1-src-release/dspace/target/dspace-installer/build.xml:1061: ERROR occurred while checking Solr index version: Exception in thread main java.io.IOException: Could not read Lucene segments files in /cul/app/dspace/solr/statistics/data/index I know the 5.1 documentation states that you can manually update the Solr indexes. Should that be done in my 4.3 upgrade before I try to go to 5.1 or should it be done in the 5.1 upgrade? George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1
George just in case … dspace was misbehaving mysteriously and badly for me - I just fixed this by doing chmod +w /Users/monikam/Installations/tstspace/solr/search/conf/schema.xml If your problems continue - have a look at the access settings on your solr directory Monika Monika Mevenkamp phone: 609-258-4161 Lewis Library,Washington Road and Ivy Lane, Princeton University, Princeton, NJ 08544 On Mar 26, 2015, at 10:11 AM, George Stanley Kozak g...@cornell.edumailto:g...@cornell.edu wrote: Thank you, Monika. I do check to see that tomcat is shutdown. Also, I have been finding and deleting the write.lock file, as well, but I’ll keep digging. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 From: Monika C. Mevenkamp [mailto:moni...@princeton.edu] Sent: Thursday, March 26, 2015 9:59 AM To: Andrea Schweer Cc: George Stanley Kozak; dspace-tech@lists.sourceforge.netmailto:dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 my tomcat sometimes refuses to shutdown to make sure it is really gone I do ps -ef | frep omcat this may need to be ps -aux | fgrep omcat on your system . Maybe for some strange reason the locks were not removed properly ? Look for the lock files: ls DSPACE_INSTALLDIR/solr/*/data/index/write.lock ls DSPACE_INSTALLDIR/solr/*/data/spellchecker/write.lock you can use lsof command to see which process ‘owns’ a file lsof DSPACE_INSTALLDIR/solr/*/data/index/write.lock Monika Monika Mevenkamp phone: 609-258-4161 Lewis Library,Washington Road and Ivy Lane, Princeton University, Princeton, NJ 08544 On Mar 25, 2015, at 8:15 PM, Andrea Schweer schw...@waikato.ac.nzmailto:schw...@waikato.ac.nz wrote: Hi George, On 26/03/15 10:55, George Stanley Kozak wrote: Yes, it has me stumped. If I restore the Solr indexes and the postgres database back to the DSpace 4.2 versions and bring up DSpace 4.2, everything works fine. I do shutdown tomcat when I do the ant step. Here are my steps, and let me know if I have forgotten anything: That all looks good so far 1) Shutdown tomcat 2) Run mvn –U clear package (I also tried with the Mirage2 option) 3) After successful build, I go to the target/dspace-install directory and run “ant update” As which user do you run ant? Same as the one that tomcat is running as? 4) After that is successful, I copy the webpass over to tomcat 5) Restart tomcat 6) Go to view the system The system comes up and I have access to the database records, but all browses and searches generate the index lock error. When you look for the write locks after step 5, what are their permissions? You should be able to find them (assuming a Linux system) using find [dspace]/solr -name *.lock cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.netmailto:DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1
my tomcat sometimes refuses to shutdown to make sure it is really gone I do ps -ef | frep omcat this may need to be ps -aux | fgrep omcat on your system . Maybe for some strange reason the locks were not removed properly ? Look for the lock files: ls DSPACE_INSTALLDIR/solr/*/data/index/write.lock ls DSPACE_INSTALLDIR/solr/*/data/spellchecker/write.lock you can use lsof command to see which process ‘owns’ a file lsof DSPACE_INSTALLDIR/solr/*/data/index/write.lock Monika Monika Mevenkamp phone: 609-258-4161 Lewis Library,Washington Road and Ivy Lane, Princeton University, Princeton, NJ 08544 On Mar 25, 2015, at 8:15 PM, Andrea Schweer schw...@waikato.ac.nzmailto:schw...@waikato.ac.nz wrote: Hi George, On 26/03/15 10:55, George Stanley Kozak wrote: Yes, it has me stumped. If I restore the Solr indexes and the postgres database back to the DSpace 4.2 versions and bring up DSpace 4.2, everything works fine. I do shutdown tomcat when I do the ant step. Here are my steps, and let me know if I have forgotten anything: That all looks good so far 1) Shutdown tomcat 2) Run mvn –U clear package (I also tried with the Mirage2 option) 3) After successful build, I go to the target/dspace-install directory and run “ant update” As which user do you run ant? Same as the one that tomcat is running as? 4) After that is successful, I copy the webpass over to tomcat 5) Restart tomcat 6) Go to view the system The system comes up and I have access to the database records, but all browses and searches generate the index lock error. When you look for the write locks after step 5, what are their permissions? You should be able to find them (assuming a Linux system) using find [dspace]/solr -name *.lock cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.netmailto:DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1
George, Just a note: you may want to see if any other errors occur in the DSpace or Tomcat logs when you restart tomcat (your step #5). I'm especially curious if any errors are thrown besides those index lock errors. They may provide hints as to what may be going on. - Tim On 3/25/2015 4:55 PM, George Stanley Kozak wrote: Andrea: Yes, it has me stumped. If I restore the Solr indexes and the postgres database back to the DSpace 4.2 versions and bring up DSpace 4.2, everything works fine. I do shutdown tomcat when I do the ant step. Here are my steps, and let me know if I have forgotten anything: 1)Shutdown tomcat 2)Run mvn –U clear package (I also tried with the Mirage2 option) 3)After successful build, I go to the target/dspace-install directory and run “ant update” 4)After that is successful, I copy the webpass over to tomcat 5)Restart tomcat 6)Go to view the system The system comes up and I have access to the database records, but all browses and searches generate the index lock error. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 *From:*Andrea Schweer [mailto:schw...@waikato.ac.nz] *Sent:* Wednesday, March 25, 2015 5:03 PM *To:* George Stanley Kozak; dspace-tech@lists.sourceforge.net *Subject:* Re: [Dspace-tech] Error after install of DSpace 5.1 Hi George, On 26/03/15 03:42, George Stanley Kozak wrote: So, I double checked the permissions on my solr index and tomcat. They are both owned by the DSpace user. If I bring my system up in DSpace 4.2 (after a restore), I have no problems. If I bring it up in 5.1, I get the error: “SolrCore 'search' is not available due to init failure: Index locked for write for core search,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Index locked for write for core search” I can’t even do an re-index. I have to say that I’m kind of stumped here. Works OK in 4.2, doesn’t work in 5.1 How odd. Are you running the ant step (with the 5.1 code) as the same user as tomcat? Also, when you run ant for the first time using the 5.1 code, tomcat *must not* be running -- ant will upgrade the solr indexes to the version used in DSpace 5.1 and it needs to have exclusive access to the index while that runs (and the mechanism for that is the write lock, which makes me suspect it's involved in the problem you're seeing). You have presumably checked already that the ant step with the 5.1 code doesn't throw any errors. cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1
Thank you, Monika. I do check to see that tomcat is shutdown. Also, I have been finding and deleting the write.lock file, as well, but I'll keep digging. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 From: Monika C. Mevenkamp [mailto:moni...@princeton.edu] Sent: Thursday, March 26, 2015 9:59 AM To: Andrea Schweer Cc: George Stanley Kozak; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 my tomcat sometimes refuses to shutdown to make sure it is really gone I do ps -ef | frep omcat this may need to be ps -aux | fgrep omcat on your system . Maybe for some strange reason the locks were not removed properly ? Look for the lock files: ls DSPACE_INSTALLDIR/solr/*/data/index/write.lock ls DSPACE_INSTALLDIR/solr/*/data/spellchecker/write.lock you can use lsof command to see which process 'owns' a file lsof DSPACE_INSTALLDIR/solr/*/data/index/write.lock Monika Monika Mevenkamp phone: 609-258-4161 Lewis Library,Washington Road and Ivy Lane, Princeton University, Princeton, NJ 08544 On Mar 25, 2015, at 8:15 PM, Andrea Schweer schw...@waikato.ac.nzmailto:schw...@waikato.ac.nz wrote: Hi George, On 26/03/15 10:55, George Stanley Kozak wrote: Yes, it has me stumped. If I restore the Solr indexes and the postgres database back to the DSpace 4.2 versions and bring up DSpace 4.2, everything works fine. I do shutdown tomcat when I do the ant step. Here are my steps, and let me know if I have forgotten anything: That all looks good so far 1) Shutdown tomcat 2) Run mvn -U clear package (I also tried with the Mirage2 option) 3) After successful build, I go to the target/dspace-install directory and run ant update As which user do you run ant? Same as the one that tomcat is running as? 4) After that is successful, I copy the webpass over to tomcat 5) Restart tomcat 6) Go to view the system The system comes up and I have access to the database records, but all browses and searches generate the index lock error. When you look for the write locks after step 5, what are their permissions? You should be able to find them (assuming a Linux system) using find [dspace]/solr -name *.lock cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.netmailto:DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1
Thanks, Tim: I didn't notice any errors, but I will check. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 -Original Message- From: Tim Donohue [mailto:tdono...@duraspace.org] Sent: Thursday, March 26, 2015 10:04 AM To: George Stanley Kozak; Andrea Schweer; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 George, Just a note: you may want to see if any other errors occur in the DSpace or Tomcat logs when you restart tomcat (your step #5). I'm especially curious if any errors are thrown besides those index lock errors. They may provide hints as to what may be going on. - Tim On 3/25/2015 4:55 PM, George Stanley Kozak wrote: Andrea: Yes, it has me stumped. If I restore the Solr indexes and the postgres database back to the DSpace 4.2 versions and bring up DSpace 4.2, everything works fine. I do shutdown tomcat when I do the ant step. Here are my steps, and let me know if I have forgotten anything: 1)Shutdown tomcat 2)Run mvn -U clear package (I also tried with the Mirage2 option) 3)After successful build, I go to the target/dspace-install directory and run ant update 4)After that is successful, I copy the webpass over to tomcat 5)Restart tomcat 6)Go to view the system The system comes up and I have access to the database records, but all browses and searches generate the index lock error. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 *From:*Andrea Schweer [mailto:schw...@waikato.ac.nz] *Sent:* Wednesday, March 25, 2015 5:03 PM *To:* George Stanley Kozak; dspace-tech@lists.sourceforge.net *Subject:* Re: [Dspace-tech] Error after install of DSpace 5.1 Hi George, On 26/03/15 03:42, George Stanley Kozak wrote: So, I double checked the permissions on my solr index and tomcat. They are both owned by the DSpace user. If I bring my system up in DSpace 4.2 (after a restore), I have no problems. If I bring it up in 5.1, I get the error: SolrCore 'search' is not available due to init failure: Index locked for write for core search,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Index locked for write for core search I can't even do an re-index. I have to say that I'm kind of stumped here. Works OK in 4.2, doesn't work in 5.1 How odd. Are you running the ant step (with the 5.1 code) as the same user as tomcat? Also, when you run ant for the first time using the 5.1 code, tomcat *must not* be running -- ant will upgrade the solr indexes to the version used in DSpace 5.1 and it needs to have exclusive access to the index while that runs (and the mechanism for that is the write lock, which makes me suspect it's involved in the problem you're seeing). You have presumably checked already that the ant step with the 5.1 code doesn't throw any errors. cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1
Hello, Andrea (and everyone else): So, I double checked the permissions on my solr index and tomcat. They are both owned by the DSpace user. If I bring my system up in DSpace 4.2 (after a restore), I have no problems. If I bring it up in 5.1, I get the error: SolrCore 'search' is not available due to init failure: Index locked for write for core search,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Index locked for write for core search I can't even do an re-index. I have to say that I'm kind of stumped here. Works OK in 4.2, doesn't work in 5.1 George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 From: Andrea Schweer [mailto:schw...@waikato.ac.nz] Sent: Tuesday, March 24, 2015 5:26 PM To: George Stanley Kozak; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 Hi George, On 25/03/15 10:06, George Stanley Kozak wrote: Thank you for your suggestions. I did check and my tomcat (tomcat 7) is owned by my dspace user as is my solr index, so tomcat should be able to write to the solr directory (it works OK for 4.2). So, I am not sure what is causing the permission problem, but I will keep checking. Have you run a chown with the -c flag on your solr directory? That will tell you about any changes it makes. Might be worth it just so you can go from tomcat should be able to write to the solr directory to tomcat is definitely able to write to the solr directory. Just on the off-chance this is happening to you: I've had nasty surprises when the solr directory was on a different partition and symlinked into [dspace], chown -R didn't work across the symlink (it changed ownership of the symlink but not of the symlinked directory). Also, if the permissions have been wrong in the past at some point, sometimes the write.lock file sticks around with wrong write permissions. In that case, it helps to stop tomcat, delete the write.lock file, then start tomcat. cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1
Hi George, On 26/03/15 03:42, George Stanley Kozak wrote: So, I double checked the permissions on my solr index and tomcat. They are both owned by the DSpace user. If I bring my system up in DSpace 4.2 (after a restore), I have no problems. If I bring it up in 5.1, I get the error: “SolrCore 'search' is not available due to init failure: Index locked for write for core search,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Index locked for write for core search” I can’t even do an re-index. I have to say that I’m kind of stumped here. Works OK in 4.2, doesn’t work in 5.1 How odd. Are you running the ant step (with the 5.1 code) as the same user as tomcat? Also, when you run ant for the first time using the 5.1 code, tomcat must not be running -- ant will upgrade the solr indexes to the version used in DSpace 5.1 and it needs to have exclusive access to the index while that runs (and the mechanism for that is the write lock, which makes me suspect it's involved in the problem you're seeing). You have presumably checked already that the ant step with the 5.1 code doesn't throw any errors. cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1
Hi George, On 26/03/15 10:55, George Stanley Kozak wrote: Yes, it has me stumped. If I restore the Solr indexes and the postgres database back to the DSpace 4.2 versions and bring up DSpace 4.2, everything works fine. I do shutdown tomcat when I do the ant step. Here are my steps, and let me know if I have forgotten anything: That all looks good so far 1) Shutdown tomcat 2) Run mvn –U clear package (I also tried with the Mirage2 option) 3) After successful build, I go to the target/dspace-install directory and run “ant update” As which user do you run ant? Same as the one that tomcat is running as? 4) After that is successful, I copy the webpass over to tomcat 5) Restart tomcat 6) Go to view the system The system comes up and I have access to the database records, but all browses and searches generate the index lock error. When you look for the write locks after step 5, what are their permissions? You should be able to find them (assuming a Linux system) using find [dspace]/solr -name "*.lock" cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1
Andrea: Yes, it has me stumped. If I restore the Solr indexes and the postgres database back to the DSpace 4.2 versions and bring up DSpace 4.2, everything works fine. I do shutdown tomcat when I do the ant step. Here are my steps, and let me know if I have forgotten anything: 1) Shutdown tomcat 2) Run mvn -U clear package (I also tried with the Mirage2 option) 3) After successful build, I go to the target/dspace-install directory and run ant update 4) After that is successful, I copy the webpass over to tomcat 5) Restart tomcat 6) Go to view the system The system comes up and I have access to the database records, but all browses and searches generate the index lock error. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 From: Andrea Schweer [mailto:schw...@waikato.ac.nz] Sent: Wednesday, March 25, 2015 5:03 PM To: George Stanley Kozak; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 Hi George, On 26/03/15 03:42, George Stanley Kozak wrote: So, I double checked the permissions on my solr index and tomcat. They are both owned by the DSpace user. If I bring my system up in DSpace 4.2 (after a restore), I have no problems. If I bring it up in 5.1, I get the error: SolrCore 'search' is not available due to init failure: Index locked for write for core search,trace=org.apache.solr.common.SolrException: SolrCore 'search' is not available due to init failure: Index locked for write for core search I can't even do an re-index. I have to say that I'm kind of stumped here. Works OK in 4.2, doesn't work in 5.1 How odd. Are you running the ant step (with the 5.1 code) as the same user as tomcat? Also, when you run ant for the first time using the 5.1 code, tomcat must not be running -- ant will upgrade the solr indexes to the version used in DSpace 5.1 and it needs to have exclusive access to the index while that runs (and the mechanism for that is the write lock, which makes me suspect it's involved in the problem you're seeing). You have presumably checked already that the ant step with the 5.1 code doesn't throw any errors. cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1
Thanks for the tips, Andrea. I will definitely check it out. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 From: Andrea Schweer [mailto:schw...@waikato.ac.nz] Sent: Tuesday, March 24, 2015 5:26 PM To: George Stanley Kozak; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 Hi George, On 25/03/15 10:06, George Stanley Kozak wrote: Thank you for your suggestions. I did check and my tomcat (tomcat 7) is owned by my dspace user as is my solr index, so tomcat should be able to write to the solr directory (it works OK for 4.2). So, I am not sure what is causing the permission problem, but I will keep checking. Have you run a chown with the -c flag on your solr directory? That will tell you about any changes it makes. Might be worth it just so you can go from tomcat should be able to write to the solr directory to tomcat is definitely able to write to the solr directory. Just on the off-chance this is happening to you: I've had nasty surprises when the solr directory was on a different partition and symlinked into [dspace], chown -R didn't work across the symlink (it changed ownership of the symlink but not of the symlinked directory). Also, if the permissions have been wrong in the past at some point, sometimes the write.lock file sticks around with wrong write permissions. In that case, it helps to stop tomcat, delete the write.lock file, then start tomcat. cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1
Hi George, On 25/03/15 10:06, George Stanley Kozak wrote: Thank you for your suggestions. I did check and my tomcat (tomcat 7) is owned by my dspace user as is my solr index, so tomcat should be able to write to the solr directory (it works OK for 4.2). So, I am not sure what is causing the permission problem, but I will keep checking. Have you run a chown with the -c flag on your solr directory? That will tell you about any changes it makes. Might be worth it just so you can go from "tomcat should be able to write to the solr directory" to "tomcat is definitely able to write to the solr directory". Just on the off-chance this is happening to you: I've had nasty surprises when the solr directory was on a different partition and symlinked into [dspace], chown -R didn't work across the symlink (it changed ownership of the symlink but not of the symlinked directory). Also, if the permissions have been wrong in the past at some point, sometimes the write.lock file sticks around with wrong write permissions. In that case, it helps to stop tomcat, delete the write.lock file, then start tomcat. cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1
Hi If you are using Tomcat7 the commands below should solve sudo chown tomcat7:tomcat7 [your_dspace_dir] -R sudo service tomcat7 restart Regards Royopa On Tue, Mar 24, 2015 at 4:59 PM, Andrea Schweer schw...@waikato.ac.nz wrote: Hi George, On 25/03/15 06:42, George Stanley Kozak wrote: I upgraded our DSpace 4.2 XMLUI to DSpace 5.1 (using Mirage 2). The install went fine, but once the system came up, if I try to view any records, I get: “The server encountered an internal error that prevented it from fulfilling this request.” and Caused by: org.apache.solr.common.SolrException: Index locked for write for core search and Caused by: org.apache.lucene.store.LockObtainFailedException: Index locked for write for core search Does anyone have any suggestions as to what I am doing wrong? This is a permissions problem. Tomcat needs to have write permissions on your [dspace]/solr directory and all its subdirectories. Change the permissions and restart tomcat -- hopefully that will solve the issue. cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- Rodrigo Prado de Jesus http://about.me/royopa roy...@gmail.com -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1
Hi George, On 25/03/15 06:42, George Stanley Kozak wrote: I upgraded our DSpace 4.2 XMLUI to DSpace 5.1 (using Mirage 2). The install went fine, but once the system came up, if I try to view any records, I get: “The server encountered an internal error that prevented it from fulfilling this request.” and Caused by: org.apache.solr.common.SolrException: Index locked for write for core search and Caused by: org.apache.lucene.store.LockObtainFailedException: Index locked for write for core search Does anyone have any suggestions as to what I am doing wrong? This is a permissions problem. Tomcat needs to have write permissions on your [dspace]/solr directory and all its subdirectories. Change the permissions and restart tomcat -- hopefully that will solve the issue. cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] Error after install of DSpace 5.1
Andrea and Royopa: Thank you for your suggestions. I did check and my tomcat (tomcat 7) is owned by my dspace user as is my solr index, so tomcat should be able to write to the solr directory (it works OK for 4.2). So, I am not sure what is causing the permission problem, but I will keep checking. George Kozak Digital Library Specialist Cornell University Library Information Technologies (CUL-IT) 218 Olin Library Cornell University Ithaca, NY 14853 607-255-8924 From: Andrea Schweer [mailto:schw...@waikato.ac.nz] Sent: Tuesday, March 24, 2015 4:00 PM To: George Stanley Kozak; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Error after install of DSpace 5.1 Hi George, On 25/03/15 06:42, George Stanley Kozak wrote: I upgraded our DSpace 4.2 XMLUI to DSpace 5.1 (using Mirage 2). The install went fine, but once the system came up, if I try to view any records, I get: The server encountered an internal error that prevented it from fulfilling this request. and Caused by: org.apache.solr.common.SolrException: Index locked for write for core search and Caused by: org.apache.lucene.store.LockObtainFailedException: Index locked for write for core search Does anyone have any suggestions as to what I am doing wrong? This is a permissions problem. Tomcat needs to have write permissions on your [dspace]/solr directory and all its subdirectories. Change the permissions and restart tomcat -- hopefully that will solve the issue. cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand -- Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette