Re: [Dspace-tech] DSpace 1.6 Handle server
I removed solr, and that didn't appear to really change anything.The hangup still happened. I did watch catalina.out again, and I saw these lines right at the hangup. INFO: Starting Servlet Engine: Apache Tomcat/5.5.26 Apr 2, 2010 1:54:21 PM org.apache.catalina.core.StandardHost start INFO: XML validation disabled log4j:WARN No appenders could be found for logger (org.apache.commons.digester.Digester.sax). log4j:WARN Please initialize the log4j system properly. It seems it's hanging at that line. I looked at my log4j.properties, and there is nothing strange in it. It looks like the standard DSpace-supplied log4j.properties file. I googled around for an answer to the problem, but couldn't find anything definite. -- Jason Fowler, CA, MSLS From: Stuart Lewis [s.le...@auckland.ac.nz] Sent: Friday, April 02, 2010 1:12 AM To: Jason Fowler Cc: Mark Diggory; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] DSpace 1.6 Handle server Hi Jason, You can disable the solr webapp by removing the whole solr directory from [tomcat]/webapps. Your storage server may be slowing down the loading of the solr index, but 10 minutes sounds quite severe. You could test it by moving the indexes to the local disk. They shouldn't get too big, so hopefully you'll have the space. They are normally kept in [dspace]/solr so move them to a local disk, then edit [tomcat]/solr/WEB-INF/web.xml Look for: env-entry env-entry-namesolr/home/env-entry-name env-entry-typejava.lang.String/env-entry-type env-entry-value/dspace/solr/env-entry-value /env-entry And edit /dspace/solr as appropriate for the new location of your solr index. Thanks, Stuart Lewis IT Innovations Analyst and Developer Te Tumu Herenga The University of Auckland Library Auckland Mail Centre, Private Bag 92019, Auckland 1142, New Zealand Ph: 64 9 373-7599 x81928 http://www.library.auckland.ac.nz/ On 2/04/2010, at 5:29 PM, Jason Fowler wrote: Mark, I'm not sure exactly how to disable the solr webapplication. Any suggestions? The file access is pretty quick. We've never had any trouble before. We have an app server for running the webapps and a storage server for everything else. --Jason Fowler, CA, MSLS From: mdigg...@gmail.com [mdigg...@gmail.com] On Behalf Of Mark Diggory [mdigg...@atmire.com] Sent: Thursday, April 01, 2010 11:31 PM To: Jason Fowler Cc: Stuart Lewis; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] DSpace 1.6 Handle server It looks like the delay is before the solr servlet init log entry is being called, IE before solr is loaded, disable the solr webapplication from loading and test if the delay is still there Likewise, what is the storage mount situation you are using here? If file access is slow across this mount, that be slowing Lucene access within Solr. You may try, moving your solr data directory onto the local file system and testing if that speeds your loading time. Finally, we have rather large solr indexes of statistics in production that do not have load times anywhere near this degree of slowness. Mark On Thu, Apr 1, 2010 at 7:52 PM, Jason Fowler jfow...@sbts.edu wrote: Stuart, I have confirmed from watching catalina.out that it consistently takes nearly10 minutes to start. Here's the section of the file where the hangup occurs. Apr 1, 2010 9:23:28 PM org.apache.coyote.http11.Http11BaseProtocol init INFO: Initializing Coyote HTTP/1.1 on http-8080 Apr 1, 2010 9:23:28 PM org.apache.catalina.startup.Catalina load INFO: Initialization processed in 537 ms Apr 1, 2010 9:23:28 PM org.apache.catalina.core.StandardService start INFO: Starting service Catalina Apr 1, 2010 9:23:28 PM org.apache.catalina.core.StandardEngine start INFO: Starting Servlet Engine: Apache Tomcat/5.5.26 Apr 1, 2010 9:23:28 PM org.apache.catalina.core.StandardHost start INFO: XML validation disabled Apr 1, 2010 9:33:12 PM org.apache.solr.servlet.SolrDispatchFilter init INFO: SolrDispatchFilter.init() Apr 1, 2010 9:33:12 PM org.apache.solr.core.SolrResourceLoader locateInstanceDir INFO: Using JNDI solr.home: /mnt/storage/dspace/solr Apr 1, 2010 9:33:12 PM org.apache.solr.core.CoreContainer$Initializer initialize INFO: looking for solr.xml: /mnt/storage/dspace/solr/solr.xml Apr 1, 2010 9:33:12 PM org.apache.solr.core.SolrResourceLoader init INFO: Solr home set to '/mnt/storage/dspace/solr/' For some reason, it hangs at org.apache.catalina.core.StandardHost start -- Jason Fowler, CA, MSLS From: Stuart Lewis [s.le...@auckland.ac.nz] Sent: Thursday, April 01, 2010 7:30 PM To: Jason Fowler Cc: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] DSpace 1.6 Handle server Hi Jason, How slow is slow? (Is it the handle server that is slow to startup, or tomcat
[Dspace-tech] DSpace 1.6 Handle server
I'm having a puzzling problem. I am updating my system from DSpace 1.52 to 1.6. I'm running RHEL Server release 5.3 (Tikanga). My Java version is 1.5.0_20 My Tomcat version is 5.5.26 and I'm using mod_proxy_ajp to redirect to port 80. I have upgraded correctly on my live server and everything works fine except handle (well, startup is a little slower, but I'm not sure why that is.). Whenever I run the handle server and visit a handle, the whole screen goes white and DSpace goes out in the weeds. Nothing works at that point. The whole system crashes. What makes it worse is that I don't see any cause for it in any of the log files. Does anyone have any ideas what causes this and how I might fix it? I tested the upgrade pretty rigorously on a Fedora 12 virtual machine with no problems prior to upgrading. Of course, handle was the one thing I didn't have a way to test. Blessings, Jason Fowler, CA, MSLS Archives and Special Collections Librarian The Southern Baptist Theological Seminary Vice President, ALABI jfow...@sbts.edu -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] DSpace 1.6 Handle server
Jason, Check your hosts file comparing with your virtual machine. I had same problem that you my handle server is not worked, the hostname had resolving a internal IP and we need to resolve a external IP (internet IP). [ ]'s Antoanne Pontes On Thu, Apr 1, 2010 at 4:48 PM, Jason Fowler jfow...@sbts.edu wrote: I'm having a puzzling problem. I am updating my system from DSpace 1.52 to 1.6. I'm running RHEL Server release 5.3 (Tikanga). My Java version is 1.5.0_20 My Tomcat version is 5.5.26 and I'm using mod_proxy_ajp to redirect to port 80. I have upgraded correctly on my live server and everything works fine except handle (well, startup is a little slower, but I'm not sure why that is.). Whenever I run the handle server and visit a handle, the whole screen goes white and DSpace goes out in the weeds. Nothing works at that point. The whole system crashes. What makes it worse is that I don't see any cause for it in any of the log files. Does anyone have any ideas what causes this and how I might fix it? I tested the upgrade pretty rigorously on a Fedora 12 virtual machine with no problems prior to upgrading. Of course, handle was the one thing I didn't have a way to test. Blessings, Jason Fowler, CA, MSLS Archives and Special Collections Librarian The Southern Baptist Theological Seminary Vice President, ALABI jfow...@sbts.edu -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Att, Antoanne Pontes http://antoanne.com - Internet Design -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] DSpace 1.6 Handle server
I think I have my problem solved. It all had to do with the way I had dspace.BaseUrl configured. I removed the xmlui from the end, and everything went back to working like normal. The startup is still painfully slow, though. Would anyone have any idea why? -- Jason Fowler, CA, MSLS _ From: Jason Fowler [jfow...@sbts.edu] Sent: Thursday, April 01, 2010 3:48 PM To: dspace-tech@lists.sourceforge.net Subject: [Dspace-tech] DSpace 1.6 Handle server I'm having a puzzling problem. I am updating my system from DSpace 1.52 to 1.6. I'm running RHEL Server release 5.3 (Tikanga). My Java version is 1.5.0_20 My Tomcat version is 5.5.26 and I'm using mod_proxy_ajp to redirect to port 80. I have upgraded correctly on my live server and everything works fine except handle (well, startup is a little slower, but I'm not sure why that is.). Whenever I run the handle server and visit a handle, the whole screen goes white and DSpace goes out in the weeds. Nothing works at that point. The whole system crashes. What makes it worse is that I don't see any cause for it in any of the log files. Does anyone have any ideas what causes this and how I might fix it? I tested the upgrade pretty rigorously on a Fedora 12 virtual machine with no problems prior to upgrading. Of course, handle was the one thing I didn't have a way to test. Blessings, Jason Fowler, CA, MSLS Archives and Special Collections Librarian The Southern Baptist Theological Seminary Vice President, ALABI jfow...@sbts.edu -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] DSpace 1.6 Handle server
Hi Jason, How slow is slow? (Is it the handle server that is slow to startup, or tomcat?) DSpace 1.6 will take slightly longer than earlier versions to start up as it now has to start a new webapp (solr) and 'warm up' the solr indexes. Somewhere between 20 seconds and a minute should be normal. Thanks, Stuart Lewis IT Innovations Analyst and Developer Te Tumu Herenga The University of Auckland Library Auckland Mail Centre, Private Bag 92019, Auckland 1142, New Zealand Ph: 64 9 373-7599 x81928 http://www.library.auckland.ac.nz/ On 2/04/2010, at 11:56 AM, Jason Fowler wrote: I think I have my problem solved. It all had to do with the way I had dspace.BaseUrl configured. I removed the xmlui from the end, and everything went back to working like normal. The startup is still painfully slow, though. Would anyone have any idea why? -- Jason Fowler, CA, MSLS _ From: Jason Fowler [jfow...@sbts.edu] Sent: Thursday, April 01, 2010 3:48 PM To: dspace-tech@lists.sourceforge.net Subject: [Dspace-tech] DSpace 1.6 Handle server I'm having a puzzling problem. I am updating my system from DSpace 1.52 to 1.6. I'm running RHEL Server release 5.3 (Tikanga). My Java version is 1.5.0_20 My Tomcat version is 5.5.26 and I'm using mod_proxy_ajp to redirect to port 80. I have upgraded correctly on my live server and everything works fine except handle (well, startup is a little slower, but I'm not sure why that is.). Whenever I run the handle server and visit a handle, the whole screen goes white and DSpace goes out in the weeds. Nothing works at that point. The whole system crashes. What makes it worse is that I don't see any cause for it in any of the log files. Does anyone have any ideas what causes this and how I might fix it? I tested the upgrade pretty rigorously on a Fedora 12 virtual machine with no problems prior to upgrading. Of course, handle was the one thing I didn't have a way to test. Blessings, Jason Fowler, CA, MSLS Archives and Special Collections Librarian The Southern Baptist Theological Seminary Vice President, ALABI jfow...@sbts.edu -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] DSpace 1.6 Handle server
As a tangent, You can run the webapps like solr on separate tomcat instances if this becomes a concern. They seldom need to be restarted as often as DSpace does. In fact, there are very good scalability strategies that are oriented around replicating solr instances across a cluster of tomcat nodes and sharding the indexes to improve the performance and scalability of the application. Mark On Thu, Apr 1, 2010 at 4:30 PM, Stuart Lewis s.le...@auckland.ac.nz wrote: Hi Jason, How slow is slow? (Is it the handle server that is slow to startup, or tomcat?) DSpace 1.6 will take slightly longer than earlier versions to start up as it now has to start a new webapp (solr) and 'warm up' the solr indexes. Somewhere between 20 seconds and a minute should be normal. Thanks, Stuart Lewis IT Innovations Analyst and Developer Te Tumu Herenga The University of Auckland Library Auckland Mail Centre, Private Bag 92019, Auckland 1142, New Zealand Ph: 64 9 373-7599 x81928 http://www.library.auckland.ac.nz/ On 2/04/2010, at 11:56 AM, Jason Fowler wrote: I think I have my problem solved. It all had to do with the way I had dspace.BaseUrl configured. I removed the xmlui from the end, and everything went back to working like normal. The startup is still painfully slow, though. Would anyone have any idea why? -- Jason Fowler, CA, MSLS _ From: Jason Fowler [jfow...@sbts.edu] Sent: Thursday, April 01, 2010 3:48 PM To: dspace-tech@lists.sourceforge.net Subject: [Dspace-tech] DSpace 1.6 Handle server I'm having a puzzling problem. I am updating my system from DSpace 1.52 to 1.6. I'm running RHEL Server release 5.3 (Tikanga). My Java version is 1.5.0_20 My Tomcat version is 5.5.26 and I'm using mod_proxy_ajp to redirect to port 80. I have upgraded correctly on my live server and everything works fine except handle (well, startup is a little slower, but I'm not sure why that is.). Whenever I run the handle server and visit a handle, the whole screen goes white and DSpace goes out in the weeds. Nothing works at that point. The whole system crashes. What makes it worse is that I don't see any cause for it in any of the log files. Does anyone have any ideas what causes this and how I might fix it? I tested the upgrade pretty rigorously on a Fedora 12 virtual machine with no problems prior to upgrading. Of course, handle was the one thing I didn't have a way to test. Blessings, Jason Fowler, CA, MSLS Archives and Special Collections Librarian The Southern Baptist Theological Seminary Vice President, ALABI jfow...@sbts.edu -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Mark R. Diggory Head of U.S. Operations - @mire http://www.atmire.com - Institutional Repository Solutions http://www.togather.eu - Before getting together, get t...@ther -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] DSpace 1.6 Handle server
Stuart, I have confirmed from watching catalina.out that it consistently takes nearly10 minutes to start. Here's the section of the file where the hangup occurs. Apr 1, 2010 9:23:28 PM org.apache.coyote.http11.Http11BaseProtocol init INFO: Initializing Coyote HTTP/1.1 on http-8080 Apr 1, 2010 9:23:28 PM org.apache.catalina.startup.Catalina load INFO: Initialization processed in 537 ms Apr 1, 2010 9:23:28 PM org.apache.catalina.core.StandardService start INFO: Starting service Catalina Apr 1, 2010 9:23:28 PM org.apache.catalina.core.StandardEngine start INFO: Starting Servlet Engine: Apache Tomcat/5.5.26 Apr 1, 2010 9:23:28 PM org.apache.catalina.core.StandardHost start INFO: XML validation disabled Apr 1, 2010 9:33:12 PM org.apache.solr.servlet.SolrDispatchFilter init INFO: SolrDispatchFilter.init() Apr 1, 2010 9:33:12 PM org.apache.solr.core.SolrResourceLoader locateInstanceDir INFO: Using JNDI solr.home: /mnt/storage/dspace/solr Apr 1, 2010 9:33:12 PM org.apache.solr.core.CoreContainer$Initializer initialize INFO: looking for solr.xml: /mnt/storage/dspace/solr/solr.xml Apr 1, 2010 9:33:12 PM org.apache.solr.core.SolrResourceLoader init INFO: Solr home set to '/mnt/storage/dspace/solr/' For some reason, it hangs at org.apache.catalina.core.StandardHost start -- Jason Fowler, CA, MSLS From: Stuart Lewis [s.le...@auckland.ac.nz] Sent: Thursday, April 01, 2010 7:30 PM To: Jason Fowler Cc: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] DSpace 1.6 Handle server Hi Jason, How slow is slow? (Is it the handle server that is slow to startup, or tomcat?) DSpace 1.6 will take slightly longer than earlier versions to start up as it now has to start a new webapp (solr) and 'warm up' the solr indexes. Somewhere between 20 seconds and a minute should be normal. Thanks, Stuart Lewis IT Innovations Analyst and Developer Te Tumu Herenga The University of Auckland Library Auckland Mail Centre, Private Bag 92019, Auckland 1142, New Zealand Ph: 64 9 373-7599 x81928 http://www.library.auckland.ac.nz/ On 2/04/2010, at 11:56 AM, Jason Fowler wrote: I think I have my problem solved. It all had to do with the way I had dspace.BaseUrl configured. I removed the xmlui from the end, and everything went back to working like normal. The startup is still painfully slow, though. Would anyone have any idea why? -- Jason Fowler, CA, MSLS _ From: Jason Fowler [jfow...@sbts.edu] Sent: Thursday, April 01, 2010 3:48 PM To: dspace-tech@lists.sourceforge.net Subject: [Dspace-tech] DSpace 1.6 Handle server I'm having a puzzling problem. I am updating my system from DSpace 1.52 to 1.6. I'm running RHEL Server release 5.3 (Tikanga). My Java version is 1.5.0_20 My Tomcat version is 5.5.26 and I'm using mod_proxy_ajp to redirect to port 80. I have upgraded correctly on my live server and everything works fine except handle (well, startup is a little slower, but I'm not sure why that is.). Whenever I run the handle server and visit a handle, the whole screen goes white and DSpace goes out in the weeds. Nothing works at that point. The whole system crashes. What makes it worse is that I don't see any cause for it in any of the log files. Does anyone have any ideas what causes this and how I might fix it? I tested the upgrade pretty rigorously on a Fedora 12 virtual machine with no problems prior to upgrading. Of course, handle was the one thing I didn't have a way to test. Blessings, Jason Fowler, CA, MSLS Archives and Special Collections Librarian The Southern Baptist Theological Seminary Vice President, ALABI jfow...@sbts.edu -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See
Re: [Dspace-tech] DSpace 1.6 Handle server
It looks like the delay is before the solr servlet init log entry is being called, IE before solr is loaded, disable the solr webapplication from loading and test if the delay is still there Likewise, what is the storage mount situation you are using here? If file access is slow across this mount, that be slowing Lucene access within Solr. You may try, moving your solr data directory onto the local file system and testing if that speeds your loading time. Finally, we have rather large solr indexes of statistics in production that do not have load times anywhere near this degree of slowness. Mark On Thu, Apr 1, 2010 at 7:52 PM, Jason Fowler jfow...@sbts.edu wrote: Stuart, I have confirmed from watching catalina.out that it consistently takes nearly10 minutes to start. Here's the section of the file where the hangup occurs. Apr 1, 2010 9:23:28 PM org.apache.coyote.http11.Http11BaseProtocol init INFO: Initializing Coyote HTTP/1.1 on http-8080 Apr 1, 2010 9:23:28 PM org.apache.catalina.startup.Catalina load INFO: Initialization processed in 537 ms Apr 1, 2010 9:23:28 PM org.apache.catalina.core.StandardService start INFO: Starting service Catalina Apr 1, 2010 9:23:28 PM org.apache.catalina.core.StandardEngine start INFO: Starting Servlet Engine: Apache Tomcat/5.5.26 Apr 1, 2010 9:23:28 PM org.apache.catalina.core.StandardHost start INFO: XML validation disabled Apr 1, 2010 9:33:12 PM org.apache.solr.servlet.SolrDispatchFilter init INFO: SolrDispatchFilter.init() Apr 1, 2010 9:33:12 PM org.apache.solr.core.SolrResourceLoader locateInstanceDir INFO: Using JNDI solr.home: /mnt/storage/dspace/solr Apr 1, 2010 9:33:12 PM org.apache.solr.core.CoreContainer$Initializer initialize INFO: looking for solr.xml: /mnt/storage/dspace/solr/solr.xml Apr 1, 2010 9:33:12 PM org.apache.solr.core.SolrResourceLoader init INFO: Solr home set to '/mnt/storage/dspace/solr/' For some reason, it hangs at org.apache.catalina.core.StandardHost start -- Jason Fowler, CA, MSLS From: Stuart Lewis [s.le...@auckland.ac.nz] Sent: Thursday, April 01, 2010 7:30 PM To: Jason Fowler Cc: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] DSpace 1.6 Handle server Hi Jason, How slow is slow? (Is it the handle server that is slow to startup, or tomcat?) DSpace 1.6 will take slightly longer than earlier versions to start up as it now has to start a new webapp (solr) and 'warm up' the solr indexes. Somewhere between 20 seconds and a minute should be normal. Thanks, Stuart Lewis IT Innovations Analyst and Developer Te Tumu Herenga The University of Auckland Library Auckland Mail Centre, Private Bag 92019, Auckland 1142, New Zealand Ph: 64 9 373-7599 x81928 http://www.library.auckland.ac.nz/ On 2/04/2010, at 11:56 AM, Jason Fowler wrote: I think I have my problem solved. It all had to do with the way I had dspace.BaseUrl configured. I removed the xmlui from the end, and everything went back to working like normal. The startup is still painfully slow, though. Would anyone have any idea why? -- Jason Fowler, CA, MSLS _ From: Jason Fowler [jfow...@sbts.edu] Sent: Thursday, April 01, 2010 3:48 PM To: dspace-tech@lists.sourceforge.net Subject: [Dspace-tech] DSpace 1.6 Handle server I'm having a puzzling problem. I am updating my system from DSpace 1.52 to 1.6. I'm running RHEL Server release 5.3 (Tikanga). My Java version is 1.5.0_20 My Tomcat version is 5.5.26 and I'm using mod_proxy_ajp to redirect to port 80. I have upgraded correctly on my live server and everything works fine except handle (well, startup is a little slower, but I'm not sure why that is.). Whenever I run the handle server and visit a handle, the whole screen goes white and DSpace goes out in the weeds. Nothing works at that point. The whole system crashes. What makes it worse is that I don't see any cause for it in any of the log files. Does anyone have any ideas what causes this and how I might fix it? I tested the upgrade pretty rigorously on a Fedora 12 virtual machine with no problems prior to upgrading. Of course, handle was the one thing I didn't have a way to test. Blessings, Jason Fowler, CA, MSLS Archives and Special Collections Librarian The Southern Baptist Theological Seminary Vice President, ALABI jfow...@sbts.edu -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] DSpace 1.6 Handle server
Mark, I'm not sure exactly how to disable the solr webapplication. Any suggestions? The file access is pretty quick. We've never had any trouble before. We have an app server for running the webapps and a storage server for everything else. --Jason Fowler, CA, MSLS From: mdigg...@gmail.com [mdigg...@gmail.com] On Behalf Of Mark Diggory [mdigg...@atmire.com] Sent: Thursday, April 01, 2010 11:31 PM To: Jason Fowler Cc: Stuart Lewis; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] DSpace 1.6 Handle server It looks like the delay is before the solr servlet init log entry is being called, IE before solr is loaded, disable the solr webapplication from loading and test if the delay is still there Likewise, what is the storage mount situation you are using here? If file access is slow across this mount, that be slowing Lucene access within Solr. You may try, moving your solr data directory onto the local file system and testing if that speeds your loading time. Finally, we have rather large solr indexes of statistics in production that do not have load times anywhere near this degree of slowness. Mark On Thu, Apr 1, 2010 at 7:52 PM, Jason Fowler jfow...@sbts.edu wrote: Stuart, I have confirmed from watching catalina.out that it consistently takes nearly10 minutes to start. Here's the section of the file where the hangup occurs. Apr 1, 2010 9:23:28 PM org.apache.coyote.http11.Http11BaseProtocol init INFO: Initializing Coyote HTTP/1.1 on http-8080 Apr 1, 2010 9:23:28 PM org.apache.catalina.startup.Catalina load INFO: Initialization processed in 537 ms Apr 1, 2010 9:23:28 PM org.apache.catalina.core.StandardService start INFO: Starting service Catalina Apr 1, 2010 9:23:28 PM org.apache.catalina.core.StandardEngine start INFO: Starting Servlet Engine: Apache Tomcat/5.5.26 Apr 1, 2010 9:23:28 PM org.apache.catalina.core.StandardHost start INFO: XML validation disabled Apr 1, 2010 9:33:12 PM org.apache.solr.servlet.SolrDispatchFilter init INFO: SolrDispatchFilter.init() Apr 1, 2010 9:33:12 PM org.apache.solr.core.SolrResourceLoader locateInstanceDir INFO: Using JNDI solr.home: /mnt/storage/dspace/solr Apr 1, 2010 9:33:12 PM org.apache.solr.core.CoreContainer$Initializer initialize INFO: looking for solr.xml: /mnt/storage/dspace/solr/solr.xml Apr 1, 2010 9:33:12 PM org.apache.solr.core.SolrResourceLoader init INFO: Solr home set to '/mnt/storage/dspace/solr/' For some reason, it hangs at org.apache.catalina.core.StandardHost start -- Jason Fowler, CA, MSLS From: Stuart Lewis [s.le...@auckland.ac.nz] Sent: Thursday, April 01, 2010 7:30 PM To: Jason Fowler Cc: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] DSpace 1.6 Handle server Hi Jason, How slow is slow? (Is it the handle server that is slow to startup, or tomcat?) DSpace 1.6 will take slightly longer than earlier versions to start up as it now has to start a new webapp (solr) and 'warm up' the solr indexes. Somewhere between 20 seconds and a minute should be normal. Thanks, Stuart Lewis IT Innovations Analyst and Developer Te Tumu Herenga The University of Auckland Library Auckland Mail Centre, Private Bag 92019, Auckland 1142, New Zealand Ph: 64 9 373-7599 x81928 http://www.library.auckland.ac.nz/ On 2/04/2010, at 11:56 AM, Jason Fowler wrote: I think I have my problem solved. It all had to do with the way I had dspace.BaseUrl configured. I removed the xmlui from the end, and everything went back to working like normal. The startup is still painfully slow, though. Would anyone have any idea why? -- Jason Fowler, CA, MSLS _ From: Jason Fowler [jfow...@sbts.edu] Sent: Thursday, April 01, 2010 3:48 PM To: dspace-tech@lists.sourceforge.net Subject: [Dspace-tech] DSpace 1.6 Handle server I'm having a puzzling problem. I am updating my system from DSpace 1.52 to 1.6. I'm running RHEL Server release 5.3 (Tikanga). My Java version is 1.5.0_20 My Tomcat version is 5.5.26 and I'm using mod_proxy_ajp to redirect to port 80. I have upgraded correctly on my live server and everything works fine except handle (well, startup is a little slower, but I'm not sure why that is.). Whenever I run the handle server and visit a handle, the whole screen goes white and DSpace goes out in the weeds. Nothing works at that point. The whole system crashes. What makes it worse is that I don't see any cause for it in any of the log files. Does anyone have any ideas what causes this and how I might fix it? I tested the upgrade pretty rigorously on a Fedora 12 virtual machine with no problems prior to upgrading. Of course, handle was the one thing I didn't have a way to test. Blessings, Jason Fowler, CA, MSLS Archives and Special Collections Librarian The Southern Baptist Theological Seminary Vice President, ALABI jfow...@sbts.edu
Re: [Dspace-tech] DSpace 1.6 Handle server
Hi Jason, You can disable the solr webapp by removing the whole solr directory from [tomcat]/webapps. Your storage server may be slowing down the loading of the solr index, but 10 minutes sounds quite severe. You could test it by moving the indexes to the local disk. They shouldn't get too big, so hopefully you'll have the space. They are normally kept in [dspace]/solr so move them to a local disk, then edit [tomcat]/solr/WEB-INF/web.xml Look for: env-entry env-entry-namesolr/home/env-entry-name env-entry-typejava.lang.String/env-entry-type env-entry-value/dspace/solr/env-entry-value /env-entry And edit /dspace/solr as appropriate for the new location of your solr index. Thanks, Stuart Lewis IT Innovations Analyst and Developer Te Tumu Herenga The University of Auckland Library Auckland Mail Centre, Private Bag 92019, Auckland 1142, New Zealand Ph: 64 9 373-7599 x81928 http://www.library.auckland.ac.nz/ On 2/04/2010, at 5:29 PM, Jason Fowler wrote: Mark, I'm not sure exactly how to disable the solr webapplication. Any suggestions? The file access is pretty quick. We've never had any trouble before. We have an app server for running the webapps and a storage server for everything else. --Jason Fowler, CA, MSLS From: mdigg...@gmail.com [mdigg...@gmail.com] On Behalf Of Mark Diggory [mdigg...@atmire.com] Sent: Thursday, April 01, 2010 11:31 PM To: Jason Fowler Cc: Stuart Lewis; dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] DSpace 1.6 Handle server It looks like the delay is before the solr servlet init log entry is being called, IE before solr is loaded, disable the solr webapplication from loading and test if the delay is still there Likewise, what is the storage mount situation you are using here? If file access is slow across this mount, that be slowing Lucene access within Solr. You may try, moving your solr data directory onto the local file system and testing if that speeds your loading time. Finally, we have rather large solr indexes of statistics in production that do not have load times anywhere near this degree of slowness. Mark On Thu, Apr 1, 2010 at 7:52 PM, Jason Fowler jfow...@sbts.edu wrote: Stuart, I have confirmed from watching catalina.out that it consistently takes nearly10 minutes to start. Here's the section of the file where the hangup occurs. Apr 1, 2010 9:23:28 PM org.apache.coyote.http11.Http11BaseProtocol init INFO: Initializing Coyote HTTP/1.1 on http-8080 Apr 1, 2010 9:23:28 PM org.apache.catalina.startup.Catalina load INFO: Initialization processed in 537 ms Apr 1, 2010 9:23:28 PM org.apache.catalina.core.StandardService start INFO: Starting service Catalina Apr 1, 2010 9:23:28 PM org.apache.catalina.core.StandardEngine start INFO: Starting Servlet Engine: Apache Tomcat/5.5.26 Apr 1, 2010 9:23:28 PM org.apache.catalina.core.StandardHost start INFO: XML validation disabled Apr 1, 2010 9:33:12 PM org.apache.solr.servlet.SolrDispatchFilter init INFO: SolrDispatchFilter.init() Apr 1, 2010 9:33:12 PM org.apache.solr.core.SolrResourceLoader locateInstanceDir INFO: Using JNDI solr.home: /mnt/storage/dspace/solr Apr 1, 2010 9:33:12 PM org.apache.solr.core.CoreContainer$Initializer initialize INFO: looking for solr.xml: /mnt/storage/dspace/solr/solr.xml Apr 1, 2010 9:33:12 PM org.apache.solr.core.SolrResourceLoader init INFO: Solr home set to '/mnt/storage/dspace/solr/' For some reason, it hangs at org.apache.catalina.core.StandardHost start -- Jason Fowler, CA, MSLS From: Stuart Lewis [s.le...@auckland.ac.nz] Sent: Thursday, April 01, 2010 7:30 PM To: Jason Fowler Cc: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] DSpace 1.6 Handle server Hi Jason, How slow is slow? (Is it the handle server that is slow to startup, or tomcat?) DSpace 1.6 will take slightly longer than earlier versions to start up as it now has to start a new webapp (solr) and 'warm up' the solr indexes. Somewhere between 20 seconds and a minute should be normal. Thanks, Stuart Lewis IT Innovations Analyst and Developer Te Tumu Herenga The University of Auckland Library Auckland Mail Centre, Private Bag 92019, Auckland 1142, New Zealand Ph: 64 9 373-7599 x81928 http://www.library.auckland.ac.nz/ On 2/04/2010, at 11:56 AM, Jason Fowler wrote: I think I have my problem solved. It all had to do with the way I had dspace.BaseUrl configured. I removed the xmlui from the end, and everything went back to working like normal. The startup is still painfully slow, though. Would anyone have any idea why? -- Jason Fowler, CA, MSLS _ From: Jason Fowler [jfow...@sbts.edu] Sent: Thursday, April 01, 2010 3:48 PM To: dspace-tech@lists.sourceforge.net Subject: [Dspace-tech] DSpace
