I've seen queries to both suggest and dedicated in the logs around the times when it goes down - But I assume most users start by using the search suggest, and then move to the dedicated page if they don't find what they need, so search suggest is probably triggered in most circumstances.
For what it's worth, my short term fix has been to hide the search box (which would force users to go to Main.Search if they want to search for something), and it's been much more stable. I haven't seen any search-related issues so far on 6.2 RC1, so I'm hopeful that upgrading this farm will deal with the issue as well. I'm pretty sure I just used the deb package for 5.4.3, so it's unlikely I have old search suggest sources, but who knows. Thanks for the info, aaron On Mon, Sep 15, 2014 at 8:38 AM, Marius Dumitru Florea < [email protected]> wrote: > On Fri, Sep 5, 2014 at 8:13 PM, Aaron Hardy at AC > <[email protected]> wrote: > > Hi, > > > > I'm experiencing frequent crashes that I think are related to Solr, and > I'd > > appreciate any advice on how to fix them. > > > > I have a fairly large wiki (with several thousand attachments) running > > 5.4.3 on tomcat7, and it goes down multiple times a day. Sometimes it > will > > remain unresponsive for a few minutes then come back on its own, other > > times it stays down until I restart tomcat. Usage runs anywhere from a > few > > active users to several thousand, depending on time of day. Crashes > happen > > even when the number of users is low. > > > > Checking catalina.out and the localhost_access logs, there almost always > > appears to be requests related to solr around the time it goes down. > > Sometimes catalina.out shows a java heap out of memory error, but other > > times there are no errors in the logs at all, the site is just > unresponsive. > > > > > I can usually reproduce the issue by running several searches back to > back, > > while browsing pages in another tab. While the searches are in progress, > > the responsiveness of browsing pages steadily drops until it times out. > It > > usually only takes 5 or 10 searches before this happens. > > Are you referring to the live search (a.k.a. search suggest) from the > top right corner of each page or are you using the dedicated search > page, Main.Search? One big difference is that the live search performs > multiple queries (one per each search suggest source configured in the > administration) while the dedicated search page performs a single > query. Also, the default search suggest sources have been improved in > 5.4.2 as part of http://jira.xwiki.org/browse/XWIKI-10051 . Maybe you > kept the old search suggest sources when you upgraded to 5.4.3? > > Hope this helps, > Marius > > > > > The other thing that makes me suspect Solr is that even when the site > > doesn't crash, while running a search in one tab, requests in the other > tab > > don't finish until the search is complete - it (subjectively, I know) > feels > > like other requests are suspended while Solr is thinking. > > > > Potentially related is that I occasionally also see JDBC connection pool > > related errors in the logs when it goes down - I have tried tweaking my > > connection pool size settings, to no avail. > > > > My current java memory settings are: > > JAVA_OPTS="-Djava.awt.headless=true -Xms2500m -Xmx2500m -XX:PermSize=64m > > -XX:MaxPermSize=256m -XX:+UseConcMarkSweepGC" > > > > I've used everything from the 800m xwiki.org settings to 3GB+, and it > > doesn't seem to make a difference. > > > > I have also reduced the cache settings from 3000 back to 100 in > > xwiki.properties, which hasn't helped. > > > > My hibernate settings are: > > connection.pool.size = 100 > > statement_cache.size = 50 > > dbcp.maxActive = 100 > > dbcp.maxIdle = 10 > > dbcp.maxWait = 300000 > > > > Mysql max_connections is set to 290 > > > > I have changed nearly all of those numbers at some point, but nothing > seems > > to help. > > > > Any assistance is greatly appreciated - even just tips on how to > > troubleshoot what's actually taking the site down, since the logs aren't > > always useful. > > > > thanks, > > > > aaron > > _______________________________________________ > > users mailing list > > [email protected] > > http://lists.xwiki.org/mailman/listinfo/users > _______________________________________________ > users mailing list > [email protected] > http://lists.xwiki.org/mailman/listinfo/users > _______________________________________________ users mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/users
