Re: Review Board 3.0.3 goes down often without showing any error

2018-07-22 Thread Christian Hammond
Hi Hema, Whoosh is good for smaller servers, but I imagine you're dealing with a pretty substantial search index (though that kind of CPU usage doesn't sound correct even for Whoosh). You're probably best off using a compatible version of Elasticsearch instead, which is designed to scale. See

Re: Review Board 3.0.3 goes down often without showing any error

2018-07-22 Thread Hema
Hello Christian, We ensured that we are running the newer Djblets. But still we face the server overload issue. Then we got to know that the "Full Text Search" is causing this. We use whoosh and we do update index every 10 minutes as recommended. When we hit enter in search box, it forks out

Re: Review Board 3.0.3 goes down often without showing any error

2018-07-10 Thread Christian Hammond
I'd start by making 100% sure that you're running against the newer Djblets. That is the only source of deadlocks we've encountered. Make sure Apache has been completely killed and restarted. I'd also advise getting the latest versions of both Review Board and Djblets, as there may be fixes that

Re: Review Board 3.0.3 goes down often without showing any error

2018-07-09 Thread Hema
Hello Christian, As per your suggestion on the other email thread, we upgraded Djblets from 1.0.3 to 1.0.4 and its dependencies. But even after upgrading it, the server load average often reaches 300+ and it goes unresponsive. We don't find any error in the logs. Could you please let us know

Re: Review Board 3.0.3 goes down often without showing any error

2018-06-20 Thread Christian Hammond
No, I’m sorry. You’ll need to upgrade. The only other option is frequent restarts of Apache. The upgrade shouldn’t take more than a couple minutes, though. Christian On Tue, Jun 19, 2018 at 22:57 Hema wrote: > Hello Christian, > > Thanks for the reply. > > We cannot do another upgrade, so

Re: Review Board 3.0.3 goes down often without showing any error

2018-06-19 Thread Christian Hammond
Hi Hema, Yes, this is the main symptom of the deadlock bug. Christian On Tue, Jun 19, 2018 at 9:50 PM Hema wrote: > *2018-06-20 01:47:52,607 - CRITICAL - - root - Unable to load > SiteConfiguration: (2013, "Lost connection to MySQL server at 'reading > authorization packet', system error:

Re: Review Board 3.0.3 goes down often without showing any error

2018-06-19 Thread Hema
*2018-06-20 01:47:52,607 - CRITICAL - - root - Unable to load SiteConfiguration: (2013, "Lost connection to MySQL server at 'reading authorization packet', system error: 0")* *OperationalError: (2013, "Lost connection to MySQL server at 'reading authorization packet', system error: 0")*

Re: Review Board 3.0.3 goes down often without showing any error

2018-06-14 Thread Christian Hammond
3.0.3 contains a deadlock bug, which you *will* hit. It will bring down the server periodically. This is fixed in 3.0.4. We always recommend the latest 3.0.x release, though. If there's a major bug, we work to fix it quickly, but I haven't seen any major issues with the most recent release

Re: Review Board 3.0.3 goes down often without showing any error

2018-06-14 Thread Hema
Thanks for the reply. We were running 2.0 in another server. Recently, we had setup 3.0 server; connected it to 2.0's database and upgraded. I'll check whether 2.0 server is conflicting with the new one even though we've turned off apache. 1) Are you running Review Board on a single server, or

Re: Review Board 3.0.3 goes down often without showing any error

2018-06-14 Thread Christian Hammond
Hi, This is definitely not normal, and points to a conflict somewhere in the infrastructure. Is there any other instance anywhere, any tooling, any test server that's talking to the same database? It looks as if somewhere, a 2.0.12 instance is running and setting its version in the database.

Review Board 3.0.3 goes down often without showing any error

2018-06-14 Thread Hema
Review Board 3.0.3 goes down often and it shows the attached error page. When I checked, both the web server and rb-site uses the same Python version. Error logs (httpd, ssl, reviewboard) are clean and also we don't get any error e-mail. And when it shows this page, just the apache restart