Re: Review board hanging / access issue..
Hi Chris, Thank you for your assistance. Looks like the Powers-that-be around here are reasonable keen to see us upgrade to 1.6.4.1 and change to using MySQL as the database (from 1.5.4 using SQLite.) Do you have any suggestions on the best way to do this? - I will have the opportunity to migrate reviewboard to a fresh install of CentOS 6.2. - Do you have any suggestions on migrating the data from SQLite to MySQL? - Is is possible to change a name of a 'site' after installation eg. Migrate data 'reviewboard' - 'reviewboard-dev' (for testing) Disable old instance of 'reviewboard' and rename 'reviewboard-dev' to 'reviewboard' Cheers, Paul On Fri, Feb 17, 2012 at 11:28 AM, Paul Schulz p...@mawsonlakes.org wrote: Hi Christian, Happy to be included in any migration script testing you may need. On Fri, Feb 17, 2012 at 11:21 AM, Christian Hammond chip...@chipx86.com wrote: Hi Paul, patch actually doesn't listen. What's happening is that an Apache thread running Review Board has spun off a patch instance, but due to a quirk and bug involving spinning off processes, it's possible they could block on waiting for input. Since it stalls, its parent Apache thread also stalls, and Apache is what's listening on port 80. This stall is what we've since fixed. So, that's an issue, but you see database locking because of sqlite. That should never be used in production, as it's not designed for concurrency. You'll start to hit these issues with normal usage as the number of users go up, and every write will block everyone else's read. You should look into moving to MySQL or something. The next release of RB 1.6.x will have scripts to help in the process of moving databases over. Christian -- Christian Hammond - chip...@chipx86.com Review Board - http://www.reviewboard.org VMware, Inc. - http://www.vmware.com On Thu, Feb 16, 2012 at 4:30 PM, Paul Schulz p...@mawsonlakes.org wrote: Hi Christian, Thank you for your replay.. I had a look in the bugs but wasn't able to find anything quickly. (I didn't look that hard.) I am using sqlite3 as the database on an old version on Ubuntu. (9.04) Upgrading in place has just worked, up to now. I will look into upgrading. (I still have one question.. how or why does patch end up listeing on port 80?) Thanks again, Paul On Fri, Feb 17, 2012 at 10:37 AM, Christian Hammond chip...@chipx86.com wrote: Hi Paul, What database are you using? The patch problem is possibly a cause. It blocks Apache threads from closing. We've seen this before, and have a fix in a release. Unfortunately, it requires some newer stuff in order to prevent the problem. You'll need the latest Review Board 1.6.x release, plus Python 2.6+, in order to fix it. Christian -- Christian Hammond - chip...@chipx86.com Review Board - http://www.reviewboard.org VMware, Inc. - http://www.vmware.com On Wed, Feb 15, 2012 at 9:13 PM, Paul Schulz p...@mawsonlakes.org wrote: Greetings, I have an interesting access issue with my Reviewboard instance. I have a dedicated VM running reviewboard 1.5.4. The Problem: Every now and again users have issues using the system and I get Admin alert emails with the details below. The load on the machine goes up to approximate 4 (single processor).. more observations below. (Email Message to Root) Traceback (most recent call last): File /usr/local/lib/python2.6/dist-packages/Django-1.2.5-py2.6.egg/django/core/handlers/base.py, line 100, in get_response response = callback(request, *callback_args, **callback_kwargs) File /usr/local/lib/python2.6/dist-packages/ReviewBoard-1.5.4-py2.6.egg/reviewboard/accounts/decorators.py, line 27, in _check return view_func(*args, **kwargs) File /usr/local/lib/python2.6/dist-packages/ReviewBoard-1.5.4-py2.6.egg/reviewboard/accounts/decorators.py, line 46, in _check_valid_prefs return view_func(request, *args, **kwargs) File /usr/local/lib/python2.6/dist-packages/ReviewBoard-1.5.4-py2.6.egg/reviewboard/reviews/views.py, line 141, in review_detail visited.save() File /usr/local/lib/python2.6/dist-packages/Django-1.2.5-py2.6.egg/django/db/models/base.py, line 458, in save self.save_base(using=using, force_insert=force_insert, force_update=force_update) File /usr/local/lib/python2.6/dist-packages/Django-1.2.5-py2.6.egg/django/db/models/base.py, line 558, in save_base transaction.commit_unless_managed(using=using) File /usr/local/lib/python2.6/dist-packages/Django-1.2.5-py2.6.egg/django/db/transaction.py, line 175, in commit_unless_managed connection._commit() File /usr/local/lib/python2.6/dist-packages/Django-1.2.5-py2.6.egg/django/db/backends/__init__.py, line 32, in _commit return self.connection.commit() OperationalError: database is locked ModPythonRequest path:/r/1371/,
Re: Review board hanging / access issue..
Hi Paul,
What database are you using?
The patch problem is possibly a cause. It blocks Apache threads from
closing. We've seen this before, and have a fix in a release.
Unfortunately, it requires some newer stuff in order to prevent the
problem. You'll need the latest Review Board 1.6.x release, plus Python
2.6+, in order to fix it.
Christian
--
Christian Hammond - chip...@chipx86.com
Review Board - http://www.reviewboard.org
VMware, Inc. - http://www.vmware.com
On Wed, Feb 15, 2012 at 9:13 PM, Paul Schulz p...@mawsonlakes.org wrote:
Greetings,
I have an interesting access issue with my Reviewboard instance.
I have a dedicated VM running reviewboard 1.5.4.
The Problem:
Every now and again users have issues using the system and I get Admin
alert emails with the details below. The load on the machine goes up
to approximate 4 (single processor).. more observations below.
(Email Message to Root)
Traceback (most recent call last):
File
/usr/local/lib/python2.6/dist-packages/Django-1.2.5-py2.6.egg/django/core/handlers/base.py,
line 100, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File
/usr/local/lib/python2.6/dist-packages/ReviewBoard-1.5.4-py2.6.egg/reviewboard/accounts/decorators.py,
line 27, in _check
return view_func(*args, **kwargs)
File
/usr/local/lib/python2.6/dist-packages/ReviewBoard-1.5.4-py2.6.egg/reviewboard/accounts/decorators.py,
line 46, in _check_valid_prefs
return view_func(request, *args, **kwargs)
File
/usr/local/lib/python2.6/dist-packages/ReviewBoard-1.5.4-py2.6.egg/reviewboard/reviews/views.py,
line 141, in review_detail
visited.save()
File
/usr/local/lib/python2.6/dist-packages/Django-1.2.5-py2.6.egg/django/db/models/base.py,
line 458, in save
self.save_base(using=using, force_insert=force_insert,
force_update=force_update)
File
/usr/local/lib/python2.6/dist-packages/Django-1.2.5-py2.6.egg/django/db/models/base.py,
line 558, in save_base
transaction.commit_unless_managed(using=using)
File
/usr/local/lib/python2.6/dist-packages/Django-1.2.5-py2.6.egg/django/db/transaction.py,
line 175, in commit_unless_managed
connection._commit()
File
/usr/local/lib/python2.6/dist-packages/Django-1.2.5-py2.6.egg/django/db/backends/__init__.py,
line 32, in _commit
return self.connection.commit()
OperationalError: database is locked
ModPythonRequest
path:/r/1371/,
GET:QueryDict: {},
POST:QueryDict: {},
COOKIES:{'rbsessionid': '803f0ef200329d8921c1b70a9b7eb8ff'},
META:{'AUTH_TYPE': None,
'CONTENT_LENGTH': 0,
'CONTENT_TYPE': None,
'GATEWAY_INTERFACE': 'CGI/1.1',
'HTTP_ACCEPT':
'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'HTTP_ACCEPT_ENCODING': 'gzip, deflate',
'HTTP_ACCEPT_LANGUAGE': 'en-us,en;q=0.5',
'HTTP_CONNECTION': 'keep-alive',
'HTTP_COOKIE': 'rbsessionid=803f0ef200329d8921c1b70a9b7eb8ff',
'HTTP_HOST': 'review..***',
'HTTP_IF_NONE_MATCH': ':2012-01-16 10:46:18.283850::2012-02-16
14:12:08.745932:0:1299464299',
'HTTP_REFERER': 'http://review.rocksoft.com/dashboard/',
'HTTP_USER_AGENT': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64;
rv:10.0.1) Gecko/20100101 Firefox/10.0.1',
'PATH_INFO': u'/r/1371/',
'PATH_TRANSLATED': None,
'QUERY_STRING': None,
'REMOTE_ADDR': '.***.***.***',
'REMOTE_HOST': None,
'REMOTE_IDENT': None,
'REMOTE_USER': None,
'REQUEST_METHOD': 'GET',
'SCRIPT_NAME': '',
'SERVER_NAME': 'review.***.***.***',
'SERVER_PORT': 80,
'SERVER_PROTOCOL': 'HTTP/1.1',
'SERVER_SOFTWARE': 'mod_python'}
While trying to restart apache (and this is the wierd bit), patch
appears to get started and open 'port 80' (yeah, I don't understand
this either).
# netstat -ntlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address
State PID/Program name
...
tcp0 0 0.0.0.0:80 0.0.0.0:*
LISTEN 23275/patch
...
# ps aux | grep patch
www-data 23275 0.0 0.0 4036 492 ?SJan12 0:00 patch
-o /tmp/reviewboard.VDhhfR/tmp0HLn60-new
/tmp/reviewboard.VDhhfR/tmp0HLn60
www-data 23276 0.0 0.0 4036 484 ?SJan12 0:00 patch
-o /tmp/reviewboard.xTIxjP/tmpN7YjTq-new
/tmp/reviewboard.xTIxjP/tmpN7YjTq
www-data 32634 0.0 0.0 4036 528 ?SJan17 0:00 patch
-o /tmp/reviewboard.Mhy4yq/tmpqytjmk-new
/tmp/reviewboard.Mhy4yq/tmpqytjmk
www-data 32635 0.0 0.0 4036 532 ?SJan17 0:00 patch
-o /tmp/reviewboard.NrBcSF/tmpq5ehYD-new
/tmp/reviewboard.NrBcSF/tmpq5ehYD
Question 1: Why does 'patch' have port 80 open? ( I have some
thoughts, but it requires more knowledge on how
reviewboard calls patch ).
Question 2: Is this what would cause issues accessing the apache
server? (It certainly doesn't allow the apache server to restart.)
Cheers,
Paul
--
Want to help the Review Board project? Donate today at
Re: Review board hanging / access issue..
Hi Christian,
Happy to be included in any migration script testing you may need.
On Fri, Feb 17, 2012 at 11:21 AM, Christian Hammond chip...@chipx86.com wrote:
Hi Paul,
patch actually doesn't listen. What's happening is that an Apache thread
running Review Board has spun off a patch instance, but due to a quirk and
bug involving spinning off processes, it's possible they could block on
waiting for input. Since it stalls, its parent Apache thread also stalls,
and Apache is what's listening on port 80. This stall is what we've since
fixed.
So, that's an issue, but you see database locking because of sqlite. That
should never be used in production, as it's not designed for concurrency.
You'll start to hit these issues with normal usage as the number of users go
up, and every write will block everyone else's read. You should look into
moving to MySQL or something. The next release of RB 1.6.x will have scripts
to help in the process of moving databases over.
Christian
--
Christian Hammond - chip...@chipx86.com
Review Board - http://www.reviewboard.org
VMware, Inc. - http://www.vmware.com
On Thu, Feb 16, 2012 at 4:30 PM, Paul Schulz p...@mawsonlakes.org wrote:
Hi Christian,
Thank you for your replay.. I had a look in the bugs but wasn't able
to find anything quickly. (I didn't look that hard.)
I am using sqlite3 as the database on an old version on Ubuntu. (9.04)
Upgrading in place has just worked, up to now.
I will look into upgrading.
(I still have one question.. how or why does patch end up listeing on port
80?)
Thanks again,
Paul
On Fri, Feb 17, 2012 at 10:37 AM, Christian Hammond chip...@chipx86.com
wrote:
Hi Paul,
What database are you using?
The patch problem is possibly a cause. It blocks Apache threads from
closing. We've seen this before, and have a fix in a release.
Unfortunately,
it requires some newer stuff in order to prevent the problem. You'll
need
the latest Review Board 1.6.x release, plus Python 2.6+, in order to fix
it.
Christian
--
Christian Hammond - chip...@chipx86.com
Review Board - http://www.reviewboard.org
VMware, Inc. - http://www.vmware.com
On Wed, Feb 15, 2012 at 9:13 PM, Paul Schulz p...@mawsonlakes.org
wrote:
Greetings,
I have an interesting access issue with my Reviewboard instance.
I have a dedicated VM running reviewboard 1.5.4.
The Problem:
Every now and again users have issues using the system and I get Admin
alert emails with the details below. The load on the machine goes up
to approximate 4 (single processor).. more observations below.
(Email Message to Root)
Traceback (most recent call last):
File
/usr/local/lib/python2.6/dist-packages/Django-1.2.5-py2.6.egg/django/core/handlers/base.py,
line 100, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File
/usr/local/lib/python2.6/dist-packages/ReviewBoard-1.5.4-py2.6.egg/reviewboard/accounts/decorators.py,
line 27, in _check
return view_func(*args, **kwargs)
File
/usr/local/lib/python2.6/dist-packages/ReviewBoard-1.5.4-py2.6.egg/reviewboard/accounts/decorators.py,
line 46, in _check_valid_prefs
return view_func(request, *args, **kwargs)
File
/usr/local/lib/python2.6/dist-packages/ReviewBoard-1.5.4-py2.6.egg/reviewboard/reviews/views.py,
line 141, in review_detail
visited.save()
File
/usr/local/lib/python2.6/dist-packages/Django-1.2.5-py2.6.egg/django/db/models/base.py,
line 458, in save
self.save_base(using=using, force_insert=force_insert,
force_update=force_update)
File
/usr/local/lib/python2.6/dist-packages/Django-1.2.5-py2.6.egg/django/db/models/base.py,
line 558, in save_base
transaction.commit_unless_managed(using=using)
File
/usr/local/lib/python2.6/dist-packages/Django-1.2.5-py2.6.egg/django/db/transaction.py,
line 175, in commit_unless_managed
connection._commit()
File
/usr/local/lib/python2.6/dist-packages/Django-1.2.5-py2.6.egg/django/db/backends/__init__.py,
line 32, in _commit
return self.connection.commit()
OperationalError: database is locked
ModPythonRequest
path:/r/1371/,
GET:QueryDict: {},
POST:QueryDict: {},
COOKIES:{'rbsessionid': '803f0ef200329d8921c1b70a9b7eb8ff'},
META:{'AUTH_TYPE': None,
'CONTENT_LENGTH': 0,
'CONTENT_TYPE': None,
'GATEWAY_INTERFACE': 'CGI/1.1',
'HTTP_ACCEPT':
'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'HTTP_ACCEPT_ENCODING': 'gzip, deflate',
'HTTP_ACCEPT_LANGUAGE': 'en-us,en;q=0.5',
'HTTP_CONNECTION': 'keep-alive',
'HTTP_COOKIE': 'rbsessionid=803f0ef200329d8921c1b70a9b7eb8ff',
'HTTP_HOST': 'review..***',
'HTTP_IF_NONE_MATCH': ':2012-01-16 10:46:18.283850::2012-02-16
14:12:08.745932:0:1299464299',
'HTTP_REFERER': 'http://review.rocksoft.com/dashboard/',
'HTTP_USER_AGENT': 'Mozilla/5.0 (X11; Ubuntu;
