Re: Error 500 when posting comments
Hi Sara. Some questions for you.
When you say a reboot, was it of the full server, or just the web server? Did
this include the database server? (What type of database?)
When did the upgrade of Review Board to 2.0.4 happen, relative to the reboot?
I wouldn’t expect these errors unless you had some data corruption somewhere.
Was the reboot a clean reboot?
Are you getting that exact same error traceback for all review requests?
Are you able to see reviews on existing review requests?
Can you also make sure that people are actually logged in when posting this? I
want to rule out a session-related issue.
Christian
--
Christian Hammond - christ...@beanbaginc.com
Review Board - http://www.reviewboard.org
Beanbag, Inc. - http://www.beanbaginc.com
On July 25, 2014 at 10:29:17 AM, Sara Rayburn (sararayb...@gmail.com) wrote:
Hi all,
After a reboot of my reviewboard server, I am now getting Error 500 internal
server errors any time a user tries to submit a comment on a review request. I
have verified that I can add new review requests, and perform other edits on
those review requests. Here is a sample error:
Traceback (most recent call last):
File
/usr/lib/python2.6/site-packages/Django-1.6.5-py2.6.egg/django/core/handlers/base.py,
line 112, in get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File
/usr/lib/python2.6/site-packages/Django-1.6.5-py2.6.egg/django/views/decorators/cache.py,
line 52, in _wrapped_view_func
response = view_func(request, *args, **kwargs)
File
/usr/lib/python2.6/site-packages/Django-1.6.5-py2.6.egg/django/views/decorators/vary.py,
line 19, in inner_func
response = func(*args, **kwargs)
File
/usr/lib/python2.6/site-packages/Djblets-0.8.7-py2.6.egg/djblets/webapi/resources.py,
line 494, in __call__
request, method, view, api_format=api_format, *args, **kwargs)
File
/usr/lib/python2.6/site-packages/Djblets-0.8.7-py2.6.egg/djblets/webapi/resources.py,
line 565, in call_method_view
return view(request, *args, **kwargs)
File
/usr/lib/python2.6/site-packages/Djblets-0.8.7-py2.6.egg/djblets/webapi/resources.py,
line 747, in post
return self.create(*args, **kwargs)
File
/usr/lib/python2.6/site-packages/Djblets-0.8.7-py2.6.egg/djblets/webapi/decorators.py,
line 117, in _call
return view_func(*args, **kwargs)
File
/usr/lib/python2.6/site-packages/ReviewBoard-2.0.4-py2.6.egg/reviewboard/webapi/decorators.py,
line 110, in _check
return view_func(*args, **kwargs)
File
/usr/lib/python2.6/site-packages/Djblets-0.8.7-py2.6.egg/djblets/webapi/decorators.py,
line 117, in _call
return view_func(*args, **kwargs)
File
/usr/lib/python2.6/site-packages/Djblets-0.8.7-py2.6.egg/djblets/webapi/decorators.py,
line 138, in _checklogin
return view_func(*args, **kwargs)
File
/usr/lib/python2.6/site-packages/Djblets-0.8.7-py2.6.egg/djblets/webapi/decorators.py,
line 117, in _call
return view_func(*args, **kwargs)
File
/usr/lib/python2.6/site-packages/Djblets-0.8.7-py2.6.egg/djblets/webapi/decorators.py,
line 117, in _call
return view_func(*args, **kwargs)
File
/usr/lib/python2.6/site-packages/Djblets-0.8.7-py2.6.egg/djblets/webapi/decorators.py,
line 287, in _validate
return view_func(*args, **new_kwargs)
File
/usr/lib/python2.6/site-packages/ReviewBoard-2.0.4-py2.6.egg/reviewboard/webapi/resources/base_review.py,
line 169, in create
**self.get_base_reply_to_field(*args, **kwargs))
File
/usr/lib/python2.6/site-packages/Djblets-0.8.7-py2.6.egg/djblets/db/managers.py,
line 53, in get_or_create
return self.get(**kwargs)
File
/usr/lib/python2.6/site-packages/Django-1.6.5-py2.6.egg/django/db/models/manager.py,
line 151, in get
return self.get_queryset().get(*args, **kwargs)
File
/usr/lib/python2.6/site-packages/Django-1.6.5-py2.6.egg/django/db/models/query.py,
line 310, in get
self.model._meta.object_name)
DoesNotExist: Review matching query does not exist.
WSGIRequest
path:/api/review-requests/3677/reviews/,
GET:QueryDict: {},
POST:QueryDict: {u'body_top': [u''], u'text_type': [u'plain'], u'body_bottom':
[u''], u'api_format': [u'json'], u'ship_it': [u'false']},
COOKIES:{'collapsediffs': 'True',
'csrftoken': 'v9zdlH2rkdux8ABIrOYXIoEZ3UOY3vB0',
'rbsessionid': '3vha1z6fnrabst0eneet4qjvgjji9b0i'},
META:{'CONTENT_LENGTH': '68',
'CONTENT_TYPE': 'application/x-www-form-urlencoded; charset=UTF-8',
'DOCUMENT_ROOT': '/var/www/reviewboard/htdocs',
'GATEWAY_INTERFACE': 'CGI/1.1',
'HTTP_ACCEPT': 'application/json, text/javascript, */*; q=0.01',
'HTTP_ACCEPT_ENCODING': 'gzip, deflate',
'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.5',
'HTTP_CACHE_CONTROL': 'no-cache',
'HTTP_CONNECTION': 'keep-alive',
'HTTP_COOKIE': 'csrftoken=v9zdlH2rkdux8ABIrOYXIoEZ3UOY3vB0;
rbsessionid=3vha1z6fnrabst0eneet4qjvgjji9b0i; collapsediffs=True',
'HTTP_HOST': 'reviewboard.cctechnol.com',
'HTTP_PRAGMA': 'no-cache',
'HTTP_REFERER':
Re: Error 500 when posting comments
Hi Christian, See responses inline. I'm afraid this is a server corruption thing. If that's your opinion, is there some way to repair the db? On Friday, July 25, 2014 2:32:32 PM UTC-5, Christian Hammond wrote: Hi Sara. Some questions for you. When you say a reboot, was it of the full server, or just the web server? Did this include the database server? (What type of database?) Full server reboot, including the db. All requisite services are running, and there is read/write access to the db since we can create/view review requests. When did the upgrade of Review Board to 2.0.4 happen, relative to the reboot? Within a couple of days, however the server VM has been rebooted since the install. (Our VM host keeps dying) I wouldn’t expect these errors unless you had some data corruption somewhere. Was the reboot a clean reboot? This is not out of the question. The server is running in a VM on a dying server. I shutdown the VM to do an emergency back up, and when I relaunched it, all these problems occurred. Are you getting that exact same error traceback for all review requests? Yes, with the exception of the review request number Are you able to see reviews on existing review requests? Yes Can you also make sure that people are actually logged in when posting this? I want to rule out a session-related issue. I'm not sure how to verify this. I have logged out, logged back in, and tried to post. Our site is not public. -- Get the Review Board Power Pack at http://www.reviewboard.org/powerpack/ --- Sign up for Review Board hosting at RBCommons: https://rbcommons.com/ --- Happy user? Let us know at http://www.reviewboard.org/users/ --- You received this message because you are subscribed to the Google Groups reviewboard group. To unsubscribe from this group and stop receiving emails from it, send an email to reviewboard+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Error 500 when posting comments
On July 25, 2014 at 12:38:06 PM, Sara Rayburn (sararayb...@gmail.com) wrote: Hi Christian, See responses inline. I'm afraid this is a server corruption thing. If that's your opinion, is there some way to repair the db? Possibly, depending on what’s wrong. I don’t know enough about the state of your database to say. A few more questions for now: 1) Before upgrading to 2.0.4, what version were you running? 2) Did you have these problems immediately after upgrading, or just after the reboot? 3) Are these review requests on private repositories/groups? 4) Does this only break when replying to reviews? Can new reviews be posted? Christian On Friday, July 25, 2014 2:32:32 PM UTC-5, Christian Hammond wrote: Hi Sara. Some questions for you. When you say a reboot, was it of the full server, or just the web server? Did this include the database server? (What type of database?) Full server reboot, including the db. All requisite services are running, and there is read/write access to the db since we can create/view review requests. When did the upgrade of Review Board to 2.0.4 happen, relative to the reboot? Within a couple of days, however the server VM has been rebooted since the install. (Our VM host keeps dying) I wouldn’t expect these errors unless you had some data corruption somewhere. Was the reboot a clean reboot? This is not out of the question. The server is running in a VM on a dying server. I shutdown the VM to do an emergency back up, and when I relaunched it, all these problems occurred. Are you getting that exact same error traceback for all review requests? Yes, with the exception of the review request number Are you able to see reviews on existing review requests? Yes Can you also make sure that people are actually logged in when posting this? I want to rule out a session-related issue. I'm not sure how to verify this. I have logged out, logged back in, and tried to post. Our site is not public. -- Get the Review Board Power Pack at http://www.reviewboard.org/powerpack/ --- Sign up for Review Board hosting at RBCommons: https://rbcommons.com/ --- Happy user? Let us know at http://www.reviewboard.org/users/ --- You received this message because you are subscribed to the Google Groups reviewboard group. To unsubscribe from this group and stop receiving emails from it, send an email to reviewboard+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- Get the Review Board Power Pack at http://www.reviewboard.org/powerpack/ --- Sign up for Review Board hosting at RBCommons: https://rbcommons.com/ --- Happy user? Let us know at http://www.reviewboard.org/users/ --- You received this message because you are subscribed to the Google Groups reviewboard group. To unsubscribe from this group and stop receiving emails from it, send an email to reviewboard+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Error 500 when posting comments
Hi Sara, Okay. So there are a couple options here for figuring this out. One will require a support contract (either a one-time support session for an hourly fee, or a one-year contract that would include such sessions). Through that, we could share more information over a private tracker, and we could set up either a live chat or phone call, along with screen sharing. This would be the quickest way to get things figured out. Otherwise, we can continue on with e-mail here. I can send you some commands that would help provide more information, so I could at least see what’s going wrong. There will be some back-and-forth going, but I’ll be around for the next several hours now. Let me know what sounds best for you! Christian -- Christian Hammond - christ...@beanbaginc.com Review Board - http://www.reviewboard.org Beanbag, Inc. - http://www.beanbaginc.com On July 25, 2014 at 3:01:27 PM, Sara Rayburn (sararayb...@gmail.com) wrote: Responses inline. On Jul 25, 2014 4:43 PM, Christian Hammond christ...@beanbaginc.com wrote: On July 25, 2014 at 12:38:06 PM, Sara Rayburn (sararayb...@gmail.com) wrote: Hi Christian, See responses inline. I'm afraid this is a server corruption thing. If that's your opinion, is there some way to repair the db? Possibly, depending on what’s wrong. I don’t know enough about the state of your database to say. A few more questions for now: 1) Before upgrading to 2.0.4, what version were you running? 2.0.3 2) Did you have these problems immediately after upgrading, or just after the reboot? This specific problem is new since the reboot, but we were having occasional issues even at .3 . I posted another question last week with another error message that no one responded to. 3) Are these review requests on private repositories/groups? They do belong to groups. 4) Does this only break when replying to reviews? Can new reviews be posted? Yes. All other functionality seems to work. Sara Christian On Friday, July 25, 2014 2:32:32 PM UTC-5, Christian Hammond wrote: Hi Sara. Some questions for you. When you say a reboot, was it of the full server, or just the web server? Did this include the database server? (What type of database?) Full server reboot, including the db. All requisite services are running, and there is read/write access to the db since we can create/view review requests. When did the upgrade of Review Board to 2.0.4 happen, relative to the reboot? Within a couple of days, however the server VM has been rebooted since the install. (Our VM host keeps dying) I wouldn’t expect these errors unless you had some data corruption somewhere. Was the reboot a clean reboot? This is not out of the question. The server is running in a VM on a dying server. I shutdown the VM to do an emergency back up, and when I relaunched it, all these problems occurred. Are you getting that exact same error traceback for all review requests? Yes, with the exception of the review request number Are you able to see reviews on existing review requests? Yes Can you also make sure that people are actually logged in when posting this? I want to rule out a session-related issue. I'm not sure how to verify this. I have logged out, logged back in, and tried to post. Our site is not public. -- Get the Review Board Power Pack at http://www.reviewboard.org/powerpack/ --- Sign up for Review Board hosting at RBCommons: https://rbcommons.com/ --- Happy user? Let us know at http://www.reviewboard.org/users/ --- You received this message because you are subscribed to the Google Groups reviewboard group. To unsubscribe from this group and stop receiving emails from it, send an email to reviewboard+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- Get the Review Board Power Pack at http://www.reviewboard.org/powerpack/ --- Sign up for Review Board hosting at RBCommons: https://rbcommons.com/ --- Happy user? Let us know at http://www.reviewboard.org/users/ --- You received this message because you are subscribed to the Google Groups reviewboard group. To unsubscribe from this group and stop receiving emails from it, send an email to reviewboard+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
