Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 3513 by ray.l.ri...@gmail.com: Unicode characters in source file prevent review requests from being posted correctly (or at all)
http://code.google.com/p/reviewboard/issues/detail?id=3513

What version are you running?
2.0.2

What's the URL of the page containing the problem?

https://hopper.rose.hp.com/r/new/ (Page behind firewall)

What steps will reproduce the problem?
1. Find a file with at least one non-ASCII character (e.g. single quote ' instead of ' in a code comment).
2. Check it in into the repository.
3. Try to post a review request for the file in question.

What is the expected output? What do you see instead?

If using the browser, then instead of creating a new review request, it gets stuck with the loading animation. If using rbt post, then a new review requests is created, but it always fails to upload the diff.

What operating system are you using? What browser?
Confirmed on Ubuntu 14.04 GNU/Linux, Firefox 31.0, Chromium 34.0, Windows 7 with the same browsers and the rbt post command in RHEL 6.5.

Please provide any additional information below.

The issue is related to the *content* of the file and how RB reacts to non-ASCII characters in it. The workaround I found was to:
1. Remove the non-ASCII characters from the files.
2. Check in the files into the repository *without* posting any review requests. 3. Now that the non-ASCII chars have been removed, review requests can be successfully posted.

This was found when trying to post a review request for a file that had been reviewed and checked in 1+ year(s) ago in an older version of RB (*likely* on the 1.6.xx series) without incident.

So it was particularly time-consuming to track down the problem because RB will encounter the issue whenever it tries to read/process the remote content in the repository host. This makes it non-obvious because this happens even if the diff you're trying to upload is 100% ASCII, but as mentioned above, it fails when RB tries to process what's already checked in, which is not immediately clear to user.

While reviewing the rbt post source code, I did not find any explicit statement where the decode('ascii') method is invoked on a string or character --at least not explicitly.

I've attached a simple Python script I wrote while I was trying to reproduce the error message displayed by rbt post, which is how I was able to better confirm what I suspected. It tries to decode('ascii') each character in a file.


Error displayed on Firefox's JavaScript console after pressing 'OK' to create the request with selected diff and base directory:

SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data


Debug output and stack trace of rbt post command on RHEL system:

$ rbt post --diff-filename=$HOME/current.diff -d
RBTools 0.6.2
Python 2.6.6 (r266:84292, Nov 21 2013, 10:50:32)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)]
Running on Linux-2.6.32-431.5.1.el6.x86_64-x86_64-with-redhat-6.5-Santiago
Home = /home/ray
Current directory = /home/ray/cti/windjammer/common/lib/Connector
Checking for a Subversion repository...
Running: svn info --non-interactive
Running: diff --version
repository info: Path: https://csvnhou-pro.houston.hp.com:18490/svn/cfe-cti, Base path: /trunk/cti/windjammer/common/lib/Connector, Supports changesets: False
Making HTTP GET request to http://hopper.rose.hp.com/api/
Making HTTP GET request to https://hopper.rose.hp.com/api/review-requests/ Making HTTP POST request to https://hopper.rose.hp.com/api/review-requests/ Making HTTP GET request to https://hopper.rose.hp.com/api/review-requests/8600/diffs/ Making HTTP POST request to https://hopper.rose.hp.com/api/review-requests/8600/diffs/
Got API Error 105 (HTTP code 400): One or more fields had errors
Error data: {u'fields': {u'path': [u"'ascii' codec can't decode byte 0xe2 in position 2154: ordinal not in range(128)"]}, u'stat': u'fail', u'err': {u'msg': u'One or more fields had errors', u'code': 105}}
Traceback (most recent call last):
  File "/usr/bin/rbt", line 9, in <module>
    load_entry_point('RBTools==0.6.2', 'console_scripts', 'rbt')()
File "/usr/lib/python2.6/site-packages/RBTools-0.6.2-py2.6.egg/rbtools/commands/main.py", line 134, in main
    command.run_from_argv([RB_MAIN, command_name] + args)
File "/usr/lib/python2.6/site-packages/RBTools-0.6.2-py2.6.egg/rbtools/commands/__init__.py", line 416, in run_from_argv
    exit_code = self.main(*args) or 0
File "/usr/lib/python2.6/site-packages/RBTools-0.6.2-py2.6.egg/rbtools/commands/post.py", line 784, in main
    submit_as=self.options.submit_as)
File "/usr/lib/python2.6/site-packages/RBTools-0.6.2-py2.6.egg/rbtools/commands/post.py", line 551, in post_request
    raise CommandError(u'\n'.join(error_msg))
rbtools.commands.CommandError: Error uploading diff


One or more fields had errors (HTTP 400, API Error 105)

path: 'ascii' codec can't decode byte 0xe2 in position 2154: ordinal not in range(128)

Your review request still exists, but the diff is not attached.

https://hopper.rose.hp.com/r/8600/




Attachments:
        dec.py  360 bytes

--
You received this message because this project is configured to send all issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

--
You received this message because you are subscribed to the Google Groups 
"reviewboard-issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to reviewboard-issues+unsubscr...@googlegroups.com.
To post to this group, send email to reviewboard-issues@googlegroups.com.
Visit this group at http://groups.google.com/group/reviewboard-issues.
For more options, visit https://groups.google.com/d/optout.

Reply via email to