------------------------------------------------------------------------------
To reply, visit https://hellosplat.com/s/beanbag/tickets/4596/
------------------------------------------------------------------------------
New ticket #4596 by yaqian
For Beanbag, Inc. > Review Board
Status: New
Tags: Priority:Medium, Type:Defect
------------------------------------------------------------------------------
Error displaying diff for utf-8 files on Perforce
==============================================================================
# What version are you running?
ReviewBoard 2.5.13.1
RBTools 0.7.10
p4 executable and pypython package 2017.1
Perforce Server 2015.2
# What's the URL of the page containing the problem?
https://<review-board-server-name>/r/203074/diff/1
# What steps will reproduce the problem?
1. Find an utf-8 file from the perforce server, make some changes
2. Use the rbt tool to post the changes to reviewboard
3. Go to reviewboard to look at the diff, it will show "There was an error
displaying this diff."
# What is the expected output? What do you see instead?
Expected: A visible diff view
Actual result: There was an error displaying this diff.
Error message:
The patch to '//user/yaqian/tmp/utf8files/demo.xml' didn't apply cleanly. The
temporary files have been left in '/tmp/reviewboard.kMIK0e' for debugging
purposes.
`patch` returned:
This may be a bug in the software, a temporary outage, or an issue
with the format of your diff.
Please try again, and if you still have trouble,
contact support.
Details
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/reviewboard/diffviewer/views.py", line
299, in get
response = renderer.render_to_response(request)
File "/usr/lib/python2.7/site-packages/reviewboard/diffviewer/renderers.py",
line 56, in render_to_response
return HttpResponse(self.render_to_string(request))
File "/usr/lib/python2.7/site-packages/reviewboard/diffviewer/renderers.py",
line 74, in render_to_string
large_data=True)
File "/usr/lib/python2.7/site-packages/djblets/cache/backend.py", line 298,
in cache_memoize
compress_large_data))
File "/usr/lib/python2.7/site-packages/djblets/cache/backend.py", line 252,
in cache_memoize_iter
items = items_or_callable()
File "/usr/lib/python2.7/site-packages/djblets/cache/backend.py", line 295,
in <lambda>
lambda: [lookup_callable()],
File "/usr/lib/python2.7/site-packages/reviewboard/diffviewer/renderers.py",
line 73, in <lambda>
lambda: self.render_to_string_uncached(request),
File "/usr/lib/python2.7/site-packages/reviewboard/diffviewer/renderers.py",
line 87, in render_to_string_uncached
request=request)
File "/usr/lib/python2.7/site-packages/reviewboard/diffviewer/diffutils.py",
line 669, in populate_diff_chunks
chunks = list(generator.get_chunks())
File
"/usr/lib/python2.7/site-packages/reviewboard/diffviewer/chunk_generator.py",
line 786, in get_chunks
for chunk in super(DiffChunkGenerator, self).get_chunks(cache_key):
File
"/usr/lib/python2.7/site-packages/reviewboard/diffviewer/chunk_generator.py",
line 107, in get_chunks
large_data=True)
File "/usr/lib/python2.7/site-packages/djblets/cache/backend.py", line 298,
in cache_memoize
compress_large_data))
File "/usr/lib/python2.7/site-packages/djblets/cache/backend.py", line 252,
in cache_memoize_iter
items = items_or_callable()
File "/usr/lib/python2.7/site-packages/djblets/cache/backend.py", line 295,
in <lambda>
lambda: [lookup_callable()],
File
"/usr/lib/python2.7/site-packages/reviewboard/diffviewer/chunk_generator.py",
line 106, in <lambda>
lambda: list(self.get_chunks_uncached()),
File
"/usr/lib/python2.7/site-packages/reviewboard/diffviewer/chunk_generator.py",
line 793, in get_chunks_uncached
new = get_patched_file(old, self.filediff, self.request)
File "/usr/lib/python2.7/site-packages/reviewboard/diffviewer/diffutils.py",
line 233, in get_patched_file
return patch(diff, buffer, filediff.dest_file, request)
File "/usr/lib/python2.7/site-packages/reviewboard/diffviewer/diffutils.py",
line 169, in patch
'output': stderr,
Exception: The patch to '//user/yaqian/tmp/utf8files/demo.xml' didn't apply
cleanly. The temporary files have been left in '/tmp/reviewboard.kMIK0e' for
debugging purposes.
`patch` returned:
# What operating system are you using? What browser?
The Reviewboard server is running on Linux (Centos 7.4);
I'm using Chrome on Windows.
# Please provide any additional information below.
1. The error is from the `patch` command. I investigated and found it's because
rbtool and reviewboard synced different content for the utf8 file, to be
specific:
* the rbt tool is calling 'p4 print' with the -o option which reserves the BOM
characters.
https://github.com/reviewboard/rbtools/blob/master/rbtools/clients/perforce.py#L1317
* but the reviewboard is calling 'p4 print' without the -o option which removed
the BOM.
https://github.com/reviewboard/reviewboard/blob/master/reviewboard/scmtools/perforce.py#L556
You can use the following python code to test.
import P4
p4 = P4.P4()
# update the following according to your actual perforce settings
p4.port = 'perforce:1666'
p4.user = 'yaqian'
#p4.password = ''
depot_file = '//user/yaqian/tmp/utf8files/demo.xml'
p4.connect()
print "p4 print -q '%s'" % depot_file
res = p4.run_print('-q', depot_file)
first5c = res[-1][0:5]
print first5c
print ':'.join(x.encode('hex') for x in first5c)
print "p4 print -q -o /tmp/test.utf8.out '%s'" % depot_file
p4.run_print('-q', '-o', '/tmp/test.utf8.out', depot_file)
first5c = open('/tmp/test.utf8.out', 'rb').read()[0:5]
print first5c
print ':'.join(x.encode('hex') for x in first5c)
p4.disconnect()
2. This issue is only reproduciable with Perforce 2015.2+ which added the utf8
server file type - see the release notes -
https://www.perforce.com/perforce/r15.2/user/relnotes.txt.
> Major new functionality in 2015.2
> ...
> #998379 * **
> *UTF8 server file type added*. Files detected to hold utf8 textual
> contents can be stored as a utf8 file. These files will have
> BOMs (Byte order marks) removed when stored on the server
> and *added back when synced*.
3. To fix the issue, rbtool and reviewboard should have sync behavior
------------------------------------------------------------------------------
--
You received this message because you are subscribed to the Google Groups
"reviewboard-issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/reviewboard-issues.
For more options, visit https://groups.google.com/d/optout.
