I re-ran the migration, and before doing anything, set the DEBUG flag to True before fetching any diffs.
I can confirm that it fails on *every* diff, not just random ones here and there. Running with DEBUG = False, the warning doesn't throw an exception, but I do see an entry in the log: WARNING:py.warnings:/usr/lib64/python2.7/site-packages/django/db/backends/mysql/base.py:124: Warning: Invalid utf8 character string: '81FE48' After I've viewed the diffs of a review request once, the warnings stop appearing (consistent with the data being migrated to compressed form.). Switching back to DEBUG = True, and I no longer see failures for the review request diffs that I looked at while DEBUG = False Trying a different approach, I ran "rb-site manage ______ -- condensediffs", and that also generated the same warnings. Here's a sample: WARNING:py.warnings:/usr/lib64/python2.7/site-packages/django/db/backends/mysql/base.py:124: Warning: Invalid utf8 character string: 'D76700' return self.cursor.execute(query, args) WARNING:py.warnings:/usr/lib64/python2.7/site-packages/django/db/backends/mysql/base.py:124: Warning: Invalid utf8 character string: 'A9C813' return self.cursor.execute(query, args) ... This generated 169000+ lines of output, corresponding to 56435 individual warning messages out of a total of 76032 diff files condensed. This gave me an idea. I ran "rb-site ... condensediffs" before the upgrade, then once again after the upgrade. Problem went away. Weird. Do you want me to try to find out more, and if so, what? Eric. On Friday, May 13, 2016 at 1:49:53 PM UTC-7, Christian Hammond wrote: > > Hi Eric, > > Hmm, we'll need to look into that. Is there a way you'd be able to send us > the diff for that? (I can help you find it.) We will need a copy in order > to diagnose this. We can sign an NDA for it. > > Christian > > > On Friday, May 13, 2016, eric via reviewboard <revie...@googlegroups.com > <javascript:>> wrote: > >> After I migrated my server to 2.5.4, I'm seeing a weird error. I >> restarted both memcached and apache2, and then browse to a specific review >> request. >> >> Then I click on the "Diff" tab. (After I turned on DEBUG = True in the >> settings_local.py file) I see this instead of diffs. >> >> Traceback (most recent call last): >> File >> "/usr/lib64/python2.7/site-packages/ReviewBoard-2.5.4-py2.7.egg/reviewboard/diffviewer/views.py", >> >> line 275, in get >> response = renderer.render_to_response(request) >> File >> "/usr/lib64/python2.7/site-packages/ReviewBoard-2.5.4-py2.7.egg/reviewboard/diffviewer/renderers.py", >> >> line 56, in render_to_response >> return HttpResponse(self.render_to_string(request)) >> File >> "/usr/lib64/python2.7/site-packages/ReviewBoard-2.5.4-py2.7.egg/reviewboard/diffviewer/renderers.py", >> >> line 74, in render_to_string >> large_data=True) >> File >> "/usr/lib64/python2.7/site-packages/Djblets-0.9.3-py2.7.egg/djblets/cache/backend.py", >> >> line 295, in cache_memoize >> compress_large_data)) >> File >> "/usr/lib64/python2.7/site-packages/Djblets-0.9.3-py2.7.egg/djblets/cache/backend.py", >> >> line 249, in cache_memoize_iter >> items = items_or_callable() >> File >> "/usr/lib64/python2.7/site-packages/Djblets-0.9.3-py2.7.egg/djblets/cache/backend.py", >> >> line 292, in <lambda> >> lambda: [lookup_callable()], >> File >> "/usr/lib64/python2.7/site-packages/ReviewBoard-2.5.4-py2.7.egg/reviewboard/diffviewer/renderers.py", >> >> line 73, in <lambda> >> lambda: self.render_to_string_uncached(request), >> File >> "/usr/lib64/python2.7/site-packages/ReviewBoard-2.5.4-py2.7.egg/reviewboard/diffviewer/renderers.py", >> >> line 87, in render_to_string_uncached >> request=request) >> File >> "/usr/lib64/python2.7/site-packages/ReviewBoard-2.5.4-py2.7.egg/reviewboard/diffviewer/diffutils.py", >> >> line 429, in populate_diff_chunks >> chunks = list(generator.get_chunks()) >> File >> "/usr/lib64/python2.7/site-packages/ReviewBoard-2.5.4-py2.7.egg/reviewboard/diffviewer/chunk_generator.py", >> >> line 756, in get_chunks >> for chunk in super(DiffChunkGenerator, self).get_chunks(cache_key): >> File >> "/usr/lib64/python2.7/site-packages/ReviewBoard-2.5.4-py2.7.egg/reviewboard/diffviewer/chunk_generator.py", >> >> line 107, in get_chunks >> large_data=True) >> File >> "/usr/lib64/python2.7/site-packages/Djblets-0.9.3-py2.7.egg/djblets/cache/backend.py", >> >> line 295, in cache_memoize >> compress_large_data)) >> File >> "/usr/lib64/python2.7/site-packages/Djblets-0.9.3-py2.7.egg/djblets/cache/backend.py", >> >> line 249, in cache_memoize_iter >> items = items_or_callable() >> File >> "/usr/lib64/python2.7/site-packages/Djblets-0.9.3-py2.7.egg/djblets/cache/backend.py", >> >> line 292, in <lambda> >> lambda: [lookup_callable()], >> File >> "/usr/lib64/python2.7/site-packages/ReviewBoard-2.5.4-py2.7.egg/reviewboard/diffviewer/chunk_generator.py", >> >> line 106, in <lambda> >> lambda: list(self.get_chunks_uncached()), >> File >> "/usr/lib64/python2.7/site-packages/ReviewBoard-2.5.4-py2.7.egg/reviewboard/diffviewer/chunk_generator.py", >> >> line 763, in get_chunks_uncached >> new = get_patched_file(old, self.filediff, self.request) >> File >> "/usr/lib64/python2.7/site-packages/ReviewBoard-2.5.4-py2.7.egg/reviewboard/diffviewer/diffutils.py", >> >> line 230, in get_patched_file >> diff = tool.normalize_patch(filediff.diff, filediff.source_file, >> File >> "/usr/lib64/python2.7/site-packages/ReviewBoard-2.5.4-py2.7.egg/reviewboard/diffviewer/models.py", >> >> line 218, in _get_diff >> self._migrate_diff_data() >> File >> "/usr/lib64/python2.7/site-packages/ReviewBoard-2.5.4-py2.7.egg/reviewboard/diffviewer/models.py", >> >> line 421, in _migrate_diff_data >> diff_hash_is_new = self._set_diff(self.legacy_diff_hash.binary) >> File >> "/usr/lib64/python2.7/site-packages/ReviewBoard-2.5.4-py2.7.egg/reviewboard/diffviewer/models.py", >> >> line 225, in _set_diff >> RawFileDiffData.objects.get_or_create_from_data(diff) >> File >> "/usr/lib64/python2.7/site-packages/ReviewBoard-2.5.4-py2.7.egg/reviewboard/diffviewer/managers.py", >> >> line 345, in get_or_create_from_data >> 'compression': compression, >> File "/usr/lib64/python2.7/site-packages/django/db/models/manager.py", >> line 154, in get_or_create >> return self.get_queryset().get_or_create(**kwargs) >> File "/usr/lib64/python2.7/site-packages/django/db/models/query.py", >> line 383, in get_or_create >> obj.save(force_insert=True, using=self.db) >> File "/usr/lib64/python2.7/site-packages/django/db/models/base.py", >> line 545, in save >> force_update=force_update, update_fields=update_fields) >> File "/usr/lib64/python2.7/site-packages/django/db/models/base.py", >> line 573, in save_base >> updated = self._save_table(raw, cls, force_insert, force_update, >> using, update_fields) >> File "/usr/lib64/python2.7/site-packages/django/db/models/base.py", >> line 654, in _save_table >> result = self._do_insert(cls._base_manager, using, fields, update_pk, >> raw) >> File "/usr/lib64/python2.7/site-packages/django/db/models/base.py", >> line 687, in _do_insert >> using=using, raw=raw) >> File "/usr/lib64/python2.7/site-packages/django/db/models/manager.py", >> line 232, in _insert >> return insert_query(self.model, objs, fields, **kwargs) >> File "/usr/lib64/python2.7/site-packages/django/db/models/query.py", >> line 1514, in insert_query >> return query.get_compiler(using=using).execute_sql(return_id) >> File >> "/usr/lib64/python2.7/site-packages/django/db/models/sql/compiler.py", line >> 903, in execute_sql >> cursor.execute(sql, params) >> File >> "/usr/lib64/python2.7/site-packages/Djblets-0.9.3-py2.7.egg/djblets/log/middleware.py", >> >> line 32, in execute >> return self.cursor.execute(sql, params) >> File >> "/usr/lib64/python2.7/site-packages/django/db/backends/mysql/base.py", line >> 124, in execute >> return self.cursor.execute(query, args) >> File "/usr/lib64/python2.7/site-packages/MySQLdb/cursors.py", line 207, >> in execute >> if not self._defer_warnings: self._warning_check() >> File "/usr/lib64/python2.7/site-packages/MySQLdb/cursors.py", line 117, >> in _warning_check >> warn(w[-1], self.Warning, 3) >> Warning: Invalid utf8 character string: '890600' >> >> >> Note that if I set DEBUG = False, then I see the diffs in the browser, >> but I still see warnings in the log file. I'm nervous that migrating my >> production system forward to 2.5.4 will corrupt data. So now I'm holding >> off. >> >> Looking back through the stack trace, I see there's some logic in there >> about "_migrate_diff_data" - code which didn't exist in 2.0.X. >> >> I suspect this is a manifestation of this bug: >> https://hellosplat.com/s/beanbag/tickets/4007/ >> >> What can I do? >> >> Eric. >> >> >> -- >> Supercharge your Review Board with Power Pack: >> https://www.reviewboard.org/powerpack/ >> Want us to host Review Board for you? Check out RBCommons: >> https://rbcommons.com/ >> Happy user? Let us know! https://www.reviewboard.org/users/ >> --- >> You received this message because you are subscribed to the Google Groups >> "reviewboard" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to reviewboard+unsubscr...@googlegroups.com. >> For more options, visit https://groups.google.com/d/optout. >> > > > -- > -- > Christian Hammond > President/CEO of Beanbag <https://www.beanbaginc.com/> > Makers of Review Board <https://www.reviewboard.org/> > > -- Supercharge your Review Board with Power Pack: https://www.reviewboard.org/powerpack/ Want us to host Review Board for you? Check out RBCommons: https://rbcommons.com/ Happy user? Let us know! https://www.reviewboard.org/users/ --- You received this message because you are subscribed to the Google Groups "reviewboard" group. To unsubscribe from this group and stop receiving emails from it, send an email to reviewboard+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
