Re: A weird unicode puzzle after upgrade from 2.0.20 to 2.5.4

My devserver is indeed a staging server. Also, I have written a couple of extensions for our specific workflow in the organization. For that too I need a devserver. I validated that MySQL 5.6 does not produce this warning. I will upgrade to MySQL 5.6 soon. Thanks Koushik On Friday, December

Re: A weird unicode puzzle after upgrade from 2.0.20 to 2.5.4

What's the purpose of your dev server? Depending on needs, SQLite might be a better option, but probably not if this is a staging server. You might want to try a newer version of MySQL. I might be wrong, but I don't believe I've hit this issue with 5.6. When in DEBUG = True mode, warnings will

Re: A weird unicode puzzle after upgrade from 2.0.20 to 2.5.4

condemsediffs only needs to be done once. It's optional, but can reduce database size significantly. You don't need to run a periodic condense operation. We also condense individual diffs when manually viewing them as well. I'll try to find out more info about which versions of MySQL are impacted

Re: A weird unicode puzzle after upgrade from 2.0.20 to 2.5.4

MySQL 5.5 Thanks for the explanation So it is not required to condense diff before db migration, right? In fact, condense diff happens on the fly. Is there any advantage to do condense diff from time to time? On Dec 21, 2016 7:19 PM, "Christian Hammond" wrote: > Hey,

Re: A weird unicode puzzle after upgrade from 2.0.20 to 2.5.4

Hey, So the good news is, this is not a corruption issue, nor is it related to the contents of any of your files. This error is occurring when saving diff data using our new compressed storage mechanism. We populate a binary field with compressed diff data when uploading a new diff, viewing an

Re: A weird unicode puzzle after upgrade from 2.0.20 to 2.5.4

Christian, As promised, more information in the file attached. Please HELP, HELP!! With the utf8 conversion warning, MY CONCERN IS DATA LOSS/CORRUPTION. If the text contains only plain English, then may be we are fine. But where multi-byte encoding is required, is there any guarantee

Re: A weird unicode puzzle after upgrade from 2.0.20 to 2.5.4

Yes, the database upgrade was complete without errors, after changing the tables to InnoDB. I will soon give you the full information. My db size is too large (15GB). I am trying to create a smaller test DB to reproduce the problem. Thanks, On Monday, December 19, 2016 at 7:35:44 PM UTC-5,

Re: A weird unicode puzzle after upgrade from 2.0.20 to 2.5.4

Hi, Sorry you're hitting this too. To confirm, did the actual database upgrade itself complete without errors, once the table type and such were adjusted? Can you show me the full error information? Thanks, Christian On Mon, Dec 19, 2016 at 15:23 Koushik Roy wrote:

Re: A weird unicode puzzle after upgrade from 2.0.20 to 2.5.4

Christian, I too, have faced this problem while upgrading from 2.0.21 to 2.5.7 The 'invalid utf8 character' may be only a symptom of the problem. My only guess is around the following - - the database was using 'latin1' as default charset, RB 2.5.x expects 'utf8'. So some

Re: A weird unicode puzzle after upgrade from 2.0.20 to 2.5.4

Hi Eric, I'd still really love to get a copy of those entries from your database so I can replicate the problem. I simply can't make it happen here, and have no idea why you'd be seeing what you're seeing. It should not be necessary to run condensediffs prior to upgrading. Christian --

Re: A weird unicode puzzle after upgrade from 2.0.20 to 2.5.4

I have finally re-run my entire automated migration process from start to finish. After running condensediffs *before* the upgrade, I was then able to run condensediffs after the upgrade as well, all without any warnings or errors. Perhaps you should add a step to the upgrade guide, advising

Re: A weird unicode puzzle after upgrade from 2.0.20 to 2.5.4

I re-ran the migration, and before doing anything, set the DEBUG flag to True before fetching any diffs. I can confirm that it fails on *every* diff, not just random ones here and there. Running with DEBUG = False, the warning doesn't throw an exception, but I do see an entry in the log:

Re: A weird unicode puzzle after upgrade from 2.0.20 to 2.5.4

Hi Eric, Hmm, we'll need to look into that. Is there a way you'd be able to send us the diff for that? (I can help you find it.) We will need a copy in order to diagnose this. We can sign an NDA for it. Christian On Friday, May 13, 2016, eric via reviewboard

A weird unicode puzzle after upgrade from 2.0.20 to 2.5.4

After I migrated my server to 2.5.4, I'm seeing a weird error. I restarted both memcached and apache2, and then browse to a specific review request. Then I click on the "Diff" tab. (After I turned on DEBUG = True in the settings_local.py file) I see this instead of diffs. Traceback (most