| Ok after a few more hours of fiddling around:
It seems like originally all the data was in Latin 1 encoding. When I change everything back to Latin 1, the € symbol appears fine again. However I still see all the '?' in any data the comes from a BLOB. This seems to indicate that all those '?' are actually now part of the data and are not actually indicating an unknown code. If that's the case, I have the lovely task of going through each piece of data by hand and trying to figure out what that '?' was before. I think when the sql server was upgraded, something must have gone terribly wrong.
So I am going to move to FrontBase for this. To hell with mySQL I say. So I have another question which is probably fairly straight forward:
If I add a duplicate EOModel to my project which uses a different adaptor setting to connect to FrontBase, can I connect to mySQL with one of the EOModels and FrontBase with the other and using the custom java classes just load the data from mySQL via one and write to FrontBase via the other? I presume I need two editing contexts, each one using a different EOModel with the relevant adaptor settings but I don't know how to create the second editing context using the other EOModel.
Can I do it this way and if so, can someone tell me how to create the 2nd connection...?
Thanks, David. On 14 Jun 2006, at 13:57, David Griffith wrote: Ok I have just written a little routine to load each BLOB in WO and then write that back to a TEXT column instead. What is happening now is that WO is receiving the text but with '?' instead of special characters. When I then write that back to the TEXT column, the '?' is written instead of the original character. So now I end up with all the text with loads of '?' scattered throughout it.
Any idea how to avoid that?
Then, if I go to the database using Navicat and I change one of the '?' to a • (bullet in case that did not appear correctly), it saves and displays fine in Navicat but on my web pages it shows: 'â?¢'.
This is the same problem I am having with the euro symbol. Any ideas?
Kind regards, David. On 13 Jun 2006, at 17:07, David Griffith wrote: Just let me be sure that I am clear on all of this:
The JDBC adaptor uses a default encoding of UTF-8 so I should not need to specify anything on the connection URL.
I have set the page wrapper of my site to have the content encoding set to UTF8.
I have included the code that someone posted on here to put in Application and Session classes to force UTF8.
I need to convert (somehow) the BLOBs to TEXT fields in mySQL with UTF-8 encoding, which is what I was trying to do but it is truncating them all.
Do I need to specifically convert all columns in the database to UTF8? Or does it matter?
Is there anything else I need to do when working with mySQL 4.1.x apart from using 3.0.x Connector-J?
If not, I am stumped.... I have done all this except of the BLOB to TEXT conversion.
Kind regards, David.
On 13 Jun 2006, at 17:00, David Griffith wrote:
HI Xavier,
Sorry I see what you mean about the email, I'll start a fresh one in the future to avoid problems..
I'm talking about types in mySQL. I am using mySQL 4.1.19 with connector-j 3.0.17. Having read on the list (and the problem with BLOBs returning hex instead of text) I reverted to using 3.0.17 from 3.1.7. I cannot get any 3.1.x connector-j to work correctly with BLOBs.
I have tried converting the BLOB columns in mySQL to TEXT and MEDIUMTEXT but it truncates so much that it destroys the data. I can't figure any other reasonable way of converting them. If I can convert them to TEXT columns in mySQL successfully, that will be a start. However I am now getting the Euro symbol (which is in a VARCHAR) displaying as rubbish as well where that used to be fine before. I think that has only happened since I added the UTF8 code (application and session) and changed the header in the HTML pages to force UTF8.
It's a real nightmare.. I was actually in the process of trying to convert the whole thing over to FrontBase as I just cannot figure out what to do about this. The problem is the conversion to Frontbase will take a while too with all the referenital integrity stuff so I have to be careful and I don't want to lose any data either.
Kind regards, David.
On 13 Jun 2006, at 16:52, Dev WO wrote:
Hi Xavier,
Thank you for your reply. I'm not sure what you meant about starting a new email, I thought I did, but anyway I'll bear that in mind.
You probably first did "reply" to the post about "Handi Wikibook link" the change the subject. But all the references about the "previous" email where still in the mail header so that's why it's in the same thread (when messages are sorted by thread).
The problem is that I tried to convert the columns from BLOB to TEXT already but it truncated virtually every field. Can you tell me how you did it?
Are you talking about types in EOModeler or types in your database? Which version of MySQL are you using? 4.1.x? I had the same issue, I just have to recall what I did...I'll search the list to see if I recall something.
I've done virtually everything else you've said below and no difference at all, hence the reason I want to switch to TEXT fields. I used TEXT rather than MEDIUMTEXT but TEXT should be long enough, should it not?
TEXT is already 65535 characters long (I think MySQL 4.1.x handles in number of characters) so it's not the issue.
Kind regards, David.
On 13 Jun 2006, at 16:01, Dev WO wrote:
Hello David, so first of all, you should start a new email when asking a new question, if you don't, thread don't work correctly:)
I had the same issue as you when moving from MySQL 4.0.x to 4.1.x. The solution was to change all the BLOB to mediumtext (or longtext, but medium is already bigger than what you'd really want), apply a caracter encoding to those fields (which should be UTF-8). Then make sure you don't have any specific encoding in your connection dictionary (or UTF-8). Make sure your web page header and the "head" declaration of your pages is also UTF-8 And finally you should be OK.
If you have Practical WO at hand, there's a couple method to add in the Application.java and in the Session.java to make sure everything is UTF-8 (it's in the chapter about localization). I'm sure it was already posted on this list, maybe search for UTF-8.
You should be going before the end of the day:)
Xavier
HI all,
I have a database (using mySQL) with a number of BLOB fields that store long text values. Apparently BLOBs have no character encoding so I presume this is applied at the time it is retrieved from the database.
Previously everything was working fine but now I am getting all sorts of strange behaviour. For example the Euro symbol (which is stored in a field which is of type VARCHAR) is not displaying correctly - have a look at http://www.ipodshoppers.com if you want to see what I mean.
All the data from the BLOB fields are displaying ? instead of certain symbols and I cannot figure out what to do. I have spent about 3 days already trying to figure it out. I really need some help here as I am unable to update any of the data.
I appears that the data is read from the BLOB, is not interpreted correctly and displays wrong, then when I try to write back to the database it can't update because of the ? in the text, which do not exist in the BLOB.
It's an absolute nightmare. I have tried applying UTF-8 encoding everywhere I can but it makes no difference at all. I even tried converting the BLOBs to TEXT fields but that only served to truncate 90% of the fields and lose most of the data.
Does anyone have any idea what I can do?
Kind regards, David. _______________________________________________ Do not post admin requests to the list. They will be ignored. Help/Unsubscribe/Update your Subscription:
_______________________________________________ Do not post admin requests to the list. They will be ignored. Help/Unsubscribe/Update your Subscription:
_______________________________________________ Do not post admin requests to the list. They will be ignored. Help/Unsubscribe/Update your Subscription:
_______________________________________________ Do not post admin requests to the list. They will be ignored. Help/Unsubscribe/Update your Subscription:
|
