URL:
<http://gna.org/task/?func=detailitem&item_id=1506>
Summary: Branch DEV_2005-03-28_UTF-8
Project: Savane
Submitted by: toddy
Submitted on: jeu 31.03.2005 à 23:00
Should Start On: lun 28.03.2005 à 00:00
Should be Finished on: ven 01.07.2005 à 00:00
Category: i18n
Priority: 5 - Immediate
Status: Ready For Test
Privacy: Public
Assigned to: toddy
Open/Closed: Open
Planned Release: 1.0.7
_______________________________________________________
Details:
This item is for tracking the progress of the switch to UTF-8, needed for
correct I18N of Savane.
What has happened already?
- Initial version of README for update
- Generation of needed locales (UTF-8 enabled version)
- Conversion of site-specific files into UTF-8
- Switch in the PHP frontend to display only UTF-8
What needs still to be done?
- Complete README for update
- Conversion of database into UTF-8
_______________________________________________________
Follow-up Comments:
-------------------------------------------------------
Date: sam 18.06.2005 à 14:13 By: Mathieu Roy <yeupou>
I think it would be quite unusual (possible though) to have a mysqld running
without mysql command being available.
As Savane installation process already relies on it, we should not be worried
about using it.
However, the fact that we have a perl script relying on Savane libs is in
fact helpful, since the user is not forced to type password. Have you checked
out my changes?
Finally, yes, the problem occurs only with error/success messages. That's
what I found strange.
-------------------------------------------------------
Date: sam 18.06.2005 à 13:55 By: Tobias Toedter <toddy>
Hi,
I'll try to answer some of the comments below.
> BTW, have original database file (in sql/mysql) have been
> converted for fresh installs?
It doesn't seem to be necessary. I converted them into UTF-8 and diff'ed the
resulting files against the repository copies. There were no differences. I
guess everything in the files is ASCII anyway.
> Looking at your script more carefully, I'm not sure to
> understand why you use perl to do the mysql insert instead of
> a simple dumb
> mysql < my_converted_file.dump
That's because I assumed that I can only rely on Perl being installed, since
it's required for Savane anyway. "mysql" on the other hand is a command line
client which is not necessary to run the mysql server. If we can rely on the
client program mysql being installed, that's certainly the way to go. We
should drop the Perl script then.
> So with my little changes to your script, now it seems to
> work. Everything seems to be in UTF-8 but I have an issue
> with the success/error messages.
Does this encoding problem only occur with the success and error messages?
The rest of your screenshot shows correct French accents, it seems.
-------------------------------------------------------
Date: ven 17.06.2005 à 15:45 By: Mathieu Roy <yeupou>
There\'s also an issue with the encoding of mails we should sort out before
the release, mentioned in bug #2440
-------------------------------------------------------
Date: ven 17.06.2005 à 15:43 By: Mathieu Roy <yeupou>
So with my little changes to your script, now it seems to work. Everything
seems to be in UTF-8 but I have an issue with the success/error messages.
See the attached screenshot.
Any clues?
-------------------------------------------------------
Date: ven 17.06.2005 à 15:23 By: Mathieu Roy <yeupou>
Looking at your script more carefully, I'm not sure to understand why you use
perl to do the mysql insert instead of a simple dumb
mysql < my_converted_file.dump
I think it gets messy because dbi add quotes when it considers to be
necessary but a mysql dumpfile is supposed to be correctly escaped already.
-------------------------------------------------------
Date: ven 17.06.2005 à 15:15 By: Mathieu Roy <yeupou>
We definitely have to speed up this release that have been delayed for too
long because of me. I'm now able to spend the necessary time to test it.
-------------------------------------------------------
Date: ven 17.06.2005 à 15:14 By: Mathieu Roy <yeupou>
BTW, have original database file (in sql/mysql) have been converted for fresh
installs?
-------------------------------------------------------
Date: ven 17.06.2005 à 15:13 By: Mathieu Roy <yeupou>
[EMAIL PROTECTED]
./convert_database.pl --file=database-backup_001.sql
Converting your database from ISO-8859-1 into UTF-8
DBD::mysql::db do failed: You have an error in your SQL syntax. Check the
manual that corresponds to your MySQL server version for the right syntax to
use near ''' at line 1 at ./convert_database.pl line 71.
DBD::mysql::db do failed: You have an error in your SQL syntax. Check the
manual that corresponds to your MySQL server version for the right syntax to
use near ''' at line 1 at ./convert_database.pl line 71.
Still the same with the stuff from the head.
Attached to this mail the database dump that cause the trouble.
-------------------------------------------------------
Date: dim 12.06.2005 à 22:41 By: Mathieu Roy <yeupou>
Hum, will be done by the end of this week, promise (urf, got an already big
list of things to do for this week, but this one has been on the list for too
long already)
-------------------------------------------------------
Date: dim 12.06.2005 à 20:23 By: Tobias Toedter <toddy>
Hi,
what about some testing of this branch to get the next release going? The
"should be finished on"-deadline comes closer :)
-------------------------------------------------------
Date: jeu 12.05.2005 à 09:03 By: Mathieu Roy <yeupou>
No, I'm sorry, I havent tested further since then (my brother was in
Strasbourg for a few days, I was more or less off the net during that period
and I have some backlog to handle right now).
-------------------------------------------------------
Date: mer 11.05.2005 à 20:26 By: Tobias Toedter <toddy>
Mathieu,
just a short ping in case you've lost my question in your inbox ...
Did the HEAD version of Util.pm (i.e. quoting of the newline character) fix
the error you've encountered, or does the script still fail?
-------------------------------------------------------
Date: mer 04.05.2005 à 18:32 By: Tobias Toedter <toddy>
"In fact, you should not commit on the HEAD a fix that have to go on the
branch."
Right, but the fix was not specific to UTF-8, and the function seems to be
used in some other places as well, so I thought it would be sensible to
commit to HEAD.
Apart from that, did the HEAD version of Util.pm fix your problem?
-------------------------------------------------------
Date: mer 04.05.2005 à 18:10 By: Mathieu Roy <yeupou>
" BTW: Should I commit the exact same fix in HEAD and the branch twice? I
thought that this could create some confusion, so I refrained from doing
so."
In fact, you should not commit on the HEAD a fix that have to go on the
branch. The proper way to get such fix on the HEAD is to merge the file with
the branch. Otherwise, you only have two option, the one you picked (the most
sensible; commiting only on the HEAD) which breaks the branch or commiting on
both HEAD and branch which will ends up in merge conflicts later.
-------------------------------------------------------
Date: mer 04.05.2005 à 17:48 By: Tobias Toedter <toddy>
Hm, that's strange. I didn't add any code for handling quotes, because the
script operates on a database dump. In this dump (which is SQL) I thought all
quotes are already escaped.
However, maybe the failure was not with a quote, but a newline. I changed the
file Util.pm in the lib subdirectory a few weeks ago, to handle newlines as
well. To make the merge easier, I didn't include the fix in the UTF-8 branch;
it is only in HEAD. So could you update your file lib/Savannah/Util.pm back to
HEAD and try that script again?
BTW: Should I commit the exact same fix in HEAD and the branch twice? I
thought that this could create some confusion, so I refrained from doing so.
-------------------------------------------------------
Date: mer 04.05.2005 à 16:43 By: Mathieu Roy <yeupou>
I've got:
Converting your database from ISO-8859-1 into UTF-8
DBD::mysql::db do failed: You have an error in your SQL syntax. Check the
manual that corresponds to your MySQL server version for the right syntax to
use near ''Replace this paragraph by the technical description of your' a at
./convert_database.pl line 71.
Are you sure you dealth with quotes? Looks like it failed with a quote.
-------------------------------------------------------
Date: mar 05.04.2005 à 14:04 By: Mathieu Roy <yeupou>
I'll try to test this stuff before the end of the week. If I cant, I'm afraid
I wont be able to do it the next week.
-------------------------------------------------------
Date: lun 04.04.2005 à 17:40 By: Tobias Toedter <toddy>
I've just completed the README for the conversion and committed three more
Perl scripts for the database conversion into UTF-8.
In my opinion, this should be all which needs to be done to prepare the
switch to UTF-8.
I'd like people to review my scripts (and general approach) in the branch,
read the README, and, if possible, even try to migrate a test install (please
don't do this with real data yet).
Comments are welcome.
_______________________________________________________
File Attachments:
-------------------------------------------------------
Date: ven 17.06.2005 à 15:43 Name: prob1.jpg Size: 326,78Ko By: yeupou
<http://gna.org/task/download.php?item_id=1506&item_file_id=80>
-------------------------------------------------------
Date: ven 17.06.2005 à 15:13 Name: database-backup_001.sql Size: 126,52Ko
By: yeupou
<http://gna.org/task/download.php?item_id=1506&item_file_id=79>
_______________________________________________________
Reply to this item at:
<http://gna.org/task/?func=detailitem&item_id=1506>
_______________________________________________
Message posté via/par Gna!
http://gna.org/
