Re: [Freeciv-Dev] (PR#40601) bad char in city name crashes GTK civclient 2.1.7 and 2.1.8
URL: http://bugs.freeciv.org/Ticket/Display.html?id=40601 On Tue, Dec 16, 2008 at 8:25 PM, Madeline Book madeline.b...@gmail.com wrote: So my best guess is that somehow the contents of the save file got munged into the present state (invalid utf8) outside of freeciv. Are you sure you did not open and re-save it in an editor or something like that? I did not. Any munging was accomplished by Freeciv. Could be as simple as a string pointer error somewhere, given that the bizarre character is at the end of the string. I don't know if it's related, but Freeciv GTK consistently fails on Vista after the game has been going a long time. The GUI windows change colors and become illegible like they're being bitblitted incorrectly, then civclient freezes. For this reason I always play with set saveturns 1. After such a failure, I can always load the most recent autosaved game. I haven't submitted the bug because I don't have a deterministic reproducer for it. I just know that it will happen after several hours of play. I guess my point is, Freeciv can munge things. ___ Freeciv-dev mailing list Freeciv-dev@gna.org https://mail.gna.org/listinfo/freeciv-dev
Re: [Freeciv-Dev] (PR#40601) bad char in city name crashes GTK civclient 2.1.7 and 2.1.8
URL: http://bugs.freeciv.org/Ticket/Display.html?id=40601 On Tue, Dec 16, 2008 at 11:32 PM, Madeline Book madeline.b...@gmail.com wrote: By the way, do you play with the prompt for city names local option enabled? No. Also, what happens when you play a nation like Brazillian or Turk? Do the utf8 characters in the start or middle of the city names get corrupted too? I will try these and other things you suggested and get back to you. ___ Freeciv-dev mailing list Freeciv-dev@gna.org https://mail.gna.org/listinfo/freeciv-dev
Re: [Freeciv-Dev] (PR#40601) bad char in city name crashes GTK civclient 2.1.7 and 2.1.8
URL: http://bugs.freeciv.org/Ticket/Display.html?id=40601 On Wed, Dec 17, 2008 at 11:13 AM, Brandon Van Every bvanev...@gmail.com wrote: On Tue, Dec 16, 2008 at 11:32 PM, Madeline Book madeline.b...@gmail.com wrote: By the way, do you play with the prompt for city names local option enabled? No. I spoke too quickly. Yes, I do, as it is the default option. I'm just so habituated to hitting OK without even thinking about it, that it's psychologically invisible to me. On Tue, Dec 16, 2008 at 11:32 PM, Madeline Book madeline.b...@gmail.com wrote: Finally, if you were to play as the Catalan nation using the sdl gui and build cities until you get Perpinyà suggested, would the name get garbled? I played another game as Catalan and reproduced the error. 1) Load the attached game catalan-1400.sav. 2) play as Catalan 3) hit Turn Done, since the settler doesn't have any moves remaining 4) build the Settler. The What should we call our new city? dialogue box appears. It contains the garbled string Perpiny[X]. 5) Selecting OK will crash the game. So, the name generator is somehow at fault, and it is specific to utf8 chars at the end of the string. Also, what happens when you play a nation like Brazillian or Turk? Do the utf8 characters in the start or middle of the city names get corrupted too? Brazilian, in the middle, no problem. Didn't try Turk or at the beginning of the city name. How about if you edit the save game manually and put in the missing a0 (that's the character with value 0xa0, or 160 in decimal). What happens if you load the game now? Didn't bother to try it. Or what if you just paste in Perpinyà into the city name suggestion popup or when renaming the city, what does that do (hopefully not just rejected by the server)? No problem here. ___ Freeciv-dev mailing list Freeciv-dev@gna.org https://mail.gna.org/listinfo/freeciv-dev
Re: [Freeciv-Dev] (PR#40601) bad char in city name crashes GTK civclient 2.1.7 and 2.1.8
URL: http://bugs.freeciv.org/Ticket/Display.html?id=40601 Forgot the attachment. catalan-1400.sav.gz Description: GNU Zip compressed data ___ Freeciv-dev mailing list Freeciv-dev@gna.org https://mail.gna.org/listinfo/freeciv-dev
Re: [Freeciv-Dev] (PR#40601) bad char in city name crashes GTK civclient 2.1.7 and 2.1.8
URL: http://bugs.freeciv.org/Ticket/Display.html?id=40601 On Wed, Dec 17, 2008 at 1:33 PM, Brandon Van Every bvanev...@gmail.com wrote: 4) build the Settler. The What should we call our new city? dialogue box appears. It contains the garbled string Perpiny[X]. Additional weirdness: if you try to edit the Perpiny[X] string, Perpiny is editable but the garbled [X] is not. You cannot erase it, the cursor will not move over it. A workaround is to hit Cancel and then take the next city name selection, in this case La Seu d'Urgell. ___ Freeciv-dev mailing list Freeciv-dev@gna.org https://mail.gna.org/listinfo/freeciv-dev
Re: [Freeciv-Dev] (PR#40601) bad char in city name crashes GTK civclient 2.1.7 and 2.1.8
URL: http://bugs.freeciv.org/Ticket/Display.html?id=40601 I must say that this only happens on Windows GTK2 client... not on gtk2 linux client... ___ Freeciv-dev mailing list Freeciv-dev@gna.org https://mail.gna.org/listinfo/freeciv-dev
[Freeciv-Dev] (PR#40601) bad char in city name crashes GTK civclient 2.1.7 and 2.1.8
URL: http://bugs.freeciv.org/Ticket/Display.html?id=40601 [bvanev...@gmail.com - Wed Dec 17 21:30:18 2008]: On Wed, Dec 17, 2008 at 4:28 PM, Jordi Negrevernis i Font jnegrever...@jnfprogramari.com wrote: URL: http://bugs.freeciv.org/Ticket/Display.html?id=40601 I must say that this only happens on Windows GTK2 client... not on gtk2 linux client... So it's probably an #ifdef in a name generation string handler. Ring any bells anyone? The name generation does not work that way. The server reads the city name suggestions from the nation ruleset file on startup and sends city name suggestions to the client in PACKET_CITY_NAME_SUGGESTION_INFO. The received name is passed on to gtk code that puts it into the entry box. The only assumptions this code makes is that the string is NULL terminated and less than 32 bytes in length, which is the case with Perpinyà (even garbled) from what I can tell from the savefile. What I suspect is that when the string is placed into the entry box, gtk tries to convert it to an encoding other than utf-8, or chops off the last character resulting in the invalid utf-8. This could be proven if you turn off the prompt for city name option and build a city with the Perpinya settler. If the name is not garbled and the program does not crash then we would know it is a problem with the string being put into the entry, and hence a bug in the particular gtk libraries used by the freeciv exe on windows. You could also try inputting other valid utf-8 (e.g. my sig) into the name suggestion box, the city rename popup, or just the chat input line to see what it does. On linux these all work fine, so they should too on windows. --- 私のためにいい家を見つけてください。 ___ Freeciv-dev mailing list Freeciv-dev@gna.org https://mail.gna.org/listinfo/freeciv-dev
Re: [Freeciv-Dev] (PR#40601) bad char in city name crashes GTK civclient 2.1.7 and 2.1.8
URL: http://bugs.freeciv.org/Ticket/Display.html?id=40601 On Wed, Dec 17, 2008 at 4:28 PM, Jordi Negrevernis i Font jnegrever...@jnfprogramari.com wrote: URL: http://bugs.freeciv.org/Ticket/Display.html?id=40601 I must say that this only happens on Windows GTK2 client... not on gtk2 linux client... So it's probably an #ifdef in a name generation string handler. Ring any bells anyone? ___ Freeciv-dev mailing list Freeciv-dev@gna.org https://mail.gna.org/listinfo/freeciv-dev
[Freeciv-Dev] (PR#40601) bad char in city name crashes GTK civclient 2.1.7 and 2.1.8
URL: http://bugs.freeciv.org/Ticket/Display.html?id=40601 [bvanev...@gmail.com - Wed Dec 17 23:43:51 2008]: On Wed, Dec 17, 2008 at 5:42 PM, Madeline Book madeline.b...@gmail.com wrote: This could be proven if you turn off the prompt for city name option and build a city with the Perpinya settler. If the name is not garbled and the program does not crash then we would know it is a problem with the string being put into the entry, and hence a bug in the particular gtk libraries used by the freeciv exe on windows. I turned off prompt for city name. When building the settler, the game now simply crashes. So, the string is being corrupted before it is delivered to the dialog box. You could also try inputting other valid utf-8 (e.g. my sig) into the name suggestion box, the city rename popup, or just the chat input line to see what it does. On linux these all work fine, so they should too on windows. 私のためにいい家を見つけてください。 Your .sig is truncated for length, but it inputs just fine. This is very good, both results imply that gtk is not to blame and that the corruption may be occurring when the city name is being transferred from the client to the server. When you start a client from a dos window, it should print a line like: Encodings: Data=UTF-8, Local=UTF-8, Internal=UTF-8 Does yours match the one above, or is it different? Also, are you using any kind of special character set on your system (e.g. iso_8859-1 or something similar)? --- 我々は隠れ家まであれらの文字をたどった。 ___ Freeciv-dev mailing list Freeciv-dev@gna.org https://mail.gna.org/listinfo/freeciv-dev
Re: [Freeciv-Dev] (PR#40601) bad char in city name crashes GTK civclient 2.1.7 and 2.1.8
URL: http://bugs.freeciv.org/Ticket/Display.html?id=40601 On Wed, Dec 17, 2008 at 5:42 PM, Madeline Book madeline.b...@gmail.com wrote: What I suspect is that when the string is placed into the entry box, gtk tries to convert it to an encoding other than utf-8, or chops off the last character resulting in the invalid utf-8. This could be proven if you turn off the prompt for city name option and build a city with the Perpinya settler. If the name is not garbled and the program does not crash then we would know it is a problem with the string being put into the entry, and hence a bug in the particular gtk libraries used by the freeciv exe on windows. I turned off prompt for city name. When building the settler, the game now simply crashes. So, the string is being corrupted before it is delivered to the dialog box. You could also try inputting other valid utf-8 (e.g. my sig) into the name suggestion box, the city rename popup, or just the chat input line to see what it does. On linux these all work fine, so they should too on windows. --- 私のためにいい家を見つけてください。 Your .sig is truncated for length, but it inputs just fine. ___ Freeciv-dev mailing list Freeciv-dev@gna.org https://mail.gna.org/listinfo/freeciv-dev
[Freeciv-Dev] (PR#40601) bad char in city name crashes GTK civclient 2.1.7 and 2.1.8
URL: http://bugs.freeciv.org/Ticket/Display.html?id=40601 The default Windows character set in western countries is CP1252: http://en.wikipedia.org/wiki/CP1252 In this character set 0xA0 means non-breakable space, so this character gets stripped by remove_trailing_spaces() when reading the nation file. ___ Freeciv-dev mailing list Freeciv-dev@gna.org https://mail.gna.org/listinfo/freeciv-dev
[Freeciv-Dev] (PR#40601) bad char in city name crashes GTK civclient 2.1.7 and 2.1.8
URL: http://bugs.freeciv.org/Ticket/Display.html?id=40601 [cproc - Thu Dec 18 04:42:10 2008]: The default Windows character set in western countries is CP1252: http://en.wikipedia.org/wiki/CP1252 Ah thanks for that, the last piece of the puzzle. ;) In this character set 0xA0 means non-breakable space, so this character gets stripped by remove_trailing_spaces() when reading the nation file. Indeed, if I start the client with a different local encoding e.g. $ LANG=de_DE ./civ then the trailling space eating behavior occurs. This suggests that it might be enough to just start the client in a locale encoded with utf8: C:\ set LANG=en_US.UTF-8 C:\ civclient.exe (Of course this should be done in the directory where the program resides instead of C:.) But the behavior of remove_trailing_spaces() is troubling. Would it be safe to just have it remove only ' ', '\t', '\r', '\v' and '\n', instead of anything for which isspace returns TRUE? Or would this then cause other problems when other kinds of whitespace characters are used in the user's locale? Perhaps the server should just set its own (local) encoding to utf8 when it reads the ruleset files, then set it back to what it was originally... :? --- 宇宙の向こうから来て、空間そのものを食ってしまっていた。 ___ Freeciv-dev mailing list Freeciv-dev@gna.org https://mail.gna.org/listinfo/freeciv-dev
[Freeciv-Dev] (PR#40601) bad char in city name crashes GTK civclient 2.1.7 and 2.1.8
URL: http://bugs.freeciv.org/Ticket/Display.html?id=40601 [bvanev...@gmail.com - Thu Dec 18 04:26:29 2008]: On Wed, Dec 17, 2008 at 9:04 PM, Madeline Book madeline.b...@gmail.com wrote: When you start a client from a dos window, it should print a line like: Encodings: Data=UTF-8, Local=UTF-8, Internal=UTF-8 Does yours match the one above, or is it different? Nothing at all is printed on Vista. civclient --help also prints nothing. In fact, it doesn't matter what command line options I give, nothing is printed. Documentation on client command line options appears to be quite out of date. How recently have you done this with a dos box on Windows? What version of Windows? Sorry, this was my mistake. That line is only printed if the client was configured with debugging support enabled, which I realize is probably not true for the windows packages. --- 腹が減っているんだろうね。 ___ Freeciv-dev mailing list Freeciv-dev@gna.org https://mail.gna.org/listinfo/freeciv-dev
[Freeciv-Dev] (PR#40601) bad char in city name crashes GTK civclient 2.1.7 and 2.1.8
URL: http://bugs.freeciv.org/Ticket/Display.html?id=40601 [bvanev...@gmail.com - Mon Dec 15 02:25:20 2008]: In civclient 2.1.7, a random city name was generated, Perpiny[X]. The [X] stands for some weird graphic of a presumably unprintable character. Clicking on this city crashes both civclient 2.1.7 and 2.1.8 on Windows Vista SP1. I've attached a saved game file, garbagechar, that reproduces the problem. Somehow or another, the city name in the save file got turned into the invalid utf8 sequence Perpinyc3. When this game is loaded there are numerous gtk error messages about invalid utf8 and the widgets that are supposed to display the city name are blank. Now I looked in the catalan ruleset file where this city name is defined and there it is correctly Perpinyc3a0. I also tried starting a new game (in trunk though) and making Catalan cities with the editor until Perpinya was created, then saving, quitting, and reloading the game. The save file contained the valid utf-8 and the loaded game had the right name. Then I checked the saving and loading code, and as far as I can see it should work regardless of the encoding (so it does not seem likely it chopped off the last byte or something like that). So my best guess is that somehow the contents of the save file got munged into the present state (invalid utf8) outside of freeciv. Are you sure you did not open and re-save it in an editor or something like that? --- 明らかに点突然変異のです。 ___ Freeciv-dev mailing list Freeciv-dev@gna.org https://mail.gna.org/listinfo/freeciv-dev
[Freeciv-Dev] (PR#40601) bad char in city name crashes GTK civclient 2.1.7 and 2.1.8
URL: http://bugs.freeciv.org/Ticket/Display.html?id=40601 [bvanev...@gmail.com - Wed Dec 17 03:17:43 2008]: On Tue, Dec 16, 2008 at 8:25 PM, Madeline Book madeline.b...@gmail.com wrote: So my best guess is that somehow the contents of the save file got munged into the present state (invalid utf8) outside of freeciv. Are you sure you did not open and re-save it in an editor or something like that? I did not. Any munging was accomplished by Freeciv. Could be as simple as a string pointer error somewhere, given that the bizarre character is at the end of the string. Yes unfortunately that the particular character is at the end of the string does not help to narrow down the cause. By the way, do you play with the prompt for city names local option enabled? And if so, does it still make the bad city name if you turn it off? Also, what happens when you play a nation like Brazillian or Turk? Do the utf8 characters in the start or middle of the city names get corrupted too? How about if you edit the save game manually and put in the missing a0 (that's the character with value 0xa0, or 160 in decimal). What happens if you load the game now? Or what if you just paste in Perpinyà into the city name suggestion popup or when renaming the city, what does that do (hopefully not just rejected by the server)? Finally, if you were to play as the Catalan nation using the sdl gui and build cities until you get Perpinyà suggested, would the name get garbled? I would run these test myself, but on linux they all pass. :( I don't know if it's related, but Freeciv GTK consistently fails on Vista after the game has been going a long time. The GUI windows change colors and become illegible like they're being bitblitted incorrectly, then civclient freezes. Maybe GDI objects are being leaked and running out, as was once suggested in a forum posting regarding the freeland tileset. :) For this reason I always play with set saveturns 1. After such a failure, I can always load the most recent autosaved game. I haven't submitted the bug because I don't have a deterministic reproducer for it. I just know that it will happen after several hours of play. I guess my point is, Freeciv can munge things. Well, on windows we have the added joy of dealing with problems caused by incompatibilities or deficiencies in the gtk libraries used to build the executables. So in cases like this it is possible that the origin of the bug lies in vista features or the win32 gtk code (as much as I would prefer it to be in freeciv, since then I could actually fix it). --- これらの文字は繁殖しているかも。 ___ Freeciv-dev mailing list Freeciv-dev@gna.org https://mail.gna.org/listinfo/freeciv-dev
[Freeciv-Dev] (PR#40601) bad char in city name crashes GTK civclient 2.1.7 and 2.1.8
URL: http://bugs.freeciv.org/Ticket/Display.html?id=40601 In civclient 2.1.7, a random city name was generated, Perpiny[X]. The [X] stands for some weird graphic of a presumably unprintable character. Clicking on this city crashes both civclient 2.1.7 and 2.1.8 on Windows Vista SP1. I've attached a saved game file, garbagechar, that reproduces the problem. 1) start civclient 2.1.8. 2) click Load Saved Game 3) click garbagechar 4) click Catalan (Human) player 5) double click the city Perpiny[X]. Immediately one gets the message civclient.exe has stopped working. garbagechar.sav.gz Description: GNU Zip compressed data ___ Freeciv-dev mailing list Freeciv-dev@gna.org https://mail.gna.org/listinfo/freeciv-dev
