>>> (From your previous post I found out that e.g. gui_mch_draw_string() >>> will pass strings in the encoding specified by 'enc', thus I can look >>> at output_conv to determine if I need to convert the string to utf-8 >>> before drawing it. However, I cannot use the same method to decide if >>> conversion is necessary for the titles of menu items.) >> Menu functions don't deal with more than one encoding. The Vim script >> is written in a certain encoding. It must either match the value of >> 'encoding' or specify the encoding actually used with the >> :scriptencoding command. >> >> Thus in all the gui_*.c functions you can safely assume that all menu >> text is in 'encoding'.
So it looks like you *can* use the same method, actually. What you can't assume is that the text is *valid*. You also can't assume that 'enc=utf-8', as the user could have changed it. > This presents a problem with scripts that do not use :scriptencoding > like the HTML plugin. It does present a problem, but it is primarily the script writer's problem, not Vim's or MacVim's. > It (I'm only using it as an example since its > the only problem I'm aware of) defines menu items whose titles contain > characters with the 8th bit set (in latin-1 encoding). The problem is that it does this without :scriptencoding and by hackily temporarily setting 'enc' to 'latin1', so the characters are not validated, and indeed become invalid when 'enc' is later restored to utf-8. It shouldn't do this. > With 'enc=utf-8' these characters are not converted to utf-8 so when > gui_mch_add_menu_item() is called MacVim chokes because it is fed menu > titles with invalid utf-8 strings (since they are in fact latin-1). It is at the point where 'enc' is restored to 'utf-8' that you could argue this should happen. It doesn't, and the Vim help even notes this (implicitly by noting that you need to reload menus if you change 'enc'). > I can work around this problem by falling back to using latin-1 encoding > when the utf-8 is invalid, but this doesn't seem like a good solution. That seems like a perfectly valid solution. If a plugin author doesn't declare script encoding, there is no defined 'right way' to interpret their menus and they can expect garbage on some systems. What kind of garbage they get isn't really relevant, and using 'latin-1' is probably more likely to do what the plugin author wanted than any other encoding. To my mind, as long as there isn't a crash, it can do whatever it likes in this case. It is the plugin author's responsibility to use :scriptencoding and not change 'encoding'; by doing so everything will work (because the menu items will be translated into 'encoding' when the script is read and not invalidated by a later change in 'encoding'). > I tried understanding how the other GUI ports deal with this issue in > gui_mch_add_menu_item() but they all seem to assume that the value of > 'enc' is respected. According to Bram's comment, this is the right thing to do. > I don't know why they don't fail since I'm not > familiar with the APIs used on those platforms. Evidently those OSes do their own kind of fallback or other graceful failure when they are fed invalid text. > Have you got any suggestions how to deal with scripts that fail to set > :scriptencoding? The only think I can think of that would be worth doing would be to check that when Vim converts a script that uses :scriptencoding, it uses ++bad=drop or ++bad=X and not ++bad=keep. Then as long as 'encoding' isn't changed, menus will be valid. If 'encoding' changes, the help already says they need reloading. > It seems to me this is a real problem since it makes > the encoding of (e.g.) menu titles unpredictable, and its thus up to > the OS APIs to deal with the issue (not very reliable since each OS > behaves differently). I think it's merely the responsibility to ensure the application doesn't crash when passed invalid data, which is always a possibility. This is OS specific. But what kind of garbage you should get is surely undefined. Ben. --~--~---------~--~----~------------~-------~--~----~ You received this message from the "vim_dev" maillist. For more information, visit http://www.vim.org/maillist.php -~----------~----~----~----~------~----~------~--~---
