Re: [Iup-users] Request
Ola Antonio, Mucho gracias. I know what you mean by "taking so long". I do that all the time too, so don't worry about it. I almost forgot about this issue, but since then I've did some more research and as a result, this has become my Windows programming philosophy: Windows is native to UTF-16, not UTF-8 or UTF-GB (GB18035) UTF-8 and UTF-GB are both ASCII/ANSI compatible, so they can be substituted in most cases for ASCII functions EVERYTHING that Unicode evangelists claim as a benefit for UTF-8, is also THE SAME EXACT BENEFIT that UTF-GB offers, and because UTF-GB is legacy compatible for Chinese, UTF-GB should always be promoted as the official Unicode encoding of choice. To not do so can only be explained by a petty racist bias instead of sound logic Windows has a "multibyte" conversion function for UTF-8/UTF-GB, so while it over-complicates things a little, it isn't hopelessly over-complicated, probably adding about an extra eight lines of code whenever needed (like it is for filenames). Speaking of filenames, path lengths will be limited to 256 bytes unless you use the "\\?\" prefix The relevant Window API functions for those who are interested in having TRULY international applications is: MultiByteToWideChar() WideCharToMultiByte() WideCharToMultiByte() I don't like it, but I most certainly can easily live with it. Signed, Andres PS -- The problem with certain overlapping characters was due to the wrong DPI being set for my monitor. As usual, it was a simple problem that was not at all easy to find. The lesson learned is not all display monitors should default to 96dpi, so all you people out there with 2560x1600 monitors out there, be warned! On 2020-05-01 at 7:27 AM, Antonio Scuri wrote: Hi, Sorry taking so long to get back to this. I run some tests here and it seems to be working now: This was in Windows 10 with visual styles enabled. Here is in Windows 7 with visual styles enabled: And Windows 7 using Classic Theme: All build from latest SVN. Maybe something changed that fixed this. But anyway it seems to be something in the native system. We don't have much control over that inside an IupText. Best, Scuri Em qui., 6 de fev. de 2020 às 16:25, Antonio Scuri escreveu: Ok got it Em qui, 6 de fev de 2020 11:54, Andrew Robinson escreveu: See attached On 2020-02-06 at 8:51 AM, Antonio Scuri wrote: Please send me a small text file with the string. Next week I'll be back to work and check this. Best, Scuri Em qui, 6 de fev de 2020 10:44, Andrew Robinson escreveu: No problemo, The context was: IupSetStrAttribute(txtAsmCode,'VALUE',pCodeArray) where txtAsmCode is a pointer to the multiline textbox control. Regards, Andres On 2020-02-06 at 7:37 AM, Antonio Scuri wrote: Hi, I need more information about the context when the problem occurs. Were you typing, or setting the VALUE attribute? Or pasting from clipboard? Best, Scuri Em qui, 6 de fev de 2020 09:09, Andrew Robinson escreveu: Hello Antonio, I just want to remind you that IUP still doesn't appear to support UTF-8 properly in textboxes, as evidenced by the screen capture below of the right-pointer arrow (➔ U+279c) overlapping the characters u in "u8" and the character $ in "$DstStr" Regards, Andrew ___ Iup-users mailing list Iup-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/iup-users
Re: [Iup-users] Request
Hi, Sorry taking so long to get back to this. I run some tests here and it seems to be working now: [image: image.png] This was in Windows 10 with visual styles enabled. Here is in Windows 7 with visual styles enabled: [image: image.png] And Windows 7 using Classic Theme: [image: image.png] All build from latest SVN. Maybe something changed that fixed this. But anyway it seems to be something in the native system. We don't have much control over that inside an IupText. Best, Scuri Em qui., 6 de fev. de 2020 às 16:25, Antonio Scuri escreveu: > Ok got it > > Em qui, 6 de fev de 2020 11:54, Andrew Robinson > escreveu: > >> See attached >> >> On 2020-02-06 at 8:51 AM, Antonio Scuri wrote: >> >> Please send me a small text file with the string. Next week I'll be >> back to work and check this. >> >> Best, >> Scuri >> >> >> Em qui, 6 de fev de 2020 10:44, Andrew Robinson >> escreveu: >> >>> No problemo, >>> >>> The context was: IupSetStrAttribute(txtAsmCode,'VALUE',pCodeArray) where >>> txtAsmCode is a pointer to the multiline textbox control. >>> >>> Regards, >>> Andres >>> >>> >>> On 2020-02-06 at 7:37 AM, Antonio Scuri wrote: >>> >>> Hi, >>> >>> I need more information about the context when the problem occurs. >>> >>> Were you typing, or setting the VALUE attribute? Or pasting from >>> clipboard? >>> >>> Best, >>> Scuri >>> >>> >>> Em qui, 6 de fev de 2020 09:09, Andrew Robinson >>> escreveu: >>> Hello Antonio, I just want to remind you that IUP still doesn't appear to support UTF-8 properly in textboxes, as evidenced by the screen capture below of the right-pointer arrow (➔ U+279c) overlapping the characters u in "u8" and the character $ in "$DstStr" Regards, Andrew >>> >>> >> ___ Iup-users mailing list Iup-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/iup-users
Re: [Iup-users] Request
Ok got it Em qui, 6 de fev de 2020 11:54, Andrew Robinson escreveu: > See attached > > On 2020-02-06 at 8:51 AM, Antonio Scuri wrote: > > Please send me a small text file with the string. Next week I'll be back > to work and check this. > > Best, > Scuri > > > Em qui, 6 de fev de 2020 10:44, Andrew Robinson > escreveu: > >> No problemo, >> >> The context was: IupSetStrAttribute(txtAsmCode,'VALUE',pCodeArray) where >> txtAsmCode is a pointer to the multiline textbox control. >> >> Regards, >> Andres >> >> >> On 2020-02-06 at 7:37 AM, Antonio Scuri wrote: >> >> Hi, >> >> I need more information about the context when the problem occurs. >> >> Were you typing, or setting the VALUE attribute? Or pasting from >> clipboard? >> >> Best, >> Scuri >> >> >> Em qui, 6 de fev de 2020 09:09, Andrew Robinson >> escreveu: >> >>> Hello Antonio, >>> >>> I just want to remind you that IUP still doesn't appear to support UTF-8 >>> properly in textboxes, as evidenced by the screen capture below of the >>> right-pointer arrow (➔ U+279c) overlapping the characters u in "u8" and the >>> character $ in "$DstStr" >>> >>> Regards, >>> Andrew >>> >>> >>> >> >> > ___ Iup-users mailing list Iup-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/iup-users
Re: [Iup-users] Request
See attached On 2020-02-06 at 8:51 AM, Antonio Scuri wrote: Please send me a small text file with the string. Next week I'll be back to work and check this. Best, Scuri Em qui, 6 de fev de 2020 10:44, Andrew Robinson escreveu: No problemo, The context was: IupSetStrAttribute(txtAsmCode,'VALUE',pCodeArray) where txtAsmCode is a pointer to the multiline textbox control. Regards, Andres On 2020-02-06 at 7:37 AM, Antonio Scuri wrote: Hi, I need more information about the context when the problem occurs. Were you typing, or setting the VALUE attribute? Or pasting from clipboard? Best, Scuri Em qui, 6 de fev de 2020 09:09, Andrew Robinson escreveu: Hello Antonio, I just want to remind you that IUP still doesn't appear to support UTF-8 properly in textboxes, as evidenced by the screen capture below of the right-pointer arrow (➔ U+279c) overlapping the characters u in "u8" and the character $ in "$DstStr" Regards, Andrew Sample.txt Description: Binary data ___ Iup-users mailing list Iup-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/iup-users
Re: [Iup-users] Request
Please send me a small text file with the string. Next week I'll be back to work and check this. Best, Scuri Em qui, 6 de fev de 2020 10:44, Andrew Robinson escreveu: > No problemo, > > The context was: IupSetStrAttribute(txtAsmCode,'VALUE',pCodeArray) where > txtAsmCode is a pointer to the multiline textbox control. > > Regards, > Andres > > > On 2020-02-06 at 7:37 AM, Antonio Scuri wrote: > > Hi, > > I need more information about the context when the problem occurs. > > Were you typing, or setting the VALUE attribute? Or pasting from > clipboard? > > Best, > Scuri > > > Em qui, 6 de fev de 2020 09:09, Andrew Robinson > escreveu: > >> Hello Antonio, >> >> I just want to remind you that IUP still doesn't appear to support UTF-8 >> properly in textboxes, as evidenced by the screen capture below of the >> right-pointer arrow (➔ U+279c) overlapping the characters u in "u8" and the >> character $ in "$DstStr" >> >> Regards, >> Andrew >> >> >> > > ___ Iup-users mailing list Iup-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/iup-users
Re: [Iup-users] Request
No problemo, The context was: IupSetStrAttribute(txtAsmCode,'VALUE',pCodeArray) where txtAsmCode is a pointer to the multiline textbox control. Regards, Andres On 2020-02-06 at 7:37 AM, Antonio Scuri wrote: Hi, I need more information about the context when the problem occurs. Were you typing, or setting the VALUE attribute? Or pasting from clipboard? Best, Scuri Em qui, 6 de fev de 2020 09:09, Andrew Robinson escreveu: Hello Antonio, I just want to remind you that IUP still doesn't appear to support UTF-8 properly in textboxes, as evidenced by the screen capture below of the right-pointer arrow (➔ U+279c) overlapping the characters u in "u8" and the character $ in "$DstStr" Regards, Andrew ___ Iup-users mailing list Iup-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/iup-users
Re: [Iup-users] Request
Hi, I need more information about the context when the problem occurs. Were you typing, or setting the VALUE attribute? Or pasting from clipboard? Best, Scuri Em qui, 6 de fev de 2020 09:09, Andrew Robinson escreveu: > Hello Antonio, > > I just want to remind you that IUP still doesn't appear to support UTF-8 > properly in textboxes, as evidenced by the screen capture below of the > right-pointer arrow (➔ U+279c) overlapping the characters u in "u8" and the > character $ in "$DstStr" > > Regards, > Andrew > > > ___ Iup-users mailing list Iup-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/iup-users
Re: [Iup-users] Request
Hello Antonio, I just want to remind you that IUP still doesn't appear to support UTF-8 properly in textboxes, as evidenced by the screen capture below of the right-pointer arrow (➔ U+279c) overlapping the characters u in "u8" and the character $ in "$DstStr" Regards, Andrew ___ Iup-users mailing list Iup-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/iup-users
Re: [Iup-users] Request
Hi Andrew, On Sun, 1 Dec 2019 17:09:57 -0700 "Andrew Robinson" wrote: > Antonio, > > If you don't feel like having this discussion, let me know and I can > save it for another time... Frankly, I don't want to really enter into this discussion. Just I strongly disagree. > [...] > work, otherwise someone would have done it already. If I were to pick > sides (and I have had to pick sides) I would choose native Windows > and hobble UTF-16 in Linux because Linux represents less than 3% of > the entire market that IUP could ever appeal to. Without good > internationalization support, only Westerners will want to use IUP. > You know this because you've seen the complaints about this already, > and it will only get worse. IUp's unique selling point never was to be yet another low code tool to create applications for Windows. There are enough out there. Its USP was portability. I'd rather look into supporting Android and iOS via IUp. This kills those 3%. Others research pointet out that UTF-8 is usually the better thing internally. IUp wraps Windows with a compatibility layer anyway, doesn't it? Why should leak low level encoding information to the application layer? Best Regards Jörg ___ Iup-users mailing list Iup-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/iup-users
Re: [Iup-users] Request
PS -- This is a secret, so don't tell anybody listening: You can leave out Unicode support in IUP, and still support Unicode. For example, without Unicode support enabled, most compilers will translate GetCommandLine into GetCommandLineA (the ANSI version), and with Unicode enabled most compilers will translate GetCommandLine into GetCommandLineW (the UTF-16 version). If your compiler won't let you bypass that, then use the "asm" keyword and call it directly. You only need to do this is a few key places, and leave the rest of the code alone. On 2019-12-01 at 5:09 PM, Andrew Robinson wrote: Antonio, If you don't feel like having this discussion, let me know and I can save it for another time... To avoid confusion, let's get our terminology right: There is no such thing as a Unicode strings type. Unicode is a standardized enumeration that is not used anywhere because it is a 21-bit enumeration, which is not a nice neat integer multiple of 8-bits. Instead there has to be standardized Unicode encodings which are 8-bit multiples, such as UTF-8, UTF-16, UTF-32, and GB-18030. Therefore when you say IUP uses "Unicode strings", that has no meaning (even the C-language itself has no string data-type, but rather hijacks a pointer to an undeclared array of unsigned chars). From what I can tell, IUP follows the C-model and uses a hybrid zero-terminated array of bytes consisting of ANSI and/or UTF-8. This issue isn't only about fopen, there are 711 other functions in Windows that do not natively support UTF-8 functions. Microsoft even said they will not support UTF-8 encodings and using it can CRASH some functions if you do, so they recommend not doing it. Why would you ignore Microsoft? Why use it in ways it wasn't intended to be used and is unsupported? You have a very difficult decision to make here, Antonio. I know because I've been there and done that. You want IUP to be universal but in order to do that, you need to write code that will compile for both Linux and Windows. The problem is internationalization. Linux natively uses UTF-8 and Windows natively uses UTF-16 and they do not go well together. So either you hobble Windows together to support UTF-8 or you hobble Linux to support UTF-16. I guarantee you it won't work, otherwise someone would have done it already. If I were to pick sides (and I have had to pick sides) I would choose native Windows and hobble UTF-16 in Linux because Linux represents less than 3% of the entire market that IUP could ever appeal to. Without good internationalization support, only Westerners will want to use IUP. You know this because you've seen the complaints about this already, and it will only get worse. That's because no matter what you do, I will always have to do translations between UTF-8 and UTF-16 and GB-18030 and God knows what else. I don't even want to have to think about that unless I'm reading or writing text to a file, otherwise I want one encoding standard, not three or four encoding standards. It will make it confusing, error-prone, and almost useless. So let's talk about ways to get IUP to support UTF-16 for Windows and UTF-8 for Linux, and we can start by letting everyone know that not everything needs to be UTF-8 or UTF-16 in IUP. Internally IUP works just fine with ANSI or ASCII text. It is only the textbox and file dialog I/O that need different code. Oh, and the menu functions. Is there anything else? That isn't a lot, is it? And the file dialog should be simple -- just return an UTF-16 string when Unicode is enabled and tell the programmers to use wfopen instead of fopen. Since Linux doesn't support anything other than UTF-8, Linux can remain as is, without any "Unicode" directive. Also, since this does not involve code pages, that issue can be skipped entirely. Regards, Andrew On 2019-12-01 at 1:28 PM, Antonio Scuri wrote: Yes, the problem is not the conversion itself. IUP already use Unicode strings when setting and retrieving native element strings. The problem is we can NOT change the IUP API without breaking compatibility with several thousands of lines of code of existing applications. To add a new API in parallel is something we don't have time to do. So we are still going to use char* in our API for some time. UTF-16 would imply in using wchar* in everything. What we can do now is to provide some mechanism for the application to be able to use the returned string in IupFileDlg in another control and in fopen. Probably a different attribute will be necessary. Some applications I know are simply using UTF8MODE_FILE=NO, so they can use it in fopen, and before displaying it in another control, converting that string to UTF-8. Best, Scuri Em dom., 1 de dez. de 2019 às 12:43, Andrew Robinson escreveu: Antonio, It is a trivial thing to translate between UTF encodings in Windows using the MultiByteToWideChar() function, but the problem is, there are 711 API functions in Windows, none of which support UTF-8. They only
Re: [Iup-users] Request
This isn't a competition, is it? It takes at most, four lines of code to convert from one UTF encoding to any other encoding supported by Microsoft. This is not rocket science. On 2019-11-30 at 11:21 PM, Pete Lomax via Iup-users wrote: On Saturday, 30 November 2019, 18:49:55 GMT, Antonio Scuri wrote: need conversion functions to/from UTF-8 and the filesystem encoding. Would be nice to have a solution for that inside IUP, but for now we still don't have one. My own routines in Phix are 360 lines, a quick search yielded this: https://www.montefusco.com/qs1rboostsrv/ConvertUTF.c which is 540 lines. Not saying that's the very best, or the right licence, etc, but it should all be reasonably straightforward?? Regards, Pete ___ Iup-users mailing list Iup-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/iup-users
Re: [Iup-users] Request
Hi, Andrew wrote: >TIP: IUP is just a GUI framework and should not be involved in any UTF >>standards battles. IUP should just stick to the native encoding of the >underlying >OS (which is ANSI or UTF-16) and let the programmer be able to >seamlessly >call native Windows Unicode compliant functions without any >additional and >unnecessary overhead. I agreed. The simpler the better. Best regards, Ranier Vilela ___ Iup-users mailing list Iup-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/iup-users
Re: [Iup-users] Request
Antonio, If you don't feel like having this discussion, let me know and I can save it for another time... To avoid confusion, let's get our terminology right: There is no such thing as a Unicode strings type. Unicode is a standardized enumeration that is not used anywhere because it is a 21-bit enumeration, which is not a nice neat integer multiple of 8-bits. Instead there has to be standardized Unicode encodings which are 8-bit multiples, such as UTF-8, UTF-16, UTF-32, and GB-18030. Therefore when you say IUP uses "Unicode strings", that has no meaning (even the C-language itself has no string data-type, but rather hijacks a pointer to an undeclared array of unsigned chars). From what I can tell, IUP follows the C-model and uses a hybrid zero-terminated array of bytes consisting of ANSI and/or UTF-8. This issue isn't only about fopen, there are 711 other functions in Windows that do not natively support UTF-8 functions. Microsoft even said they will not support UTF-8 encodings and using it can CRASH some functions if you do, so they recommend not doing it. Why would you ignore Microsoft? Why use it in ways it wasn't intended to be used and is unsupported? You have a very difficult decision to make here, Antonio. I know because I've been there and done that. You want IUP to be universal but in order to do that, you need to write code that will compile for both Linux and Windows. The problem is internationalization. Linux natively uses UTF-8 and Windows natively uses UTF-16 and they do not go well together. So either you hobble Windows together to support UTF-8 or you hobble Linux to support UTF-16. I guarantee you it won't work, otherwise someone would have done it already. If I were to pick sides (and I have had to pick sides) I would choose native Windows and hobble UTF-16 in Linux because Linux represents less than 3% of the entire market that IUP could ever appeal to. Without good internationalization support, only Westerners will want to use IUP. You know this because you've seen the complaints about this already, and it will only get worse. That's because no matter what you do, I will always have to do translations between UTF-8 and UTF-16 and GB-18030 and God knows what else. I don't even want to have to think about that unless I'm reading or writing text to a file, otherwise I want one encoding standard, not three or four encoding standards. It will make it confusing, error-prone, and almost useless. So let's talk about ways to get IUP to support UTF-16 for Windows and UTF-8 for Linux, and we can start by letting everyone know that not everything needs to be UTF-8 or UTF-16 in IUP. Internally IUP works just fine with ANSI or ASCII text. It is only the textbox and file dialog I/O that need different code. Oh, and the menu functions. Is there anything else? That isn't a lot, is it? And the file dialog should be simple -- just return an UTF-16 string when Unicode is enabled and tell the programmers to use wfopen instead of fopen. Since Linux doesn't support anything other than UTF-8, Linux can remain as is, without any "Unicode" directive. Also, since this does not involve code pages, that issue can be skipped entirely. Regards, Andrew On 2019-12-01 at 1:28 PM, Antonio Scuri wrote: Yes, the problem is not the conversion itself. IUP already use Unicode strings when setting and retrieving native element strings. The problem is we can NOT change the IUP API without breaking compatibility with several thousands of lines of code of existing applications. To add a new API in parallel is something we don't have time to do. So we are still going to use char* in our API for some time. UTF-16 would imply in using wchar* in everything. What we can do now is to provide some mechanism for the application to be able to use the returned string in IupFileDlg in another control and in fopen. Probably a different attribute will be necessary. Some applications I know are simply using UTF8MODE_FILE=NO, so they can use it in fopen, and before displaying it in another control, converting that string to UTF-8. Best, Scuri Em dom., 1 de dez. de 2019 às 12:43, Andrew Robinson escreveu: Antonio, It is a trivial thing to translate between UTF encodings in Windows using the MultiByteToWideChar() function, but the problem is, there are 711 API functions in Windows, none of which support UTF-8. They only directly support ANSI or UTF-16. Read that quote from Microsoft I posted the other day, and notice how they said that passing a UTF-8 encoded string WILL cause some API functions to crash your application. The problem is, Microsoft has not published a list of all of those unstable API functions BECAUSE MICROSOFT HAS SAID IT WILL ONLY SUPPORT UTF-16 and ANSI, and only certain versions of Windows 10 can support UTF-8. Code should be not only be readable, maintainable, and modular, it should also make sense, so if Microsoft choose UTF-16 as their Unicode standard of choice, why complicate things by
Re: [Iup-users] Request
Antonio, It is a trivial thing to translate between UTF encodings in Windows using the MultiByteToWideChar() function, but the problem is, there are 711 API functions in Windows, none of which support UTF-8. They only directly support ANSI or UTF-16. Read that quote from Microsoft I posted the other day, and notice how they said that passing a UTF-8 encoded string WILL cause some API functions to crash your application. The problem is, Microsoft has not published a list of all of those unstable API functions BECAUSE MICROSOFT HAS SAID IT WILL ONLY SUPPORT UTF-16 and ANSI, and only certain versions of Windows 10 can support UTF-8. Code should be not only be readable, maintainable, and modular, it should also make sense, so if Microsoft choose UTF-16 as their Unicode standard of choice, why complicate things by choosing a different standard? Adding 711 UTF-8-->UTF-16 and 711 UTF-16-->UTF-8 translations to those 711 native Windows API functions would make it unreasonably complicated and clutter the code. The only time translations should be an "issue" is when you need to translate text files from and to a different UTF encoding, but like I said, that is a trivial thing to do. TIP: IUP is just a GUI framework and should not be involved in any UTF standards battles. IUP should just stick to the native encoding of the underlying OS (which is ANSI or UTF-16) and let the programmer be able to seamlessly call native Windows Unicode compliant functions without any additional and unnecessary overhead. Regards, Andrew On 2019-11-30 at 11:21 PM, Pete Lomax via Iup-users wrote: On Saturday, 30 November 2019, 18:49:55 GMT, Antonio Scuri wrote: need conversion functions to/from UTF-8 and the filesystem encoding. Would be nice to have a solution for that inside IUP, but for now we still don't have one. My own routines in Phix are 360 lines, a quick search yielded this: https://www.montefusco.com/qs1rboostsrv/ConvertUTF.c which is 540 lines. Not saying that's the very best, or the right licence, etc, but it should all be reasonably straightforward?? Regards, Pete ___ Iup-users mailing list Iup-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/iup-users
Re: [Iup-users] Request
Might be worth checking out https://bitbucket.org/knight666/utf8rewind/src/default/ as well... On Sunday, 1 December 2019, 06:41:57 GMT, Pete Lomax via Iup-users wrote: On Saturday, 30 November 2019, 18:49:55 GMT, Antonio Scuri wrote: ... need conversion functions to/from UTF-8 and the filesystem encoding. Would be nice to have a solution for that inside IUP, but for now we still don't have one. My own routines in Phix are 360 lines, a quick search yielded this: https://www.montefusco.com/qs1rboostsrv/ConvertUTF.c which is 540 lines. Not saying that's the very best, or the right licence, etc, but it should all be reasonably straightforward?? Regards, Pete ___ Iup-users mailing list Iup-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/iup-users ___ Iup-users mailing list Iup-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/iup-users
Re: [Iup-users] Request
On Saturday, 30 November 2019, 18:49:55 GMT, Antonio Scuri wrote: ... need conversion functions to/from UTF-8 and the filesystem encoding. Would be nice to have a solution for that inside IUP, but for now we still don't have one. My own routines in Phix are 360 lines, a quick search yielded this: https://www.montefusco.com/qs1rboostsrv/ConvertUTF.c which is 540 lines. Not saying that's the very best, or the right licence, etc, but it should all be reasonably straightforward?? Regards, Pete ___ Iup-users mailing list Iup-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/iup-users
Re: [Iup-users] Request
Hi Antonio, Thanks for the professional reply. "The problem, for now, resumes to convert the string returned by IupFileDlg to a format that fopen can successfully use" The problem there is using fopen instead of _wfopen. Those are the only two basic C-STD choices. "When ... you need to convert it" My point exactly! Why do all that back and forth encoding and decoding, when you can just keep everything in UTF-16 and have hundreds of Unicode specific functions at your disposal in all versions of Windows, with no excessive encoding and decoding needed? That way they only other time you have to worry about Unicode, is for reading and writing foreign text files. "The GTK documentation describe this" GTK isn't Windows. In Windows, a great many functions end in either an "A" or a "W", such as AddAtom, AddConsoleAlias, AddLocalAlternateComputerName, BeginUpdateResourceBuildCommDCBAndTimeouts, BuildCommDCB, CallNamedPipe, CheckNameLegalDOS8Dot3CommConfigDialog, CompareString, CopyFileEx, CopyFile, CreateActCtx, ... etcWindows is UTF-16 centric but Linux is only UTF-8 capable, but I'm just interested in the Windows API for now, since Linux represents less than 3% of the desktop market. Internally Windows is either ASCII or UTF-16 and I want to work with Windows, not against it. So ... UTF-16 for Windows and UTF-8 for Linux. I think this would be a huge step to making IUP internationally friendly as well, as it would cut down the considerable amount of time it can sometimes take for language translations, especially for UTF-GB18030. I'm not sure about menu items, as I haven't experimented with that. Yet. I'll let you know unless you beat me to it. Much Thanks, Andrew On 2019-11-30 at 11:49 AM, Antonio Scuri wrote: Hi Andrew, I still think that we are in the right path. So, the most promising situation is: > Action: So then I set UTF8MODE=Yes and UTF8MODE_FILE=Yes and I get the following: > Result: All Unicode characters displays properly but I still cannot open the file with fopen() using the filename provided by IupFileDlg(). The problem, for now, resumes to convert the string returned by IupFileDlg to a format that fopen can successfully use. When UTF8MODE_FILE=Yes, the string returned by IupFileDlg is in UTF-8, so you need to convert it to the filesystem encoding. When UTF8MODE_FILE=No, the string returned by IupFileDlg is in the filesystem encoding, there is not need for conversion, but you can not display the returned value in an interface element, so you need to convert it to UTF-8. Either way, you need conversion functions to/from UTF-8 and the filesystem encoding. Would be nice to have a solution for that inside IUP, but for now we still don't have one. The GTK documentation describe this in: https://developer.gnome.org/glib/stable/glib-Character-Set-Conversion.html Best, Scuri Em sex., 22 de nov. de 2019 às 19:16, Andrew Robinson escreveu: Antonio, I have a file on a Windows 7 computer (with the default code page for English), that I rename by cutting and pasting a UTF-8 encoded character from an Internet page into the filename. What happens then? The browser copies and converts it to UTF-16 and places it in the clipboard, which I then paste into the filename. Good News: Even though my version Windows is English only, the filename displays properly in Windows Bad News: IUP will not display Unicode properly in all textboxes Comment: IUP displays the ellipsis (… U+2026) correctly, but the textarea doesn't display the right-pointer arrow (➔ U+279c) properly Action: What happens when I set UTF8MODE=Yes? Result: The first textbox incorrectly displays the ellipsis character as unknown/error, but now the textarea displays the correct right-pointer character, although the character incorrectly overlap other characters. Action: If I set UTF8MODE_FILE=Yes, I get the following: Bad News:: No Unicode characters display correctly anywhere. Also, I cannot open the file with fopen() using the filename provided by IupFileDlg(). Action: So then I set UTF8MODE=Yes and UTF8MODE_FILE=Yes and I get the following: Result: All Unicode characters displays properly but I still cannot open the file with fopen() using the filename provided by IupFileDlg(). Discussion: Windows NTFS supports UTF-16 and only UTF-16 filenames, so no matter what your code page is set to, Windows will always translate filenames from UTF-16 to your code page encoding (and back again). If I retrieve that filename using IUP's very convenient IupFileDlg(), it returns the filename string in ANSI or UTF-8, but Windows fopen() does not accept UTF-8 or UTF-16, it only accepts the current ANSI code page. I can use _wfopen(), but only if the string is encoded as UTF-16, which Iup does not support for this. It would be nice to have everything in the Windows version of Iup in native UTF-16, as their are hundreds of internal functions that directly support that format. I would only have to worry
Re: [Iup-users] Request
Hi Andrew, I still think that we are in the right path. So, the most promising situation is: > Action: So then I set UTF8MODE=Yes and UTF8MODE_FILE=Yes and I get the following: > Result: All Unicode characters displays properly but I still cannot open the file with fopen() using the filename provided by IupFileDlg(). The problem, for now, resumes to convert the string returned by IupFileDlg to a format that fopen can successfully use. When UTF8MODE_FILE=Yes, the string returned by IupFileDlg is in UTF-8, so you need to convert it to the filesystem encoding. When UTF8MODE_FILE=No, the string returned by IupFileDlg is in the filesystem encoding, there is not need for conversion, but you can not display the returned value in an interface element, so you need to convert it to UTF-8. Either way, you need conversion functions to/from UTF-8 and the filesystem encoding. Would be nice to have a solution for that inside IUP, but for now we still don't have one. The GTK documentation describe this in: https://developer.gnome.org/glib/stable/glib-Character-Set-Conversion.html Best, Scuri Em sex., 22 de nov. de 2019 às 19:16, Andrew Robinson escreveu: > Antonio, > > I have a file on a Windows 7 computer (with the default code page for > English), that I rename by cutting and pasting a UTF-8 encoded character > from an Internet page into the filename. What happens then? The browser > copies and converts it to UTF-16 and places it in the clipboard, which I > then paste into the filename. > > *Good News*: Even though my version Windows is English only, the filename > displays properly in Windows > > *Bad News*: IUP will not display Unicode properly in all textboxes > *Comment*: IUP displays the ellipsis (… U+2026) correctly, but the > textarea doesn't display the right-pointer arrow (➔ U+279c) properly > > *Action*: What happens when I set UTF8MODE=Yes? > *Result*: The first textbox incorrectly displays the ellipsis character > as unknown/error, but now the textarea displays the correct > right-pointer character, although the character incorrectly overlap other > characters. > > *Action*: If I set UTF8MODE_FILE=Yes, I get the following: > *Bad News*:: No Unicode characters display correctly anywhere. Also, I > cannot open the file with fopen() using the filename provided by > IupFileDlg(). > > *Action*: So then I set UTF8MODE=Yes and UTF8MODE_FILE=Yes and I get the > following: > *Result*: All Unicode characters displays properly but I still cannot > open the file with fopen() using the filename provided by IupFileDlg(). > > *Discussion*: Windows NTFS supports UTF-16 and only UTF-16 filenames, so > no matter what your code page is set to, Windows will > always translate filenames from UTF-16 to your code page encoding (and back > again). If I retrieve that filename using IUP's very convenient > IupFileDlg(), it returns the filename string in ANSI or UTF-8, but Windows > fopen() does not accept UTF-8 or UTF-16, it only accepts the current ANSI > code page. I can use _wfopen(), but only if the string is encoded as > UTF-16, which Iup does not support for this. It would be nice to have > everything in the Windows version of Iup in native UTF-16, as their are > hundreds of internal functions that directly support that format. I would > only have to worry about translating filenames or certain file contents > when supporting other languages and code pages. > > Per https://en.wikipedia.org/wiki/Unicode_in_Microsoft_Windows#UTF-8, it > explains how, "Microsoft Windows has a code page designated for UTF-8, code > page 65001. Prior to Windows 10 insider build 17035 (November 2017), it was > impossible to set the locale code page to 65001, leaving this code page > only available for (a) explicit conversion functions such as > MultiByteToWideChar and/or (b) the Win32 console command chcp 65001 to > translate stdin/out between UTF-8 and UTF-16. > > Microsoft said that a UTF-8 locale might break some functions (a possible > example is _mbsrev) as they were written to assume multibyte encodings used > no more than 2 bytes per character, thus code pages with more bytes such as > GB 18030 (cp54936) and UTF-8 could not be set as the locale. > > This means that "narrow" functions, in particular fopen (which opens > files), cannot be called with UTF-8 strings, and in fact there is no way to > open all possible files using fopen no matter what the locale is set to > and/or what bytes are put in the string, as none of the available locales > can produce all possible UTF-16 characters. This problem also applies to > all other api that takes or returns 8 bit strings, including Windows ones > such as SetWindowText" > > *Recommendation*: Get rid of UTF-8 Windows support since it isn't very > useful (Linux should be okay though). Use Microsoft's internal default of > UTF-16. There are hundreds of functions in Windows that support ASCII or > UTF-16, but there are none that natively support any other encoding. Doing > this
