Re: [fpc-devel] Save the current FPC UnicodeString!
I see that this should not be discussed here :) . Thanks ! -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
I did not yet try or consider to use the GUI part of MSEGUJI, as my intention is about doing embedded stuff (e.g. with NoGUIApplication). So sorry for may ignorance about this seemingly already thoroughly discussed issue. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
Michael Schnell wrote: So MSE-GUI creates it's own Widget set instead of using something like GTK. Is this really advantageous ? Definitely - depending on your needs. That's the whole reason we started fpGUI Toolkit as well. LCL, GTK, etc simply did not do what we needed. In creating our own GUI toolkit, we could implement everything that was required and get same behaviour and look on all supported platforms (another requirement for our projects). Regards, - Graeme - -- fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
I see. Thanks, -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
On Fri, 13 Nov 2009, Martin Schreiber wrote: On Friday 13 November 2009 15:58:10 Michael Schnell wrote: So MSE-GUI creates it's own Widget set instead of using something like GTK. Is this really advantageous ? Michael, you are joking, right? ;-) Please join us on NNTP: news://news.grid-sky.com/public.mseide-msegui.talk to get answers to the question. Well, at least one or two views out of many. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
Florian Klaempfl wrote: Well, I guess you're talking about lazarus ;) I don't see yet however, if and how the ansistring abuse in the lcl will bite one ... Why should the (future) lcl use ANSIString. I suppose using ANSIString for UTF-8 data was just a nasty hack to do Unicode with the _current_ FPC. (In contrast the MSE-GUI hack was using Widestring, which is less ambiguous and seems to work better on Windows, but needs more conversions on Linux.) Of course the future LCL should use the dynamically encodes strings to allow for using utf-16 on Windows and utf-8 on Linux with the same source code. There should be appropriate means to define the string constants (in the LCL and in the user program) to be stored so that conversions are avoided (maybe on Linux, for optimization, the user would need to have the string constants meant for the GUI saved as UTF-8 and those meant for database handling in utf-16 I'm quite illiterate with db-programming ). -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
Martin Schreiber wrote: On Linux xlib and xft have utf-16 interfaces. What exactly are xlib and xft and why does MSE-GUI seemingly use those while LCL seemingly uses something else ? -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
On Friday 13 November 2009 12:51:58 Michael Schnell wrote: Martin Schreiber wrote: On Linux xlib and xft have utf-16 interfaces. What exactly are xlib and xft and why does MSE-GUI seemingly use those while LCL seemingly uses something else ? xlib is the C-library used under Linux to communicate with the X-server, X-server is the graphic and user interface (keyboard, mouse...) engine for a common Linux installation. xft is a library for advanced font handling and rendering, based on xlib and xrender. xrender is an advanced rendering engine extension for the X-server. xft can render antialiased fonts for example. Lazarus - contrary to fpGUI and MSEgui - does not talk to the X-server directly but uses an intermediate layer, the widgetset. Some of the widgetsets have an utf-16 interface, the Lazarus main widgetset (GTK2) has an utf-8 interface AFAIK. Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
On Friday 13 November 2009 13:26:29 Graeme Geldenhuys wrote: Michael Schnell wrote: Martin Schreiber wrote: On Linux xlib and xft have utf-16 interfaces. What exactly are xlib and xft and why does MSE-GUI seemingly use those while LCL seemingly uses something else ? Martin answered this already. But just to be clear, xlib and xft have both utf-8 and utf-16 interfaces. fpGUI uses the utf-8 interfaces, whereas MSEgui uses the utf-16 interfaces. Often strings are converted to UCS4 somewhere in the chain anyway so it doesn't matter. utf-16-UCS4 can be a little faster maybe. Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
Michael Schnell wrote: Martin Schreiber wrote: On Linux xlib and xft have utf-16 interfaces. What exactly are xlib and xft and why does MSE-GUI seemingly use those while LCL seemingly uses something else ? Martin answered this already. But just to be clear, xlib and xft have both utf-8 and utf-16 interfaces. fpGUI uses the utf-8 interfaces, whereas MSEgui uses the utf-16 interfaces. Regards, - Graeme - -- fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
Zitat von Martin Schreiber fp...@bluewin.ch: On Friday 13 November 2009 13:26:29 Graeme Geldenhuys wrote: Michael Schnell wrote: Martin Schreiber wrote: On Linux xlib and xft have utf-16 interfaces. What exactly are xlib and xft and why does MSE-GUI seemingly use those while LCL seemingly uses something else ? Martin answered this already. But just to be clear, xlib and xft have both utf-8 and utf-16 interfaces. fpGUI uses the utf-8 interfaces, whereas MSEgui uses the utf-16 interfaces. Often strings are converted to UCS4 somewhere in the chain anyway so it doesn't matter. utf-16-UCS4 can be a little faster maybe. For graphic apps the text conversion is hardly measurable. UTF-16 is not sufficient under linux for file names, because Linux uses UTF-8 and allows invalid characters. This happens often when transferring files from Windows. That's why the RTL file operations must use the system encoding under Linux and not UTF-16. Afaik Mac OS X does not allow invalid UTF-8 characters for filenames. Windows has its codepages, with characters that have no unicode equivalent. But afaik this is a rare case, so for almost all programs UTF-16 is sufficient. Mattias ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
So MSE-GUI creates it's own Widget set instead of using something like GTK. Is this really advantageous ? -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
On Fri, 13 Nov 2009, Michael Schnell wrote: So MSE-GUI creates it's own Widget set instead of using something like GTK. Is this really advantageous ? Oh boy, this promises to start a long thread... ... which Jonas is likely to end if he gets fed up with it :-) Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
2009/11/13 Michael Van Canneyt mich...@freepascal.org: On Fri, 13 Nov 2009, Michael Schnell wrote: So MSE-GUI creates it's own Widget set instead of using something like GTK. Is this really advantageous ? Oh boy, this promises to start a long thread... ... which Jonas is likely to end if he gets fed up with it :-) If Martin had a euro for every time someone asked him this, he'd be a very rich man, indeed :) Henry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
On Friday 13 November 2009 15:58:10 Michael Schnell wrote: So MSE-GUI creates it's own Widget set instead of using something like GTK. Is this really advantageous ? Michael, you are joking, right? ;-) Please join us on NNTP: news://news.grid-sky.com/public.mseide-msegui.talk to get answers to the question. Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
On Tuesday 10 November 2009 10:33:07 Florian Klaempfl wrote:
So please don't destroy this ideal solution by dropping current FPC
UnicodeString in favour of the Delphi string which is complicated,
Who says that? If you don't mess with code pages, the only different
you'll might see is that UnicodeString gets two new fields: encoding and
char size. However, this information is usually only used if you pass
the string to a RawString parameters. Normal Unicodestring routines
initialize these fields and that's it.
I can confirm there is not much overhead for the new UnicodeString. I was
mislead by the Delphi {$stringchecks on} option and a misinterpreted comment
from a FPC developer that it is not possible to check codepage compatibility
at compiletime, sorry for that.
Some guesswork gained form my experiments with the cpstrnew branch, Win32,
Russian locale, source in utf-8, {$codepage utf8}, please correct me if I am
wrong:
UnicodeString
- always utf-16 encoded.
- str:= 'abc'; length(str) = 6, stringcodepage(str) = 1200.
- str:= 'abä'; length(str) = 6, stringcodepage(str) = 1200.
- no encoding checks by concanteation, concatenation does not work because of
wrong length() value.
- setlength() of empty string creates CP 1200.
UTF8String
- str:= 'abc'; length(str) = 3, stringcodepage(str) = 65001.
- str:= 'abä'; length(str) = 4, stringcodepage(str) = 65001.
Runtime widestringmanager.Wide2AnsiMoveProc().
- encoding checked by concatenation.
- utf8string:= utf8string + '123' needs conversion to UnicodeString and back.
- setlength() of empty string creates CP 1251.
String1251
- str:= 'abc'; length(str) = 3, stringcodepage(str) = 65001.
- str:= 'abä'; length(str) = 3, stringcodepage(str) = 1251.
Runtime widestringmanager.Wide2AnsiMoveProc().
- str:= 'abc'; str:= str + '123'; needs conversion to UnicodeString and back.
- setlength() of empty string creates CP 1251.
AnsiString
- str:= 'abc'; length(str) = 3, stringcodepage(str) = 65001.
- str:= 'abä'; length(str) = 0, stringcodepage(str) = 1251.
Runtime widestringmanager.Wide2AnsiMoveProc().
- str:= 'abc'; str:= str + '123'; needs conversion to UnicodeString and back.
- setlength() of empty string creates CP 1251.
RawByteString
- str:= 'abc'; length(str) = 3, stringcodepage(str) = 65001.
- str:= 'abä'; length(str) = 0, stringcodepage(str) = 1251.
Runtime widestringmanager.Wide2AnsiMoveProc().
- str:= 'abc'; str:= str + '123'; needs conversion to UnicodeString and back.
- setlength() of empty string creates CP 1251.
- utf8str1:= 'abc'; cp1251str1:= utf8str1; needs conversion to UnicodeString
and back.
- utf8str1:= 'abc'; ansistr1:= utf8str1; no conversion. CP ansistr1 = 65001.
- ansistr1:= 'abc'; utf8str1:= ansistr1; no conversion. CP utf8str1 = 1251.
What are the differences of AnsiString and RawByteString?
Please report when you think cpstrnew branch is stable enough to be tested
with MSEgui.
Thanks, Martin
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
Martin Schreiber schrieb:
On Tuesday 10 November 2009 10:33:07 Florian Klaempfl wrote:
So please don't destroy this ideal solution by dropping current FPC
UnicodeString in favour of the Delphi string which is complicated,
Who says that? If you don't mess with code pages, the only different
you'll might see is that UnicodeString gets two new fields: encoding and
char size. However, this information is usually only used if you pass
the string to a RawString parameters. Normal Unicodestring routines
initialize these fields and that's it.
I can confirm there is not much overhead for the new UnicodeString. I was
mislead by the Delphi {$stringchecks on} option and a misinterpreted comment
from a FPC developer that it is not possible to check codepage compatibility
at compiletime, sorry for that.
Some guesswork gained form my experiments with the cpstrnew branch, Win32,
Russian locale, source in utf-8, {$codepage utf8}, please correct me if I am
wrong:
I'am not sure how far these things are already fixed/supposed to work.
What are the differences of AnsiString and RawByteString?
Ansistring: system encoding
RawByteString: variable encoding, cannot be checked at compile time,
when working with RawByteStrings, you've to take care of the newly
introduced encoding field
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
On Thursday 12 November 2009 19:26:06 Florian Klaempfl wrote: What are the differences of AnsiString and RawByteString? Ansistring: system encoding System encoding at compile time or run time? ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
Martin Schreiber schrieb: On Thursday 12 November 2009 19:26:06 Florian Klaempfl wrote: What are the differences of AnsiString and RawByteString? Ansistring: system encoding System encoding at compile time or run time? Runtime, as it is currently. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
On Thu, Nov 12, 2009 at 9:26 PM, Florian Klaempfl flor...@freepascal.org wrote: Ansistring: system encoding RawByteString: variable encoding, cannot be checked at compile time, when working with RawByteStrings, you've to take care of the newly introduced encoding field Let's say there's a library that works with AnsiString as a string type with no encoding (as it's now). Let's assume that the new branche is officially accepted. The library doesn't want the new feature, but it also doesn't want to change its code from AnsiString to RawByteString. Would it be possible to make the following type declaration: unit MainLibTypes; type AnsiString = RawByteString; to avoid using new auto-converters? thanks, dmitry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
dmitry boyarintsev schrieb: On Thu, Nov 12, 2009 at 9:26 PM, Florian Klaempfl flor...@freepascal.org wrote: Ansistring: system encoding RawByteString: variable encoding, cannot be checked at compile time, when working with RawByteStrings, you've to take care of the newly introduced encoding field Let's say there's a library that works with AnsiString as a string type with no encoding (as it's now). Well, an ansistring has an implicit encoding: system. This won't change and if one uses only unicodestrings and ansistrings, no change will be noticed. Let's assume that the new branche is officially accepted. The library doesn't want the new feature, but it also doesn't want to change its code from AnsiString to RawByteString. Would it be possible to make the following type declaration: unit MainLibTypes; type AnsiString = RawByteString; to avoid using new auto-converters? No, because it would avoid auto-conversion from/to unicodestring as well. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
On Fri, Nov 13, 2009 at 12:44 AM, Florian Klaempfl flor...@freepascal.org wrote: Well, an ansistring has an implicit encoding: system. This won't change and if one uses only unicodestrings and ansistrings, no change will be noticed. Thanks. It's always nice to hear No changes required :) No, because it would avoid auto-conversion from/to unicodestring as well. does your answer means, that: var r : RawByteString; w : WideString; r:=w; // is impossible because of unknown encoding of RawByteString? thanks, dmitry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
dmitry boyarintsev schrieb: On Fri, Nov 13, 2009 at 12:44 AM, Florian Klaempfl flor...@freepascal.org wrote: Well, an ansistring has an implicit encoding: system. This won't change and if one uses only unicodestrings and ansistrings, no change will be noticed. Thanks. It's always nice to hear No changes required :) Well, I guess you're talking about lazarus ;) I don't see yet however, if and how the ansistring abuse in the lcl will bite one ... No, because it would avoid auto-conversion from/to unicodestring as well. does your answer means, that: var r : RawByteString; w : WideString; r:=w; // is impossible because of unknown encoding of RawByteString? No, it is possible becuase the encoding is known because it is stored in RawByteString, but the appropriate conversion will be selected at runtime. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
On Fri, Nov 13, 2009 at 12:55 AM, Florian Klaempfl flor...@freepascal.org wrote: Well, I guess you're talking about lazarus ;) I don't see yet however, if and how the ansistring abuse in the lcl will bite one ... not really. But it's good to know that LCL is safe as well :) thanks, dmitry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
- There is only one encoding in MSEgui framework, it is the same on all platforms. Which is performance wise neither a good decision. - In the majority of cases the fast and convenient character access by index can be used. This is important for beginners. - The current FPC UnicodeString is fast, simple, has little overhead and is well tested. So please don't destroy this ideal solution by dropping current FPC UnicodeString in favour of the Delphi string which is complicated, Who says that? If you don't mess with code pages, the only different you'll might see is that UnicodeString gets two new fields: encoding and char size. However, this information is usually only used if you pass the string to a RawString parameters. Normal Unicodestring routines initialize these fields and that's it. less performant, In which regard? will have many new bugs and is inscrutable by the average user! Please remember how many time it took to fix WideString. I don't want to go this ordeal again. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
In our previous episode, Martin Schreiber said: - The current FPC UnicodeString is fast, simple, has little overhead and is well tested. Do you have any benchmark data on the overhead that delphi adds ? Because I don't think it is much slower if all strings are initialized on the same encoding (like they are on your system) So please don't destroy this ideal solution by dropping current FPC UnicodeString in favour of the Delphi string which is complicated, less performant, will have many new bugs and is inscrutable by the average user! Please remember how many time it took to fix WideString. I don't want to go this ordeal again. I'm no compiler devel, so I have no say in this, but even if widestring is retained there might be a significant revalidation. It will significantly turn the whole stringhandling upside down with many new routines and overloads. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
I don't see why using D2009 strings would hurt that much. If there is a compiler option to encode string constants as utf-16 or maybe an additional plain 16 Bit String type, that encoding would automatically be used for the Strings and thus all have the same encodings and in fact are reference counting WideStrings. The string handling functions would need to compare the character size and encoding (both are stored in a 32 bit word), see that they are equal and now call the original wide string handling functions. So maybe we just additionally need that plain 16 Bit encoding type for the D2009 Strings. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Save the current FPC UnicodeString!
On Tue, Nov 10, 2009 at 7:33 AM, Florian Klaempfl flor...@freepascal.org wrote: - There is only one encoding in MSEgui framework, it is the same on all platforms. Which is performance wise neither a good decision. Could be, if are writing a specialized command line tool or something like that. But for GUI software the conversion speed impact is tiny, and the head-ache caused be a string type whose encoding is not know is by far much worse. Tons of string related code will stop working and need to be adapted at the gain of a couple of microseconds. I don't see why FPC is going in the contrary direction of most of it's users and offering the contrary of what is desired (= fixed encoding strings). Sure you can claim you know better then us what is good for our software, but I don't see how that will end up well. Further Pascal has always been very easy to use, but now we make it harder to use to gain a couple of CPU clocks, while the global tendency in the world is the opposite. Most inovations out there are designed to make software easier to build, not more performant. Garbage Collection for example, Java's virtual environment, etc. Because it is more important to be easier to use then being high performant for the vast majority of uses. Now is any one pushing into harder to use software but more performant? Does it make sense to do such a thing? -- Felipe Monteiro de Carvalho ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
